Edwin Pan

Graduate Research Assistant @ SAIL

Interactive Perception and Robotics Laboratory

Stanford Vision and Learning Lab

Partnership in AI-Assisted Care

About

Hello there! My name is Edwin, and I’m a graduate student at Stanford University working on intelligent sensing through computer vision and robotics.

In short, teaching robots to act based on what they see.

I received my Bachelor of Science in Electrical Engineering with Highest Honors from The University of Illinois at Urbana-Champaign in 2020, along with a minor in Computer Science focused on artificial intelligence. I led a start-up company called PreSense through acquisition as part of the Alchemy Deep Tech Incubator Program, with advising from Professor Sanjay Patel, Professor Wen-mei Hwu, and Jinjun Xiong.

My primary research explores video-based scene understanding, it’s usefulness for understanding deformable objects, and it’s applications in robotic manipulation.

Long term, I want to enable robotic care-givers designed to do no harm.

Interests

Computer Vision
Machine Learning
Robotic Manipulation

Education

MS in Electrical Engineering, 2023

Stanford University
BS in Electrical Engineering (CS Minor), 2020

University of Illinois at Urbana-Champaign

Experience

Graduate Research Assistantship

Stanford Artificial Intelligence Laboratory

Mar 2022 – Present Stanford, California

I am building robotic systems that can complete tasks with deformable objects.

Computer Vision Research Intern

DawnLight Technologies

Jun 2021 – Sep 2021 Palo Alto, California

I built object detection networks for the healthcare domain.

Perception Intern

Jun 2021 – Jun 2021 Mountain View, California

Systems Engineering Intern

May 2017 – Aug 2020 Austin, Texas

Summer 2017, 2018, 2020

Produced a Bayesian approach to occupancy grid mapping for an autonomous robot system. The Uhnder digital radar provides rich 4D information, making it an attractive platform for distilling perception insights.

Publications

Transformers with Topological Attention (TTA)

Submission is in progress

Edwin Pan, others

Single-Shot Object Action Recognition (SOAR)

Submission is in progress

Edwin Pan, others

Object Classification Using Range-Doppler Plots from a High Density PMCW MIMO mmWave Radar

Classification of detections as either running pedestrians or driving cars is achieved using only the range-doppler information from a single forward facing automotive PMCW radar system.

Edwin Pan

Object Classification Using Range-Doppler Plots from a High Density PMCW MIMO mmWave Radar

OpenRadar: A Toolkit for Prototyping mmWave Radar Applications

Open source toolkit for improved integration of machine learning with Texas Instruments millimeter wave FMCW radar sensor systems.

Arjun Gupta, Dashiell Kosaka, Edwin Pan, Jingning Tang, Ruihao Yao, Sanjay Patel

OpenRadar: A Toolkit for Prototyping mmWave Radar Applications

Projects

Blackjack: Are the ”Basic Strategies” Sufficient?

This project compares the performance of a Q-learning based reinforcement learning approach with the basic strategy advised by online guides for learning to play Blackjack. The results show that the Q-learning algorithm significantly outperformed the basic strategy, achieving a higher win rate and average return over a large number of simulated games. The findings suggest that Q-learning can be an effective approach to learning to play Blackjack, offering a better strategy than the basic strategy recommended by online guides.

Blackjack: Are the ”Basic Strategies” Sufficient?

ViPER：Virtual Reality Pose Estimator and Viewer

3D human pose tracking is difficult. Recent works in the computer vision space have made major strides in estimating 3D human motion from monocular video. The core aspect of our project is the question：can motions captured from video be used in games? To answer this question, we built a VR game in Unity that simultaneously utilizes both video-distilled motions and traditional motion capture motions. This allows us to evaluate the accuracy and usability of these motions in their native 3D environment.

ViPER：Virtual Reality Pose Estimator and Viewer

Imitation Learning with iGibson Non-Embodied Demonstrations

Humans posess a vast sensorimotor knowledge base that must be utilized to accelerate robot policy learning. While existing methods of demonstration generally ask humans to directly manipulate the robot’s state, this work utilizes a more natural demonstration capture platform (i.e. mouse and keyboard) to train deep learning networks on a motion planning task. Through qualitative and quantitative analysis, we demonstrate that such methods can successfully supervise imitation learning policies.

Imitation Learning with iGibson Non-Embodied Demonstrations

MetaHDR：Model-Agnostic Meta-Learning for HDR Image Reconstruction

By learning the common structure between all LDR-to-HDR conversion tasks, our model is able to adapt it’s predictions given extra exposures of a scene. This novel approach reframes LDR-to-HDR conversion as a meta-learning problem.

MetaHDR：Model-Agnostic Meta-Learning for HDR Image Reconstruction

Using Spherical Harmonics to Generate Believable Lighting for Arbitrary Environments

Realistic global illumination can achieved through sequential techniques like ray tracing. However, can be prohibitably expensive and inefficient in realtime applications. With Spherical Harmonic lighting (first proposed in SIGGRAPH 2002 by Sloan, Kautz and Snyder), we can compute global illumination for dynamic 3D scenes in realtime. This project reimplements this technique.

Using Spherical Harmonics to Generate Believable Lighting for Arbitrary Environments

Meta-Regularization by Enforcing Mutual-Exclusiveness

This work proposes a regularization technique for meta-learning models that gives the model designer more control over the information flow during meta-training. Our method consists of a regularization function that is constructed by maximizing the distance between task-summary statistics, in the case of black-box models and task specific network parameters in the case of optimization based models during meta-training. Our proposed regularization function shows an accuracy boost of ∼ 36% on the Omniglot dataset for 5-way, 1-shot classification using black-box method and for 20-way, 1-shot classification problem using optimization-based method.

Meta-Regularization by Enforcing Mutual-Exclusiveness

Generating Segmentation Masks for Autonomous Vehicle Application via Style Transfer Framework

This final project explores the feasibility of using a GAN based style-transfer approach to generating semantic segmentation masks for autonomous driving application. Specifically, CycleGAN and pix2pix are benchmarked against a more traditional U-Net architecture built off of Resnet-18. The project uses a subset of the image data from the ApolloScape open dataset.

Generating Segmentation Masks for Autonomous Vehicle Application via Style Transfer Framework

Automotive mmWave FMCW Radar-based Micro-Doppler Signature Feature Extraction and Classification

Millimeter-wave radar has the potential to offer a weather-agnostic sensing solution to pedestrian classification. To this end, I applied a CNN architecture to micro-doppler spectrogram data simulated through Matlab in an attempt to distinguish between a pedestrian radar motion spectrogram and a bicycle radar motion spectrogram.

Automotive mmWave FMCW Radar-based Micro-Doppler Signature Feature Extraction and Classification

Adventures

20+ mile hike over the span of 10+ hours, starting at 2AM. Pristine Milky Way, a pre-dawn calm, gradual sunrise, all out in the middle of Yosemite Valley with hardly a soul in sight.

Edwin Pan, Anthony

Last updated on Aug 11, 2021 3 min read Hike, Nature

Half Dome Hike

Pekka the Curious Canine

A running chronicle of Pekka’s exploits

Edwin Pan, Pekka

Last updated on Aug 10, 2021 1 min read Pet, Dog, German Shepherd

Pekka the Curious Canine