Deep Learning for Behavior Prediction

Visit my Linkedin profile and see Waymo's recent publications to learn more about my roles and what we do at Waymo to predict agent behaviors for autonomous vehicles. 

Learning to Track Dynamic Targets in Partially Known Environments

In this project, we solve active target tracking using a deep reinforcement learning (RL) approach. In particular, we introduce Active Tracking Target Network (ATTN), a unified RL policy that is capable of solving major sub-tasks of active target tracking -- in-sight tracking, navigation, and exploration. The policy shows robust behavior for tracking agile and anomalous targets with a partially known target model. Additionally, the same policy is able to navigate in obstacle environments to reach distant targets as well as explore the environment when targets are positioned in unexpected locations.


Learning Q-network for Active Information Acquisition

This project proposes a novel RL framework for the active information acquisition problem and developed a detailed approach for solving the active target tracking problem with a Q-network-based RL algorithm.

The experimental results demonstrated that the RL-based method can outperform the search-based planning algorithm.


H. Jeong, B. Schlotfeldt, H. Hassani, M. Morari, D. D. Lee, and G. J. Pappas, "Learning Q-network for Active Information Acquisition," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 2019 (pdf)

ADFQ (Assumed Density Filtering Q-learning)


Assumed Density Filtering is a Bayesian counterpart method of Q-learning. ADFQ demonstrated that it could improve some of the issues from the greedy update of Q-learning by showing the quicker convergence to the optimal Q-values than Q-learning and surpassing DQN and Double DQN in various Atari games especially in stochastic domains and/or domains with a large action set.


H. Jeong, C. Zhang, G. J. Pappas, and D. D. Lee "Assumed Density Filtering Q-learning," the 28th International Joint Conference on Artificial Intelligence (IJCAI), Macao, China, 2019 (pdf)

Pod Storage with Deep Reinforcement Learning


During my internship at Amazon Robotics in the Research & Development team, I worked on solving Amazon's pod storage problem using a state-of-the-art deep reinforcement learning - Trust Region Policy Optimization (TRPO). 

Learning Stand-up Motion for Humanoid Robots

Standing up after falling is an essential ability for humanoid robots in order to resume their tasks without help from humans. In this research, we applied Q-learning in order to learn stand-up motions for humanoid robots. We discretized the continuous state and action spaces using a clustering method, the Expectation-Maximization (EM) algorithm for Gaussian Mixtures.  

We implemented this method on a DarwIn-OP humanoid robot. With the optimal policy learned by this method, the robot was able to successfully and efficiently stand up from different fallen positions while the manually designed stand-up motions failed or took longer to get up.



H. Jeong and D. D. Lee, "Efficient Learning of Stand-up Motion for Humanoid Robots with Bilateral Symmetry," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 2016 

H. Jeong and D. D. Lee, "Learning Complex Stand-up Motion for Humanoid Robots," in Proc. of the 30th AAAI Conference on Artificial Intelligence (AAAI-16), Phoenix, AZ, 2016 (pdf)

Robocup 2015 : Humanoid Adult-size League Winner


Robocup is an annual international robotics competition where different kinds of robots play soccer and compete with each other.  

In 2015, UPenn's Team THORwIn participated in the adult-size humanoid league and won the championship.


S. Yi, S. G. McGill, H. Jeong, J. Huh, M. Missura, H. Yi, M. Ahn, S. Cho, K. Liu, D. Hong and D. D. Lee, “RoboCup 2015 Humanoid Adult-Size League Winner,” RoboCup 2015: Robot World Cup XIX, vol. 9513, pp.132-143, Springer International Publishing, 2015 (pdf

DARPA Robotics Challenge Finals


The goal of DARPA Robotics Challenge (DRC) was to develop human-supervised ground robots capable of executing complex tasks in dangerous, degraded, human-engineered environments. Team THOR, the joint team of UPenn and UCLA, passed the DRC trial in 2013 and compete in DRC Finals in 2015. 

The challenge consisted of eight tasks including driving a car, rotating a valve, and climbing stairs. 


S. G. McGill, S. Yi, H. Yi, M. Ahn, S. Cho, K. Liu, D. Sun, B. Lee, H. Jeong, J. Huh, D. Hong and D. D. Lee, "Team THOR's Entry in the DARPA Robotics Challenge Finals 2015," Journal of Field Robotics (link)

PIBOT : A humanoid robot flying an airplane


In this project, we developed a framework of automating a vehicle, an airplane particular, using a humanoid robot. In our experiment, we used Bioloid premium humanoid robot by ROBOTIS, Inc. The robot successfully completed basic flight maneuvers from take-off to landing using control equipment designed for humans and the X-plane flight simulator. 


H. Jeong, D. H. Shim and S. Cho, "A Robot-Machine Interface for Full-functionality Automation using a Humanoid," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, IL, 2014 (pdf)

H. Jeong, J. Kim and D. H. Shim, "Development of an Optimality Piloted Vehicle using a Humanoid Robot," AIAA SciTech: 52nd Aerospace Science Meeting, Washington, DC, 2014 (pdf)