ADFQ (Assumed Density Filtering Q-learning)
Assumed Density Filtering is a Bayesian counterpart method of Q-learning. ADFQ demonstrated that it could improve some of the issues from the greedy update of Q-learning by showing the quicker convergence to the optimal Q-values than Q-learning and surpassing DQN and Double DQN in various Atari games especially in stochastic domains and/or domains with a large action set.
H. Jeong, C. Zhang, G. J. Pappas, and D. D. Lee "Assumed Density Filtering Q-learning," the 28th International Joint Conference on Artificial Intelligence (IJCAI), Macao, China, 2019 (pdf)
Learning Q-network for Active Information Acquisition
This project proposes a novel RL framework for the active information acquisition problem and developed a detailed approach for solving the active target tracking problem with a Q-network-based RL algorithm.
The experimental results demonstrated that the RL-based method can outperform the search-based planning algorithm.
H. Jeong, B. Schlotfeldt, H. Hassani, M. Morari, D. D. Lee, and G. J. Pappas, "Learning Q-network for Active Information Acquisition," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 2019 (pdf)
Learning Stand-up Motion for Humanoid Robots
Standing up after falling is an essential ability for humanoid robots in order to resume their tasks without help from humans. In this research, we applied Q-learning in order to learn stand-up motions for humanoid robots. We discretized the continuous state and action spaces using a clustering method, the Expectation-Maximization (EM) algorithm for Gaussian Mixtures.
We implemented this method on a DarwIn-OP humanoid robot. With the optimal policy learned by this method, the robot was able to successfully and efficiently stand up from different fallen positions while the manually designed stand-up motions failed or took longer to get up.
H. Jeong and D. D. Lee, "Efficient Learning of Stand-up Motion for Humanoid Robots with Bilateral Symmetry," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 2016
H. Jeong and D. D. Lee, "Learning Complex Stand-up Motion for Humanoid Robots," in Proc. of the 30th AAAI Conference on Artificial Intelligence (AAAI-16), Phoenix, AZ, 2016 (pdf)
Robocup 2015 : Humanoid Adult-size League Winner
Robocup is an annual international robotics competition where different kinds of robots play soccer and compete with each other.
In 2015, UPenn's Team THORwIn participated in the adult-size humanoid league and won the championship.
S. Yi, S. G. McGill, H. Jeong, J. Huh, M. Missura, H. Yi, M. Ahn, S. Cho, K. Liu, D. Hong and D. D. Lee, “RoboCup 2015 Humanoid Adult-Size League Winner,” RoboCup 2015: Robot World Cup XIX, vol. 9513, pp.132-143, Springer International Publishing, 2015 (pdf)
DARPA Robotics Challenge Finals
The goal of DARPA Robotics Challenge (DRC) was to develop human-supervised ground robots capable of executing complex tasks in dangerous, degraded, human-engineered environments. Team THOR, the joint team of UPenn and UCLA, passed the DRC trial in 2013 and compete in DRC Finals in 2015.
The challenge consisted of eight tasks including driving a car, rotating a valve, and climbing stairs.
S. G. McGill, S. Yi, H. Yi, M. Ahn, S. Cho, K. Liu, D. Sun, B. Lee, H. Jeong, J. Huh, D. Hong and D. D. Lee, "Team THOR's Entry in the DARPA Robotics Challenge Finals 2015," Journal of Field Robotics (link)
PIBOT : A humanoid robot flying an airplane
In this project, we developed a framework of automating a vehicle, an airplane particular, using a humanoid robot. In our experiment, we used Bioloid premium humanoid robot by ROBOTIS, Inc. The robot successfully completed basic flight maneuvers from take-off to landing using control equipment designed for humans and the X-plane flight simulator.
H. Jeong, D. H. Shim and S. Cho, "A Robot-Machine Interface for Full-functionality Automation using a Humanoid," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, IL, 2014 (pdf)
H. Jeong, J. Kim and D. H. Shim, "Development of an Optimality Piloted Vehicle using a Humanoid Robot," AIAA SciTech: 52nd Aerospace Science Meeting, Washington, DC, 2014 (pdf)