Publications
Conference Papers
-
Deep Multimodal Embedding: Manipulating Novel Objects with Point-clouds, Language, Trajectories
International Conference on Robotics and Automation (ICRA), 2017
Best Cognitive Robotics Paper Finalist@inproceedings{sung_dme_2017, title = {Deep Multimodal Embedding: Manipulating Novel Objects with Point-clouds, Language and Trajectories}, author = {Jaeyong Sung and Ian Lenz and Ashutosh Saxena}, booktitle = {International Conference on Robotics and Automation (ICRA)}, year = {2017}, } -
Learning to Represent Haptic Feedback for Partially-Observable Tasks
International Conference on Robotics and Automation (ICRA), 2017
Best Cognitive Robotics Paper Finalist@inproceedings{sung_haptics_2017, author = {Jaeyong Sung and J. Kenneth Salisbury and Ashutosh Saxena}, title = {Learning to Represent Haptic Feedback for Partially-Observable Tasks}, booktitle = {International Conference on Robotics and Automation (ICRA)}, year = {2017}, } -
Robobarista: Object Part-based Transfer of Manipulation Trajectories from Crowd-sourcing in 3D Pointclouds
International Symposium on Robotics Research (ISRR), 2015
@inproceedings{sung_robobarista_2015, title={Robobarista: Object Part-based Transfer of Manipulation Trajectories from Crowd-sourcing in 3D Pointclouds}, author={Jaeyong Sung and Seok Hyun Jin and Ashutosh Saxena}, booktitle={International Symposium on Robotics Research (ISRR)}, year={2015} } -
Tell Me Dave: Context-Sensitive Grounding of Natural Language to Mobile Manipulation Instructions
Robotics: Science and Systems (RSS), 2014
@inproceedings{misra_tellmedave_2014, title={Tell Me Dave: Context-Sensitive Grounding of Natural Language to Mobile Manipulation Instructions}, author={Dipendra Misra and Jaeyong Sung and Kevin Lee and Ashutosh Saxena}, booktitle={Robotics: Science and Systems (RSS)}, year={2014} } -
Synthesizing Manipulation Sequences for Under-Specified Tasks using Unrolled Markov Random Fields
IEEE/RSJ International Conference on Intelligent Robots and Systems, 2014
@inproceedings{sung_synthesizingsequences_2014, title={Synthesizing Manipulation Sequences for Under-Specified Tasks using Unrolled Markov Random Fields}, author={Jaeyong Sung and Bart Selman and Ashutosh Saxena}, booktitle={IEEE/RSJ International Conference on Intelligent Robots and Systems}, year={2014} } -
Unstructured Human Activity Detection from RGBD Images
International Conference on Robotics and Automation (ICRA), 2012
@inproceedings{sung_rgbdactivity_2012, title={Unstructured Human Activity Detection from RGBD Images}, author={Jaeyong Sung and Colin Ponce and Bart Selman and Ashutosh Saxena}, booktitle={International Conference on Robotics and Automation (ICRA)}, year={2012} }
Journal
-
Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions
International Journal of Robotics Research (IJRR), 2015
@inproceedings{misra_tellmedave_ijrr_2015, title={Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions}, author={Dipendra Misra and Jaeyong Sung and Kevin Lee and Ashutosh Saxena}, booktitle={International Journal of Robotics Research (IJRR)}, year={2015} }
Workshop Papers
-
Robobarista: Object Part-based Transfer of Manipulation Trajectories from Crowd-sourcing in 3D Pointclouds
AAAI-RSS Special Workshop on the 50th Anniversary of Shakey, 2015
Blue Sky Ideas Award Winner!@inproceedings{sung_robobarista_2015, title={Robobarista: Object Part-based Transfer of Manipulation Trajectories from Crowd-sourcing in 3D Pointclouds}, author={Jaeyong Sung and Seok Hyun Jin and Ashutosh Saxena}, booktitle={International Symposium on Robotics Research (ISRR)}, year={2015} } -
Learning Sequences of Controllers for Complex Manipulation Tasks
International Conference on Machine Learnong (ICML) workshop on Prediction with Sequential Models, 2013
@inproceedings{sung_learningsequences_2013, title={Learning Sequences of Controllers for Complex Manipulation Tasks}, author={Jaeyong Sung and Bart Selman and Ashutosh Saxena}, booktitle={International Conference on Machine Learnong (ICML) workshop on Prediction with Sequential Models}, year={2013} } -
Human Activity Detection from RGBD Images
Association for the Advancement of Artificial Intelligence (AAAI) workshop on Pattern, Activity and Intent Recognition, 2011
@inproceedings{sung_rgbdactivity_2011, title={Human Activity Detection from RGBD Images}, author={Jaeyong Sung and Colin Ponce and Bart Selman and Ashutosh Saxena}, booktitle={Association for the Advancement of Artificial Intelligence (AAAI) workshop on Pattern, Activity and Intent Recognition}, year={2011} }
Videos
Education & Experience
- 2012 - 2017
Cornell University
PhD in Computer Science
Thesis: Learning to Manipulate Novel Objects for Assistive Robots
Committee: Ashutosh Saxena, J. Kenneth Salisbury, Bart Selman, Francois Guimbretiere, Steve Marschner - 2007 - 2011
Cornell University
Bachelor of Science in Computer Science (concentration: Artificial Intelligence)
Minor: Applied Mathematics - 08/2011 - 08/2012
Amazon.com
Software Development Engineer - Herndon, VA- worked on VM Import/Export service of Amazon Elastic Compute Cloud (EC2)
- took lead in designing/implementing major workflow as well as database design prior to launching AWS VM Export
- took lead in integration/deployment/end-to-end testing prior to successful VM Export service launch
- took lead in developing fast detection of bad machine images for VM Import which greatly reduced customer frustration
- identified problem in deployment process for VM Import/Export and reduced deployment time from almost full day of an engineer to a few line of command line tools
- developed internal automatic deployment prep tools which prepares deployment of service component in all AWS regions
- built components leveraging other AWS services including S3, EC2, EBS, and RDS
- 05/2010 - 08/2010
Google
Software Engineering Intern - Mountain View, CA- worked at Framework / Latin IME Team under Android Team
- improved word prediction algorithm for soft keyboard of Android OS
- expanded word prediction model from unigram model to bigram model
- generated tools for filtering massive English dictionary and generated compact dictionary
- 12/2009 - 01/2010
Samsung Electronics
Intern - Suwon, Korea- worked at Modem S/W Lab under Modem SoC Team
- learned to develop embedded system using ARM processor
- implemented different types of ARQ (Automatic Repeat Request) feedback for mobile WiMAX chipset and analyzed performance of each type
Teaching & Talks
- Teaching
- TA for CS 4758/6758: Robot Learning (Spring 2013)
- TA for CS 4700: Foundations of Artificial Intelligence (Fall 2012)
- Consultant for CS 2110: Object-Oriented Programming and Data Structures (Spring 2011)
- TA for CS 2046: Intro to Mobile Application Development (Fall 2010)
- Talks
- OpenAI (Nov. 2016)
- Google Tech Talk (Oct. 2015)
- UC Berkeley (Sep. 2015)
Popular Press
Robobarista, 2015
- IEEE Spectrum · PR2 Robot Figures Out How to Make a Latte
- Wired · Crowdsourcing a coffee-pouring, juice-making robot
- Popular Science · Robobarista can learn how to operate your coffee machine
- CNet · Robobarista autonomously makes latte, taught by the Internet
- Huffington Post · Cats, Robot Baristas, Tricorders, and the Future of Deep Learning
Tell Me Dave, 2014
- LA Times · New robot learns from plain speech, not computer code
- New Scientist · Make robots useful by teaching them to talk like us
- Wired · Help train robots to understand natural language
- IEEE Spectrum · Robot Responds to Natural Language Instructions, Brings You Fancy Ice Cream
- CNet · The robot that can be programmed by talking to it
- Washington Post · This robot can make you ice cream
- GigaOm · New research shows robots can learn quite a bit when fed a lot of data
- TechCrunch · Tell Me Dave Lets You Train A Robot To Respond To Complex Commands
- Venture Beat · Cornell team creates robot that understands casual commands, makes dessert
- Smithsonian Magazine · Robots Are Smart -- But Can They Understand Us?
Lab feature, 2013
Human Activity Detection, 2011
- E&T Magazine · Gaming technology helps robots recognise human activity
- CornellChronicle · Researchers teach robots to recognize what we're doing
- Phys.org · Teaching robots to identify human activities
- R&D Magazine · Teaching robots to identify human activities
- Gizmag · Kinect-based system developed that recognizes what you're doing
- i-programmer · Kinect knows what you are doing...
- GizmoWatch · Kinect-based system keeps all your activities under observation
- WonderHowTo · Watch Out! Kinect-Based AI System Knows What You're Up To
- myScience · Researchers teach robots to recognize what we're doing