Jaeyong Sung

I am a postdoctoral scholar at Stanford AI Lab working with Prof. Ken Salisbury. I received my Ph.D. in Computer Science from Cornell University under supervision of Prof. Ashutosh Saxena (Robot Learning Lab).

Publications

Deep Multimodal Embedding: Manipulating Novel Objects with Point-clouds, Language, Trajectories,
Jaeyong Sung, Ian Lenz, Ashutosh Saxena. In International Conference on Robotics and Automation (ICRA), 2017.
Best Cognitive Robotics Paper Finalist
[PDF] [Project Web] [Robobarista Data] bibtex 
@inproceedings{sung_dme_2017,
  title     = {Deep Multimodal Embedding: Manipulating Novel Objects with Point-clouds,
               Language and Trajectories},
  author    = {Jaeyong Sung and Ian Lenz and Ashutosh Saxena},
  booktitle = {International Conference on Robotics and Automation (ICRA)},               
  year      = {2017},
}

Learning to Represent Haptic Feedback for Partially-Observable Tasks,
Jaeyong Sung, J. Kenneth Salisbury, Ashutosh Saxena. In International Conference on Robotics and Automation (ICRA), 2017.
Best Cognitive Robotics Paper Finalist
[PDF] [Project Web] bibtex 
@inproceedings{sung_haptics_2017,
  author    = {Jaeyong Sung and J. Kenneth Salisbury and Ashutosh Saxena},
  title     = {Learning to Represent Haptic Feedback for Partially-Observable Tasks},
  booktitle = {International Conference on Robotics and Automation (ICRA)},           
  year      = {2017},
}

Robobarista: Object Part-based Transfer of Manipulation Trajectories from Crowd-sourcing in 3D Pointclouds,
Jaeyong Sung, Seok Hyun Jin, Ashutosh Saxena. In International Symposium on Robotics Research (ISRR), 2015.
[PDF] [Project Web] [Robobarista Data] bibtex 
@inproceedings{sung_robobarista_2015,
    title={Robobarista: Object Part-based Transfer of Manipulation Trajectories from Crowd-sourcing in 3D Pointclouds},
    author={Jaeyong Sung and Seok Hyun Jin and Ashutosh Saxena},
    booktitle={International Symposium on Robotics Research (ISRR)},
    year={2015}
}

Tell Me Dave: Context-Sensitive Grounding of Natural Language to Mobile Manipulation Instructions,
Dipendra Misra, Jaeyong Sung, Kevin Lee, Ashutosh Saxena. In Robotics: Science and Systems (RSS), 2014.
[PDF] [Project Web] bibtex 
@inproceedings{misra_tellmedave_2014,
    title={Tell Me Dave: Context-Sensitive Grounding of Natural Language to Mobile Manipulation Instructions},
    author={Dipendra Misra and Jaeyong Sung and Kevin Lee and Ashutosh Saxena},
    booktitle={Robotics: Science and Systems (RSS)},
    year={2014}
}

Synthesizing Manipulation Sequences for Under-Specified Tasks using Unrolled Markov Random Fields,
Jaeyong Sung, Bart Selman, Ashutosh Saxena. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2014.
[PDF] [Project Web] bibtex 
@inproceedings{sung_synthesizingsequences_2014,
    title={Synthesizing Manipulation Sequences for Under-Specified Tasks using Unrolled Markov Random Fields},
    author={Jaeyong Sung and Bart Selman and Ashutosh Saxena},
    booktitle={IEEE/RSJ International Conference on Intelligent Robots and Systems},
    year={2014}
}

Unstructured Human Activity Detection from RGBD Images,
Jaeyong Sung, Colin Ponce, Bart Selman, Ashutosh Saxena. In International Conference on Robotics and Automation (ICRA), 2012.
[PDF] [Project Web] [CAD-60 Data] [Code] bibtex 
@inproceedings{sung_rgbdactivity_2012,
    title={Unstructured Human Activity Detection from RGBD Images},
    author={Jaeyong Sung and Colin Ponce and Bart Selman and Ashutosh Saxena},
    booktitle={International Conference on Robotics and Automation (ICRA)},
    year={2012}
}



Journal Paper

Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions,
Dipendra Misra, Jaeyong Sung, Kevin Lee, Ashutosh Saxena. In International Journal of Robotics Research (IJRR), 2015.
bibtex 
@inproceedings{misra_tellmedave_ijrr_2015,
    title={Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions},
    author={Dipendra Misra and Jaeyong Sung and Kevin Lee and Ashutosh Saxena},
    booktitle={International Journal of Robotics Research (IJRR)},
    year={2015}
}



Workshop Papers

Robobarista: Object Part-based Transfer of Manipulation Trajectories from Crowd-sourcing in 3D Pointclouds,
Jaeyong Sung, Seok Hyun Jin, Ashutosh Saxena. In AAAI-RSS Special Workshop on the 50th Anniversary of Shakey, 2015.
Blue Sky Ideas Award Winner!
[PDF] [Project Web] bibtex 
@inproceedings{sung_robobarista_2015,
    title={Robobarista: Object Part-based Transfer of Manipulation Trajectories from Crowd-sourcing in 3D Pointclouds},
    author={Jaeyong Sung and Seok Hyun Jin and Ashutosh Saxena},
    booktitle={International Symposium on Robotics Research (ISRR)},
    year={2015}
}

Learning Sequences of Controllers for Complex Manipulation Tasks,
Jaeyong Sung, Bart Selman, Ashutosh Saxena. In International Conference on Machine Learnong (ICML) workshop on Prediction with Sequential Models, 2013.
[arXiv] [PDF] [Project Web] bibtex 
@inproceedings{sung_learningsequences_2013,
    title={Learning Sequences of Controllers for Complex Manipulation Tasks},
    author={Jaeyong Sung and Bart Selman and Ashutosh Saxena},
    booktitle={International Conference on Machine Learnong (ICML) workshop on Prediction with Sequential Models},
    year={2013}
}

Human Activity Detection from RGBD Images,
Jaeyong Sung, Colin Ponce, Bart Selman, Ashutosh Saxena. In Association for the Advancement of Artificial Intelligence (AAAI) workshop on Pattern, Activity and Intent Recognition, 2011.
[arXiv] [PDF] [CAD-60 Data] [Code] [Project Web] bibtex 
@inproceedings{sung_rgbdactivity_2011,
    title={Human Activity Detection from RGBD Images},
    author={Jaeyong Sung and Colin Ponce and Bart Selman and Ashutosh Saxena},
    booktitle={Association for the Advancement of Artificial Intelligence (AAAI) workshop on Pattern, Activity and Intent Recognition},
    year={2011}
}


Videos

Education

Cornell University

2012 - 2017

PhD in Computer Science
Thesis: Learning to Manipulate Novel Objects for Assistive Robots
Committee: Ashutosh Saxena, J. Kenneth Salisbury, Bart Selman, Francois Guimbretiere, Steve Marschner

Cornell University

2007 - 2011

Bachelor of Science in Computer Science (concentration: Artificial Intelligence)
Minor: Applied Mathematics

Work Experience

Amazon.com

Software Development Engineer - Herndon, VA

08/2011 - 08/2012
  • worked on VM Import/Export service of Amazon Elastic Compute Cloud (EC2)
  • took lead in designing/implementing major workflow as well as database design prior to launching AWS VM Export
  • took lead in integration/deployment/end-to-end testing prior to successful VM Export service launch
  • took lead in developing fast detection of bad machine images for VM Import which greatly reduced customer frustration
  • identified problem in deployment process for VM Import/Export and reduced deployment time from almost full day of an engineer to a few line of command line tools
  • developed internal automatic deployment prep tools which prepares deployment of service component in all AWS regions
  • built components leveraging other AWS services including S3, EC2, EBS, and RDS

Google

Software Engineering Intern - Mountain View, CA

05/2010 - 08/2010
  • worked at Framework / Latin IME Team under Android Team
  • improved word prediction algorithm for soft keyboard of Android OS
  • expanded word prediction model from unigram model to bigram model
  • generated tools for filtering massive English dictionary and generated compact dictionary

Samsung Electronics

Intern - Suwon, Korea

12/2009 - 01/2010
  • worked at Modem S/W Lab under Modem SoC Team
  • learned to develop embedded system using ARM processor
  • implemented different types of ARQ (Automatic Repeat Request) feedback for mobile WiMAX chipset and analyzed performance of each type

Teaching Experience

  • TA for CS 4758/6758: Robot Learning (Spring 2013)
  • TA for CS 4700: Foundations of Artificial Intelligence (Fall 2012)
  • Consultant for CS 2110: Object-Oriented Programming and Data Structures (Spring 2011)
  • TA for CS 2046: Intro to Mobile Application Development (Fall 2010)

Talks

  • OpenAI (Nov. 2016)
  • Google Tech Talk (Oct. 2015)
  • UC Berkeley (Sep. 2015)

Popular Press