Robot task learning by active tracking of hands

Project members

Prof. Dr. Thomas Brox
Prof. Dr. Wolfram Burgard
Christian Zimmermann
Gabriel Oliveira

Abstract

The goal of this project is to develop the technology for service robots that learn from observation of human interaction with objects. With current techniques, a robot learns a task such as moving a bottle from the shelf to the table, by using kinesthetic training, manual scripting, or lengthy programming procedures. This proposal aims for a system that will render the teaching process easy also to the non-expert. It emphasizes the idea of \textit{learning-through-observation} in a largely unconstrained environment. The robot observes the actions of the human operator by tracking their hands and by automatically detecting and tracking the manipulated object. From this data, the robot will learn a model to repeat the task in an unsupervised way. By using a mobile tablet device, the robot can be commanded to perform a learned task or to complete a learned action started by a human. To this end, new methods from computer vision as well as robot learning and manipulation will be developed, and the performance of existing methods will be improved.

Publications