FlowControl: Optical Flow Based Visual Servoing

Max Argus, L. Hermann, J. Long, Thomas Brox
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020
Abstract: One-shot imitation is the vision of robot programming from a single demonstration, rather than by tedious construction of computer code. We present a practical method for realizing one-shot imitation for manipulation tasks, exploiting modern learning-based optical flow to perform real-time visual servoing. Our approach, which we call FlowControl, continuously tracks a demonstration video, using a specified foreground mask to attend to an object of interest. Using RGB-D observations, FlowControl requires no 3D object models, and is easy to set up. FlowControl inherits great robustness to visual appearance from decades of work in optical flow. We exhibit FlowControl on a range of problems, including ones requiring very precise motions, and ones requiring the ability to generalize.

Other associated files : video-flowcontrol.pdf [5.9MB]  

Images and movies


See also

BibTex reference

  author       = "M. Argus and L. Hermann and J. Long and T. Brox",
  title        = "FlowControl: Optical Flow Based Visual Servoing",
  booktitle    = "IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)",
  month        = " ",
  year         = "2020",
  url          = "http://lmb.informatik.uni-freiburg.de/Publications/2020/AB20"

Other publications in the database