Motion segmentation and multiple object tracking by correlation co-clustering

Margret Keuper, S. Tang, B. Andres, Thomas Brox, B. Schiele
IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(1): 140-153, Jan 2020
Abstract: Models for computer vision are commonly defined either w.r.t. low-level concepts such as pixels that are to be grouped, or w.r.t. high-level concepts such as semantic objects that are to be detected and tracked. Combining bottom-up grouping with top-down detection and tracking, although highly desirable, is a challenging problem. We state this joint problem as a co-clustering problem that is principled and tractable by existing algorithms. We demonstrate the effectiveness of this approach by combining bottom-up motion segmentation by grouping of point trajectories with high-level multiple object tracking by clustering of bounding boxes. We show that solving the joint problem is beneficial at the low-level, in terms of the FBMS59 motion segmentation benchmark, and at the high-level, in terms of the Multiple Object Tracking benchmarks MOT15, MOT16 and the MOT17 challenge, and is state-of-the-art in some metrics.
Publisher's link

Other associated files : keuper_tpami20.pdf [7.2MB]  

Images and movies


BibTex reference

  author       = "M. Keuper and S. Tang and B. Andres and T. Brox and B. Schiele",
  title        = "Motion segmentation and multiple object tracking by correlation co-clustering",
  journal      = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
  number       = "1",
  volume       = "42",
  pages        = "140-153",
  month        = "Jan",
  year         = "2020",
  url          = "http://lmb.informatik.uni-freiburg.de/Publications/2020/KB20"

Other publications in the database