Robust Semantic Segmentation using Deep Fusion

A. Valada, Gabriel Leivas Oliveira, Thomas Brox, W. Burgard
Robotics: Science and Systems (RSS 2016) Workshop, Are the Sceptics Right? Limits and Potentials of Deep Learning in Robotics, 2016
Abstract: Robust semantic scene understanding of unstructured environments is critical for robots operating in the real world. Several inherent natural factors such as shadows, glare and snow make this problem highly challenging, especially using RGB images. In this paper, we propose the use of multispectral and multimodal images to increase robustness of segmentation in real-world outdoor environments. Deep Convolutional Neural Network (DCNN) architectures define the state of the art in various segmentation tasks, however, architectures that incorporate fusion has not been sufficiently explored. We explore early and late fusion architectures for dense pixel-wise segmentation from RGB, Near-InfraRed (NIR) channels, and depth data. We identify data augmentation strategies that enable training of very deep fusion models using small datasets. We qualitatively and quantitatively evaluate our approach and show it exceeds several other state of the art architectures. In addition, we present experimental results for segmentation under challenging real-world conditions. Demo and dataset is publicly available at http://deepscene.cs.uni-freiburg.de.

Other associated files : valada16rssws.pdf [5.8MB]  

Images and movies


BibTex reference

  author       = "A. Valada and G. Oliveira and T.Brox and W. Burgard",
  title        = "Robust Semantic Segmentation using Deep Fusion",
  booktitle    = "Robotics: Science and Systems (RSS 2016) Workshop, Are the Sceptics Right{\i} Limits and Potentials of Deep Learning in Robotics",
  month        = " ",
  year         = "2016",
  url          = "http://lmb.informatik.uni-freiburg.de/Publications/2016/OB16d"

Other publications in the database