Far Away in the Deep Space: Dense Nearest-Neighbor-Based Out-of-Distribution Detection

IEEE International Conference on Computer Vision (ICCV), Workshops, 2023
Abstract: The key to out-of-distribution detection is density estimation of the in-distribution data or of its feature representations. This is particularly challenging for dense anomaly detection in domains where the in-distribution data has a complex underlying structure. Nearest-Neighbors approaches have been shown to work well in object-centric data domains, such as industrial inspection and image classification. In this paper, we show that nearest-neighbor approaches also yield state-of-the-art results on dense novelty detection in complex driving scenes when working with an appropriate feature representation. In particular, we find that transformer-based architectures produce representations that yield much better similarity metrics for the task. We identify the multi-head structure of these models as one of the reasons, and demonstrate a way to transfer some of the improvements to CNNs. Ultimately, the approach is simple and non-invasive, i.e., it does not affect the primary segmentation performance, refrains from training on examples of anomalies, and achieves state-of-the-art results on RoadAnomaly, StreetHazards, and SegmentMeIfYouCan-Anomaly.
Paper Poster

Images and movies


BibTex reference

  author       = "S. Galesso and M. Argus and T. Brox",
  title        = "Far Away in the Deep Space: Dense Nearest-Neighbor-Based Out-of-Distribution Detection",
  booktitle    = "IEEE International Conference on Computer Vision (ICCV), Workshops",
  month        = " ",
  year         = "2023",
  url          = "http://lmb.informatik.uni-freiburg.de/Publications/2023/GAB23"

Other publications in the database