I-search

I-Search Project

Project Members
Introduction
References and Links
Publications
Demos and Tools

Project Members

Introduction

Visual information retrieval is a new subject of research in information technology. Its purpose is to retrieval from a database, images or image sequences that are relevant to a query. It is an extension of traditional information retrieval designed to include visual media.

The variety of knowledge required in visual information retrieval is large. Different research fields, which have involved separately, provide valuable contributions to this new research subject. Information retrieval, visual data modeling and representation, image/video analysis and processing, pattern recognition and computer vision, multimedia database organization, multidimensional indexing, psychological modeling of user behavior, man-machine interaction and data visualization, are the most important research fields that contribute to visual information retrieval.

New-generation visual information retrieval systems support full retrieval by visual content. Access to visual information is not only performed at a conceptual level, using keywords as in the textual domain, but also at a perceptual level, using objective measurements of visual content and appropriate similarity models. The contents include:

Perceptual properties, like color, texture, shape and spatial relationships
Semantic primitives corresponding to abstractions, like objects, roles and scenes
Impressions, emotions and meaning associated with the combination of perceptual features

In typical content-based image retrieval systems (Figure 1), the visual contents of the images in the database are extracted and described by multi-dimensional feature vectors. The
feature vectors of the images in the database form a feature database. To retrieve images, users provide the retrieval system with example images or sketched figures. The system then changes these examples into its internal representation of feature vectors. The similarities /distances between the feature vectors of the query example or sketch and those of the images in the database are then calculated and retrieval is performed with the aid of an indexing scheme. The indexing scheme provides an efficient way to search for the image database. Recent retrieval systems have incorporated users' relevance feedback to modify the retrieval process in order to generate perceptually and semantically more meaningful retrieval results.

Following are the problems to be solved in a CBIR system:

Feature representation How to represent a feature from an image?
Similarity measure How to design a similarity measure algorithm?
User interaction How to create a mathematically description of a query, how to select a feature subset for a specific query, and how to improve performance by relevance feedback?
Evaluation How to evaluation the performance of a CBIR system?

Methods

SIMBA

We have developed a demo image retrieval system-SIMBA(Search IMages By Appearance). Our approach is based on invariant features, i.e. features that do not vary if the image is transformed by some transformation group (we will consider translation and rotation here). Schulz-Mirbach introduced an algorithm for the construction of invariant features [Schulz-Mirbach:1995] which is very suitable because of its robustness to slight topological deformations and even to independent motion of objects within the image. The major advantage is that it does not require the extraction of objects (segmentation), or distinct points (key-points) from the image, but can be applied directly to the original image data.

However, in order to improve the algorithm's robustness in an image retrieval application - especially for supporting partial matches - we had to modify it, so that more local information is preserved in the final features. Thus we constructed feature histograms [Siggelkow, Burkhardt:1998], which are very similar to the well known color histograms but consider features drawn from a local neighborhood of each pixel instead of just using the color value of each pixel only. Thus we incorporate also textural information.

Recently the method was further enhanced by a fast estimation of the features instead of a tedious calculation. Thus the extracted features will have a small error which, however, can be well estimated [Siggelkow, Schael:1999].

Enhancement during I-Search

The I-Search project aims at the construction of a content-based image retrieval system, which is successfully implemented. Our contribution is the development and enhancement of image representation and image similarity algorithms. We developed a broad band feature extraction library capable to help solving content-based image queries. Hence, we can significantly improve the quality of our chair's local invariant texture features [4][7]. Further, we could develop a new method, based on inter-structural information [6][10]. A further module of our algorithmic palette specifies image features based on the JPEG2000 image-standard [9]. Moreover we implemented a feature selection method, which finds 'the best' features out of many for an arbitrary image query, leading to better results [8]. The choice of different feature methods is concluded by an inter-active algorithm, which is known in the literature as Relevance Feedback. Relevance Feedback enables through a distinct selection of images by the user an iterative enhancement of the image retrieval results. Where, the selected images should be similar to the query image. The inter-active enhancement of the results is obtained by a Support Vector Machine, which learns the user's image selection [5]. Thus, one can improve the original results by several iterations. In the scope of I-Search we develop several algorithms for content-based image retrieval tasks. The methods might be of help for different image query problems. We would like to stress, that there is no single best algorithm, it is much more the smart choice or proper combination of different methods, leading to a satisfactory result.

References

Björn Johansson: A Survey on: Contents Based Search in Image Databases.
R.C.Veltkamp,M.Tanase: Content-Based Image Retrieval Systems: A Survey
Alberto Del Bimbo, Alberto Del Bimbo:Visual Information Retrieval.1999
Fuhui Long, Hongjiang Zhang, David D. Feng: Fundamentals of Content-based Image retrieval, in Multimedia Information Retrieval and Management - Technological Fundamentals and Applications, D. Feng, W.C. Siu, and H.J.Zhang. (ed.), Springer, 2002.

Links of related projects

QBIC by IBM
Visual Information Retrieval by Virage, Inc.
Photobook by MIT
CANDID by Los Alamos National Laboratory
Informedia by CMU
ADL by U of California Santa Barbara
CHABOT by U of California, Berkley
Video and Image Libraries by MIT
COMPASS by ITC-irst

Publications

[Schulz-Mirbach:1995]
H. Schulz-Mirbach.
Invariant features for gray scale images.
In G. Sagerer, S. Posch, and F. Kummert, editors, 17. DAGM - Symposium ``Mustererkennung'', pages 1-14, Bielefeld, 1995. Reihe Informatik aktuell, Springer. DAGM-Preis.

[Siggelkow,Burkhardt:1998]
S. Siggelkow and H. Burkhardt.
Image retrieval based on local invariant features.
In Proceedings of the IASTED International Conference on Signal and Image Processing (SIP) 1998, pages 369-373, Las Vegas, Nevada, USA, October 1998. IASTED.

[Sigglekow,Schael:1999]
S. Siggelkow and M. Schael.
Fast estimation of invariant features.
In W. Förstner, J. M. Buhmann, A. Faber, and P. Faber, editors, Mustererkennung, DAGM 1999, Informatik aktuell, pages 181-188, Bonn, September 1999. Springer. DAGM-Preis.

[ha:hal:bu:icpr04]
B. Haasdonk, A. Halawani and H. Burkhardt.
Adjustable invariant features by partial Haar-integration.
In Proceedings of the 17th International Conference on Pattern Recognition (ICPR), Vol. 2, pages 769-774, Cambridge, United Kingdom, Aug. 2004.
Get the pdf file here

[se:ick:bu:mva05]
L. Setia and J. Ick and H. Burkhardt.
SVM-based Relevance Feedback in Image Retrieval using Invariant Feature Histograms.
IAPR Workshop on Machine Vision Applications (MVA2005), Pages 542-545, Tsukuba Science City, Japan, 16-18 May 2005.
Get the pdf file here

[br:bu:mva05]
G. Brunner and H. Burkhardt.
Building Classification of Terrestrial Images by Generic Geometric Hierarchical Cluster Analysis Features.
IAPR Workshop on Machine Vision Applications (MVA2005), Pages 136-139, Tsukuba Science City, Japan, 16-18 May 2005.
Get the pdf file here

[hal:bu:mva05]
A. Halawani and H. Burkhardt.
On using Histograms of Local Invariant Features for Image Retrieval.
IAPR Workshop on Machine Vision Applications (MVA2005), Pages 538-541, Tsukuba Science City, Japan, 16-18 May 2005.
Get the pdf file here

[mei:br:se:bu:ideal05]
L. Mei, G. Brunner, L. Setia, H. Burkhardt.
Kernel Biased Discriminant Analysis using Histogram Intersection Kernel for Content-Based Image Retrieval.
Sixth International Conference on Intelligent Data Engineering and Automated Learning (IDEAL'05), in number 3578 LNCS, Springer, Pages 63-70, Brisbane Queensland, Australia, 6th - 8th July, 2005.
Get the pdf file here

[te:mue:kow:visual05]
A. Teynor, W. Müller, W. Kowarschik.
Compressed Domain Image Retrieval Using JPEG2000 and Gaussian Mixture Models.
In Proceedings of Conference on Visual Information Retrieval (VISUAL2005), Amsterdam, The Netherlands, July 5th, 2005.

[br:bu:dagm05]
G. Brunner and H. Burkhardt.
Structure Features for Content-Based Image Retrieval.
In Proceedings of the 27th DAGM Symposium, in number 3663 LNCS, Springer, Vienna, Austria, 30.8 - 2.9. 2005.
Get the pdf file here

Demos and Tools

SIMBA (Search IMages By Appearance)

This page is maintained by Lin Mei(mei@informatik.uni-freiburg.de) & Gerd Brunner(gbrunner@informatik.uni-freiburg.de)