Mathematics and Computer Science

Unsupervised Content-Based Image Retrieval (CBIR) and categorization (No. T4-1647)

Lead Researcher: Prof. Michal Irani


Novel algorithms developed at the Weizmann Institute of Science for Content-Based Image Retrieval (CBIR) can enhance search engines by crowd-sourcing and improved clustering.Discovering visual categories among collection of images is a long standing challenge in computer vision, which limits images-based search engines. Existing approaches are searching for a common cluster model. They are focused on identifying shared visual properties (such as a shared object) and subsequently grouping the images into meaningful clusters based upon these shared properties. Such methods are likely to fail once encountering a highly variable set of images or a fairly limited number of images per category.Researchers form Prof. Michal Irani lab suggest a novel approach based on ‘similarity by composition’. This technology detects statistically significant regions which co-occur across images, which reveals strong and meaningful affinities, even if they appear only in few images. The outcome is a reliable cluster in which each image has high affinity to many images in the cluster, and weak affinity to images outside the cluster.


  • Images search engines - can be applied for collaborative search between users.

  • Detecting abnormalities in medical imaging.

  • Quality assurance in the fields of agriculture, food, pharmaceutical industry etc.

  • Security industry- from counting people up to identifying suspicious acts.

  • Computer games and brain machine interface.
  • Advantages

    • Can be applied to very few images, as well as benchmark datasets, and yields state-of-the-art results.• Handles large diversity in appearance.• The search is not a global search, it requires no semantic query, tagging or pre-existing knowledge.• The multi-images collaboration significantly speeds up the process, reducing the number of random samples and iterations.• Set of images are obtained in time which is nearly linear in the size of the image collection.

    Technology's Essence

    In “clustering by composition”, a good cluster is referred as one in which each image can be easily composed using statistically significant pieces from other images in the cluster while is difficult to compose from images outside the cluster. Multiple images exploit their ‘wisdom of crowds’ to further improve the process. Using a collaborative randomized search algorithm images can be composed from each other simultaneously and efficiently. This enables each image to direct the other images where to search for similar regions within the image collection. The resulted sets of images affinities are sparse yet meaningful and reliable.