Visual information retrieval
In this project we investigate a promising content based retrieval paradigm called interactive search or relevance feedback. In relevance feedback methods, the user is a key component of the search process and provides positive and negative feedback on the results which the system uses to iteratively improve the set of candidate results. In our approach we closely integrate synthetic imagery with a niching evolution strategy algorithm where the synthetic imagery is used to clarify uncertain regions in the relevance space. As such this project proposes novel research on the application of synthetic imagery in content based retrieval, and defines a new fundamental paradigm: Artificial Imagination (AIm).

Artificial imagination
One of the main areas which is explored in this project is the way in which artificial imagination by intelligently computed examples can be used to aid a computer algorithm in learning new visual concepts, in particular the correct solutions to a user query for a multimedia database. In particular, we intend to endow the computer with the ability to ask whether a particular synthesized example (which is not in the database) is relevant. The synthesized example would presumably target a particular feature which could be important to the query.

Industrial cooperation
We collaborate with a number of industrial partners including Philips Research, Google, and Microsoft. Specific collaborations exist with Ruoyun Gao, Ling Shao and Devrim Unay of Philips Research, on content analysis techniques and image synthesis using image quilting and with Roger Fujii of Helios Technologies, on visual content understanding and Internet search engines.

International cooperation
In the context of BRICKS, we actively collaborate with a set of international research groups from University of Illinois at Urbana-Champaign and Dublin City University. Further, collaborations take place with Michael Emmerich, Assistant Professor at Leiden University, on optimization using natural computing; Alan Smeation, Professor at Dublin City University, on video understanding and analysis, ongoing and including a colloboration trip in Summer of 2008; and Kiyo Aizawa, Professor at University of Tokyo, on moving object segmentation in video, planned for summer of 2009.

Highlights 2006
Economic & societal impact
Our project is planned to create new technology for searching through digital libraries, the WWW which will significantly improve upon the existing state of the art. We expect worldwide usage of the technology.

Future work 2007-2009
This project has recently started and is writing a survey of the state of the art in multimedia information retrieval using relevance feedback. We are also analyzing generalized PCA techniques toward investigating the artificial imagination which will synthesize the imagery.

Several novel challenges exist in the artificial imagination paradigm. First, we need to have a method to synthesize a virtual example based on a point in relevance space. The synthesis problem can be approached on a statistical level or directly by using transformations such as the Karhunen-Loeve Transform (KLT) [Therrien 1989]. In the case of the KLT, the feature space would be created by N coefficients from a KLT representation of a dataset. When our relevance space analysis decides that a particular point in feature space is of significant interest to us, we first note that the point in feature space is a set of coefficients for the eigenvectors in the KLT representation. We also plan on synthesizing the images from the feature space by altering an existing database image so that it statistically conforms to the point in feature space or in making regularity assumptions such as: the color or texture is expected to be similar to adjacent areas in an image. Within the context of relevance feedback based retrieval methods, the synthesis problem is relatively unexplored and will require investigation within this project.

Ongoing development with Helios Technologies of the "Noteworthy" Internet Image Search Engine which will use the techniques from the VIRSI project. From past experience, we expect this to be a widely used system - i.e. over a million queries per month.

Ongoing collaboration in image synthesis with Philips Research.

IS6 Researchers funded by BRICKS

• Dr. Mark Huiskes (UL)
• Dr. Floris Sicking (UL)

Other researchers involved

• Dr. Michael Lew (UL)
• Dr. Erwin M. Bakker (UL)
• Drs. Bart Thomee (UL)

For more information, please refer to the publications and posters of this project.