Generative probabilistic models for multimedia retrieval: query generation against document generation

Abstract:  Abstract This paper presents the use of generative probabilistic models for multimedia retrieval. Gaussian mixture models are estimated to describe the visual content of images (or video) and are explored in different ways of using them for retrieval. So-called query generation (how likely is the query given the document model) and document generation (how likely is the document given the query model) approaches are considered and how both fit in a common probabilistic framework is explained. Query generation is shown to be theoretically superior, and confirmed experimentally on the Trecvid search task. However, it is found that in some cases a document generation approach gives better results. Especially in the cases where queries are narrow and visual results are combined with textual results, the document generation approach seems to be better at setting a visual context than the query generation variant.