A Research Prototype Image Retrieval System S. Nepal, M.V. Ramakrishna & J.A. Thom Department of Computer Science RMIT University, GPO Box 2476V, Melbourne VIC 3001 fnepal,rama,
[email protected]
Abstract Many content based image retrieval systems have been developed in recent years. Some of these systems capture feature information (such as color and texture) and leave semantics to the users. While posing queries the users can express semantics in terms of feature information [2, 5]. In other systems the semantic information is captured in the system's design [1, 3]; these are suitable and designed for domain speci c applications such as a collection of images of human faces. Neither of these approaches allow modeling of semantic information for a collection of unconstrained images. In addition, these systems do not support semantic queries. Capturing and storing of semantic information, such as sunset and mountain, is trivial for humans but is a challenge for content based image retrieval systems. To address this problem, we designed a fuzzy system for content based image retrieval [4]. We have developed a prototype system using the color feature. The color feature is extracted from each image and stored. The system provides a set of semantic terms on color feature values. This has been done by mapping the color spectrum into nine color terms: red, green, blue, black, purple, yellow, pink, orange, and white. A set of semantic fuzzy linguistic terms such as few, many and mostly can be used to express the amount of color contained in an image. The users can then pose queries using the fuzzy linguistic query language. An atomic query retrieves all images with mostly red color. Complex queries can be generated by combining atomic queries using fuzzy and and fuzzy or operators. The system also provides an interactive feedback system that allows users to de ne concepts such as sunset based on similarity using the color feature to one (or more) query images containing examples of sunset. These concepts can then be combined by the user in more complex queries, such as nd images containing mountain with mostly blue color (mountain and ) (see Figure 1) and nd images containing mountain and sunset (sunset and mountain). We are currently designing a more powerful query language FOQL that can be incorporated into an extension of our prototype. FOQL is a fuzzy object oriented query language. Permission to make digital/hard copy of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for pro t or commercial advantage, the copyright notice, the title of the publication and its date appear, and notice is given that copying is by permission of ACM, Inc. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speci c permission and/or c 1998 ACM 1-58113-015-5 fee. SIGIR'98, Melbourne, Australia 8/98 $5.00.
Figure 1: Result of the query.
References
[1] A. F. Cardenas, I. T. Ieong, R. K. Taira, R. Barker, and C. M. Breant. The knowledge-based object-oriented PICQUERY+ language. IEEE Transactions on Knowledge and Data Engineering, 5(4):644{657, August 1993. [2] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker. Query by image and video content: The QBIC system. Computer, 28(9):23{32, September 1995. [3] C.-C. Hsu, W. W. Chu, and R. K. Taira. A knowledge-based approach for retrieving images by content. IEEE Transactions on Knowledge and Data Engineering, 8(4):533{539, August 1996. [4] S. Nepal, M. V. Ramakrishna, and J. A. Thom. A fuzzy system for content based image retrieval. In Proc. 2nd IEEE International Conference on Intelligent Processing Systems, Gold Coast, Australia, August 1998. To Appear. [5] A. Pentland, R. W. Picard, and S. Sclaro . Photobook: Tools for content-based manipulation of image databases. International Journal of Computer Vision, 18(3):233{254, 1996.
