isk-daemon: Content-based image database server

We’ve been working steady on improvements to the current available implementation, and according to the current roadmap,  release 0.9 (due April/2008) should have support for image classification based on visual content similarity. The API would be extended to provide a method for querying which group an image belongs to and for training the internal classification engine by specifying a group on the moment an image is added to the system.

This would potentially allow isk-daemon to be used for image copyright tracking and filtering.

The roadmap for the upcoming months including the major milestones is as follows. Bug fixes shall be available immediately as intermediary releases.

Release 0.7 – March/2010

1. Basic support for a cluster of daemons, improving scalability and performance. See more details on the initial cluster architecture.
2. Support for cluster configuration on web admin interface.
3. Improved support for multiple database spaces on the web admin interface.
4. Optionally save image filenames associated to imported images.
5. Alternative querying modes: by average color and improved flexibility for similarity queries (parameters for the weight given to color and shape similarity)
6. Improvements to Database management
7. Command to remove databases
8. Database access statistics
9. Documentation for using isk-daemon modules as a library for building image similarity applications (so in a sense isk-daemon would be basically an XML-RPC/SOAP/POX frontend to this library)
10. On Windows, image filenames with extended characters won’t get imported
11. saveDbAs() is not remembering the supplied filename, so further calls to saveDb() will save to a file named “not yet saved” instead of the previous one.
12. allow batch operations in order to minimize number of remote calls during normal operation
13. addMultipleImages
14. removeAll
15. removeIdRange
16. queryIDs

Release 0.8 – December/2010

1. Caching mechanism for improved performance and lower network usage (memcached support and bloom filters for image id exchange between server instances)
2. Improved network usage for cluster synchronization
3. Auto-discovery for cluster instances, so cluster instances don’t need to know by configuration time which other instances are available on the local network. Failover should then be completely transparent.
4. Support for querying based on signature processed at client-side, enabling web-based search by sketch. One example sketch applet shall be provided.
5. Improved API examples
6. Improved image querying operatios to the web admin interface, showing image thumbnails for results.
7. UTF-8 image filename support
8. design and implement REST interface
9. write pydocs for keyword-related methods
10. fix random kwd query bug
11. implement deleteImage method
12. more admin screenshots to the website and documentation
13. prettify C++ code and improve documentation/comments
14. references on documentation to POX (plain old XML) should be rephrased as XML-POST, POX-over-http or something like that to make terminology more clear
15. rerun all tests and sample clients, compose test suites
16. restructure imgdb.cpp into modules (major feature sets)
17. handle remote exceptions properly
18. more descriptive return codes for image management operations

Release 0.9 – June/2011

1. Support for image classification based on content similarity. The API would be extended to provide a method for querying which group an image belongs to and for training the internal classification engine by providing a group on the moment an image is added to the system. This should be based on some heuristic for determining set membership.

Release 1.0 – October/2011

1. segment video into scenes and index each scene as being just another image on database.
2. reuse code from VideoQuery
3. video support for some codecs (mpeg2, xvid, divx) and formats (mpg, mov, avi)

These are the basic features for the server side daemon:

• Query for images similar to one already indexed by the database, returning a similarity degree for the images on database that most resemble the target query image.
• Query for images similar to one described by its signature. A client-side widget may generate such signature from what a user sketched and submit it to the daemon.
• XML+HTTP interface for easy integration with other web and desktop or rich-client applications
• Fast indexing of images one-by-one or in batch
• Quickly remove images from database one-by-one or in batch
• Key-based security for API users
• Built-in web-based admin interface with statistics and ad-hoc maintenance commands/API testing
• Optimized image processing code implemented in C++

More upcoming features can be found on the development roadmap.

When telling the daemon to index a file, you can either inform it a local file path with image data (plenty of image formats are currently supported), or a remote HTTP URL, in which case it would download the image to memory.

In both cases, as soon as the image is processed, it can be removed from disk or memory, thus removing copyright concerns when storing images. The image similarity database will only store a signature of the image (which is actually, just a few bytes long).

This signature can be seen merely as metadata describing the image, just like you probably already store image dimensions, source url, source gallery url, etc.

The provided API enables the image processing (as part of the process to add it to the image database) to be done on an adhoc basis (for example when user submits an image to your system) or in offline batches.