isk-daemon: Content-based image database server

When telling the daemon to index a file, you can either inform it a local file path with image data (plenty of image formats are currently supported), or a remote HTTP URL, in which case it would download the image to memory.

In both cases, as soon as the image is processed, it can be removed from disk or memory, thus removing copyright concerns when storing images. The image similarity database will only store a signature of the image (which is actually, just a few bytes long).

This signature can be seen merely as metadata describing the image, just like you probably already store image dimensions, source url, source gallery url, etc.

The provided API enables the image processing (as part of the process to add it to the image database) to be done on an adhoc basis (for example when user submits an image to your system) or in offline batches.

isk-daemon and imgSeek are opensource and licensed under the GPL by Ricardo Niederberger Cabral.

Among other remote procedure call technologies, the image similarity database daemon is capable of handling commands and queries through HTTP requests with POST payloads consisting of pure XML data.

WARNING: this communication interface is deprecated. New functionalities not listed below will not be available through the XML+HTTP interface on future versions .

Check the set of XML examples of API usage under docs/POX_details/pox-examples.txt for getting an idea of how to:

• reply for a query for similar images
• request for populating images found on the images repository on disk
• reply for a request for populating the similarity database with new images
• query the database for images similar to an image found on the database.

Notice that images are identified by a numeric ID supplied by you. These same IDs are used for replying back to you similarity search results.

If you want more details check the XML Schema for the API (docs/POX_details/isk.xsd)

Disk space usage for the image database grows linearly with the number of images. For a comparison, a database with 162.000 images takes 230mb of disk space and RAM memory.

Adding images to a database has a constant complexity, ie, it always takes the same time, regardless of how big is your collection.

And when it comes to querying for similar images, theoretically, the complexity is also linear with the size of your collection. So, if your collection with 10.000 images takes 5 seconds to query, and you increase your collection to 100.000 images, then it should take 50 seconds to query.

This diagram (click for bigger version) shows how the web server (Apache etc), web programming code (PHP, ASP etc) and the content-based image database (isk-daemon) interacts.

It depicts the two most common processes (shown as the two vertical lanes):

1. Adding images to your existing image database (a process you probably already have in place) followed by getting it indexed by isk-daemon
2. Querying for similar images.

This architecture assumes that the following components (brown boxes on the diagram) are already implemented by you:

• Web server: for handling user input and running PHP or your web application logic for presenting search results and helping the user perform queries.
  
• Image meta-data engine: be it a PHP module or some logic embedded on your pages to query a relational database management system for any image meta-data needed on your application. The architecture assumes you have such a database in place in order to store image IDs associated to its location in order to present search results and know which ID to supply to the image similarity engine given user input.
• Image meta-data: a relational database with additional data for each image stored on disk. Would typically associate image IDs to its location on disk, title, dimensions, etc.
• images: stored on a hard disk drive, readable from the web server and from the “Image similarity engine”. The image similarity engine only makes use of image pixel data when adding images to its database. Which means that right after processing, these images can be deleted from disk.

The processes for indexing and querying are detailed below:

Index images into your website:

1) Site administrator submits a request to add an image. Again, this is a process that is probably already implemented as part of your content publishing system. Alternatively, images may be getting into your system from user submission forms. In this case, the server side logic for handling these submissions should be extended to also do step 2.c) below.

2) PHP class or page handles the admin user (or an end-user request from a photo submission HTML form) request and does the following:

• a) Save image metadata on RDBMS, determining this image unique id;
• b) Save image thumbnail to disk;
• c) Request image to be indexed by isk-daemon

3) An XML-RPC or SOAP call is made to isk-daemon service interface asking it to index an image. This call should supply an image ID (an integer - probably the same unique image id generated on step 2) and a file path for the image thumbnail.

4) Image thumbnail is loaded from disk. Pixel data is summarized and a very small signature is calculated

5) Visual signature is efficiently stored on main memory

6) And later persisted to disk. This has a configurable behavior: either have them saved after the Nth image is added, or saved every N minutes or have it manually saved from a service call or manual action from the isk-daemon admin interface.

7) User requests random image by clicking on a button at your Flash or HTML interface. This click translates into a remote call as follows.

8 ) XML or JSon HTTP POST is made to an image controller (some logic you have running on the HTTP server). Probably a PHP, ASP class/page or a plain servlet.

9) Your image controller makes a request to the isk-daemon service interface for random image ids for all indexed images. This request could also be made to the RDBMS holding image metadata since only random image IDs are necessary.

10) Image controller wraps image ids and thumbnail URLs into JSon or XML data which is returned to and interpreted by a Flash applet.

11) User requests images similar to one he is currently seeing in details

12) PHP code calls isk-daemon service interface asking for the most similar images by supplying an image id.

13) isk-daemon checks if this query is already cached, if not, proceeds to step 14

14) all images are scored according to their similarity to the query target image

15) PHP code receives XML-RPC results and wraps all the results image ids and thumbnail URLs into JSon or XML data to be returned to a Flash applet.

16) Flash applet shows query results.