Archive for Scalability

Cluster architecture

isk-daemon cluster architecture

The diagram above depicts the interaction between an existing web application and a cluster of isk-daemon instances.

All isk-daemon instances communicate to each other automatically balancing the load of adding images to its database and querying for similarity. This way, the web server(s) acting as a client to the isk-daemon cluster is isolated from the fact that there is a cluster of image similarity engines and can make calls to any instance.

Optionally, isk-daemon can make use of memcached instances to cache query requests in order to increase querying throughput.

Comments

Scalability

Disk space usage for the image database grows linearly with the number of images. For a comparison, a database with 162.000 images takes 230mb of disk space and RAM memory.

Adding images to a database has a constant complexity, ie, it always takes the same time, regardless of how big is your collection.

And when it comes to querying for similar images, theoretically, the complexity is also linear with the size of your collection. So, if your collection with 10.000 images takes 5 seconds to query, and you increase your collection to 100.000 images, then it should take 50 seconds to query.

Comments