Distributed searching with SPHINX

When we performing full-text searching through a number of large indexes it is important that we get results in shortest time possible. To ensure this SPHINX provides feature of “Distributed Searching”.

Today we will look how we will look on SPHINX’s ability to perform distributed searching. If you don’t know what sphinx is please refer to my other post Introduction to Sphinx Search.

For example if you have a large index you can easily distribute. You can create this index in chunks and assign each chunk to each sphinx agent. You will query to this index and sphinx will do the searching in parallel and give you final results. This dramatically improves the speed of searching. This concept similar to table partitioning in mysql.

Here is the example to setup a distributed index
index mycompleteindex
{
type = distributed
local = chunk0
agent = localhost:3312:chunk2
agent = localhost:3312:chunk3
agent = localhost:3312:chunk4
}

Here type = distributed tells that this is not a normal index. As we only have one searchd instance installed so we are using same instance localhost:3312 and declaring it an agent.

With each agent are have specified an chunk to be served by this agent. The example shows distributed searching in one system the same can be achieved by using separate server for each chunk.

Note that this post is only introduction to this feature for further details refer to sphinx documentation.

http://wasimasif.wordpress.com/2009/09/09/distributed-searching-with-sphinx/