Sphinx SetFilter limitation

Few days back, i encountered an sphinx issue that, if you use $sphinx->setfilter() with exclude fields option, it does not allow more than 4096 items in exlusion array.


Add multiple $sphinx->setfilter() calls with same filter field and spliting exclusion array in chunks of 4096 items.
It may be limitation in PHP implementation of sphinx, but multiple calls worked.

The other solution was to create proper index with exlusion rule, which i was not in mood of.

Just shared to help someone!

Distributed searching with SPHINX

When we performing full-text searching through a number of large indexes it is important that we get results in shortest time possible. To ensure this SPHINX provides feature of “Distributed Searching”.

Today we will look how we will look on SPHINX’s ability to perform distributed searching. If you don’t know what sphinx is please refer to my other post Introduction to Sphinx Search.

For example if you have a large index you can easily distribute. You can create this index in chunks and assign each chunk to each sphinx agent. You will query to this index and sphinx will do the searching in parallel and give you final results. This dramatically improves the speed of searching. This concept similar to table partitioning in mysql.

Here is the example to setup a distributed index
index mycompleteindex
type = distributed
local = chunk0
agent = localhost:3312:chunk2
agent = localhost:3312:chunk3
agent = localhost:3312:chunk4

Here type = distributed tells that this is not a normal index. As we only have one searchd instance installed so we are using same instance localhost:3312 and declaring it an agent.

With each agent are have specified an chunk to be served by this agent. The example shows distributed searching in one system the same can be achieved by using separate server for each chunk.

Note that this post is only introduction to this feature for further details refer to sphinx documentation.