How to Build Sphinx from Source

The following excerpt from Introduction to Search with Sphinx shows you how to build Sphinx without installing a binary packages.

---

Binary packages aren't always an option. They might not be available for your platform or for a given version, or they might be built with a different set of compile-time options from what you need. We provide this section in case you need to build Sphinx from the source.

Let's begin with a quick-start that takes literally one minute (maybe two) from the tarball to the search results.

Quick Build

For the sake of this primer, we assume that you’re installing Sphinx on a Unix variant under a plain account called sphinx, and that its home directory is /home/sphinx. We also assume that a C++ compiler and libraries, the MySQL server, the MySQL development libraries, and a command-line PHP binary are all already installed and ready to use. Enter the following commands, substituting the proper version number for X.Y.Z in the file and directory names:

$ cd /home/sphinx
$ tar xzvf sphinx-X.Y.Z.tar.gz<-- (1)
$ cd sphinx-X.Y.Z<-- (1)
$ ./configure --prefix=/home/sphinx<-- (1)
$ make install<-- (1)
$ cd /home/sphinx/etc
$ cp sphinx-min.conf.dist sphinx.conf<-- (2)
$ mysql -u root test < example.sql<-- (2)
$ cd /home/sphinx/bin
$ ./indexer --all<-- (3)
$ ./searchd
$ cd /home/sphinx/sphinx-X.Y.Z
$ php test.php -i test1 test<-- (5)

If all the requirements were met and everything went well, the last line should print some search results.

Now let’s give all the steps (installation, configuration, and API usage) a somewhat closer look!

Source Build Requirements

Generally, any reasonably modern system with a reasonably modern C++ compiler and make program should suffice, and there are no other strict requirements. Systems that are known to work include various versions of Linux, Windows, FreeBSD, NetBSD, Solaris, Mac OS, and AIX. The minimum version of GCC compiler that is tested for compatibility on a regular basis is 3.4.6. Older versions of GCC might still work.

Sphinx requires a C++ compiler, not just a plain C compiler. It should be straightforward to install one from your OS binary packages. Typical package names for the GNU C++ compiler can be g++, gcc-c++, c++, and so on. You will also need a standard C++ library, which sometimes does not get automatically installed. Specific package names for the Red Hat/Fedora/CentOS family are gcc-c++, libstdc++, and libstdc++-devel. Specific names for the Ubuntu/Debian family are g++, libstdc++6, and libstdc++6-dev (your version ID may vary).

By default, the binaries are built with MySQL support in indexer, and thus MySQL client libraries are required. Respective package names are mysql-devel on Red Hat, libmysqlclient15-dev on Ubuntu, and the like on other systems (mysql-client, libmysqlclient, mysql-dev, etc.). However, this requirement can be disabled when configuring the build. In that case, indexer will no longer be able to draw data from MySQL. However, you still will be able to use SphinxQL, because searchd does not depend on MySQL for that; SphinxQL is an entirely independent implementation.

Note that the server itself is never required; just the client libraries. But for the sake of completeness (and because many sites use Sphinx together with MySQL on the same box), the server package name is mysql on Red Hat, and mysql-server on Ubuntu.

Configuring Sources and Building Binaries

Sphinx uses the GNU build system. So long as all the required library and header files are installed, the build process is as simple as follows:

$ tar xzvf sphinx-X.Y.Z.tar.gz
$ cd sphinx-X.Y.Z
$ ./configure
$ make
$ make install

These five commands respectively extract the contents of the source archive (tar); change the current directory to the extracted source root directory (cd); perform the build configuration (configure); build binaries (make); and finally, install the binaries and other support files to their proper location (make install). Of these, the most interesting stage is the build configuration, because a number of options can be tweaked only during that stage. The most important ones are:

--prefix
Specifies the Sphinx installation root (e.g., --prefix=/usr/local/sphinx).

--with-mysql
Specifies where to search for the MySQL client library and header files (useful if auto-detection fails).

--without-mysql
Skips MySQL support.

--with-pgsql
Enables PostgreSQL data source support and optionally specifies where to search for PostgreSQL client library and header files (again, if auto-detection does not automatically find them).

--enable-id64
Enables 64-bit document IDs and word IDs. By default, Sphinx uses 32-bit integers to store document and keyword IDs, generating the latter using the CRC32 hashing function. While that is adequate for smaller collections of documents, on bigger ones, 32-bit document IDs might not be enough to hold the ID values, and 32-bit keyword CRCs can conflict (i.e., the same CRC can be generated for two different words). Using 64-bit IDs alleviates both issues.

--enable-libstemmer
Enables additional stemmers from a third-party libstemmer library. Sphinx comes with three built-in stemmers for English, Russian, and Czech languages. The libstemmer library (a part of the Snowball project; see http://snowball.tartarus.org/) provides stemmers for 13 more languages (Danish, Dutch, Finnish, French, German, Hungarian, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish, and Turkish).

For instance, the following lines will configure, build, and install Sphinx without MySQL support, with PostgreSQL support using automatically detected client library locations, and with support for 64-bit IDs, placing binaries and support files into default system-wide locations such as /usr/local/bin:

$ ./configure --without-mysql --with-pgsql --enable-id64
$ make
$ make install

Once the build and installation succeed, you can use the programs provided by the Sphinx package. The two most important ones are indexer, a program that pulls the data from the sources (which you specify in the configuration file) and creates full-text indexes on that data, and searchd, a program that runs in the background (a.k.a. a daemon), and handles search queries initiated by other client programs, such as your website scripts. Other programs are:

search
A simple command-line test program that directly queries the indexes (i.e., without talking to searchd and therefore without having to run it first)

spelldump
A utility that generates Sphinx-format word form files from ispell or myspell format dictionaries

indextool
A utility that can provide various debugging information about the full-text indexes, check them for consistency, and so forth

All Sphinx programs require a configuration file, called sphinx.conf by default, which contains different settings, data source declarations, and full-text index declarations. Two sample configuration files, sphinx.conf.dist and sphinx-min.conf.dist, are bundled with Sphinx and are installed to /usr/etc, /usr/local/etc, or wherever is the default location on your system. They are fully functional, and for a super-quick start you can merely rename one of them to sphinx.conf, import example.sql (also bundled) into MySQL, and immediately try indexing and querying the test1 index:

$ cd /usr/local/etc
$ cp sphinx.conf.dist sphinx.conf
$ indexer test1
$ search -i test1 test one
$ searchd
$ cd ~/sphinx-X.Y.Z/api
$ php test.php -i test1 test one

The major difference between the bundled configuration file samples is that sphinx.conf.dist lists all the available configuration options along with the default values and short descriptions, aiming to be a quick reference, while sphinx-min.conf.dist contains only those few lines that are actually required to index the test index.