Difference between revisions of "History:Development/Search server setup"
m (User:kuno/Search server setup for developers moved to Development/Search server setup for developers)
Revision as of 18:15, 31 January 2012
Server setup for developers
The following instructions are aimed at getting a search server up and running as a developer; I will assume you are installing things into your home directory, and running under your current user account. If you're looking for a more production ready setup follow the instructions in the README included with the search_server.
Some of the instructions below are taken from the search_server README.
This server can be run on linux/osx/windows, but these instructions assume that you will be running linux and will be using the tomcat6 server to host the servlet.
To run this search server you will also need to have a MusicBrainz database with the core data set loaded. Please see
FIXME: link to ngs server install here.
for details on how to setup at least a "database only install". This document assumes that you have completed the setup of this server.
You will also need a freedb dump, which you can download from http://www.freedb.org/en/download__database.10.html .
And you will need these tools:
- Java complete with JDK 1.6, not just the JRE
- Apache Tomcat 6 ( http://tomcat.apache.org )
- Maven version 2 ( http://maven.apache.org/download.html )
Ensure that your Java installation is at least 1.6, and that you have the JDK installed (not just the JRE).
If you've installed java manually (instead of using the package system of your distribution), you may need to set JAVA_HOME to the correct path in your shell.
To build you need to install Maven, if you haven't already got it. Download it from http://maven.apache.org/download.html .
Download the search server and schemas
You will obviously need the search server, but you also need the musicbrainz schema files:
svn co http://svn.musicbrainz.org/mmd-schema/trunk mmd-schema svn co http://svn.musicbrainz.org/search_server/trunk search_server
Install the schema classes
Enter the brainz-mmd2-jaxb directory in mmd-schema, and run "mvn install".
warp@bo-bomb:~/code/mmd-schema/brainz-mmd2-jaxb$ mvn install
You should see it install mmd2-2.0.0.jar in a directory where maven2 will be able to find it later:
[INFO] Installing /home/warp/code/mmd-schema/brainz-mmd2-jaxb/target/mmd2-2.0.0.jar to /home/warp/.m2/repository/org/musicbrainz/mmd2/mmd2/2.0.0/mmd2-2.0.0.jar
Build the search server servlet
Enter the search_server directory and run "mvn package" to build the servlet (you can also run "mvn package -Dmaven.test.skip=true" if you want to skip the included tests).
You should see a line like this when it's done:
[INFO] Building war: /home/warp/code/search_server/servlet/target/searchserver.war
Build the indexes
To build the indexes, the search server will need access to both your MusicBrainz database and the freedb dump. First enter the index/target directory and run "java -Xmx512M -jar index-2.0-SNAPSHOT-jar-with-dependencies.jar -?" to see the options.
Figure out your database name and login details (if you already have mb_server running you can find them in lib/DBDefs.pm in the mb_server directory), and run the indexer like this (all on one line):
warp@bo-bomb:~/code/search_server/index/target$ java -Xmx512M -jar index-2.0-SNAPSHOT-jar-with-dependencies.jar --indexes-dir /home/warp/indexdata --db-name musicbrainz_db --db-user warp --db-password mypassword --freedb-dump ~/Downloads/freedb-complete-20100201.tar.bz2 --test
Adding '--test' at the end will build a small test index. After you are done with this setup, you may want to re-run this command without the '--test' option to build the full indexes.
When it is done you should have the following index directories in the index directory you specified:
warp@bo-bomb:~/indexdata$ ls annotation_index cdstub_index label_index releasegroup_index work_index artist_index freedb_index recording_index release_index
Install the servlet
To install the servlet in Apache Tomcat, you obviously need a tomcat install. I downloaded a binary distribution and extracted in ~/opt. If you have java in your path you should be able to start it right away.
warp@bo-bomb:~/opt$ tar xfz apache-tomcat-6.0.24.tar.gz warp@bo-bomb:~/opt$ cd apache-tomcat-6.0.24 warp@bo-bomb:~/opt/apache-tomcat-6.0.24$ bin/startup.sh Using CATALINA_BASE: /home/warp/opt/apache-tomcat-6.0.24 Using CATALINA_HOME: /home/warp/opt/apache-tomcat-6.0.24 Using CATALINA_TMPDIR: /home/warp/opt/apache-tomcat-6.0.24/temp Using JRE_HOME: /usr Using CLASSPATH: /home/warp/opt/apache-tomcat-6.0.24/bin/bootstrap.jar warp@bo-bomb:~/opt$
Try connecting to http://localhost:8080, and you should see the tomcat startup page.
Shut down tomcat again by running "bin/shutdown.sh". To install the search server, enter the "webapps" directory, remove the current ROOT, and copy over the servlet as the new ROOT webapp:
warp@bo-bomb:~/opt/apache-tomcat-6.0.24$ cd webapps/ warp@bo-bomb:~/opt/apache-tomcat-6.0.24/webapps$ ls docs examples host-manager manager ROOT warp@bo-bomb:~/opt/apache-tomcat-6.0.24/webapps$ rm -rf ROOT warp@bo-bomb:~/opt/apache-tomcat-6.0.24/webapps$ cp ~/code/search_server/servlet/target/searchserver.war ROOT.war warp@bo-bomb:~/opt/apache-tomcat-6.0.24/webapps$ cd .. warp@bo-bomb:~/opt/apache-tomcat-6.0.24$ bin/startup.sh
You should see a new ROOT directory tree appear under webapps, stop tomcat again to configure the indexes for the search server.
warp@bo-bomb:~/opt/apache-tomcat-6.0.24$ bin/shutdown.sh warp@bo-bomb:~/opt/apache-tomcat-6.0.24$ $EDITOR webapps/ROOT/WEB-INF/web.xml warp@bo-bomb:~/opt/apache-tomcat-6.0.24$ bin/startup.sh warp@bo-bomb:~/opt/apache-tomcat-6.0.24$
In web.xml, you should replace the current index_dir value ("/home/search/indexdata") with the path to the index files, in my case this is "/home/warp/indexdata".
If you now query the search server, you should get some results. For example, try this: http://localhost:8080/ws/2/artist?query=%22alice%22.
If you followed the instructions so far, you should have a running search server. The README included with the search server includes some hints about tuning tomcat, you should follow those.
Make sure the follow the instructions in the README related to UTF-8. If you don't set "URIEncoding" on the connectors in server.xml non-ascii lookups will probably fail.