History:Installing MB Slave and Search Server on a New Xubuntu Machine

From MusicBrainz Wiki
Jump to navigationJump to search
Status: This Page is Glorious History!

This refers to very old MusicBrainz infrastructure including an older, barely related version of the search server.

The existing instructions are okay, but because they try and cover all bases they can be more confusing than they need to be. So I thought it would be useful to show exactly what is required for just one common scenario :ijabz.

Install software not available from apt

- Download Desktop ISO image from http://xubuntu.org/ (I used Xubuntu 12.10 but I expect later versions should work) and burn to CD

- Install Xbuntu from the CD

- Download and unzip mbslave from https://bitbucket.org/lalinsky/mbslave

- Download an unzip psycopg from http://initd.org/psycopg/ (I used pyscogs 2.4.6 but later versions should work)

Download MusicBrainz Data Dumps

Download Musicbrainz data dumps from:

cp /home/paul/Downloads/mbdump*.bz2 /home/paul/Downloads/lalinsky-mbslave*

Install Postgres

sudo apt-get install postgresql-9.1 postgresql-server-dev-9.1 postgresql-contrib-9.1
export PATH=$PATH:/usr/lib/postgresql/9.1/bin

Configue psycogs2

sudo apt-get install python python-psycopg2

Configure mbslave

cd Downloads/lalinsky-mbslave*
cp mbslave.conf.default mbslave.conf

Uncomment password line and set to musicbrainz

Setup MusicBrainz Database

sudo su - postgres

and enter root password

createuser musicbrainz

Type y when asked if want to set up as superuser

createdb -l C -E UTF-8 -T template0 -O musicbrainz musicbrainz
psql musicbrainz
ALTER ROLE musicbrainz WITH PASSWORD 'musicbrainz';

TODO Configure pg_conf file so that can connect to plsql using musicbrainz from any unix account

cd /home/paul/Downloads/lalinsky-mbslave* Put the following into a shell script, then run:

set -xv
echo 'CREATE SCHEMA musicbrainz;' | ./mbslave-psql.py -S
echo 'CREATE SCHEMA statistics;' | ./mbslave-psql.py -S
echo 'CREATE SCHEMA cover_art_archive;' | ./mbslave-psql.py -S
./mbslave-remap-schema.py <sql/CreateTables.sql | sed 's/CUBE/TEXT/' | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreateTables.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreateTables.sql | ./mbslave-psql.py
./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2 mbdump-cdstubs.tar.bz2 mbdump-cover-art-archive.tar.bz2
./mbslave-remap-schema.py <sql/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/CreateIndexes.sql | grep -vE '(collate|page_index|gist)' | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreateIndexes.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreateIndexes.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/CreateSimpleViews.sql | ./mbslave-psql.py

Exit out of postgres user


Build Search Server

sudo apt-get install subversion openjdk-7-jdk maven2 tomcat7 git
mkdir indexdata
mkdir -p code/musicbrainz
cd code/musicbrainz
git clone https://github.com/metabrainz/mmd-schema.git mmd-schema
svn co http://svn.musicbrainz.org/search_server/trunk search_server
cd mmd-schema/brainz-mmd2-jaxb
mvn install
cd code/musicbrainz/search_server/servlet/src/main/webapp/WEB-INF

Edit web.xml change to <param-value>/home/paul/indexdata</param-value>

mvn package

Configure postgressql for better performance

sudo vi /etc/postgresql/9.1/main/postgresql.conf

- set shared_buffers=800MB

- set work_mem=20MB

- set effective_cache_size = 250MB

sudo sysctl -w kernel.shmmax=17179869184
sudo sysctl -w kernel.shmall=17179869184
sudo service postgresql stop
sudo service postgresql start

TODO:Add to startup so don't have to redo on every reboot

Build Indexes

cd /home/paul/code/musicbrainz/search_server/index/target
java -Xmx500M -jar index-2.0-SNAPSHOT-jar-with-dependencies.jar --indexes-dir /home/paul/indexdata -d musicbrainz -u musicbrainz -p musicbrainz

Install Jetty server

sudo apt-get install jetty  (note only install old jetty 6, but will do for now)
vi /etc/init.d/jetty (Remove JETTY_STOP line)

Deploy Search Server onto Jetty

cd /usr/share/jetty/webapps
rm -r ROOT
sudo cp /home/paul/code/musicbrainz/search_server/servlet/target/searchserver.war ROOT.war
sudo /etc/init.d/jetty start