Difference between revisions of "Installing MB Slave and Search Server on a New Xubuntu Machine"

From MusicBrainz Wiki
(Configure postgressql for better performance)
(Configure postgressql for better performance)
Line 95: Line 95:
sudo sysctl -w kernel.shmmax=17179869184
sudo sysctl -w kernel.shmmax=17179869184
sudo sysctl -w kernel.shmall=4194304
sudo sysctl -w kernel.shmall=17179869184
sudo service postgresql stop
sudo service postgresql stop
sudo service postgresql start
sudo service postgresql start

Revision as of 07:41, 19 March 2013

The existing instructions are okay, but because they try and cover all bases they can be more confusing than they need to be. So I thought it would be useful to show exactly what is required for just one common scenario :ijabz.

Install software not available from apt

- Download ISO image from http://xubuntu.org/ and burn to CD

- Install Xubuntu 12.10 from CD

- Download and unzip mbslave from https://bitbucket.org/lalinsky/mbslave

- Download an unzip psycopg 2-2.4.6 from http://initd.org/psycopg/

Download MusicBrainz Data Dumps

cp /home/paul/Downloads/mbdump*.bz2 /home/paul/Downloads/lalinsky-mbslave*

Install Postgres

sudo apt-get install postgresql-9.1 postgresql-server-dev-9.1 postgresql-contrib-9.1
export PATH=$PATH:/usr/lib/postgresql/9.1/bin

Configue psycogs2

sudo apt-get install python-dev
cd Downloads/psycopg2-2.4.6
sudo python setup.py install

Configure mbslave

cd Downloads/lalinsky-mbslave*
cp mbslave.conf.default mbslave.conf

uncomment password line and set to musicbrainz

Setup MusicBrainz Database

sudo su - postgres
createuser musicbrainz
createdb -l C -E UTF-8 -T template0 -O musicbrainz musicbrainz
psql musicbrainz
ALTER ROLE musicbrainz WITH PASSWORD 'musicbrainz';
cd /home/paul/Downloads/lalinsky-mbslave*
echo 'CREATE SCHEMA musicbrainz;' | ./mbslave-psql.py -S
echo 'CREATE SCHEMA statistics;' | ./mbslave-psql.py -S
echo 'CREATE SCHEMA cover_art_archive;' | ./mbslave-psql.py -S
./mbslave-remap-schema.py <sql/CreateTables.sql | sed 's/CUBE/TEXT/' | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreateTables.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreateTables.sql | ./mbslave-psql.py
./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2 mbdump-cdstubs.tar.bz2 mbdump-cover-art-archive.tar.bz2
./mbslave-remap-schema.py <sql/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/CreateIndexes.sql | grep -vE '(collate|page_index|tracklist_index)' | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreateIndexes.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreateIndexes.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/CreateSimpleViews.sql | ./mbslave-psql.py

Build Search Server

sudo apt-get install subversion openjdk-7-jdk maven2 tomcat7 
mkdir indexdata
mkdir -p code/musicbrainz
cd code/musicbrainz
svn co http://svn.musicbrainz.org/mmd-schema/trunk mmd-schema
svn co http://svn.musicbrainz.org/search_server/trunk search_server
cd mmd-schema/brainz-mmd2-jaxb
mvn install
cd code/musicbrainz/search_server/servlet/src/main/webapp/WEB-INF

Edit web.xml change to <param-value>/home/paul/indexdata</param-value>

mvn package

Configure postgressql for better performance

vi /etc/postgressql/9.1/main/postgresql.conf

- set shared_buffers=800MB

- set work_mem=20MB

- set effective_cache_size = 250MB

sudo sysctl -w kernel.shmmax=17179869184
sudo sysctl -w kernel.shmall=17179869184
sudo service postgresql stop
sudo service postgresql start

Build Indexes

cd /home/paul/code/musicbrainz/search_server/index/target
java  -jar index-2.0-SNAPSHOT-jar-with-dependencies.jar --indexes-dir /home/paul/indexdata -d musicbrainz -u musicbrainz -p musicbrainz