Difference between revisions of "History:Installing MB Slave and Search Server on a New Xubuntu Machine"

From MusicBrainz Wiki
Jump to navigationJump to search
(Add new Danish mirror)
Line 12: Line 12:
==Download MusicBrainz Data Dumps==
==Download MusicBrainz Data Dumps==
Download Musicbrainz data dumps from http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/
Download Musicbrainz data dumps from:
* US mirror: http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/
* Danish mirror: http://mirrors.dotsrc.org/MusicBrainz/data/fullexport/
cp /home/paul/Downloads/mbdump*.bz2 /home/paul/Downloads/lalinsky-mbslave*
cp /home/paul/Downloads/mbdump*.bz2 /home/paul/Downloads/lalinsky-mbslave*

Revision as of 17:42, 16 August 2013

The existing instructions are okay, but because they try and cover all bases they can be more confusing than they need to be. So I thought it would be useful to show exactly what is required for just one common scenario :ijabz.

Install software not available from apt

- Download Desktop ISO image from http://xubuntu.org/ (I used Xubuntu 12.10 but I expect later versions should work) and burn to CD

- Install Xbuntu from the CD

- Download and unzip mbslave from https://bitbucket.org/lalinsky/mbslave

- Download an unzip psycopg from http://initd.org/psycopg/ (I used pyscogs 2.4.6 but later versions should work)

Download MusicBrainz Data Dumps

Download Musicbrainz data dumps from:

cp /home/paul/Downloads/mbdump*.bz2 /home/paul/Downloads/lalinsky-mbslave*

Install Postgres

sudo apt-get install postgresql-9.1 postgresql-server-dev-9.1 postgresql-contrib-9.1
export PATH=$PATH:/usr/lib/postgresql/9.1/bin

Configue psycogs2

sudo apt-get install python-dev
cd Downloads/psycopg-2.4.6
sudo python setup.py install

Configure mbslave

cd Downloads/lalinsky-mbslave*
cp mbslave.conf.default mbslave.conf

Uncomment password line and set to musicbrainz

Setup MusicBrainz Database

sudo su - postgres

and enter root password

createuser musicbrainz

Type y when asked if want to set up as superuser

createdb -l C -E UTF-8 -T template0 -O musicbrainz musicbrainz
psql musicbrainz
ALTER ROLE musicbrainz WITH PASSWORD 'musicbrainz';

TODO Configure pg_conf file so that can connect to plsql using musicbrainz from any unix account

cd /home/paul/Downloads/lalinsky-mbslave* Put the following into a shell script, then run:

set -xv
echo 'CREATE SCHEMA musicbrainz;' | ./mbslave-psql.py -S
echo 'CREATE SCHEMA statistics;' | ./mbslave-psql.py -S
echo 'CREATE SCHEMA cover_art_archive;' | ./mbslave-psql.py -S
./mbslave-remap-schema.py <sql/CreateTables.sql | sed 's/CUBE/TEXT/' | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreateTables.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreateTables.sql | ./mbslave-psql.py
./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2 mbdump-cdstubs.tar.bz2 mbdump-cover-art-archive.tar.bz2
./mbslave-remap-schema.py <sql/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/CreateIndexes.sql | grep -vE '(collate|page_index|gist)' | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreateIndexes.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreateIndexes.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/CreateSimpleViews.sql | ./mbslave-psql.py

Exit out of postgres user


Build Search Server

sudo apt-get install subversion openjdk-7-jdk maven2 tomcat7 git
mkdir indexdata
mkdir -p code/musicbrainz
cd code/musicbrainz
git clone https://github.com/metabrainz/mmd-schema.git mmd-schema
svn co http://svn.musicbrainz.org/search_server/trunk search_server
cd mmd-schema/brainz-mmd2-jaxb
mvn install
cd code/musicbrainz/search_server/servlet/src/main/webapp/WEB-INF

Edit web.xml change to <param-value>/home/paul/indexdata</param-value>

mvn package

Configure postgressql for better performance

sudo vi /etc/postgresql/9.1/main/postgresql.conf

- set shared_buffers=800MB

- set work_mem=20MB

- set effective_cache_size = 250MB

sudo sysctl -w kernel.shmmax=17179869184
sudo sysctl -w kernel.shmall=17179869184
sudo service postgresql stop
sudo service postgresql start

TODO:Add to startup so don't have to redo on every reboot

Build Indexes

cd /home/paul/code/musicbrainz/search_server/index/target
java -Xmx500M -jar index-2.0-SNAPSHOT-jar-with-dependencies.jar --indexes-dir /home/paul/indexdata -d musicbrainz -u musicbrainz -p musicbrainz

Install Jetty server

sudo apt-get install jetty  (note only install old jetty 6, but will do for now)
vi /etc/init.d/jetty (Remove JETTY_STOP line)

Deploy Search Server onto Jetty

cd /usr/share/jetty/webapps
rm -r ROOT
sudo cp /home/paul/code/musicbrainz/search_server/servlet/target/searchserver.war ROOT.war
sudo /etc/init.d/jetty start