Difference between revisions of "Installing MB Slave and Search Server on a New Xubuntu Machine"

From MusicBrainz Wiki
(Setup MusicBrainz Database)
Line 50: Line 50:
 
ALTER ROLE musicbrainz WITH PASSWORD 'musicbrainz';
 
ALTER ROLE musicbrainz WITH PASSWORD 'musicbrainz';
 
Cntl-D
 
Cntl-D
 
+
</pre>
 
TODO
 
TODO
 
Configure pg_conf file so that can connect to plsql using musicbrainz from any unix account
 
Configure pg_conf file so that can connect to plsql using musicbrainz from any unix account
  
 
cd /home/paul/Downloads/lalinsky-mbslave*
 
cd /home/paul/Downloads/lalinsky-mbslave*
</pre>
 
 
Put the following into a shell script, then run:
 
Put the following into a shell script, then run:
 
<pre>
 
<pre>

Revision as of 10:03, 1 May 2013

The existing instructions are okay, but because they try and cover all bases they can be more confusing than they need to be. So I thought it would be useful to show exactly what is required for just one common scenario :ijabz.

Install software not available from apt

- Download ISO image from http://xubuntu.org/ and burn to CD

- Install Xubuntu 12.10 from CD

- Download and unzip mbslave from https://bitbucket.org/lalinsky/mbslave

- Download an unzip psycopg 2-2.4.6 from http://initd.org/psycopg/

Download MusicBrainz Data Dumps

Download Musicbrainz data dumps from http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/

cp /home/paul/Downloads/mbdump*.bz2 /home/paul/Downloads/lalinsky-mbslave*

Install Postgres

sudo apt-get install postgresql-9.1 postgresql-server-dev-9.1 postgresql-contrib-9.1
export PATH=$PATH:/usr/lib/postgresql/9.1/bin

Configue psycogs2

sudo apt-get install python-dev
cd
cd Downloads/psycopg2-2.4.6
sudo python setup.py install

Configure mbslave

cd
cd Downloads/lalinsky-mbslave*
cp mbslave.conf.default mbslave.conf

Uncomment password line and set to musicbrainz

Setup MusicBrainz Database

sudo su - postgres
createuser musicbrainz

Type y when asked if want to set up as superuser

createdb -l C -E UTF-8 -T template0 -O musicbrainz musicbrainz
psql musicbrainz
ALTER ROLE musicbrainz WITH PASSWORD 'musicbrainz';
Cntl-D

TODO Configure pg_conf file so that can connect to plsql using musicbrainz from any unix account

cd /home/paul/Downloads/lalinsky-mbslave* Put the following into a shell script, then run:

#!/bin/bash
set -xv
echo 'CREATE SCHEMA musicbrainz;' | ./mbslave-psql.py -S
echo 'CREATE SCHEMA statistics;' | ./mbslave-psql.py -S
echo 'CREATE SCHEMA cover_art_archive;' | ./mbslave-psql.py -S
./mbslave-remap-schema.py <sql/CreateTables.sql | sed 's/CUBE/TEXT/' | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreateTables.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreateTables.sql | ./mbslave-psql.py
./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2 mbdump-cdstubs.tar.bz2 mbdump-cover-art-archive.tar.bz2
./mbslave-remap-schema.py <sql/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreatePrimaryKeys.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/CreateIndexes.sql | grep -vE '(collate|page_index|tracklist_index)' | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/statistics/CreateIndexes.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/caa/CreateIndexes.sql | ./mbslave-psql.py
./mbslave-remap-schema.py <sql/CreateSimpleViews.sql | ./mbslave-psql.py

Exit out of postgres user

Cntl-D

Build Search Server

sudo apt-get install subversion openjdk-7-jdk maven2 tomcat7 
cd
mkdir indexdata
mkdir -p code/musicbrainz
cd code/musicbrainz
svn co http://svn.musicbrainz.org/mmd-schema/trunk mmd-schema
svn co http://svn.musicbrainz.org/search_server/trunk search_server
cd mmd-schema/brainz-mmd2-jaxb
mvn install
cd
cd code/musicbrainz/search_server/servlet/src/main/webapp/WEB-INF

Edit web.xml change to <param-value>/home/paul/indexdata</param-value>

mvn package

Configure postgressql for better performance

sudo vi /etc/postgresql/9.1/main/postgresql.conf

- set shared_buffers=800MB

- set work_mem=20MB

- set effective_cache_size = 250MB

sudo sysctl -w kernel.shmmax=17179869184
sudo sysctl -w kernel.shmall=17179869184
sudo service postgresql stop
sudo service postgresql start

TODO:Add to startup so don't have to redo on every reboot

Build Indexes

cd /home/paul/code/musicbrainz/search_server/index/target
java -Xmx500M -jar index-2.0-SNAPSHOT-jar-with-dependencies.jar --indexes-dir /home/paul/indexdata -d musicbrainz -u musicbrainz -p musicbrainz

Install Jetty server

sudo apt-get install jetty  (note only install old jetty 6, but will do for now)
vi /etc/init.d/jetty (Remove JETTY_STOP line)

Deploy Search Server onto Jetty

cd /usr/share/jetty/webapps
rm -r ROOT
sudo cp /home/paul/code/musicbrainz/search_server/servlet/target/searchserver.war ROOT.war
sudo /etc/init.d/jetty start