History:Installing MB Slave and Search Server on a New Xubuntu Machine: Difference between revisions
No edit summary |
Reosarevok (talk | contribs) No edit summary |
||
(7 intermediate revisions by 4 users not shown) | |||
Line 1: | Line 1: | ||
{{Status|'''This Page is Glorious History!''' |
|||
This refers to very old MusicBrainz infrastructure including an older, barely related version of the search server. |
|||
}} |
|||
The existing instructions are okay, but because they try and cover all bases they can be more confusing than they need to be. So I thought it would be useful to show exactly what is required for just one common scenario :ijabz. |
The existing instructions are okay, but because they try and cover all bases they can be more confusing than they need to be. So I thought it would be useful to show exactly what is required for just one common scenario :ijabz. |
||
Line 12: | Line 18: | ||
==Download MusicBrainz Data Dumps== |
==Download MusicBrainz Data Dumps== |
||
Download Musicbrainz data dumps from |
Download Musicbrainz data dumps from: |
||
* US mirror: http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/ |
|||
* Danish mirror: http://mirrors.dotsrc.org/MusicBrainz/data/fullexport/ |
|||
<pre> |
<pre> |
||
cp /home/paul/Downloads/mbdump*.bz2 /home/paul/Downloads/lalinsky-mbslave* |
cp /home/paul/Downloads/mbdump*.bz2 /home/paul/Downloads/lalinsky-mbslave* |
||
Line 25: | Line 33: | ||
==Configue psycogs2== |
==Configue psycogs2== |
||
<pre> |
<pre> |
||
sudo apt-get install python- |
sudo apt-get install python python-psycopg2 |
||
cd |
|||
cd Downloads/psycopg-2.4.6 |
|||
sudo python setup.py install |
|||
</pre> |
</pre> |
||
Line 42: | Line 47: | ||
<pre> |
<pre> |
||
sudo su - postgres |
sudo su - postgres |
||
</pre> |
|||
and enter root password |
|||
<pre> |
|||
createuser musicbrainz |
createuser musicbrainz |
||
</pre> |
</pre> |
||
Line 69: | Line 77: | ||
./mbslave-remap-schema.py <sql/statistics/CreatePrimaryKeys.sql | ./mbslave-psql.py |
./mbslave-remap-schema.py <sql/statistics/CreatePrimaryKeys.sql | ./mbslave-psql.py |
||
./mbslave-remap-schema.py <sql/caa/CreatePrimaryKeys.sql | ./mbslave-psql.py |
./mbslave-remap-schema.py <sql/caa/CreatePrimaryKeys.sql | ./mbslave-psql.py |
||
./mbslave-remap-schema.py <sql/CreateIndexes.sql | grep -vE '(collate|page_index| |
./mbslave-remap-schema.py <sql/CreateIndexes.sql | grep -vE '(collate|page_index|gist)' | ./mbslave-psql.py |
||
./mbslave-remap-schema.py <sql/statistics/CreateIndexes.sql | ./mbslave-psql.py |
./mbslave-remap-schema.py <sql/statistics/CreateIndexes.sql | ./mbslave-psql.py |
||
./mbslave-remap-schema.py <sql/caa/CreateIndexes.sql | ./mbslave-psql.py |
./mbslave-remap-schema.py <sql/caa/CreateIndexes.sql | ./mbslave-psql.py |
||
Line 135: | Line 143: | ||
sudo /etc/init.d/jetty start |
sudo /etc/init.d/jetty start |
||
</pre> |
</pre> |
||
[[Category:How To]] |
Latest revision as of 14:50, 1 March 2022
Status: This Page is Glorious History!
This refers to very old MusicBrainz infrastructure including an older, barely related version of the search server. |
The existing instructions are okay, but because they try and cover all bases they can be more confusing than they need to be. So I thought it would be useful to show exactly what is required for just one common scenario :ijabz.
Install software not available from apt
- Download Desktop ISO image from http://xubuntu.org/ (I used Xubuntu 12.10 but I expect later versions should work) and burn to CD
- Install Xbuntu from the CD
- Download and unzip mbslave from https://bitbucket.org/lalinsky/mbslave
- Download an unzip psycopg from http://initd.org/psycopg/ (I used pyscogs 2.4.6 but later versions should work)
Download MusicBrainz Data Dumps
Download Musicbrainz data dumps from:
- US mirror: http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/
- Danish mirror: http://mirrors.dotsrc.org/MusicBrainz/data/fullexport/
cp /home/paul/Downloads/mbdump*.bz2 /home/paul/Downloads/lalinsky-mbslave*
Install Postgres
sudo apt-get install postgresql-9.1 postgresql-server-dev-9.1 postgresql-contrib-9.1 export PATH=$PATH:/usr/lib/postgresql/9.1/bin
Configue psycogs2
sudo apt-get install python python-psycopg2
Configure mbslave
cd cd Downloads/lalinsky-mbslave* cp mbslave.conf.default mbslave.conf
Uncomment password line and set to musicbrainz
Setup MusicBrainz Database
sudo su - postgres
and enter root password
createuser musicbrainz
Type y when asked if want to set up as superuser
createdb -l C -E UTF-8 -T template0 -O musicbrainz musicbrainz psql musicbrainz ALTER ROLE musicbrainz WITH PASSWORD 'musicbrainz'; Cntl-D
TODO Configure pg_conf file so that can connect to plsql using musicbrainz from any unix account
cd /home/paul/Downloads/lalinsky-mbslave* Put the following into a shell script, then run:
#!/bin/bash set -xv echo 'CREATE SCHEMA musicbrainz;' | ./mbslave-psql.py -S echo 'CREATE SCHEMA statistics;' | ./mbslave-psql.py -S echo 'CREATE SCHEMA cover_art_archive;' | ./mbslave-psql.py -S ./mbslave-remap-schema.py <sql/CreateTables.sql | sed 's/CUBE/TEXT/' | ./mbslave-psql.py ./mbslave-remap-schema.py <sql/statistics/CreateTables.sql | ./mbslave-psql.py ./mbslave-remap-schema.py <sql/caa/CreateTables.sql | ./mbslave-psql.py ./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2 mbdump-cdstubs.tar.bz2 mbdump-cover-art-archive.tar.bz2 ./mbslave-remap-schema.py <sql/CreatePrimaryKeys.sql | ./mbslave-psql.py ./mbslave-remap-schema.py <sql/statistics/CreatePrimaryKeys.sql | ./mbslave-psql.py ./mbslave-remap-schema.py <sql/caa/CreatePrimaryKeys.sql | ./mbslave-psql.py ./mbslave-remap-schema.py <sql/CreateIndexes.sql | grep -vE '(collate|page_index|gist)' | ./mbslave-psql.py ./mbslave-remap-schema.py <sql/statistics/CreateIndexes.sql | ./mbslave-psql.py ./mbslave-remap-schema.py <sql/caa/CreateIndexes.sql | ./mbslave-psql.py ./mbslave-remap-schema.py <sql/CreateSimpleViews.sql | ./mbslave-psql.py
Exit out of postgres user
Cntl-D
Build Search Server
sudo apt-get install subversion openjdk-7-jdk maven2 tomcat7 git cd mkdir indexdata mkdir -p code/musicbrainz cd code/musicbrainz git clone https://github.com/metabrainz/mmd-schema.git mmd-schema svn co http://svn.musicbrainz.org/search_server/trunk search_server cd mmd-schema/brainz-mmd2-jaxb mvn install cd cd code/musicbrainz/search_server/servlet/src/main/webapp/WEB-INF
Edit web.xml change to <param-value>/home/paul/indexdata</param-value>
mvn package
Configure postgressql for better performance
sudo vi /etc/postgresql/9.1/main/postgresql.conf
- set shared_buffers=800MB
- set work_mem=20MB
- set effective_cache_size = 250MB
sudo sysctl -w kernel.shmmax=17179869184 sudo sysctl -w kernel.shmall=17179869184 sudo service postgresql stop sudo service postgresql start
TODO:Add to startup so don't have to redo on every reboot
Build Indexes
cd /home/paul/code/musicbrainz/search_server/index/target java -Xmx500M -jar index-2.0-SNAPSHOT-jar-with-dependencies.jar --indexes-dir /home/paul/indexdata -d musicbrainz -u musicbrainz -p musicbrainz
Install Jetty server
sudo apt-get install jetty (note only install old jetty 6, but will do for now) vi /etc/init.d/jetty (Remove JETTY_STOP line)
Deploy Search Server onto Jetty
cd /usr/share/jetty/webapps rm -r ROOT sudo cp /home/paul/code/musicbrainz/search_server/servlet/target/searchserver.war ROOT.war sudo /etc/init.d/jetty start