Difference between revisions of "MusicBrainz NGS Database Install for Mac"

From MusicBrainz Wiki
(Setting up Postgres for Musicbrainz on a Mac System)
(use normal wiki links)
 
(16 intermediate revisions by 3 users not shown)
Line 27: Line 27:
 
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
 
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
 
/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data >logfile 2>&1 &
 
/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data >logfile 2>&1 &
/usr/local/pgsql/bin/createdb musicbrainz
+
/usr/local/pgsql/bin/createuser -U postgres musicbrainz
/usr/local/pgsql/bin/psql test
+
/usr/local/pgsql/bin/createuser -U postgres paul
 +
/usr/local/pgsql/bin/createdb -T template0 --owner musicbrainz --encoding=UNICODE musicbrainz
 
</pre>
 
</pre>
  
Line 38: Line 39:
 
pg_config
 
pg_config
 
cd /usr/local/pgsql/share/contrib
 
cd /usr/local/pgsql/share/contrib
psql -d musicbrainz -U musicbrainz -f cube.sql
+
/usr/local/pgsql/bin/psql -d musicbrainz -U musicbrainz -f cube.sql
 
</pre>
 
</pre>
  
Line 49: Line 50:
  
 
Run pgadmin check can connect to server
 
Run pgadmin check can connect to server
then create user musicbrainz
 
  
I then followed [[http://wiki.musicbrainz.org/Database_Installation]http://wiki.musicbrainz.org/Database_Installation] but had to make some changes for NGS release as follows
+
I then followed [[Database Installation]] but had to make some changes for NGS release as follows, my revised
 +
instructions are below
  
Just interested in mbdump and mpdump-cdstubs (cdstubs) and mbdump-derived (annotations, tags)
 
  
Create one database called musicbrainz and use it for both main and raw database
+
Go to Musicbrainz-server repository
 +
 
 +
<pre>
 +
rm admin/sql/CreateIndexes.sql
 +
git pull
 +
</pre>
 +
Download mbdump and mpdump-cdstubs (cdstubs) and mbdump-derived (annotations, tags)dumps
 +
Decompress the mbdump*.tar.bz2 files:
 +
<pre>
 +
tar jxf mbdump.tar.bz2
 +
tar jxf mbdump-derived.tar.bz
 +
tar jxf mbdump-cdstubs.tar.bz
 +
</pre>
  
 
Edit admin/sql/CreateIndexes.sql removing the page indexes and musicbrainz_collate indexes
 
Edit admin/sql/CreateIndexes.sql removing the page indexes and musicbrainz_collate indexes
 +
<pre>
 +
/usr/local/pgsql/bin/psql -U musicbrainz musicbrainz
 +
\i CreateTables.sql
 +
\i vertical/rawdata/CreateTables.sql
 +
\i vertical/rawdata/CreatePrimaryKeys.sql
 +
\i vertical/rawdata/CreateIndexes.sql
 +
 +
Cntl-D
 +
</pre>
  
Also need to run admin/sql/vertical/rawdata/CreateTable.sql, CreatePrimaryKeys.sql and CreateIndexes.sql
+
Import each file into the database:
  
for t in *; do echo `date` $t ; echo "\\copy $t from ./$t" | /usr/local/pgsql/bin/psql -U musicbrainz musicbrainz && mv $t ../done/; done; echo `date` Done
+
<pre>
 +
paul$ cd mbdump/
 +
paul$ mkdir ../done
 +
paul$ for t in *; do echo `date` $t ; echo "\\copy $t from ./$t" | /usr/local/pgsql/bin/psql -U musicbrainz musicbrainz && mv $t ../done/; done; echo `date` Done
 +
paul$ cd ..
 +
</pre>
 +
 
 +
Add primary keys and indexes, and optimise the database:
 +
<pre>
 +
psql -U musicbrainz musicbrainz
 +
\i admin/sql/CreatePrimaryKeys.sql
 +
\i admin/sql/CreateIndexes.sql
 +
VACUUM ANALYZE;
 +
\q
 +
</pre>
  
 
When build indexes modify IndexOptions.prepareDbConnection to not use a schema because schema is not used with the above instructions
 
When build indexes modify IndexOptions.prepareDbConnection to not use a schema because schema is not used with the above instructions
 +
 +
[[Category:To Be Reviewed]]

Latest revision as of 23:53, 30 April 2014

Setting up Postgres for Musicbrainz on a Mac System

Remove previous installation

sudo rm -r /usr/local/pgsql/ 


Install MacPorts

Now install gmake using MacPorts

port gmake


Download postgres8.3 source and unzip Build and install

./configure
gmake
gmake install
adduser postgres
mkdir /usr/local/pgsql/data
sudo chown postgres /usr/local/pgsql/data
su - postgres
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data >logfile 2>&1 &
/usr/local/pgsql/bin/createuser -U postgres musicbrainz
/usr/local/pgsql/bin/createuser -U postgres paul
/usr/local/pgsql/bin/createdb -T template0 --owner musicbrainz --encoding=UNICODE musicbrainz

Build and install Postgres and Cube extension required by Musicbrainz

cd contrib/cube
gmake 
sudo gmake install
pg_config
cd /usr/local/pgsql/share/contrib
/usr/local/pgsql/bin/psql -d musicbrainz -U musicbrainz -f cube.sql

Reboot, start Postgres Server (because if built from source wont be started automatically, Best to put this command in a script in postgres users home dir)

sudo - postgres/usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data -l logfile start

Install pgadmin

Run pgadmin check can connect to server

I then followed Database Installation but had to make some changes for NGS release as follows, my revised instructions are below


Go to Musicbrainz-server repository

rm admin/sql/CreateIndexes.sql
git pull

Download mbdump and mpdump-cdstubs (cdstubs) and mbdump-derived (annotations, tags)dumps Decompress the mbdump*.tar.bz2 files:

tar jxf mbdump.tar.bz2
tar jxf mbdump-derived.tar.bz
tar jxf mbdump-cdstubs.tar.bz

Edit admin/sql/CreateIndexes.sql removing the page indexes and musicbrainz_collate indexes

/usr/local/pgsql/bin/psql -U musicbrainz musicbrainz
\i CreateTables.sql
\i vertical/rawdata/CreateTables.sql
\i vertical/rawdata/CreatePrimaryKeys.sql
\i vertical/rawdata/CreateIndexes.sql

Cntl-D

Import each file into the database:

paul$ cd mbdump/
paul$ mkdir ../done
paul$ for t in *; do echo `date` $t ; echo "\\copy $t from ./$t" | /usr/local/pgsql/bin/psql -U musicbrainz musicbrainz && mv $t ../done/; done; echo `date` Done
paul$ cd ..

Add primary keys and indexes, and optimise the database:

psql -U musicbrainz musicbrainz
\i admin/sql/CreatePrimaryKeys.sql
\i admin/sql/CreateIndexes.sql
VACUUM ANALYZE;
\q

When build indexes modify IndexOptions.prepareDbConnection to not use a schema because schema is not used with the above instructions