Virtual MusicBrainz Server: Difference between revisions
From MusicBrainz Wiki
Jump to navigationJump to search
PavanChander (talk | contribs) mNo edit summary |
Reosarevok (talk | contribs) m (Redirecting to MusicBrainz Server/Setup) |
||
(14 intermediate revisions by 9 users not shown) | |||
Line 1: | Line 1: | ||
#REDIRECT [[MusicBrainz Server/Setup]] |
|||
{{Status|The virtual server image has been updated to contain the [[Server Release Notes/20090524|latest release]] (May 24, 2009) of the [[MusicBrainz Server]].}} |
|||
<small>[[Products]] > [[MusicBrainz Server]] > [[Server Setup|Server setup]] > Virtual MusicBrainz Server </small> |
|||
==What is the virtual MusicBrainz server?== |
|||
The virtual MusicBrainz server is a virtual machine image containing a Linux environment fully configured with the latest release of the '''mb_server''' source code. It is intended to be the fastest way to get you up and running if you want to experiment with the data or hack on some new features. |
|||
The image is built using VMware Workstation. Windows and Linux users can run it using the free [http://www.vmware.com/player/ VMware Player] or [http://www.vmware.com/server/ VMware Server] software, and Mac users can use [http://vmware.com/mac/ VMware Fusion]. The image has hardware requirements of at least 512Mb of RAM and at least 10Gb of free disk space. |
|||
==What's in the image?== |
|||
The virtual server is running the base [[WikiPedia:Debian|Debian]] stable installation. It also includes pre-configured versions of Apache, mod_perl, all the perl dependencies, and Postgresql. |
|||
There is a full check-out of the latest MusicBrainz server release, and an empty MusicBrainz database. |
|||
[TODO: Explain disk partitions. 4Gb root, 10Gb data. vmdx files will expand] |
|||
==Running the virtual MusicBrainz server== |
|||
===Before you begin=== |
|||
# Download [http://www.vmware.com/products/player VMware Player] or [http://www.vmware.com/products/fusion/ VMware Fusion] for Mac OS-X and follow the instructions for installing and setting it up. |
|||
# Download the MusicBrainz server image from [ftp://ftp.musicbrainz.org/pub/musicbrainz/MBServer-20090524.tar.bz2 ftp://ftp.musicbrainz.org/pub/musicbrainz/MBServer-20090524.tar.bz2]. |
|||
#* To extract it in Windows, you will need to use an application like [http://www.7-zip.org/ 7-Zip]. Extracting the original .bz2 file will result in a .tar from which you will extract the VMware files themselves. |
|||
#* Linux users can use <code><nowiki>tar xvjf MBServer-20090224.tar.bz2</nowiki></code>. |
|||
===First run=== |
|||
<ol> |
|||
<li>Start the virtual machine in VMware by navigating to the folder where you extracted the VMware files and choosing MBServer. VMware will now boot Linux in a virtual machine. Wait until you get the prompt: "brainzvm login:"</li> |
|||
<li>There are two users on the system with passwords: "root" and "mbserver". '''By default, the user passwords are the same as the usernames.''' So the first thing to do is login as root, and change the passwords to something a little more secure.<br/> |
|||
<pre>brainzvm:~# passwd root |
|||
Enter new UNIX password: [enter a new password] |
|||
Retype new UNIX password: [repeat it] |
|||
passwd: password updated successfully |
|||
brainzvm:~# passwd mbserver |
|||
Enter new UNIX password: [enter a new password] |
|||
Retype new UNIX password: [repeat it] |
|||
passwd: password updated successfully |
|||
</pre></li> |
|||
<li>Make sure you have an internet connection inside your virtual machine by pinging Google. (This may fail if you're behind a firewall that blocks pings.) <br /> |
|||
<pre>brainzvm:~# ping -c 3 google.com |
|||
PING google.com (216.239.37.99) 56(84) bytes of data. |
|||
64 bytes from 216.239.37.99: icmp_seq=1 ttl=244 time=17.3 ms |
|||
64 bytes from 216.239.37.99: icmp_seq=2 ttl=244 time=17.9 ms |
|||
64 bytes from 216.239.37.99: icmp_seq=3 ttl=244 time=8.85 ms |
|||
--- google.com ping statistics --- |
|||
3 packets transmitted, 3 received, 0% packet loss, time 2029ms |
|||
rtt min/avg/max/mdev = 8.854/14.718/17.950/4.153 ms |
|||
</pre></li> |
|||
<li>If this works, find out what IP address is assigned to your virtual server by running ifconfig. <br /> |
|||
<pre>brainzvm:~# ifconfig |
|||
eth0 Link encap:Ethernet HWaddr 00:0C:29:62:7F:57 |
|||
inet addr:192.168.213.42 Bcast:192.168.213.255 Mask:255.255.255.0 |
|||
[ ^^^^^^^^^^^^^^ this is the address you care about ] |
|||
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 |
|||
[snip] |
|||
</pre></li> |
|||
<li>Outside of your virtual machine, open a webbrowser and try accessing the virtual IP address as [http://ipaddress/ http://ipaddress/]. You should see the MusicBrainz homepage, but with a banner that reads "Brainzvm Server".</li> |
|||
<li>Logout.</li> |
|||
</ol> |
|||
===Loading data=== |
|||
<ol> |
|||
<li>Outside the VM, ssh into it, as the mbserver user. (Windows users can use the excellent ssh client [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]). This makes it easier to cut and paste things from this document.</li> |
|||
<li>Now let's download the latest MB data dump. Use <code><nowiki> lynx http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/ </nowiki></code>, go into the latest folder and get the ''mbdump.tar.bz2'', ''mbdump-derived.tar.bz2'' and ''mbdump-stats.tar.bz2'' files. (You can grab the other files if you want, but the data they contain is stuff like old moderations; Nothing necessary to get your server up and running) Save them to the mbserver home directory then press 'q' to exit lynx.</li> |
|||
<li>In mbserver's home directory is a directory called ''svn''. This contains the latest MusicBrainz server code at the time of release. To make sure you have an up-to-date copy: <br /> |
|||
<pre>mbserver@brainzvm:~$ cd svn/mb_server |
|||
mbserver@brainzvm:~/svn/mb_server$ svn up |
|||
</pre></li> |
|||
<li>Stop apache, drop the empty musicbrainz database, and import these dumps: <br /> |
|||
<pre>mbserver@brainzvm:~/svn/mb_server$ su - |
|||
Password: |
|||
brainzvm:~# /etc/init.d/apache-perl stop |
|||
Stopping web server: apache-perl. |
|||
brainzvm:~# exit |
|||
mbserver@brainzvm:~/svn/mb_server$ dropdb -U postgres musicbrainz_db |
|||
DROP DATABASE |
|||
mbserver@brainzvm:~/svn/mb_server$ dropdb -U postgres musicbrainz_db_raw |
|||
DROP DATABASE |
|||
mbserver@brainzvm:~/svn/mb_server$ ./admin/InitDb.pl --createdb --echo --import -- --tmp-dir=/mnt/data/tmp ~/mbdump*.tar.bz2 |
|||
<snip> -- Go get some coffee and a book, cause this'll take at least an hour to run. |
|||
Fri Oct 21 21:11:56 2005 : Initialized and imported data into the database. |
|||
Fri Oct 21 21:11:56 2005 : InitDb.pl succeeded |
|||
mbserver@brainzvm:~$ su - |
|||
Password: |
|||
brainzvm:~# /etc/init.d/apache-perl start |
|||
Starting web server: apache-perl. |
|||
brainzvm:~# exit |
|||
</pre></li> |
|||
<li>Going to [http://ipaddress/ http://ipaddress/] should now present you with your very own searchable MusicBrainz server. Hurrah!</li> |
|||
</ol> |
|||
===Troubleshooting=== |
|||
If you see an error during the data load "Schema sequence mismatch - codebase is <number>, /home/mbserver/mbdump-derived.tar.bz2 is <other number>" then see this forum message [http://forums.musicbrainz.org/viewtopic.php?pid=3516 http://forums.musicbrainz.org/viewtopic.php?pid=3516] |
|||
===Hacking on the code=== |
|||
[TODO: Link to some developer documentation] |
|||
===Working with the database=== |
|||
If you want to access Postgresql from other tools you should change two configurations files. Both are located in the configuration directory. If you use version 8.3 you'll find it under /etc/postgresql/8.3/main: |
|||
in file pg_hba.conf put the following line: <pre>host all all 0.0.0.0 0.0.0.0 md5 |
|||
</pre> |
|||
in file postgresql.conf <pre>listen_addresses = '*' |
|||
</pre> |
|||
After you are done remember to restart postgres <pre>/etc/init.d/postgresql-8.3 restart |
|||
</pre> |
|||
Also a change of the password of the postgres user is required: |
|||
<pre>mserver@brainzvm:~/svn/mb_server$ ./admin/psql SYSTEM |
|||
template1=# alter user postgres with password 'postgres'; |
|||
</pre> |
|||
===Setting up your server as a replication "Slave"=== |
|||
[[Image:Attention.png]] ''The [[Live Data Feed]] is restricted to non-commercial settings. For a commercial setting, you will need to obtain a commercial data license from the MetaBrainz Foundation'' |
|||
====Define your server as a replication "Slave"==== |
|||
Change the type of your server: edit <code>cgi-bin/DBDefs.pm</code> and change |
|||
<pre>sub REPLICATION_TYPE { RT_STANDALONE }</pre> |
|||
to |
|||
<pre>sub REPLICATION_TYPE { RT_SLAVE }</pre> |
|||
And define the READONLY Database connexion: |
|||
<pre> |
|||
MusicBrainz::Server::Database->register_all( |
|||
{ |
|||
# How to connect when we need read-write access to the database |
|||
READWRITE => { |
|||
database => "musicbrainz_db", |
|||
username => "musicbrainz_user", |
|||
password => "", |
|||
host => "", |
|||
port => "", |
|||
}, |
|||
# How to connect for read-only access. See "REPLICATION_TYPE" |
|||
READONLY => { |
|||
database => "musicbrainz_db", |
|||
username => "musicbrainz_user", |
|||
password => "", |
|||
host => "", |
|||
port => "", |
|||
}, |
|||
# How to connect for administrative access |
|||
SYSTEM => { |
|||
database => "template1", |
|||
username => "postgres", |
|||
password => "", |
|||
host => "", |
|||
port => "", |
|||
}, |
|||
</pre> |
|||
'''If you have just imported data with an RT_STANDALONE setting''', the following extra steps are required: |
|||
<pre>mserver@brainzvm:~/svn/mb_server$ ./admin/psql READWRITE < admin/sql/DropFKConstraints.sql |
|||
mserver@brainzvm:~/svn/mb_server$ ./admin/psql RAWDATA < admin/sql/vertical/rawdata/DropFKConstraints.sql |
|||
mserver@brainzvm:~/svn/mb_server$ ./admin/psql READWRITE < admin/sql/DropTriggers.sql |
|||
</pre> |
|||
====Syncing your server==== |
|||
<pre>mserver@brainzvm:~/svn/mb_server$ ./admin/replication/LoadReplicationChanges</pre> |
|||
Replication changes are created each hour, so you can add the following entry to the mbserver crontab: |
|||
<pre>0 * * * * /home/mbserver/svn/mb_server/admin/cron/slave.sh</pre> |
|||
[[Category:Development]] |
Latest revision as of 01:35, 11 November 2011
Redirect to: