Virtual MusicBrainz Server: Difference between revisions

From MusicBrainz Wiki
Jump to navigationJump to search
Line 113: Line 113:
If you want to access Postgresql from other tools you should change two configurations files. Both are located in the configuration directory. If you use version 8.3 you'll find it under /etc/postgresql/8.3/main:
If you want to access Postgresql from other tools you should change two configurations files. Both are located in the configuration directory. If you use version 8.3 you'll find it under /etc/postgresql/8.3/main:


in file pg_hba.conf put the following line: <pre>host all all 0.0.0.0 0.0.0.0 md5
in file pg_hba.conf put the following line: <pre>host all all 0.0.0.0 md5
</pre>
</pre>



Revision as of 02:17, 27 February 2010

Status: The virtual server image has been updated to contain the latest release (May 24, 2009) of the MusicBrainz Server.

Products > MusicBrainz Server > Server setup > Virtual MusicBrainz Server

What is the virtual MusicBrainz server?

The virtual MusicBrainz server is a virtual machine image containing a Linux environment fully configured with the latest release of the mb_server source code. It is intended to be the fastest way to get you up and running if you want to experiment with the data or hack on some new features.

The image is built using VMware Workstation. Windows and Linux users can run it using the free VMware Player or VMware Server software, and Mac users can use VMware Fusion. The image has hardware requirements of at least 512Mb of RAM and at least 10Gb of free disk space.

What's in the image?

The virtual server is running the base Debian stable installation. It also includes pre-configured versions of Apache, mod_perl, all the perl dependencies, and Postgresql.

There is a full check-out of the latest MusicBrainz server release, and an empty MusicBrainz database.

[TODO: Explain disk partitions. 4Gb root, 10Gb data. vmdx files will expand]

Running the virtual MusicBrainz server

Before you begin

  1. Download VMware Player or VMware Fusion for Mac OS-X and follow the instructions for installing and setting it up.
  2. Download the MusicBrainz server image from ftp://ftp.musicbrainz.org/pub/musicbrainz/MBServer-20090524.tar.bz2.
    • To extract it in Windows, you will need to use an application like 7-Zip. Extracting the original .bz2 file will result in a .tar from which you will extract the VMware files themselves.
    • Linux users can use tar xvjf MBServer-20090224.tar.bz2.

First run

  1. Start the virtual machine in VMware by navigating to the folder where you extracted the VMware files and choosing MBServer. VMware will now boot Linux in a virtual machine. Wait until you get the prompt: "brainzvm login:"
  2. There are two users on the system with passwords: "root" and "mbserver". By default, the user passwords are the same as the usernames. So the first thing to do is login as root, and change the passwords to something a little more secure.
    brainzvm:~# passwd root
    Enter new UNIX password:   [enter a new password]
    Retype new UNIX password:  [repeat it]
    passwd: password updated successfully
    brainzvm:~# passwd mbserver
    Enter new UNIX password:   [enter a new password]
    Retype new UNIX password:  [repeat it]
    passwd: password updated successfully
    
  3. Make sure you have an internet connection inside your virtual machine by pinging Google. (This may fail if you're behind a firewall that blocks pings.)
    brainzvm:~# ping -c 3 google.com
    PING google.com (216.239.37.99) 56(84) bytes of data.
    64 bytes from 216.239.37.99: icmp_seq=1 ttl=244 time=17.3 ms
    64 bytes from 216.239.37.99: icmp_seq=2 ttl=244 time=17.9 ms
    64 bytes from 216.239.37.99: icmp_seq=3 ttl=244 time=8.85 ms
    
    --- google.com ping statistics ---
    3 packets transmitted, 3 received, 0% packet loss, time 2029ms
    rtt min/avg/max/mdev = 8.854/14.718/17.950/4.153 ms
    
  4. If this works, find out what IP address is assigned to your virtual server by running ifconfig.
    brainzvm:~# ifconfig
    eth0      Link encap:Ethernet  HWaddr 00:0C:29:62:7F:57
              inet addr:192.168.213.42  Bcast:192.168.213.255  Mask:255.255.255.0
                      [ ^^^^^^^^^^^^^^ this is the address you care about ]
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
    [snip]
    
  5. Outside of your virtual machine, open a webbrowser and try accessing the virtual IP address as http://ipaddress/. You should see the MusicBrainz homepage, but with a banner that reads "Brainzvm Server".
  6. Logout.

Loading data

  1. Outside the VM, ssh into it, as the mbserver user. (Windows users can use the excellent ssh client PuTTY). This makes it easier to cut and paste things from this document.
  2. Now let's download the latest MB data dump. Use lynx http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/ , go into the latest folder and get the mbdump.tar.bz2, mbdump-derived.tar.bz2 and mbdump-stats.tar.bz2 files. (You can grab the other files if you want, but the data they contain is stuff like old moderations; Nothing necessary to get your server up and running) Save them to the mbserver home directory then press 'q' to exit lynx.
  3. In mbserver's home directory is a directory called svn. This contains the latest MusicBrainz server code at the time of release. To make sure you have an up-to-date copy:
    mbserver@brainzvm:~$ cd svn/mb_server
    mbserver@brainzvm:~/svn/mb_server$ svn up
    
  4. Stop apache, drop the empty musicbrainz database, and import these dumps:
    mbserver@brainzvm:~/svn/mb_server$ su -
    Password:
    brainzvm:~# /etc/init.d/apache-perl stop
    Stopping web server: apache-perl.
    brainzvm:~# exit
    
    mbserver@brainzvm:~/svn/mb_server$ dropdb -U postgres musicbrainz_db
    DROP DATABASE
    mbserver@brainzvm:~/svn/mb_server$ dropdb -U postgres musicbrainz_db_raw
    DROP DATABASE
    mbserver@brainzvm:~/svn/mb_server$ ./admin/InitDb.pl --createdb --echo --import  -- --tmp-dir=/mnt/data/tmp ~/mbdump*.tar.bz2
    <snip> -- Go get some coffee and a book, cause this'll take at least an hour to run.
    Fri Oct 21 21:11:56 2005 : Initialized and imported data into the database.
    Fri Oct 21 21:11:56 2005 : InitDb.pl succeeded
    mbserver@brainzvm:~$ su -
    Password:
    brainzvm:~# /etc/init.d/apache-perl start
    Starting web server: apache-perl.
    brainzvm:~# exit
    
  5. Going to http://ipaddress/ should now present you with your very own searchable MusicBrainz server. Hurrah!

Troubleshooting

If you see an error during the data load "Schema sequence mismatch - codebase is <number>, /home/mbserver/mbdump-derived.tar.bz2 is <other number>" then see this forum message http://forums.musicbrainz.org/viewtopic.php?pid=3516

Hacking on the code

[TODO: Link to some developer documentation]

Working with the database

If you want to access Postgresql from other tools you should change two configurations files. Both are located in the configuration directory. If you use version 8.3 you'll find it under /etc/postgresql/8.3/main:

in file pg_hba.conf put the following line:

host    all         all        0.0.0.0          md5

in file postgresql.conf

listen_addresses = '*'

After you are done remember to restart postgres

/etc/init.d/postgresql-8.3 restart

Also a change of the password of the postgres user is required:

mserver@brainzvm:~/svn/mb_server$ ./admin/psql SYSTEM
template1=# alter user postgres with password 'postgres';
template1=# \q

Setting up your server as a replication "Slave"

Attention.png The Live Data Feed is restricted to non-commercial settings. For a commercial setting, you will need to obtain a commercial data license from the MetaBrainz Foundation

Define your server as a replication "Slave"

Change the type of your server: edit cgi-bin/DBDefs.pm and change

sub REPLICATION_TYPE { RT_STANDALONE }

to

sub REPLICATION_TYPE { RT_SLAVE }

And define the READONLY Database connexion:

MusicBrainz::Server::Database->register_all(
    {
        # How to connect when we need read-write access to the database
        READWRITE => {
            database    => "musicbrainz_db",
            username    => "musicbrainz_user",
            password    => "",
            host        => "",
            port        => "",
        },
        # How to connect for read-only access.  See "REPLICATION_TYPE"
        READONLY => {
            database    => "musicbrainz_db",
            username    => "musicbrainz_user",
            password    => "",
            host        => "",
            port        => "",
        },
        # How to connect for administrative access
        SYSTEM  => {
            database    => "template1",
            username    => "postgres",
            password    => "",
            host        => "",
            port        => "",
        },

If you have just imported data with an RT_STANDALONE setting, the following extra steps are required:

mserver@brainzvm:~/svn/mb_server$ ./admin/psql READWRITE < admin/sql/DropFKConstraints.sql
mserver@brainzvm:~/svn/mb_server$ ./admin/psql RAWDATA < admin/sql/vertical/rawdata/DropFKConstraints.sql
mserver@brainzvm:~/svn/mb_server$ ./admin/psql READWRITE < admin/sql/DropTriggers.sql

Syncing your server

mserver@brainzvm:~/svn/mb_server$ ./admin/replication/LoadReplicationChanges

Replication changes are created each hour, so you can add the following entry to the mbserver crontab:

0 * * * * /home/mbserver/svn/mb_server/admin/cron/slave.sh