Difference between revisions of "NGS Server Setup"

From MusicBrainz Wiki
Jump to navigationJump to search
(Add European mirror to VM download link)
Line 10: Line 10:
 
Running an NGS virtual machine requires some Linux knowledge, but it vastly simpler than installing NGS from scratch. To use the virtual machine instance, follow these steps:
 
Running an NGS virtual machine requires some Linux knowledge, but it vastly simpler than installing NGS from scratch. To use the virtual machine instance, follow these steps:
   
# Start downloading the latest [ftp://ftp.musicbrainz.org/pub/musicbrainz/vm/MusicBrainz%20NGS%202011-07-20.ova virtual machine instance]. Beware: This is a large (5Gb) download!
+
# Start downloading the latest [http://ftp.musicbrainz.org/pub/musicbrainz/vm/MusicBrainz%20NGS%202011-07-20.ova virtual machine instance] ([http://ftp.uk.musicbrainz.org/pub/musicbrainz/vm/MusicBrainz%20NGS%202011-07-20.ova UK mirror]). Beware: This is a large (5Gb) download!
 
# Download and install [http://virtualbox.org Virtual Box] on your machine.
 
# Download and install [http://virtualbox.org Virtual Box] on your machine.
 
# Start Virtual Box and choose ''Import Appliance'' from the File menu. Select the downloaded file.
 
# Start Virtual Box and choose ''Import Appliance'' from the File menu. Select the downloaded file.

Revision as of 10:02, 20 October 2011

Products > MusicBrainz Server > NGS Server Setup

Next Generation MusicBrainz Server

To set up a Next Generation Schema (NGS) MusicBrainz server, you can choose to download and setup a virtual machine image of NGS or you can install the server into your own Linux instance.

MusicBrainz NGS Virtual Server Image

Running an NGS virtual machine requires some Linux knowledge, but it vastly simpler than installing NGS from scratch. To use the virtual machine instance, follow these steps:

  1. Start downloading the latest virtual machine instance (UK mirror). Beware: This is a large (5Gb) download!
  2. Download and install Virtual Box on your machine.
  3. Start Virtual Box and choose Import Appliance from the File menu. Select the downloaded file.
  4. Once Virtual Box has imported the appliance, select the imported virtual machine from the list of virtual machines and click on Start.
  5. Once the instance has started up, log in on the console using the username musicbrainz and password musicbrainz. This account has sudo privileges -- if you would like to set a root passwd, you can do that via sudo.
  6. Run ifconfig and look at the inet addr for eth0. This is the IP address of your virtual machine. Note this IP address.
  7. Optional: The console for Virtual Box is very slow. I find it faster to SSH into the virtual box with a good terminal program.
  8. Copy lib/DBDefs.pm.default to lib/DBDefs.pm, and update MB_SERVER_ROOT and DEVELOPMENT_SERVER values
  9. To start the NGS server, enter these commands:
musicbrainz@clear:~$ cd musicbrainz-server/script 
musicbrainz@clear:~/musicbrainz-server/script$ ./musicbrainz_server.pl -r

Now you can reach the MusicBrainz server by pointing your browser to port 3000 of the IP address in step 6. If your IP address from step 7 was: 10.1.1.104, then point your browser to http://10.1.1.104:3000

Troubleshooting

If you update the code base in this version of the virtual server, you may encounter an error that complains about DEVELOPMENT_SERVER not being defined, please add this line to your lib/DBDefs.pm file:

sub DEVELOPMENT_SERVER { 1 }

And then start the server.

Running Replication

This VM has replication enabled and to have the database catch up with the latest replication packets, do this:

cd musicbrainz-server/admin/replication
./LoadReplicationChanges

This will load all of the changes to the database since the VM update. To automate this, add this script to a cron job that fires off 10 minutes after each hour. NOTE: Loading replication changes might take a long time. If the VM is more than a couple of weeks old, it might be better for you to import a fresh data set. Check the INSTALL file for how to import new data.

Accessing the database

To access the main postgres database, you can do this:

cd musicbrainz-server/admin
./psql READWRITE

to accces the RAWDATA database (that also contains edits), use RAWDATA, instead of READWRITE. If you would like to access the DB from outside the virtual box, take a look at how to change postgres connection settings.

Turning the VM into development box

If you would like to use the VM to do development instead of using it as a simple database slave, you'll need to edit lib/DBDefs.pm and set REPLICATION_TYPE to RT_STANDALONE and run admin/psql READWRITE and execute the following queries:

DELETE FROM annotation WHERE editor > (SELECT max(id) FROM editor);
DELETE FROM release_annotation WHERE NOT EXISTS (SELECT 1 FROM annotation WHERE annotation.id = release_annotation.annotation);

then from the command line execute:

  1. run admin/psql READWRITE < admin/sql/CreateFKConstraints.sql
  2. run admin/psql READWRITE < admin/sql/CreateFunctions.sql

TODO: The server will probably run out of disk space during this process. We need to add instructions on how to move the DB to a new partition.

Setting up MusicBrainz NGS from source code

(First note: your mileage may vary -- this is one way, that works for some people; it's especially good if your package manager doesn't have all the perl dependencies for musicbrainz-server. However, it's possible it won't work for you at all!)

First:

  1. Get a copy of musicbrainz-server from git: git clone git://git.musicbrainz.org/musicbrainz-server.git musicbrainz-server
  2. cd musicbrainz-server

Then, the next several blocks can be done in parallel or mostly-in-parallel:

  • cp lib/DBDefs.pm.default lib/DBDefs.pm and edit lib/DBDefs.pm to set proper values for MB_SERVER_ROOT, WEB_SERVER, DEVELOPMENT_SERVER, and REPLICATION_TYPE. (DEVELOPMENT_SERVER should probably be changed to be set to 1; REPLICATION_TYPE is probably RT_STANDALONE for a development server)
    1. Install OSSP::uuid, Test::WWW::Mechanize, local::lib, and app::cpanminus, likely from a package manager. Instructions for the first two on Debian/Ubuntu are in INSTALL; local::lib is liblocal-lib-perl; cpanminus you'll likely need to install on your own, with CPAN or via curl -L http://cpanmin.us | perl - --self-upgrade (or the same plus --sudo). For a non-package-manager way to install local::lib, see the local::lib documentation
    2. Make sure the proper environment variables for local::lib are set (PERL5LIB, PERL_MM_OPT, MODULEBUILDRC, and PATH)
    3. cpanm .
      1. As of 2011-04-27, cpanm chokes on Method::Signature 0.67 -- use cpanm http://search.cpan.org/CPAN/authors/id/F/FL/FLORA/Module-Signature-0.66.tar.gz and restart cpanm . if this applies to you (at least one success, as of 2011-05-24)
      2. Dbd::Pg may also have troubles. If so, install postgresql as in INSTALL and restart
      3. If anything else fails, try to use a package manager or regular CPAN to install those things, or inspect ~/.cpanm/build.log and try to figure out what went wrong.
    4. eventually this will move past installing dependencies, at which time it'll likely fail on you. As long as all the dependencies are installed, you're set to continue.

Finally:

  1. follow the instructions in INSTALL on creating the database
  2. ./script/musicbrainz_server.pl -r will start it running for you. Presuming this works, you're set!

Note that you won't have a search server after running this; since NGS has been released, the default search server will work for you! If you're curious, you can also install and set up your own search server, but that is beyond the scope of this document.