Difference between revisions of "MusicBrainz Server/Setup"

From MusicBrainz Wiki
Jump to navigationJump to search
m
(→‎Setup MusicBrainz Server from source code: Update after MusicBrainz Docker became officially recommended)
 
(56 intermediate revisions by 13 users not shown)
Line 1: Line 1:
== MusicBrainz Server virtual machine ==
+
== Setup from Docker Compose ==
   
  +
This method is recommended as it makes install, setup, and maintenance much easier for you, to the cost of a small overhead compared to the size of the database.
Running a MusicBrainz Server as a virtual machine requires some Linux knowledge, but it is vastly simpler than installing the server from scratch. The pre-built virtual image can be imported into either VirtualBox or VMware. If you are using Amazon EC2 you can not use this virtual image and will instead have to follow the steps outlined in the source code [https://github.com/metabrainz/musicbrainz-server/blob/master/INSTALL.md INSTALL.md]. To set up a virtual machine instance, download the torrent file and follow these steps:
 
   
  +
It can be used to:
<div style="background-color:#FFFAF5; border: 1px solid #FFDAA4; padding:10px 20px; width: 400px; margin: 20px 20px 20px 0">
 
  +
* Replicate the MusicBrainz website/web service/database,
'''MusicBrainz Server Virtual Machine <br />
 
  +
* Test your own app that queries MusicBrainz web service,
Download via BitTorrent: [http://users.musicbrainz.org/~robert/MusicBrainz%20VM%202013-08-01.torrent MusicBrainz VM 2013-08-01]<br />
 
  +
* Develop MusicBrainz Server itself.
Size: 10.13GB Open Virtualization Archive (OVA)<br/>
 
(the virtual drive can extend up to 40 GB)<br/>
 
Version: 2013-08-01 (beta version)<br />
 
MD5: <small>717726cbea28a4fce51ae6ae803b8a6a</small>
 
</div>
 
   
  +
See [https://github.com/metabrainz/musicbrainz-docker MusicBrainz Docker] and its [https://github.com/metabrainz/musicbrainz-docker/releases release notes].
=== Running with VMware ===
 
# Download [http://www.vmware.com/products/player/overview.html VMware Player] for Windows/Linux or [http://www.vmware.com/products/fusion/overview.html VMware Fusion] for Mac.
 
# Import the downloaded .ova
 
   
=== Running with VirtualBox ===
+
== Setup from source code ==
  +
This can potentially be a very laborious and time consuming method of getting a functioning MusicBrainz server. Use it only if you cannot use MusicBrainz Docker for some reason (disk space, development, …).
# Download and install [http://virtualbox.org VirtualBox] on your machine.
 
# Currently (20130801 beta VM) the .ova file doesn't work directly with VirtualBox so you have to do some additional steps:
 
#* <pre>tar -xf "MusicBrainz 2013-08-01.ova" && VBoxManage clonehd MusicBrainz_2013-08-01-disk1.vmdk MusicBrainz130801.vdi --format VDI</pre> (this will create a 27 GB file that can grow later on up to 40 GB)
 
#* Create a new VirtualBox VM with Type "Linux" and "Version Ubuntu (64 bit)". You should give at least 1 GB RAM.
 
#* Add the previously generated MusicBrainz130801.vdi as hard disk
 
#* (If you try to use or import the vmdk file before running VBoxManage it might fail. Just untar the ova again.)
 
# <strike>Start VirtualBox and choose ''Import Appliance'' from the File menu. Select the downloaded file.</strike>
 
# <strike>Once VirtualBox has imported the appliance, select the imported virtual machine from the list of virtual machines and click on Start.</strike>
 
   
  +
Get a copy of musicbrainz-server from git:
=== Starting the VM ===
 
# Once the instance has started up, log in on the console using the username ''vm'' and password ''musicbrainz''. This account has sudo privileges -- if you would like to set a root password, you can do that via sudo.
 
# Get the IP address of your virtual machine. Note this down from "inet addr" with this command: <pre>ifconfig | grep eth0 -A 1</pre> NOTE: if eth0 is not configured correctly, perhaps you've encountered [http://superuser.com/questions/90584/network-interface-missing-in-ubuntu-after-changing-mac-address this problem].
 
# Optional: The console for Virtual Box is very slow. It may be faster to SSH into the virtual box with a good terminal program.
 
# The MusicBrainz server starts automatically. Now you can reach the MusicBrainz server by pointing your browser to port 5000 of the IP address in step 5.
 
#* e.g If your IP address from step 6 was: 10.1.1.104, then point your browser to http://10.1.1.104:5000
 
#* If the server looks like it started correctly and you cannot connect in your browser, try pinging it from your '''host''' operating system command line <pre>ping <inet addr></pre> If you cannot ping it, you may have a VirtualBox network configuration problem. Try the instructions [http://askubuntu.com/questions/56434/cant-ping-from-virtualbox-ubuntu-server-nat-connection here] to switch your VM to using a ''bridged connection'' rather than ''NAT''. Restart the virtual machine and find the new IP address in step 6 again.
 
 
=== Tuning your VM ===
 
 
We recommend that you give your VM 2GB of ram, if that is possible. The more RAM you give to the VM, the faster it is going to run. To change the memory settings, you will need to shut down the VM, change the settings and then re-start the VM. The Postgres database will be automatically tuned based on the available RAM when the VM starts up.
 
 
=== Running Replication ===
 
 
This VM comes "replication ready". To enable replication, and have the database catch up with the latest replication packets, do this:
 
bin/replicate now
 
 
This will load all of the changes to the database since the VM update.
 
 
NOTE: Loading replication changes might take a long time. If the VM is more than a couple of weeks old, it might be better for you to import a [[MusicBrainz Database/Download|fresh data set]]. Drop pre-filled database using command <pre>dropdb musicbrainz_db</pre> and check [https://github.com/metabrainz/musicbrainz-server/blob/master/INSTALL.md the INSTALL.md file] for how to import new data.
 
 
=== Automating Replication ===
 
 
To turn on background replication, run:
 
 
bin/replicate start
 
 
to turn it off:
 
 
bin/replicate stop
 
 
We recommend leaving replication off for the time being, until you've built search indexes for the VM.
 
 
=== Building search indexes ===
 
 
The VM comes with support to build search indexes. In order to build the indexes, log in to the account and then:
 
 
bin/reindex
 
 
Depending on your machine, this may take quite a long time. We recommend that you leave this running overnight. After the indexes are complete, you should be able to carry out indexed searches in your VM.
 
 
=== Accessing the database ===
 
 
To access the main postgres database, you can do this:
 
 
sudo su - musicbrainz
 
cd musicbrainz-server/admin
 
./psql READWRITE
 
 
If you would like to access the DB from outside the virtual box, take a look at [http://www.cyberciti.biz/tips/postgres-allow-remote-access-tcp-connection.html how to change postgres connection settings].
 
 
=== Turning the VM into development box ===
 
 
If you would like to use the VM to do development instead of using it as a simple database slave, you'll need to edit lib/DBDefs.pm and set REPLICATION_TYPE to RT_STANDALONE and run admin/psql READWRITE and execute the following queries:
 
 
DELETE FROM annotation WHERE editor > (SELECT max(id) FROM editor);
 
DELETE FROM release_annotation WHERE NOT EXISTS (SELECT 1 FROM annotation WHERE annotation.id = release_annotation.annotation);
 
 
then from the command line execute:
 
 
admin/psql READWRITE < admin/sql/CreateFKConstraints.sql
 
admin/psql READWRITE < admin/sql/CreateFunctions.sql
 
 
TODO: The server will probably run out of disk space during this process. We need to add instructions on how to move the DB to a new partition.
 
 
== Setup MusicBrainz Server from source code ==
 
This can potentially be a very laborious and time consuming method of getting a functioning MusicBrainz server. Using the virtual machine is recommended.
 
 
Get a copy of musicbrainz-server from git:
 
 
git clone --recursive https://github.com/metabrainz/musicbrainz-server.git musicbrainz-server
 
git clone --recursive https://github.com/metabrainz/musicbrainz-server.git musicbrainz-server
 
cd musicbrainz-server
 
cd musicbrainz-server
 
And follow the instructions in the [https://github.com/metabrainz/musicbrainz-server/blob/master/INSTALL.md INSTALL] file.
 
And follow the instructions in the [https://github.com/metabrainz/musicbrainz-server/blob/master/INSTALL.md INSTALL] file.
   
== Support ==
+
=== Support ===
   
The setup process may look daunting, but please don't let this discourage you; the INSTALL is thorough and contains a lot of information, and we are willing to provide assistance. If you have questions about installing, join us in the [http://webchat.freenode.net/?channels=musicbrainz-devel #musicbrainz-devel IRC channel] or post a question on the [[Developers Mailing List|developers mailing list]] and we will attempt to help you out.
+
The setup process may look daunting, but please don't let this discourage you; the INSTALL is thorough and contains a lot of information, and we are willing to provide assistance. If you have questions about installing, join us in the [http://webchat.freenode.net/?channels=metabrainz #metabrainz IRC channel] or post a question on the [https://community.metabrainz.org/tags/musicbrainz-server community forum] and we will attempt to help you out.
   
 
We recommend that you dive in and give it a try - who knows how far you'll get and what you might learn along the way!
 
We recommend that you dive in and give it a try - who knows how far you'll get and what you might learn along the way!
   
==Requirements==
+
===Requirements===
   
 
In order to set up a running MusicBrainz server with the full database you will need:
 
In order to set up a running MusicBrainz server with the full database you will need:
 
* A linux box, preferably Ubuntu.
 
* A linux box, preferably Ubuntu.
* 8GB of free disk space, (if you are a developer and only want the server code and database structure 2GB is more than enough).
+
* 60GB+ of free disk space, (if you are a developer and only want the server code and database structure 2GB+ should do, count 6GB+ if you want sample data).
* [[Git]] knowledge which will enable you to check out the source code.
+
* [[Git]] knowledge which will enable you to check out the source code.
  +
* PostgreSQL 9.5 or later (and some [https://github.com/metabrainz/musicbrainz-server/blob/master/INSTALL.md#prerequisites other dependencies])
   
 
As a developer the following knowledge/skills are beneficial:
 
As a developer the following knowledge/skills are beneficial:
* Apache, Perl, mod_perl, PostgreSQL and a number of perl modules.
+
* Perl and a number of perl modules, PostgreSQL, React/JSX.
* How to compile and install packages from source on a Linux box.
+
* How to compile and install packages from source on a Linux box.
 
* How to patch existing packages, although we can help you out if you have questions about that.
 
* How to patch existing packages, although we can help you out if you have questions about that.
   
 
Note: The server has ''never been ported to Windows'', and we suspect that it would be a fair amount of work to make that happen.
 
Note: The server has ''never been ported to Windows'', and we suspect that it would be a fair amount of work to make that happen.
  +
  +
==External Links==
  +
* [https://github.com/metabrainz/musicbrainz-server MusicBrainz Server] on GitHub
  +
** List of [https://blog.metabrainz.org/category/musicbrainz+server/?tag=changelog MusicBrainz Server updates] on the MetaBrainz blog
  +
** List of supplemental [http://blog.musicbrainz.org/category/musicbrainz+server/?tag=instructions instructions] on the MetaBrainz blog
  +
  +
* [https://github.com/metabrainz/musicbrainz-server MusicBrainz Docker] on GitHub
  +
** List of [https://blog.metabrainz.org/category/musicbrainz+virtual-machine/ MusicBrainzB Virtual Machine releases] on the MetaBrainz blog
  +
** List of [https://github.com/metabrainz/musicbrainz-server/releases MusicBrainz Docker releases] on GitHub
   
 
[[Category:WikiDocs Page]] [[Category:Download]] [[Category:Development]] [[Category:Server]]
 
[[Category:WikiDocs Page]] [[Category:Download]] [[Category:Development]] [[Category:Server]]

Latest revision as of 15:55, 15 May 2020

Setup from Docker Compose

This method is recommended as it makes install, setup, and maintenance much easier for you, to the cost of a small overhead compared to the size of the database.

It can be used to:

  • Replicate the MusicBrainz website/web service/database,
  • Test your own app that queries MusicBrainz web service,
  • Develop MusicBrainz Server itself.

See MusicBrainz Docker and its release notes.

Setup from source code

This can potentially be a very laborious and time consuming method of getting a functioning MusicBrainz server. Use it only if you cannot use MusicBrainz Docker for some reason (disk space, development, …).

Get a copy of musicbrainz-server from git:

git clone --recursive https://github.com/metabrainz/musicbrainz-server.git musicbrainz-server
cd musicbrainz-server

And follow the instructions in the INSTALL file.

Support

The setup process may look daunting, but please don't let this discourage you; the INSTALL is thorough and contains a lot of information, and we are willing to provide assistance. If you have questions about installing, join us in the #metabrainz IRC channel or post a question on the community forum and we will attempt to help you out.

We recommend that you dive in and give it a try - who knows how far you'll get and what you might learn along the way!

Requirements

In order to set up a running MusicBrainz server with the full database you will need:

  • A linux box, preferably Ubuntu.
  • 60GB+ of free disk space, (if you are a developer and only want the server code and database structure 2GB+ should do, count 6GB+ if you want sample data).
  • Git knowledge which will enable you to check out the source code.
  • PostgreSQL 9.5 or later (and some other dependencies)

As a developer the following knowledge/skills are beneficial:

  • Perl and a number of perl modules, PostgreSQL, React/JSX.
  • How to compile and install packages from source on a Linux box.
  • How to patch existing packages, although we can help you out if you have questions about that.

Note: The server has never been ported to Windows, and we suspect that it would be a fair amount of work to make that happen.

External Links