MetaBrainz:Annual Report/2013

From MusicBrainz Wiki
Jump to navigationJump to search

Year in review

2013 continued the pace of life that we had set for MusicBrainz in 2012. We continued to release new versions of the server software every two weeks and our core development team was focused on improving the site based on bug reports and feature requests from our community. After a lot of tweaking of our internal processes of how we collect bugs and feature requests and then how to implement, test and roll out new code, we've hit a stable process that is heavily community driven. This stable work method allowed the team to focus on improving the site and continuing to add new features (like Areas and Places) that we've been wanting to add for the past few years.

The Cover Art Archive continues to grow and is now the primary source for cover art for MusicBrainz. We also embarked on the journey to create a music review site as one of the Summer of Code projects and continued to work on a data repository that will contain data from 3rd party data sources. Some record labels are warming up to MusicBrainz and are considering providing us data feeds, which we will feed into our new data repository. This is intended to bridge the gap between a top-down data stream from a label and a bottom-up community like MusicBrainz.

Overall the year was somewhat quiet and normal; we had the usual 2 schema change releases and a good Summer of Code. We had our annual summit and took on a few new customers. But, we also faced some challenges on the business side as a number of customers were behind on payments. In the end, that forced some team changes and led to us sending a cake to Amazon to congratulate them on the third anniversary of an outstanding invoice. Our finances at the end of 2013 are much stronger than when 2013 started and we're looking much more solid for 2014.

New customers

We had four new customers in 2013: 7digital, BandPage, InterTech Media, and Spotify.

7digital uses MusicBrainz data in their digital content platform to provide consumers, partners and developers with open access to an extensive international catalogue of high quality digital content. BandPage uses MusicBrainz data to aid matching their data to the data of their partners. InterTech Media uses MusicBrainz to provide services to radio stations to power their "recently played" listings. Spotify uses MusicBrainz data to improve their data for their music streaming services.

13.05 schema change

This release introduced a new core entity called area. This entity allows us to capture information about countries, subdivisions, cities, and other geographic regions. We currently have information on just under 88,000 areas.

Another addition was support for ISNI codes, which make it easier to cross-reference data in MusicBrainz with other databases. We also added the capability for multiple release events and added the schema changes for instrument credits.

13.10 schema change

This release added another core entity called place. This entity allows us to capture information about physical buildings such as recording studios as well as outdoor venues. We currently have information on just under 3,000 places. Place support was added by Nicolás Tamargo (reosarevok) during our 2013 Summer of Code.

The other major change in this release was to add a recording attribute that identifies the recording as a video. This is useful for situations where a DVD or a digital release contains both audio and video recordings. We also finally removed support for PUIDs, since AcoustID had become the preferred acoustic fingerprinting solution for MusicBrainz a while ago.

Google Summer of Code

We had three students for Google Summer of Code 2013: Michael Wiencek (bitmap), Nicolás Tamargo (reosarevok), and Maciej Czerwiński (mjjc).

This was Michael's third Summer of Code with us. This year he took on the daunting challenge of improving our release editor and he, yet again, produced some amazing work. Most of his improvements were behind the scenes, but the visible result for users will be a major improvement in speed. The new release editor is still in the review and testing process and has an expected ship date of the end of February. Michael was mentored by Ian McEwen (ianmcorvidae), another MusicBrainz Summer of Code veteran.

Nicolás took on the task of adding place support. This was an extension of the area support added earlier in the year. There is an excellent introduction and visualization of places on our blog. Nicolás was mentored by Nikki.

Maciej's proposal was to create a CC-licensed repository for music reviews. This would be a site that allows anyone to write a non-neutral point of view review that could then be integrated with MusicBrainz and its data. The project completed all of its goals, but was left unfinished since releasing the project was beyond the stated goals for the summer. Maciej was mentored by Rob (ruaok).

MusicBrainz summit

The 13th MusicBrainz summit was held in Berlin, Germany, from Sept 20-22nd. The summit was attended by 15 members of the MusicBrainz community as well as Anders Arpteg from Spotify and Lydia Pintscher from Wikimedia / Wikidata. The summit was very productive and resulted in several actionable tasks. An overview of the summit is available on our blog, and there is also a detailed listing of discussed topics.

We thank Google and Spotify for sponsoring the 2013 summit, and to Wikimedia Deutschland for hosting and providing good internet!

The Amazon cake

Amazon has been a customer of MusicBrainz for a few years and our business dealings had been on quite a few levels (Amazon Associates, Live Data Feed, SoundUnwound). In the end, the process of paying invoices to MetaBrainz at Amazon had become so convoluted that the money simply stopped flowing. Things had gotten so bad that one invoice was approaching a 3-year anniversary; no matter what people at Amazon tried, payments never made it through for a variety of reasons. After many repeated proddings to get the money flowing again, we threatened to send them a nice cake for the 3rd anniversary of the invoice and then post a picture of the cake to Twitter. Still nothing materialized, so we sent the cake as promised and posted the picture to Twitter and Cory posted the picture to BoingBoing. To give Amazon a mountain of credit, the right people finally started talking and a few days later we had a conference call to clear the air, verify outstanding invoices and prepare a lump-sum payment. Before the week was out we had a check from Amazon, paying off all outstanding invoices.

Team changes

In 2013 the MusicBrainz team also hit some snags that were painful to deal with. Due to several customers not paying is in a timely fashion we found ourselves in a cash crunch situation and it was clear that we could not continue paying the team we had in place. After a lot of internal discussion the painful decision was made to let Kuno Woudt (Warp) go in order for us to survive the cash crunch and to adjust our spending rate to be less than our income rate. (We realize that in the tech world this may seem like a foreign concept, but as a non-profit we're bound by simple laws like finances and gravity).

Then at the summit in Berlin, Oliver Charles (ocharles) gave notice and said that his heart was no longer into coding in perl and that he wanted to find a Haskell team to join. Within minutes of this announcement the suggestions came flooding in that we should hire Bitmap as he was about to graduate from University. Luckily for everyone, the offer from MusicBrainz came at the perfect time, which allowed an orderly hand-off from Ocharles to Bitmap.

Looking ahead

2013 brought about 2 new projects that will play a significant role in our future:

1. Geordi -- Geordi is intended to be an intermediary data site that allows us to store data from music labels so that our community can search, match and import data from music labels. Sadly we were not able to create a fully functioning system (there was a lot of learn about how to do this right), but we're going to re-new our efforts to bring this project about in 2014. Last year the Ninja Tune label was the first label to provide us with a dump of their data for inclusion into MusicBrainz. Once Geordi is fully functional, we plan to import this data and make it available for the MusicBrainz community to import into the main database.

2. CritiqueBrainz -- This new site will allow users to write non-neutral point of view reviews of artists and releases. Sites like Wikipedia are great for getting a lot of interesting information about music, but Wikipedia's neutral point of view policy isn't well suited for a music review. We feel that music reviews should be written with passion and we hope that CritiqueBrainz will allow users to do this. All of the content in CritiqueBrainz will be Creative Commons licensed and cross-linked with MusicBrainz. The first phase of CritiqueBrainz was created as part of Summer of Code, but the scope was too broad to create a fully functioning site. We will pick up CritiqueBrainz for Summer of Code in 2014.

Traffic

MetaBrainz-annual-report-2013-traffic.png

This traffic graph conveys only the successful requests made to the main http://musicbrainz.org servers. Aside from some monthly fluctuations, our traffic has been generally stable. This is not because of the amount of traffic that is requested of us, but because how we limit traffic to the site. If we honored all of the requests made to musicbrainz.org, all of our servers would be constantly on fire. With the resources and bandwidth we have available right now, our server farm is decently loaded and still able to handle occasional traffic spikes.

Finances

Our finances in 2013 are summarized by our Profit & Loss statement:

Income
General Donations $72,042.49
PayPal Donations $6,035.07
Live Data Feed Licenses $179,510.55
Summit Sponsorship $7,500.00
Consulting $1,500.00
CC Data License $9,160.00
Web Service Access $660.00
Amazon Affiliate Program $1,048.65
Tagger Affiliate Program $10,445.05
CD Baby Affiliate Program $5.95
Bank Interest $54.55
Total Income $287,962.31
Expenses
Development $98,828.69
Officer Salary $75,429.95
Community Manager $10,893.07
Hosting $21,425.00
Bank $1,278.78
PayPal $1,056.18
WePay $21.11
Rent $854.00
Utilities $68.90
Hardware $3,331.65
Travel $3,019.87
Internet $649.43
Marketing $1,000.00
Gifts $116.48
Events $7,182.43
Filing Fees $70.00
Software $488.00
Insurance $2,258.00
Accounting $1,871.00
Total Expenses $229,842.54

In the first half of 2013 we struggled to make ends meet. This led to a forced team change (see above) and a renewed focus to bring in more money to keep operations afloat. The latter half of the year was much better as our burn rate was more realistic and income was starting to improve. The efforts to collect money in the latter half of the year also bore fruit, which allowed us to close the year with record income and in the black! We had excess income of $58,119.77 for 2013, which gave us a solid start into 2014.

In the course of 2013 we spent $3,331.65 on hardware and $21,425.00 on hosting costs for a total of $24,756.65. In the same time we served 4.20 billion hits, of which 3.94 billion were web service hits. Please note that the graph above is an aggregate of all attempted requests which is greater than the 4.20 billion served responses. Compared to 2012, our traffic in 2013 was down by 16%.

Our cost of 1 million hits was $5.89 and the cost of 1 million web service hits was $6.28. These costs increased over 2012 because we had to pay for a few hardware maintenance tasks and our overall traffic was down. These two factors combined drive up the costs for 1M requests.

In 2013 we spent $109,721.76 in salaries for developers/community management and our costs for administration was $75,429.95. We earned $179,510.55 from our Live Data Feed and $9,160.00 from Creative Commons licensed data for a total of $188,670.55, which is a 52% increase over last year. A part of this increase was some customers finally making payments in 2013, when they should've paid in 2012. Also in 2013, we added a new category of paid Web Service access and that brought in $660.00.

Our summit sponsorship income was much larger in 2013 than it was in 2012, because the summit sponsorship payment from Spotify for 2012 was wrapped into another contract from 2012 and wasn't paid until 2013. Our end user donations via PayPal came to $6,035.07, which is down 55% from last year. Direct/large donations added up to $72,042.49, including $40,000 from Google, $20,000 from AOL and $10,000 from Spotify.

Top contributors

Top voters
1 pankkake 99,925 13 mat813 13,266
2 chabreyflint 94,371 14 spl0k 12,838
3 fmera 50,083 15 murdos 12,193
4 viper666 49,076 16 jesus2099 11,455
5 Ataki 39,371 17 drsaunde 11,450
6 reosarevok 38,436 18 MeinDummy 10,248
7 Jazzy Jarilith 33,696 19 teleguise 9,513
8 MrH 33,493 20 otters61 9,129
9 salo.rock 32,098 21 monxton 8,902
10 ListMyCDs.com 22,971 22 nikki 7,003
11 Zas 22,136 23 KRSCuan 6,797
12 sbontrager 16,988 24 steinbdj 6,549
Top editors
1 m___ah 305,683 13 Sekhmetouserapis 51,980
2 reosarevok 208,415 14 SultS 48,588
3 ListMyCDs.com 192,558 15 nikki 46,673
4 Senax 146,402 16 mudcrow 46,628
5 kaik 132,484 17 MrH 45,997
6 drsaunde 117,819 18 Hawke 39,714
7 HibiscusKazeneko 85,221 19 Phil.B 38,812
8 monxton 78,047 20 jesus2099 38,728
9 loujin 69,682 21 yeeeargh 37,257
10 otters61 63,110 22 salo.rock 35,332
11 rochusw 53,965 23 dimpole 33,473
12 fmera 53,865 24 chabreyflint 32,904

In 2013, 18,710 editors made a total of 5,047,607 edits. The total number of editors is down from 20,540 in 2012, but the total number of edits has increased by over 800,000 edits. Feedback from the music tech industry confirms the good work of our editors as many people regard MusicBrainz as the best source of music metadata and stable music IDs that can be used to cross-reference music data-sets. Thanks to everyone who edited MusicBrainz in 2013!

Server farm

Metabrainz-annual-report-2013-server-farm.jpg

At the end of 2013, MusicBrainz had 20 machines in service. From the top, going down:

  • rika: User sandbox machine (mbsandbox.org)
  • lolo: An extra front end web server
  • scooby: ftp, system statistics, blog
  • pino: MetaBrainz, MusicBrainz Classic, Cover Art Archive
  • baron: Database failover server
  • hoser: VM host (forums, blog)
  • dagon: Backup server (not in use)
  • rocko: Geordi / log analysis
  • cartman: Classic search server, index builder
  • wiley: New catch all server: SVN, git, jira, wiki, trac, mail, backups
  • dib: Network router, takes up a slot, but only visible from the back.
  • carl: Redundant network gateway
  • lenny: Redundant network gateway
  • sakura: Backup server (not in use)
  • asterix: web server
  • astro: web server
  • roobarb: search server
  • pingu: web server
  • dora: search server, memcached server
  • totoro: database server

Not shown: Hobbes (Mac mini, tucked away in the back), continuous integration (jenkins).

MusicBrainz uses 14.65 Mbits of bandwidth per second on average (1.95 Mbps inbound, 12.78 Mbps outbound) and draws 31 Amps of current for a power consumption of about 3,410 Watts. MusicBrainz physically occupies 20 Us of space (half of a rack) at Digital West in San Luis Obispo, CA.

Words of appreciation

Many thanks to our editors, voters, peer reviewers, bug watchers and other members of our community -- without you MusicBrainz would not be what it is today!

We'd like to also thank our developers that pushed out dozens of releases of the site, Picard and our numerous client libraries. All of your work is critical to enabling the MusicBrainz community to do its job.

We'd also like to thank our awesome Board of Directors, Brewster Kahle and the Internet Archive, all of the donors who contributed money and all of our customers.

In particular we'd like to thank AOL, Google, and Spotify for their large donations made in 2013. These large contributions allow us to carry on with our goal of making MusicBrainz the most comprehensive music encyclopedia out there!

Thank you to everyone who contributed in 2013!