Difference between revisions of "MetaBrainz:Annual Report/2013"

From MusicBrainz Wiki
Line 361: Line 361:
<div style="clear:both;"></div>
<div style="clear:both;"></div>
Blah blah blah
In 2013, 18,710 editors made a total of 5,047,607 edits. The total number of editors is down from 20,540 in 2012, but the total number of edits has increased by over 800,000 edits. Feedback from the music tech industry confirms the good work of our editors as many people regard MusicBrainz as the best source of music metadata and stable music IDs that can be used to cross-reference music data-sets. Thanks to everyone who edited MusicBrainz in 2013!
Large donations added up to _______ including $40,000 from Google, $10,000 from AOL, and _________.
== Server farm ==
== Server farm ==

Revision as of 11:56, 24 July 2014

Year in review

New customers

We had four new customers in 2013: 7digital, BandPage, InterTech Media, and Spotify.

7digital uses MusicBrainz data in their digital content platform to provide consumers, partners and developers with open access to an extensive international catalogue of high quality digital content. BandPage uses MusicBrainz data to aid matching their data to the data of their partners. InterTech Media uses MusicBrainz to provide services to radio stations to power their "recently played" listings. Spotify uses MusicBrainz data to improve their data for their music streaming services.

13.05 schema change

This release introduced a new core entity called area. This entity allows us to capture information about countries, subdivisions, cities, and other geographic regions. We currently have information on just under 88,000 areas.

Another addition was support for ISNI codes. These codes make it easier to cross-reference data in MusicBrainz with other databases.

13.10 schema change

This release added another core entity called place. This entity allows us to capture information about physical buildings such as recording studios as well as outdoor venues. We currently have information on just under 3,000 places. Place support was added by Nicolás Tamargo (reosarevok) during our 2013 Summer of Code.

The other major change in this release was to add a recording attribute that identifies the recording as a video. This is useful for situations where a DVD contains both audio and video recordings.

Google Summer of Code

We had three students for Google Summer of Code 2013: Michael Wiencek (bitmap), Nicolás Tamargo (reosarevok), and Maciej Czerwiński (mjjc).

This was Michael's third Summer of Code with us. This year he took on the daunting challenge of improving our release editor and he, yet again, produced some amazing work. Most of his improvements were behind the scenes, but the visible result for users will be a major improvement in speed. The new release editor is still in the review and testing process and has an expected ship date of the end of February. Michael was mentored by Ian McEwen (ianmcorvidae), another MusicBrainz Summer of Code veteran.

Nicolás took on the task of adding place support. This was an extension of the area support added earlier in the year. There is an excellent introduction and visualization of places on our blog. Nicolás was mentored by Nikki.

Maciej's proposal was to create a CC-licensed repository for music reviews. This would be a site that allows anyone to write a non-neutral point of view review that could then be integrated with MusicBrainz and its data. The project completed all of its goals, but was left unfinished since releasing the project was beyond the stated goals for the summer. Maciej was mentored by Rob.

MusicBrainz summit

The 13th MusicBrainz summit was held in Berlin, Germany, from Sept 20-22nd. The summit was attended by 15 members of the MusicBrainz community as well as Anders Arpteg from Spotify and Lydia Pintscher from Wikimedia / Wikidata.

The summit was very productive and resulted in several actionable tasks. An overview of the summit is available on our blog, and there is also a detailed listing of discussed topics.

We thank Google and Spotify for sponsoring the 2013 summit, and to Wikimedia Deutschland for hosting and providing good internet!

The Amazon cake

Amazon has been a customer of MusicBrainz for a few years and our business dealings had been on quite a few levels (Amazon Associates, Live Data Feed, SoundUnwound). In the end, the process of paying invoices to MetaBrainz at Amazon had become so convoluted that the money simply stopped flowing. Things had gotten so bad that one invoice was approaching a 3-year anniversary; no matter what people at Amazon tried, payments never made it through for a variety of reasons. After many repeated proddings to get the money flowing again, I threatened to send them a nice cake for the 3rd anniversary of the invoice and then post a picture of the cake to Twitter. Still nothing materialized, so I sent the cake as promised and posted the picture to Twitter and Cory posted the picture to BoingBoing. To give Amazon a mountain of credit, the right people finally started talking and a few days later we had a conference call to clear the air, verify outstanding invoices and prepare a lump-sum payment. Before the week was out we had a check from Amazon, paying off all outstanding invoices.

Team changes

In 2013 the MusicBrainz team also hit some snags that were painful to deal with. Due to several customers not paying is in a timely fashion we found ourselves in a cash crunch situation and it was clear that we could not continue paying the team we had in place. After a lot of internal discussion the painful decision was made to let warp go in order for us to survive the cash crunch and to adjust our spending rate to be less than our income rate.

Then at the summit in Berlin Ollie gave notice and said that his heart was no longer into coding in perl and that he wanted to find a Haskell team to be a part of. Within minutes of this announcement the suggestions came flooding in that we should hire Bitmap has we was about to graduate from University. Luckily for both of us, the offer from MusicBrainz came at the perfect time, which allowed an orderly hand-off from Ollie to Bitmap.

Looking forward

Ingestion (Geordi)



This traffic graph conveys only the successful requests made to the main http://musicbrainz.org servers. Aside from some monthly fluctuations, our traffic has been generally stable. This is not because of the amount of traffic that is requested of us, but because how we limit traffic to the site. If we honored all of the requests made to musicbrainz.org, all of our servers would be constantly on fire. With the resources and bandwidth we have available right now, our server farm is decently loaded and still able to handle occasional traffic spikes.


Our finances in 2013 are summarized by our Profit & Loss statement:

General Donations $72,042.49
PayPal Donations $6,035.07
Live Data Feed Licenses $179,510.55
Summit Sponsorship $7,500.00
Consulting $1,500.00
CC Data License $9,160.00
Web Service Access $660.00
Amazon Affiliate Program $1,048.65
Tagger Affiliate Program $10,445.05
CD Baby Affiliate Program $5.95
Bank Interest $54.55
Total Income $287,962.31
Development $98,828.69
Officer Salary $75,429.95
Community Manager $10,893.07
Hosting $21,425.00
Bank $1,278.78
PayPal $1,056.18
WePay $21.11
Rent $854.00
Utilities $68.90
Hardware $3,331.65
Travel $3,019.87
Internet $649.43
Marketing $1,000.00
Gifts $116.48
Events $7,182.43
Filing Fees $70.00
Software $488.00
Insurance $2,258.00
Accounting $1,871.00
Total Expenses $229,842.54

In the first half of 2013 we struggled to make ends meet. This led to a forced team change (see above) and a renewed focus to bring in more money to keep operations afloat. The latter half of the year was much better as our burn rate was more realistic and income was starting to improve. The efforts to collect money in the latter half of the year also bore fruit, which allowed us to close the year with record income and in the black! We had excess income of $58,119.77 for 2013, which gave us a solid start into 2014.

In the course of 2013 we spent $3,331.65 on hardware and $21,425.00 on hosting costs for a total of $24,756.65. In the same time we served 4.20 billion hits, of which 3.94 billion were web service hits. Please note that the graph above is an aggregate of all attempted requests which is greater than the 4.20 billion served responses. Compared to 2012, our traffic in 2013 was down by 16%.

Our cost of 1 million hits was $5.89 and the cost of 1 million web service hits was $6.28. These costs increased over 2012 because we had to pay for a few hardware maintenance tasks and our overall traffic was down. These two factors combined drive up the costs for 1M requests.

In 2013 we spent $109,721.76 in salaries for developers/community management and our costs for administration was $75,429.95. We earned $179,510.55 from our Live Data Feed and $9,160.00 from Creative Commons licensed data for a total of $188,670.55, which is a 52% increase over last year. A part of this increase was some customers finally making payments in 2013, when they should've paid in 2012. Also in 2013, we added a new category of paid Web Service access and that brought in $660.00.

Our summit sponsorship income was much lager in 2013 than it was in 2012, because the summit sponsorship payment from Spotify for 2012 was wrapped into another contract from 2012 and wasn't paid until 2013. Our end user donations via PayPal came to $6,035.07, which is down 55% from last year. Direct/large donations added up to $72,042.49, including $40,000 from Google, $20,000 from AOL and $10,000 from Spotify.

Top contributors

Top voters
1 pankkake 99,925 13 mat813 13,266
2 chabreyflint 94,371 14 spl0k 12,838
3 fmera 50,083 15 murdos 12,193
4 viper666 49,076 16 jesus2099 11,455
5 Ataki 39,371 17 drsaunde 11,450
6 reosarevok 38,436 18 MeinDummy 10,248
7 Jazzy Jarilith 33,696 19 teleguise 9,513
8 MrH 33,493 20 otters61 9,129
9 salo.rock 32,098 21 monxton 8,902
10 ListMyCDs.com 22,971 22 nikki 7,003
11 Zas 22,136 23 KRSCuan 6,797
12 sbontrager 16,988 24 steinbdj 6,549
Top editors
1 m___ah 305,683 13 Sekhmetouserapis 51,980
2 reosarevok 208,415 14 SultS 48,588
3 ListMyCDs.com 192,558 15 nikki 46,673
4 Senax 146,402 16 mudcrow 46,628
5 kaik 132,484 17 MrH 45,997
6 drsaunde 117,819 18 Hawke 39,714
7 HibiscusKazeneko 85,221 19 Phil.B 38,812
8 monxton 78,047 20 jesus2099 38,728
9 loujin 69,682 21 yeeeargh 37,257
10 otters61 63,110 22 salo.rock 35,332
11 rochusw 53,965 23 dimpole 33,473
12 fmera 53,865 24 chabreyflint 32,904

In 2013, 18,710 editors made a total of 5,047,607 edits. The total number of editors is down from 20,540 in 2012, but the total number of edits has increased by over 800,000 edits. Feedback from the music tech industry confirms the good work of our editors as many people regard MusicBrainz as the best source of music metadata and stable music IDs that can be used to cross-reference music data-sets. Thanks to everyone who edited MusicBrainz in 2013!

Server farm


At the end of 2013, MusicBrainz had 20 machines in service. From the top, going down:

  • rika: User sandbox machine (mbsandbox.org)
  • lolo: An extra front end web server
  • scooby: ftp, system statistics, blog
  • pino: MetaBrainz, MusicBrainz Classic, Cover Art Archive
  • baron: Database failover server
  • hoser: VM host (forums, blog)
  • dagon: Backup server (not in use)
  • rocko: Geordi / log analysis
  • cartman: Classic search server, index builder
  • wiley: New catch all server: SVN, git, jira, wiki, trac, mail, backups
  • Gir: Network router (not visible)
  • carl: Redundant network gateway
  • lenny: Redundant network gateway
  • sakura: Backup server (not in use)
  • asterix: web server
  • astro: web server
  • roobarb: search server
  • pingu: web server
  • dora: search server, memcached server
  • totoro: database server

Not shown: Hobbes (Mac mini, tucked away in the back), continuous integration (jenkins), replicated search indexes

MusicBrainz uses ____ Mbits of bandwidth per second on average (____ Mbps inbound, _____ Mbps outbound) and draws ____ Amps of current for a power consumption of about _____ Watts. MusicBrainz physically occupies 20 Us of space (half of a rack) at Digital West in San Luis Obispo, CA.

Words of appreciation

Many thanks to our editors, voters, peer reviewers, bug watchers and other members of our community -- without you MusicBrainz would not be what it is today!

We'd like to also thank our developers that pushed out dozens of releases of the site, Picard and our numerous client libraries. All of your work is critical to enabling the MusicBrainz community to do its job.

We'd also like to thank our awesome Board of Directors, Brewster Kahle and the Internet Archive, all of the donors who contributed money and all of our customers.

In particular we'd like to thank AOL, Google, and Spotify for their large donations made in 2013. These large contributions allow us to carry on with our goal of making MusicBrainz the most comprehensive music encyclopedia out there!

Thank you to everyone who contributed in 2013!