Difference between revisions of "MetaBrainz:Annual Report/2013"
|Line 361:||Line 361:|
,from ,,000 from and .
== Server farm ==
== Server farm ==
Revision as of 11:56, 24 July 2014
Year in review
We had four new customers in 2013: 7digital, BandPage, InterTech Media, and Spotify.
7digital uses MusicBrainz data in their digital content platform to provide consumers, partners and developers with open access to an extensive international catalogue of high quality digital content. BandPage uses MusicBrainz data to aid matching their data to the data of their partners. InterTech Media uses MusicBrainz to provide services to radio stations to power their "recently played" listings. Spotify uses MusicBrainz data to improve their data for their music streaming services.
13.05 schema change
This release introduced a new core entity called area. This entity allows us to capture information about countries, subdivisions, cities, and other geographic regions. We currently have information on just under 88,000 areas.
Another addition was support for ISNI codes. These codes make it easier to cross-reference data in MusicBrainz with other databases.
13.10 schema change
This release added another core entity called place. This entity allows us to capture information about physical buildings such as recording studios as well as outdoor venues. We currently have information on just under 3,000 places. Place support was added by Nicolás Tamargo (reosarevok) during our 2013 Summer of Code.
The other major change in this release was to add a recording attribute that identifies the recording as a video. This is useful for situations where a DVD contains both audio and video recordings.
Google Summer of Code
We had three students for Google Summer of Code 2013: Michael Wiencek (bitmap), Nicolás Tamargo (reosarevok), and Maciej Czerwiński (mjjc).
This was Michael's third Summer of Code with us. This year he took on the daunting challenge of improving our release editor and he, yet again, produced some amazing work. Most of his improvements were behind the scenes, but the visible result for users will be a major improvement in speed. The new release editor is still in the review and testing process and has an expected ship date of the end of February. Michael was mentored by Ian McEwen (ianmcorvidae), another MusicBrainz Summer of Code veteran.
Nicolás took on the task of adding place support. This was an extension of the area support added earlier in the year. There is an excellent introduction and visualization of places on our blog. Nicolás was mentored by Nikki.
Maciej's proposal was to create a CC-licensed repository for music reviews. This would be a site that allows anyone to write a non-neutral point of view review that could then be integrated with MusicBrainz and its data. The project completed all of its goals, but was left unfinished since releasing the project was beyond the stated goals for the summer. Maciej was mentored by Rob.
The 13th MusicBrainz summit was held in Berlin, Germany, from Sept 20-22nd. The summit was attended by 15 members of the MusicBrainz community as well as Anders Arpteg from Spotify and Lydia Pintscher from Wikimedia / Wikidata.
We thank Google and Spotify for sponsoring the 2013 summit, and to Wikimedia Deutschland for hosting and providing good internet!
The Amazon cake
Amazon has been a customer of MusicBrainz for a few years and our business dealings had been on quite a few levels (Amazon Associates, Live Data Feed, SoundUnwound). In the end, the process of paying invoices to MetaBrainz at Amazon had become so convoluted that the money simply stopped flowing. Things had gotten so bad that one invoice was approaching a 3-year anniversary; no matter what people at Amazon tried, payments never made it through for a variety of reasons. After many repeated proddings to get the money flowing again, I threatened to send them a nice cake for the 3rd anniversary of the invoice and then post a picture of the cake to Twitter. Still nothing materialized, so I sent the cake as promised and posted the picture to Twitter and Cory posted the picture to BoingBoing. To give Amazon a mountain of credit, the right people finally started talking and a few days later we had a conference call to clear the air, verify outstanding invoices and prepare a lump-sum payment. Before the week was out we had a check from Amazon, paying off all outstanding invoices.
In 2013 the MusicBrainz team also hit some snags that were painful to deal with. Due to several customers not paying is in a timely fashion we found ourselves in a cash crunch situation and it was clear that we could not continue paying the team we had in place. After a lot of internal discussion the painful decision was made to let warp go in order for us to survive the cash crunch and to adjust our spending rate to be less than our income rate.
Then at the summit in Berlin Ollie gave notice and said that his heart was no longer into coding in perl and that he wanted to find a Haskell team to be a part of. Within minutes of this announcement the suggestions came flooding in that we should hire Bitmap has we was about to graduate from University. Luckily for both of us, the offer from MusicBrainz came at the perfect time, which allowed an orderly hand-off from Ollie to Bitmap.
This traffic graph conveys only the successful requests made to the main http://musicbrainz.org servers. Aside from some monthly fluctuations, our traffic has been generally stable. This is not because of the amount of traffic that is requested of us, but because how we limit traffic to the site. If we honored all of the requests made to musicbrainz.org, all of our servers would be constantly on fire. With the resources and bandwidth we have available right now, our server farm is decently loaded and still able to handle occasional traffic spikes.
Our finances in 2013 are summarized by our Profit & Loss statement:
|Live Data Feed Licenses||$179,510.55|
|CC Data License||$9,160.00|
|Web Service Access||$660.00|
|Amazon Affiliate Program||$1,048.65|
|Tagger Affiliate Program||$10,445.05|
|CD Baby Affiliate Program||$5.95|
In the first half of 2013 we struggled to make ends meet. This led to a forced team change (see above) and a renewed focus to bring in more money to keep operations afloat. The latter half of the year was much better as our burn rate was more realistic and income was starting to improve. The efforts to collect money in the latter half of the year also bore fruit, which allowed us to close the year with record income and in the black! We had excess income of $58,119.77 for 2013, which gave us a solid start into 2014.
In the course of 2013 we spent $3,331.65 on hardware and $21,425.00 on hosting costs for a total of $24,756.65. In the same time we served 4.20 billion hits, of which 3.94 billion were web service hits. Please note that the graph above is an aggregate of all attempted requests which is greater than the 4.20 billion served responses. Compared to 2012, our traffic in 2013 was down by 16%.
Our cost of 1 million hits was $5.89 and the cost of 1 million web service hits was $6.28. These costs increased over 2012 because we had to pay for a few hardware maintenance tasks and our overall traffic was down. These two factors combined drive up the costs for 1M requests.
In 2013 we spent $109,721.76 in salaries for developers/community management and our costs for administration was $75,429.95. We earned $179,510.55 from our Live Data Feed and $9,160.00 from Creative Commons licensed data for a total of $188,670.55, which is a 52% increase over last year. A part of this increase was some customers finally making payments in 2013, when they should've paid in 2012. Also in 2013, we added a new category of paid Web Service access and that brought in $660.00.
Our summit sponsorship income was much lager in 2013 than it was in 2012, because the summit sponsorship payment from Spotify for 2012 was wrapped into another contract from 2012 and wasn't paid until 2013. Our end user donations via PayPal came to $6,035.07, which is down 55% from last year. Direct/large donations added up to $72,042.49, including $40,000 from Google, $20,000 from AOL and $10,000 from Spotify.
In 2013, 18,710 editors made a total of 5,047,607 edits. The total number of editors is down from 20,540 in 2012, but the total number of edits has increased by over 800,000 edits. Feedback from the music tech industry confirms the good work of our editors as many people regard MusicBrainz as the best source of music metadata and stable music IDs that can be used to cross-reference music data-sets. Thanks to everyone who edited MusicBrainz in 2013!
At the end of 2013, MusicBrainz had 20 machines in service. From the top, going down:
- rika: User sandbox machine (mbsandbox.org)
- lolo: An extra front end web server
- scooby: ftp, system statistics, blog
- pino: MetaBrainz, MusicBrainz Classic, Cover Art Archive
- baron: Database failover server
- hoser: VM host (forums, blog)
- dagon: Backup server (not in use)
- rocko: Geordi / log analysis
- cartman: Classic search server, index builder
- wiley: New catch all server: SVN, git, jira, wiki, trac, mail, backups
- Gir: Network router (not visible)
- carl: Redundant network gateway
- lenny: Redundant network gateway
- sakura: Backup server (not in use)
- asterix: web server
- astro: web server
- roobarb: search server
- pingu: web server
- dora: search server, memcached server
- totoro: database server
Not shown: Hobbes (Mac mini, tucked away in the back), continuous integration (jenkins), replicated search indexes
MusicBrainz uses ____ Mbits of bandwidth per second on average (____ Mbps inbound, _____ Mbps outbound) and draws ____ Amps of current for a power consumption of about _____ Watts. MusicBrainz physically occupies 20 Us of space (half of a rack) at Digital West in San Luis Obispo, CA.
Words of appreciation
Many thanks to our editors, voters, peer reviewers, bug watchers and other members of our community -- without you MusicBrainz would not be what it is today!
We'd like to also thank our developers that pushed out dozens of releases of the site, Picard and our numerous client libraries. All of your work is critical to enabling the MusicBrainz community to do its job.
In particular we'd like to thank AOL, Google, and Spotify for their large donations made in 2013. These large contributions allow us to carry on with our goal of making MusicBrainz the most comprehensive music encyclopedia out there!
Thank you to everyone who contributed in 2013!