Difference between revisions of "Fingerprinting"

From MusicBrainz Wiki
Jump to navigationJump to search
(25 intermediate revisions by 9 users not shown)
Line 1: Line 1:
Musicbrainz has used several [[wikipedia:Acoustic_fingerprint|audio fingerprinting systems]] over its lifetime.
MusicBrainz has used several [[wikipedia:Acoustic_fingerprint|audio fingerprinting systems]] over its lifetime.


All of them (so far) work in essentially the same way. It is generally a two-step process of submission and lookup. First, the raw audio is used to create a fingerprint, which is then submitted to a third-party server. This server analyzes the fingerprint, compares it to other fingerprints, and decides whether it is sufficiently different from known fingerprints as to issue a new ID.
All of them (so far) work in essentially the same way based on [[wikipedia:Automatic_content_recognition|ACR]] technology. It is generally a two-step process of submission and lookup. First, the raw audio is used to create a fingerprint, which is then submitted to a third-party server. This server analyzes the fingerprint, compares it to other fingerprints, and decides whether it is sufficiently different from known fingerprints as to issue a new ID.


Once this step is done, a fingerprint can be calculated for any file and this can be used to look up the corresponding ID.
Once this step is done, a fingerprint can be calculated for any file and this can be used to look up the corresponding ID.
Line 7: Line 7:
This ID is associated with a given track (pre-NGS) or recording (post-NGS), and metadata can be gathered from there.
This ID is associated with a given track (pre-NGS) or recording (post-NGS), and metadata can be gathered from there.


==TRM==
==AcoustID==
TRM (TRM Recognizes Music) IDs were MusicBrainz’ first audio fingerprinting system. This system was created by Relatable, and added to Musicbrainz in 2000.<ref>http://www.relatable.com/tech/trm.html</ref>
[[AcoustID]] is MusicBrainz’ third and most recent audio fingerprinting system. This was created by [[User:LukasLalinsky|Lukáš Lalinský]], and made public around January 2011.<ref>http://oxygene.sk/lukas/2011/01/how-does-chromaprint-work/</ref>


AcoustID support was implemented in Picard 0.16 released in October 2011.<ref>http://blog.musicbrainz.org/?p=1096</ref>
This system was used in the [[History:Classic_Tagger|original musicbrainz tagger application]].


It has several immediate advantages:
This system worked reasonably well for finding duplicate music files on a local system, but had problems with collisions (different-sounding audio which got the same ID) and duplicates (same-sounding audio which has different IDs). Apparently the server also did not scale well<!-- to the huge number of TRMs required? to multiple/big/multicore servers? ?? -->
* It is open source.
* It is actively developed, along with supporting software.
* It gives the ability to visually compare music
* AcoustID fingerprints have their duration recorded, making it easy to discard certain incorrect links between recordings and acoustIDs.


== Older fingerprinting systems ==
==PUID==
PUIDs are Musicbrainz’ second audio fingerprinting system. This was initially operated by MusicIP, and bought by Gracenote in June 2011. Gracenote is expected to discontinue the public service soon, and it already appears to be largely non-functional.


===TRM===
This system was better than TRM, but still has several major problems:
TRM (TRM Recognizes Music) IDs were MusicBrainz’ first audio fingerprinting system. This system was created by [http://www.relatable.com/ Relatable], and added to MusicBrainz in 2000.<ref>http://www.relatable.com/tech/trm.html</ref>
* The fingerprint submission system is not open source, and as such cannot be included in Picard. (the lookup system is, and was included in Picard as of version 0.7.0, released in July 2006.<ref>http://blog.musicbrainz.org/?p=161</ref>)

This system was used in the [[History:Classic_Tagger|original MusicBrainz tagger application]]. The client-side audio fingerprinting library was open source and integrated into [http://bugs.musicbrainz.org/browser/libmusicbrainz/tags/release-2.1.5 libmusicbrainz].

This system worked reasonably well for finding duplicate music files on a local system, but had problems with collisions (different-sounding audio which got the same ID) and duplicates (same-sounding audio which has different IDs). The server was also incapable of handling the number of TRMs needed for MusicBrainz, and Relatable didn’t seem to be interested in supporting it further.<ref>http://blog.musicbrainz.org/?p=124</ref>

TRM support was removed in November 2008<ref>http://blog.musicbrainz.org/?p=363</ref>.

===PUID===
PUIDs were MusicBrainz’ second audio fingerprinting system. This was initially operated by MusicIP (later AmpliFIND Music Services), and bought by Gracenote in June 2011. Gracenote is expected to discontinue the public service soon, and it already appears to be largely non-functional.

MusicBrainz started to use these IDs in March 2006<ref>http://blog.musicbrainz.org/?p=161</ref> and they were implemented in Picard 0.7.0 released in July 2006.<ref>http://blog.musicbrainz.org/?p=182</ref>

This system was better than TRM, but still had several major problems:
* The fingerprint submission system is not open source, and as such could not be included in Picard. Only looking up existing fingerprints was possible in Picard, new fingerprints had to be created using either [http://www.amplifindmusicservices.com/what/downloads.php MusicIP Mixer] or the command line genpuid (no longer available, the latest version is mirrored on [http://ftp.musicbrainz.org/pub/musicbrainz/genpuid/ our ftp server]).
* The fingerprinting process is slow, both on the client side and the server side
* The fingerprinting process is slow, both on the client side and the server side
* Over time, the operators have become less and less interested in running the server, to the point where today it is barely working, if it works at all.
* Over time, the operators have become less and less interested in running the server, to the point where today it is barely working, if it works at all.


PUIDs are also quite opaque, being nothing more than a unique number referencing a database outside of MusicBrainz’ control. If/when that database goes away, they become useless.
PUIDs are also quite opaque, being nothing more than a unique number referencing a database outside of MusicBrainz’ control. If/when that database goes away, they become useless. See the [http://www.freshpatents.com/Audio-fingerprinting-system-and-method-dt20050324ptan20050065976.php patent application] for details on the technology. The client-side audio fingerprinting library, [http://bugs.musicbrainz.org/browser/libofa/trunk libofa], is open-source.


PUID support was removed from the MusicBrainz server in October 2013.<ref>https://blog.metabrainz.org/?p=1832</ref><ref>https://blog.metabrainz.org/?p=2197</ref>
==AcoustID==
[[AcoustID]] is Musicbrainz’ third and latest up-and-coming audio fingerprinting system. This was created by [[User:luks]], and made public around January 2011.<ref>http://oxygene.sk/lukas/2011/01/how-does-chromaprint-work/</ref>


In June 2020 <ref>https://blog.metabrainz.org/?p=8079</ref> all PUID edits were removed from the database.
It has several immediate advantages:

* It is open source.
==Other audio fingerprinting systems==
* It is actively developed, along with supporting software.

* It gives the ability to visually compare music
===Open Source===
* AcoustID fingerprints have their duration recorded, making it easy to discard certain incorrect links between recordings and acoustIDs.

* The fingerprint in Kurt Rosenfeld's [http://www.w140.com/audio/ FDMF].
* [https://github.com/dkourtesis/musicuri MusicURI], part of the [http://sourceforge.net/projects/mpeg7audiodb Mpeg-7 Audio DB] project.
* [http://code.google.com/p/audioscout/ AudioScout]. Based on the [http://www.phash.org/ pHash] audio fingerprinting library, developed by the same authors. Uses the "Philips Robust Hashing" algorithm.
* [http://open-fp.sourceforge.net/ OpenFP].
* [http://echoprint.me/ Echoprint]. Audio fingerprinting solution developed by the Echo Nest.
* ''[defunct]'' [http://code.google.com/p/libfooid/ libFooID]. An audio fingerprinting library used by, and developed for [http://www.foosic.org/ foosic].
* ''[defunct]'' [http://sourceforge.net/projects/freetantrum Freetantrum]. It seems to be a dead project (its home page was replaced with an advert for unrelated things in 2001), but it may be worth investigating and resurrecting the code they produced.
* ''[defunct]'' [http://jhears.org/ jHears] is an acoustic fingerprinting framework based on [[Future Proof Fingerprint|FutureProofFingerPrint]] design by Geoff Schmidt (formerly of Tuneprint). jHears is developed by Juha Heljoranta.

===Partially Open-Source===

* [http://blog.last.fm/2010/07/09/fingerprint-api-and-app-updated Last.fm]. The client-side audio fingerprinting library ([https://github.com/lastfm/Fingerprinter fplib]) is open-source. Uses the "[http://www.cs.cmu.edu/~yke/musicretrieval/ Computer Vision for Music Identification]" algorithm.

=== Commercial===

* [http://www.acrcloud.com/ ACRCloud]. a cloud platform that helps companies and developers integrate ACR (Automatic Content Recognition) techniques to recognize audios and videos, monitor radio streams, detect live TV contents and so on. It's high-quality and cost efficient. [https://github.com/acrcloud-demo/audio-recognition-quick-trial/ Quick Free Trial] | [http://docs.acrcloud.com/ Docs]
* [http://audiblemagic.com/ Audible Magic]. [http://www.econtentmag.com/Articles/ArticleReader.aspx?ArticleID=4473&IssueID=128 This article] compares business models for Philips & Audible Magic
* [http://business.mufin.com/solutions/audioid AudioID]. Developed at the [http://www.idmt.fraunhofer.de/eng/research_topics/audioid.htm Fraunhofer Institute for Integrated Circuits IIS and the Fraunhofer Institute for Digital Media Technology IDMT], now provided by [http://www.mufin.com/ mufin GmbH].
* [http://www.shazam.com/ Shazam]. Proprietary music fingerprinting system, currently offered directly to users over the telephone.
* [http://www.rovicorp.com/products/developers/innovate/rovi_cloud_services.htm Rovi Media Recognition Service] (formelly [[wikipedia:AMG_LASSO|AMG LASSO]]).
* [http://www.gracenote.com/products/musicid/ Gracenote MusicID].
* Philips. One of the first scalable audio fingerprint algorithms. Described [http://ismir2002.ismir.net/proceedings/02-FP04-2.pdf here] and [http://ict.ewi.tudelft.nl/pub/doets/paper027.pdf here].
* [http://spiedl.aip.org/getabs/servlet/GetabsServlet?prog=normal&id=PSISDG005242000001000055000001&idtype=cvips&gifs=yes USC]
* And possibly most ridiculously of all: [http://www.songtapper.com/s/tappingmain.bin The Song Tapper]


==References==
==References==
<references />
<references />

[[Category:Identifier]]

Revision as of 10:21, 25 August 2020

MusicBrainz has used several audio fingerprinting systems over its lifetime.

All of them (so far) work in essentially the same way based on ACR technology. It is generally a two-step process of submission and lookup. First, the raw audio is used to create a fingerprint, which is then submitted to a third-party server. This server analyzes the fingerprint, compares it to other fingerprints, and decides whether it is sufficiently different from known fingerprints as to issue a new ID.

Once this step is done, a fingerprint can be calculated for any file and this can be used to look up the corresponding ID.

This ID is associated with a given track (pre-NGS) or recording (post-NGS), and metadata can be gathered from there.

AcoustID

AcoustID is MusicBrainz’ third and most recent audio fingerprinting system. This was created by Lukáš Lalinský, and made public around January 2011.[1]

AcoustID support was implemented in Picard 0.16 released in October 2011.[2]

It has several immediate advantages:

  • It is open source.
  • It is actively developed, along with supporting software.
  • It gives the ability to visually compare music
  • AcoustID fingerprints have their duration recorded, making it easy to discard certain incorrect links between recordings and acoustIDs.

Older fingerprinting systems

TRM

TRM (TRM Recognizes Music) IDs were MusicBrainz’ first audio fingerprinting system. This system was created by Relatable, and added to MusicBrainz in 2000.[3]

This system was used in the original MusicBrainz tagger application. The client-side audio fingerprinting library was open source and integrated into libmusicbrainz.

This system worked reasonably well for finding duplicate music files on a local system, but had problems with collisions (different-sounding audio which got the same ID) and duplicates (same-sounding audio which has different IDs). The server was also incapable of handling the number of TRMs needed for MusicBrainz, and Relatable didn’t seem to be interested in supporting it further.[4]

TRM support was removed in November 2008[5].

PUID

PUIDs were MusicBrainz’ second audio fingerprinting system. This was initially operated by MusicIP (later AmpliFIND Music Services), and bought by Gracenote in June 2011. Gracenote is expected to discontinue the public service soon, and it already appears to be largely non-functional.

MusicBrainz started to use these IDs in March 2006[6] and they were implemented in Picard 0.7.0 released in July 2006.[7]

This system was better than TRM, but still had several major problems:

  • The fingerprint submission system is not open source, and as such could not be included in Picard. Only looking up existing fingerprints was possible in Picard, new fingerprints had to be created using either MusicIP Mixer or the command line genpuid (no longer available, the latest version is mirrored on our ftp server).
  • The fingerprinting process is slow, both on the client side and the server side
  • Over time, the operators have become less and less interested in running the server, to the point where today it is barely working, if it works at all.

PUIDs are also quite opaque, being nothing more than a unique number referencing a database outside of MusicBrainz’ control. If/when that database goes away, they become useless. See the patent application for details on the technology. The client-side audio fingerprinting library, libofa, is open-source.

PUID support was removed from the MusicBrainz server in October 2013.[8][9]

In June 2020 [10] all PUID edits were removed from the database.

Other audio fingerprinting systems

Open Source

  • The fingerprint in Kurt Rosenfeld's FDMF.
  • MusicURI, part of the Mpeg-7 Audio DB project.
  • AudioScout. Based on the pHash audio fingerprinting library, developed by the same authors. Uses the "Philips Robust Hashing" algorithm.
  • OpenFP.
  • Echoprint. Audio fingerprinting solution developed by the Echo Nest.
  • [defunct] libFooID. An audio fingerprinting library used by, and developed for foosic.
  • [defunct] Freetantrum. It seems to be a dead project (its home page was replaced with an advert for unrelated things in 2001), but it may be worth investigating and resurrecting the code they produced.
  • [defunct] jHears is an acoustic fingerprinting framework based on FutureProofFingerPrint design by Geoff Schmidt (formerly of Tuneprint). jHears is developed by Juha Heljoranta.

Partially Open-Source

Commercial

References