ListenBrainz/MBIDMappingDocumentation: Difference between revisions

From MusicBrainz Wiki
Jump to navigationJump to search
No edit summary
Line 1: Line 1:
== How does this mapping work? ==
= ListenBrainz MBID Mapping documentation =


In ListenBrainz we have build the MBID Mapping which automatically attempts to map incoming listens to their right tracks in MusicBrainz. This mapper is a fairly complex system that takes data from MusicBrainz, processes into form that is usable by ListenBrainz and then ships it to the ListenBrainz server that run the MBID Mapping. When a listen from a user arrives, we will look up that listen in our database and if we have never matched that data before, we will look it up in the MBID Mapping. There are two common reasons why a listen may not be found in the mapping:
In ListenBrainz we have build the MBID Mapping which automatically attempts to map incoming listens to their right tracks in MusicBrainz. This mapper is a fairly complex system that takes data from MusicBrainz, processes into form that is usable by ListenBrainz and then ships it to the ListenBrainz server that run the MBID Mapping. When a listen from a user arrives, we will look up that listen in our database and if we have never matched that data before, we will look it up in the MBID Mapping. There are two common reasons why a listen may not be found in the mapping:
Line 5: Line 5:
# The data is missing from MusicBrainz. This case happens a lot on Fridays when new music is released -- a track might be served by a music service, but no one has entered this data in MusicBrainz yet, so the MBID Mapping has no chance of mapping the track.
# The data is missing from MusicBrainz. This case happens a lot on Fridays when new music is released -- a track might be served by a music service, but no one has entered this data in MusicBrainz yet, so the MBID Mapping has no chance of mapping the track.
# The data is in MusicBrainz, but the data in the Listen differs significantly from the data in MusicBrainz. For instance, if a music service provides us a recording name of "Running up that hill - 2008 remaster" we need to work out that the "- 2008 remaster" portion is noise and should be discarded before trying to match the listen. Music services carry out all sort of data atrocities to make their service more palatable by their users and our challenge is to undo these atrocities, so you can imagine that there are some pitfalls in this process.
# The data is in MusicBrainz, but the data in the Listen differs significantly from the data in MusicBrainz. For instance, if a music service provides us a recording name of "Running up that hill - 2008 remaster" we need to work out that the "- 2008 remaster" portion is noise and should be discarded before trying to match the listen. Music services carry out all sort of data atrocities to make their service more palatable by their users and our challenge is to undo these atrocities, so you can imagine that there are some pitfalls in this process.

== What to do if your listen wasn't mapped ==


Given all of this, when a listen of your does not get mapped and the link remains grey, what can you do?
Given all of this, when a listen of your does not get mapped and the link remains grey, what can you do?
Line 16: Line 18:


[[File:manual_mapping.png|500px|frameless]]
[[File:manual_mapping.png|500px|frameless]]

== More options for diving deeper ==


Also, for people who are inclined to delve even deeper into this, we have an [https://labs.api.listenbrainz.org/explain-mbid-mapping?artist_credit_name=conny+conrad&recording_name=live+your+life+today "MBID Mapping Explain" API endpoint] that gives is (somewhat cryptic) feedback from the matching process of matching a listen. If you enter the artist_name and recording_name that is part of your listen you can see what the mapper attempts to do trying to match your data.
Also, for people who are inclined to delve even deeper into this, we have an [https://labs.api.listenbrainz.org/explain-mbid-mapping?artist_credit_name=conny+conrad&recording_name=live+your+life+today "MBID Mapping Explain" API endpoint] that gives is (somewhat cryptic) feedback from the matching process of matching a listen. If you enter the artist_name and recording_name that is part of your listen you can see what the mapper attempts to do trying to match your data.

Revision as of 13:04, 15 February 2023

How does this mapping work?

In ListenBrainz we have build the MBID Mapping which automatically attempts to map incoming listens to their right tracks in MusicBrainz. This mapper is a fairly complex system that takes data from MusicBrainz, processes into form that is usable by ListenBrainz and then ships it to the ListenBrainz server that run the MBID Mapping. When a listen from a user arrives, we will look up that listen in our database and if we have never matched that data before, we will look it up in the MBID Mapping. There are two common reasons why a listen may not be found in the mapping:

  1. The data is missing from MusicBrainz. This case happens a lot on Fridays when new music is released -- a track might be served by a music service, but no one has entered this data in MusicBrainz yet, so the MBID Mapping has no chance of mapping the track.
  2. The data is in MusicBrainz, but the data in the Listen differs significantly from the data in MusicBrainz. For instance, if a music service provides us a recording name of "Running up that hill - 2008 remaster" we need to work out that the "- 2008 remaster" portion is noise and should be discarded before trying to match the listen. Music services carry out all sort of data atrocities to make their service more palatable by their users and our challenge is to undo these atrocities, so you can imagine that there are some pitfalls in this process.

What to do if your listen wasn't mapped

Given all of this, when a listen of your does not get mapped and the link remains grey, what can you do?

  1. Check to make sure that the data exists in MusicBrainz. One page that can be helpful is the Add Missing MusicBrainz data link from the menu on the LB site: https://listenbrainz.org/user/rob/missing-data (this is my page, yours will differ). If a track is shown here, we were unable to match the data to MusicBrainz. The data may already exist in MusicBrainz, but we were unable to match it.
  2. If the data does not exist in MusicBrainz, add it. It is best to add whole releases in MusicBrainz and here is a quick how-to guide on how to do this.
  3. Once the data has been added to MusicBrainz, you'll need to wait 4-5 hours before that data is available the ListenBrainz MBID mapping.
  4. After this time, the easiest way to get the listen matched is to listen to the unmatched tracks again. When we receive a listen that we've been unable to map in the past, we'll try to map it again.
  5. We also have a process that wakes up on a periodic basis that goes back over old listens that have not been matched and attempts to match them again. However, as of this writing, this process is broken -- we're working to resolve this issue now.
  6. If all else fails, you can manually map this track. From the track that was not matched, click the three stacked dots and select "Link with MusicBrainz". Then enter the MBID of the recording that this listen should be mapped to, and you're done! We realize that this is less than ideal, but its a fallback for when all else fails!

manual mapping.png

More options for diving deeper

Also, for people who are inclined to delve even deeper into this, we have an "MBID Mapping Explain" API endpoint that gives is (somewhat cryptic) feedback from the matching process of matching a listen. If you enter the artist_name and recording_name that is part of your listen you can see what the mapper attempts to do trying to match your data.

Finally, if all else fails and you suspect there is a bug or an unhandled case in the mapping (there are many!) then you can add your case to our MBID Mapping improvement rollup ticket where we collect all of these data cases that give us headaches.

We hope that this explains our process and helps you turn those pesky grey links blue!