History:Mix O Matic

From MusicBrainz Wiki

MusicBrainz should carry a database of information about the content of the music, not just Artist, Track Number, etc.

Specificially, there should be tables of descriptive properties -- tempo, instrumentation, mood words.

If those fields existed the capability would be there to say:

"Make me a playlist of up-beat, slow Jazz from the 40s." or perhaps: "Play me all of Jurrasic 5's angry music."

The potential would be tremendous. This would be only scratching the surface. A player with this information integrated into it could not only create static playlists like I suggest above, but could work on a continually changing basis, picking one of the ten nearest rated songs available at that moment... You could begin at The Beatles and end up with Nine Inch Nails a few hours later and never have heard one song that didn't logically follow the next.

Another possibility: music recommendations: "I like this song by <some new band>. Show me another song similar to it."

-PetervanHardenberg



Bulletin

MixOMatic lives! (Kind of.)

I've spent the last few months in a crash-course tutorial on MIR with George Tzanetakis (who appears to be fairly well known in the MIR community.) The "flowing" playlist is now a reality, based on audio feature extraction. There is still much more work to do, specifically with incorporating cultural data and tag data -- the matching is ONLY on the basis of sound at the moment, but there is a working, useful interface which compiles into an end-user application that runs under Linux.

Please contact me at mixmaster at pvh dot ca if you are interested in trying it out. I'd really like to get the ball rolling more on this now that it is at a usable phase. I am particularly interested in contact from the following:

graph theory nerds (I need to find bounded-length (read: not minimal) paths through weighted graphs) audio statistics nerds (to improve the feature set and perhaps replace Marsyas, which I find hard to use) people with brilliant ideas (to improve the interface, the idea, the concept) users (just a few, who are running Linux, who don't mind recompiling source-code)

Watch this space, it's going to heat up!

-PetervanHardenberg



Discussion

MoodLogic has a fairly good implementation of this type of feature... Also, check out the rest of the bunch for similar artist queries. I think that facts like tempo and instrumentation are easily encorporated, but mood words gets hard. Music elicits different feelings in different individuals. You need to come up with a system to categorize the emotions, and then a system to moderate them ("No, you _couldn't_ have felt _happy_ when listening to _this_ song!" lol)

ScottMcClure

But you could keep everybody's (subjective) mood classifications separate and then create playlists from users whose classifications you trust.

Paul

Yeah. I agree that any implementation would be best off keeping anything that subjective personal to the user, and not trying to create some master aggregate value.

ScottMcClure

We're working on this with last.fm/Audioscrobbler, and all our data will be freely available and linked to MusicBrainz.

RussGarrett

MoodLogic's implementation of this feature is awful. 1) The database is closed. 2) The developers ignore the users. 3) The software interface is poorly implemented. 4) The software is getting more and more moneygrubbing every day. (Click here to buy now? What is this, RealPlayer?)

It is the fact that MoodLogic can't provide me with dynamic playlisting which makes me want to extend musicbrains in that direction. Perhaps there is room for a third project linked to Audioscrobbler and MusicBrainz. Third tier data. As far as subjective information goes, in my experience, the Populace is capable of identifying music reasonably well, particularly given some guidelines. Just as long as you allow the local user to override the central database's opinion, then all is well. As far as misclassification goes, I'm not too worried about it. Because it isn't factual information like the tagger, it doesn't have to be verified.

I'll keep you all posted.

PetervanHardenberg

Update: MoodLogic has patented their method of song matching. Another group have a working alternate implementation, but theirs is buggy and seems dormant. I think a hybrid approach is best.

Combine music information from musicbrainz, and perhaps audioscrobbler (though their server has not yet been up for me to see at the time of this typing), add information gathered from listening patterns, and allow the user to manipulate the specifics by associating certain genres, songs or bands.

The idea is to take all the information you can possibly gather, and then try to make intelligent predictions based on it, and learn from your mistakes.

PetervanHardenberg



Some disconnected thoughts and ideas: I'll add/modify this later but I want to start gathering my thoughts somewhereand let people start to respond to them.

There are several different models that could be used for a playlist generator.

  • Gather user information about the subjective aspects of the music (mood, energy levels etc.) and correlate those, like MoodLogic.
  • Look at an individuals listening habits and perform analysis on that to work out which songs go together, like ["SynapseAI"].
  • Gather more general listening habits, and make music reccomendations based on that, like AudioScrobbler.
  • Rate songs, artists and albums and use a random weighted playlist, based on your personal ratings and community ratings, like LaunchCast Internet Radio.

All of these have various pros and cons, the biggest con being that if you add new music to your system that the playlist generator does not know about, they will not get played. It is always down to a user somewhere to tell the system about the music.

So why not get the computer to listen to the music for you? Using feature recognition techniques and AI techniques (see http://www.aigeek.com/geek/music/ here for some good links), it should be possible to get the computer to perform sufficient analysis to categorise the music. A combination of AI techniques, user gathered data and machine gathered data should be sufficient to give the computer a good enough chance at guessing the mood/genre/whatever of new unplayed music. This data could then be improved by user profiling.



For a playlist generator to be succesful it needs to grow and evolve with the users musical tastes, it also needs to be instantly useable. A central data model is essential for the "instantly useable" part of this, and a listening habits tracker is essential for the "evolve and grow" part.



It should be possible to present a playlist as a path through an n-dimensional space where the dimensions maybe things like mood or bpm or genre, or they could be abstract audio features with no sensible semantic description. This way a user could set up waypoints along the way so a playlists mood could change over time (imagine setting up a 3 or 4 hour playlist which moves from rock/guitar bands, through dance music, and ends up at chillout by just picking out a few songs and a time that you want to reach each song by).

GavinClarkeUK



The problem with having the computer listen for you is that it doesn't know how. Musical tastes are an extremely subjective field, and the current state of the art has a difficult time telling a violin from a clarinet, let alone Paul McCartney from John Lennon with a band in the background.

While it seems entirely possible that the technology will eventually be available to do this kind of analysis, I was hoping to focus on a software package that could be written today.

You have hit the nail on the head regarding the importance of being both 'instantly usable' and 'evolve and grow'.

I had thought to overcome that by studying listening habits of users (users like me, namely me, being my target market) in a general sense.

If we can use Audioscrobbler to get associative information on a grand scale, that information could be applied on a band-by-band basis.

For example, if you like Audioslave and download a new Audioslave tune, it should be loaded into the database with much the same profile as the rest of your Audioslave songs. If the band isn't recognized, see if Audioscrobbler can help, and if Audioscrobbler can't help, mix it in with a high frequency (new music, gonna wanna hear it) until such time as a better profile can be developed.

I have recently despaired of ever being able to navigate my music as an n-dimensional space -- the cataloguing challenges are immense. Instead of trying to collect lots of data to make smart playlists, I've been focusing more on creating smart playlists from available data.

I'm still very much working on this one.

PetervanHardenberg



The n-dimensional space solution is along the lines of how the MusicMagic Mixer is implemented. There are limitations to this, certainly, which could be used in combination with more subjective data, but it's fascinating stuff, to say the least. I've actually created lists along the lines of those suggested by Gavin Clark above, but the tricky part is that the space is often sparsely populated along certain paths (depending on what music is in the collection) so it may be tricky to transition between clusters.

One amusing feature I have internally is the "Top 40 Time Machine", which lets you choose a target song and beginning/ending year, and finds the closest match from top40 songs for each year to create a playlist. But it's hard to say that the results are anything a "normal" person would expect :^)

Wendell Hicken



This is all very interesting stuff. It seems like it may be kind of obvious, and maybe this has been answered before but a question arises in my head.

What if the user's listening habits end up being determined entirely automatically? Say for instance if a user chooses a random shuffle option, or never argues with whatever playlist audioscrobbler generates? Do you end up with some gradual data degeneration where playlists are determined by feedback loops? I'm sort of interested in how one system claims to determine its ratings. It's called IMMS and it's fairly simple. If a user listens all the way through a song, the rating is raised. If they skip, the rating is lowered. I wonder if this is really enough though?

Another situation is multiple users of the same system. I wouldn't neccesarily be interested in any of the music my sister likes, and vice versa. But if she's using the same system, I may not have a choice! It's easy to get lost in daydreaming and fantasies about the perfect playlist generator, but I think it's important to look at how people actually use their music collections.

-Breton Slivka



Another project worth checking out which is related to this is MusicMobs. They also have an open source tool which connects to a basic API for querying artists and tracks. One of the advantages over something like Audioscrobbler is that musicmobs takes into account your whole music collection and database, as well as any ratings you have on songs. This means you can update your information without having to be constantly updating it in real time, which is quite redundant. It certainly seems to be more about finding relationships between songs and artists than just collecting your 'top played artists' as Audioscrobbler does. Its also worht pointing out that because Musicmobs has your whole collection on file, it is able to make recommendations about songs that you have in your library, dont have in your library, have played or not played, that other people have in their library even if they havent played it etc, etc.