Q&A with Ty Roberts of Gracenote

Ty Roberts, Gracenote

Ty Roberts is the Co-founder, Senior Vice President and Chief Technology Officer of Gracenote, keepers of the world's music and video metadata. You could say that Gracenote is the master of metadata (I know I have) and I thought it could be interesting to learn a bit more about what Gracenote does and how they do it.

This Q&A came about in an interesting way—do you recall our post about Neil Young's missing metadata from his new album Americana? Well Stephen Dupont, Gracenote's Director of Sales, Americas and AudioStream reader saw our post, passed it along to Ty who passed it along to 'Neil Young's people at Warner' who are working on getting that missing metadata put back in place. Cool, no? I'm keeping my fingers crossed for word on a HD download version and my other fingers crossed for an interview with Neil Young about the future of HD downloads. Hey, I can dream.

I'd imagine most of our readers know about Gracenote from its metadata database, mainly through ripping CDs. The Gracenote database began life as the CDDB (Compact Disc Data Base) with user-generated content but I understand that's no longer the case. Can you give us an overview of how Gracenote acquires its metadata today and how you confirm the accuracy of this data?

Gracenote actually still does take user-submitted data, but we now combine that with data collected by our editorial experts around the world as well as from direct feeds from record companies and artists. We also employ machine listening technology to determine the tempo and mood of tracks, which allows us to have large scale track-level attributes in the database.

As people move away from physical media, metadata becomes essential. It is our link between the music we enjoy and the people responsible for making it as well as the artwork that was chosen to represent the music which includes cover art, band photos and more. There's also basic factual data that we took for granted with physical media including the release date, recording venue, and more. Is there a complete listing of the metadata fields that Gracenote captures for each recording? And is Gracenote involved in capturing any other visual aspects of metadata beyond album cover art?

"Depending on its popularity, tracks can have everything from basic information like song and album title, to machine listening data on mood and tempo, to editorial data on genre, origin, era and type of music."

Because the data we gather is integrated through various means, the Gracenote database has a huge number of fields, and the number for each track varies. Depending on its popularity, tracks can have everything from basic information like song and album title, to machine listening data on mood and tempo, to editorial data on genre, origin, era and type of music. For some tracks we drill down as deep as the ISDN number and other textual information. For record labels and artists, we have a tool that allows them to input any level of information, if they wish to use it.

One metadata issue I've run into has to do with the way iTunes represents the Year. Often times this Year data for a reissue shows the date of reissue as opposed to the original recording date which I view as a large problem. Keeping music in its original historic context is vital to our understanding of it. I know that sounds dramatic but its true - Skip James "Today!" from 1966 is a very different record from Skip James "Today!" c. 2006. Is this data, original recording date and reissue date, available in the Gracenote database?

Gracenote's editorial team always attempts to capture the original issue date, as well as the release date. However, sometimes the direct feeds we get don't have this information. We are working on improving that and we fix where we can.

Gracenote's MusicID app relies on audio fingerprinting technology. One common application for this technology is identifying an unknown piece of music. Two areas of concern for many AudioStream readers is the sound quality and provenance of a given recording. With the increasing popularity (relative to starting out at 0) of high definition downloads, I was wondering if the Gracenote audio finger printing technology could be used to help determine some physical characteristics of a given recording such as dynamic range and resolution (i.e. is this a 16/44.1 recording or higher resolution). And is it possible to 'match' a given reissue to an original recorded version through its audio finger print?

"Yes, it's technically possible – fingerprinting technology can determine the difference between a re-mastered recording and the original."

Yes, it's technically possible – fingerprinting technology can determine the difference between a re-mastered recording and the original. However other simpler means often provide the same information. Remastered versions often have that text in the title, or have a extra set of bonus tracks that differentiate it from the original release.

The Internet has given us unprecedented access to more music than ever before. That's the good news. The not so good news is this creates the problem of overabundance - how do we filter and find music we like? Gracenote leverages its data to provide the technology behind Playlisting and discovery/recommendation services which I view as one of the true benefits of Internet-based music services; helping people find those needles among all those haystacks. This technology is used by iTunes, Spotify, and many others and I'd imagine it works by essentially marrying people's music listening history (from iTunes, Spotify data) with your 130 million+ song database. Can you give us an overview of the technology behind these profiling and discovery services and provide some examples of how they are put into practice?

"Gracenote technology and our database is used by iTunes Genius service, which provides personalized playlists and recommendations to iTunes users."

There are two major things at work with our discovery and recommendation services. The first is how we categorize the music - by genre, origin, era and type - which allows us to filter tracks and put them in the same or similar categories as "seed" tracks. The second is the use of machine listening technology to determine the mood and tempo of tracks. We are uniquely able to use both of these elements for discovery, and together they allow us to make recommendations with a high degree of confidence and accuracy. For instance, Gracenote technology and our database is used by iTunes Genius service, which provides personalized playlists and recommendations to iTunes users.

Playlisting and discovery/recommendation services lead us into the social aspects of musical likes and dislikes. In other words, if someone(s) out there has nearly identical playlist habits as I do even after a few hundred or thousand songs (which is an unsettling thought in some ways), it follows that we may have other things in common. Since I'm married the match-making capabilities of such data isn't relevant but I'd imagine that musical matches can be powerful bonds. Are music-matching social services part of the Gracenote portfolio of products?

At the moment most music services aren't focusing on connecting people who have similar tastes in music, but the idea of having that capability for Facebook, Twitter and other social networks is exciting and likely to happen. But it's important to remember opposites attract as well!

I recently read about your "Mood Grid" technology for the automotive industry that allows a user to select music based on mood. This is an interesting concept and I can imagine all kinds of practical implementations such as having my music player recommend a cocktail based on the music I've been playing on the way home from work. On a serious note, this is certainly a feature-rich future-world type app. Can you tell us more about the "Mood" Grid" and some of its practical applications?

"Our mood grid concept is the interface by which people can interact with music based on how they feel."

Our mood grid concept is the interface by which people can interact with music based on how they feel. It provides the technology and information to visually describe the emotional attributes of music, from “calm” to “energetic” and “dark” to “positive", and these mood characteristics are then mapped to songs in your personal music collection and across Cloud-based music services. It is based on the understanding that the kind of music we want to listen to often depends on our mood. What we want to hear when going to work is often different than what you want when you are at the gym, or getting ready to go out on a Friday night. We launched it first for the automotive industry, allowing drivers to launch mood-based playlists with one-touch or simple voice commands while still safely focusing on the road ahead. But the concept could be used for mobile and Cloud services and across our customer base.

Can you give us a peek into Gracenote's (and our) musical future?

Currently there's a need for more graphically and socially rich information being associated with music. I want to know who inside my social world knows about a band I'm following, or what people on the internet know. In the future I see more photography and video coming together with the music I'm listening to, especially in the living room. Gracenote will hopefully be helping to allow this to happen.

COMMENTS
deckeda's picture

(Former news archivist here.) Good search with relevant results can trump specialized fields ... that were left blank or missing altogether. I suppose that's the Google model.

But I'd love to see iTunes' metadata window include a separate date field for reissue date if it exists. And lets have one for both original and reissue catalog numbers and so on.

Until our software gives us more flexibility and power here, setting up a discogs.com account is a ready-made solution to cataloging what you have.

I really wish the record labels didn't treat previously-released product as if it didn't exist, by only telling us about the last iteration of a given title.

Michael Lavorgna's picture

Consumers appear to be leading the market in terms of providing detailed and accurate metadata. Discogs being one good example.

From Ty's answers (my emphasis):

For record labels and artists, we have a tool that allows them to input any level of information, if they wish to use it.

If they wish to use it and apparently they often do not. This has become a recurring theme in each of these recent Q&As - the record labels are a large part of the problem when it comes to missing, incomplete or inaccurate metadata as well as a lack of provenance information.

Regor Ladan's picture

Itunes tagging is ragged at best. Don't know if this has anything to do with Gracenote.

Michael Lavorgna's picture

So if you use iTunes to rip your CDs, its getting that metadata from Gracenote.

http://www.gracenote.com/casestudies/itunes/

Regor Ladan's picture

I would still love to know, ten years on, how to convince iTunes that a compliation or an CD with various artists is an album.

Also, I get strange underlining between characaters on certain ablums, especially classical, which can be a big mess. Artwork is total pot luck too.

I think we should all expect better from all the tagging services, but maybe this is the best it can get.

deckeda's picture

You have the option of correcting the info within iTunes while your CD is still inserted and upload your "version" back up the chain.

Michael Lavorgna's picture

You too can correct incorrect metadata right from home and earn extra...metadata karma points.

;-)

X