Standardizing Media Art Thumbnailing & Caching

Vynil covers
Since before 2009, Lanedo has been coordinating contributions on the Tracker project (a search engine and metadata storage for user’s data). After recent community debates, I created libmediaart based on the sources that existed in Tracker. Thumbnails, media art and caching thereof is something that has always been done by each desktop project independently over the years. Even now with the development in GNOME of the GNOME Music application, the maintainers need to consider how to handle media art and caching just like we do in the Tracker project.

What’s the problem?

On the desktop, you usually want to illustrate local media with thumbnails or previews. Caching those thumbnails is much faster than extracting it each time it is used. Here is an example. Say you download an MP3 file and you want to play that on your desktop. The chances are, the MP3 will have album art encoded into the MP3 (this usually depends on where you get it from, most vendors like iTunes embed the data in the file). Extracting that information when listing a lot of media is expensive and not to mention actually wasteful on system resources to do it repeatedly. How can we display media thumbnails and previews efficiently while sharing as much code as possible?

A standardized solution

Some years ago, members of the Tracker team realised that many projects were doing this process and decided to standardize it so all applications knew how the caching worked and could make use of the extracted media art. The specification currently resides on GNOME’s wiki an the MediaArtStorageSpec. Caching media art quickly becomes complicated. The common use case is with music or movies. In either case, you likely have the same media art for more than one song or video. Consider that you have the same album art for perhaps 10 song belonging to an album or even a TV series which has several episodes (perhaps one per file). You clearly don’t want to have one media art for each song or episode, instead you want to use the same media art. The naming convention generally works by creating an Md5sum (a unique identifier) for a combination of strings that identify the media. By standardizing on this, it means that other applications can check a known location for a cache of media art for their song or episode and use it if it exists. If it doesn’t then they can create the cache themselves and know how to name the cache so others can make use of it.

Isn’t there a library somewhere that can do this for me?

Until recently, most applications just implemented the specification, but after some chat amongst the community in the last few weeks, we decided that it might be a good idea to just export all the code we use in Tracker to its own library for other projects to make use of, like GNOME Music. For now it’s living in github and called libmediaart. We’re in the process of cleaning up the API (which had some Tracker dependencies) but are aiming to put out a library that is useful for a number of cases. In particular, looking up the path to a cache for your media, queueing the extraction and managing the cache itself. Here’s a quick look at the initial API we’ve taken over:

typedef enum {
  MEDIA_ART_NONE,
  MEDIA_ART_ALBUM,
  MEDIA_ART_VIDEO,
  MEDIA_ART_TYPE_COUNT
} MediaArtType;

/* Back end initialisation for GdkPixbuf, Qt, and others... */
void     media_art_plugin_init     (gint max_width);
void     media_art_plugin_shutdown (void);
gboolean media_art_process         (const unsigned char *buffer,
                                    size_t               len,
                                    const gchar         *mime,
                                    MediaArtType         type,
                                    const gchar         *artist,
                                    const gchar         *title,
                                    const gchar         *uri);

/* Buffer/file conversions */
gboolean media_art_file_to_jpeg    (const gchar         *filename,
                                    const gchar         *target);
gboolean media_art_buffer_to_jpeg  (const unsigned char *buffer,
                                    size_t               len, 
                                    const gchar         *buffer_mime,
                                    const gchar         *target);

/* Utilities to look up paths and strip useless crap */
gchar*  media_art_strip_invalid_entities (const gchar   *original);
void    media_art_get_path         (const gchar         *artist,
                                    const gchar         *album,
                                    const gchar         *prefix,
                                    const gchar         *uri,
                                    gchar              **path,
                                    gchar              **local_uri);

/* Queue and caching handling of media art */
void    media_art_remove           (void);

What next?

After speaking with the developers from GNOME Music, they’re eager to start using a shared API that everyone can contribute to and improve, not just steal code from the Tracker project. This all started actually because a bug was noticed in Tracker not following the specification exactly to the letter 🙂

Further changes to the API are needed and some other small project things, like a pkgconfig addition. We would love to hear what you have to say about the API and what’s missing for you! Feel free to email us on the Tracker mailing list.

Soon after improving the API, we hope to have an initial release too!

If you’re looking for a solution like this or like Tracker for your business, get in touch with us at Lanedo to see how we can help!

Share on Google+Tweet about this on TwitterShare on LinkedInShare on FacebookFlattr the authorBuffer this pageShare on RedditDigg thisPin on PinterestShare on StumbleUponShare on TumblrEmail this to someone

Martyn Russell has a background in the Telecommunications industry where he worked for seven years prior to becoming a software developer for Imendio. Martyn has authored open source projects such as Gossip & Tracker and been involved in other areas of GNOME since 2005. You can contact Martyn and his team for professional consulting on our contact page.

Posted in Blog, Development, Gnome Tagged with: , , , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *

*