The hidden pearls of Tracker (#2)

Megaphone

It is a very little advertized fact that Tracker can handle RSS feeds. Tracker handles RSS feeds through the mfo ontology, which defines the information structures to store feeds and all information associated to its syndicated data. This piece of the Tracker ontology was bred under the Maemo platform umbrella with the participation of Lanedo, and it was made so it is highly generic to streams of messages, and not just RSS.

If Tracker is compiled with libgrss support (although unfortunately not all distros do so), a standalone daemon called tracker-miner-rss will be built too. This application will ensure that all mfo:FeedChannels are checked regularly, and will make all the information stored in the RSS file available through Tracker.

In order to add RSS feeds of your interest so they’re indexed by Tracker, the associated mfo:FeedChannels need to be created first, let’s for example add the Lanedo blog and Planet GNOME feeds:

INSERT { 
  <gnome> a mfo:FeedChannel; 
          a nie:DataObject; 
          nie:title 'Planet GNOME'; 
          nie:url 'https://planet.gnome.org/rss20.xml';  
          mfo:feedSettings [ a mfo:FeedSettings ; 
                               mfo:updateInterval 60 ] . 
}

INSERT {
  <lanedo> a mfo:FeedChannel;
           a nie:DataObject;
           nie:title 'Lanedo blog';
           nie:url 'http://www.lanedo.com/feed/';
           mfo:feedSettings [ a mfo:FeedSettings ; 
                                mfo:updateInterval 1440 ] . 
}

You can also use tracker-miner-rss to add the feeds for you without using the SPARQL above. For example:

$ tracker-miner-rss \
    --add-feed http://www.lanedo.com/feed/ \
    --title "Lanedo Blog"
Adding feed:
  title:'Lanedo Blog'
  url:'http://www.lanedo.com/feed/'
Done

Those feeds will be then picked up by tracker-miner-rss, extracting all relevant information out of these, all the messages written there will be available as mfo:FeedMessage elements, so you can for example see the volume of indexed articles per feed with:

SELECT nie:title(?feed) count(?message) 
WHERE { 
  ?message a mfo:FeedMessage; 
           nmo:communicationChannel ?feed
} GROUP BY ?feed

Or the 20 most recent unread articles that have been indexed:

SELECT nie:title(?message) nie:url(?message) 
WHERE { 
  ?message a mfo:FeedMessage;
           nmo:isRead false .
} 
ORDER BY DESC(nie:contentCreated(?message)) 
LIMIT 20

You can also use the tracker-search command line interface to see what feeds you have and information about each one, for example:

$ tracker-search --feeds --limit=5
Feeds:
  urn:uuid:7ce13082-f8e7-b738-ef22-977cb7237560 
    (A Quest for Speed in LibreOffice)
  urn:uuid:c28f1e22-2fed-0483-1a9e-2c568697a7f5
    (Embedding OS X UI elements in the GTK+ toolkit)
  urn:uuid:0ee43f47-1fea-b91a-a239-44dc8c97e4c6
    (How your company can become a LibreOffice contributor)
  urn:uuid:0a50f42e-bb37-6148-707d-183bcbd49e50
    (Improvements for 3GPP2 Gobi 2k/3k devices in ModemManager on GNU/Linux)
  urn:uuid:9a75f74d-d486-0c96-12ed-cf835827e880
    (Improving PDF Gradients Exported By LibreOffice)
  ...

$ tracker-info urn:uuid:7ce13082-f8e7-b738-ef22-977cb7237560
Querying information for entity:'urn:uuid:7ce13082-f8e7-b738-ef22-977cb7237560'
Results:
  ...
  'rdf:type' = 'http://www.tracker-project.org/temp/mfo#FeedElement'
  'rdf:type' = 'http://www.tracker-project.org/temp/mfo#FeedMessage'
  'nie:url' = 'http://www.lanedo.com/2013/quest-for-libreoffice-speed/'
  'nie:contentCreated' = '2013-07-30T15:47:21Z'
  'nie:informationElementDate' = '2013-07-30T15:47:21Z'
  'nie:title' = 'A Quest for Speed in LibreOffice'
  'nmo:isRead' = 'false'
  'nmo:receivedDate' = '2013-10-29T15:35:55Z'
  'mfo:downloadedTime' = '2013-10-29T15:35:55Z'

As the mfo:FeedMessage class inherits from nfo:TextDocument, it has an nfo:plainTextContent, which is indexed by FTS, so advanced text search is available throughout the indexed contents of feeds, for example searching for the word “Tracker” on all those messages would be done with:

SELECT nie:title(?feed) nie:title(?message) 
       nie:url(?message) fts:snippet(?message) 
WHERE { 
  ?message a mfo:FeedMessage ; 
           nmo:communicationChannel ?feed; 
           fts:match 'tracker' 
}

Again, you can use the tracker-search command to search using FTS, for example, the word “bug” which isn’t in any title but in the RSS feed content and the word “libreoffice” which appears in a lot of articles and their content:

$ tracker-search --feeds bug
Feeds:
  urn:uuid:7ce13082-f8e7-b738-ef22-977cb7237560
    (A Quest for Speed in LibreOffice)

$ tracker-search --feeds libreoffice
Feeds:
  urn:uuid:7ce13082-f8e7-b738-ef22-977cb7237560
    (A Quest for Speed in LibreOffice)
  urn:uuid:7bb653f6-586b-8984-fb1e-c62ca530684a
    (LibreOffice Development Howto)
  urn:uuid:9a75f74d-d486-0c96-12ed-cf835827e880
    (Improving PDF Gradients Exported By LibreOffice)
  urn:uuid:4dff2615-9fa6-4c07-2781-cace516264cb
    (LibreOffice Conference 2013 Milano Highlights)
  urn:uuid:0ee43f47-1fea-b91a-a239-44dc8c97e4c6
    (How your company can become a LibreOffice contributor)
  ...

Thanks to Tracker, RSS is not something you can just read, you could for example relate feeds and posts to contacts stored in tracker, enabling cross search for multiple communication channels. Or for most simple use-cases, a UI application could be easily built around the stored information.

If you’re looking for expertise around the Tracker project, do not hesitate to contact Lanedo! You can also read another hidden pearl of Tracker about Geolocalization.

Share on Google+Tweet about this on TwitterShare on LinkedInShare on FacebookFlattr the authorBuffer this pageShare on RedditDigg thisPin on PinterestShare on StumbleUponShare on TumblrEmail this to someone

Carlos Garnacho is born and raised in Madrid. He’s been involved in Free Software since late 2001, mainly in GNOME as a contributor to many modules throughout the stack and as a maintainer of gnome-system-tools maintainer and system-tools backend on freedesktop.org. More recently Carlos has been involved in LibreOffice, in the XInput 2.x and CSS themeing work for GTK+ as well as being a core maintainer of the Tracker project. He also likes playing the guitar. You can contact Carlos and his team for professional consulting on our contact page.

Posted in Blog, Gnome, Tracker Tagged with: , , ,
2 comments on “The hidden pearls of Tracker (#2)
  1. Mattias Bengtsson says:

    Cool, do you happen to know whether Fedora has this built in?

Leave a Reply

Your email address will not be published. Required fields are marked *

*