Jump to content

How to translate an OData feed into an iCalendar feed

+ 1
  JonUdell's Photo
Posted Nov 12 2010 09:11 AM

Introduction

In this week's companion article on the Radar blog ("The iCalendar chicken-and-egg conundrum"), I bemoan the fact that content management systems typically produce events pages in HTML format without a corresponding iCalendar feed, I implore CMS vendors to make iCalendar feeds, and I point to a validator that can help make them right. Typically these content management systems store event data in a database, and flow the data through HTML templates to produce HTML views. In this installment I'll show one way that same data can be translated into an iCalendar feed.

The PDC 2010 schedule

The schedule for the 2010 Microsoft Professional Developers conference was, inadvertently, an example of a calendar made available as an HTML page but not also as a companion iCalendar feed. There was, however, an OData feed at http://odata.microso...ataSchedule.svc. When Jamie Thomson noticed that, he blogged:

Whoop-de-doo! Now we can, get this, view the PDC schedule as raw XML rather than on a web page or in Outlook or on our phone, how cool is THAT?

Seriously, I admire Microsoft's commitment to OData, both in their Creative Commons licensing of it and support of it in a myriad of products but advocating its use for things that it patently should not be used for is verging on irresponsible and using OData to publish schedule information is a classic example.

I both understood Jamie's frustration, and applauded the publication of a generic OData service that can be used in all sorts of ways. Here, for example, are some questions that you could ask and answer directly in your browser:


Q: How many speakers? (http://odata.microso...Speakers/$count)

A: 79


Q: How many Scott Hanselman sessions? (http://odata.microso...tt+Hanselman%27)

A: 1


Q: How many cloud services sessiosn? (http://odata.microso...oud+Services%27)

A: 31

General-purpose access to data is a wonderful thing. But Jamie was right, a special-purpose iCalendar feed ought to have been provided too. Why wasn't that done? It was partly just an oversight. But it's an all-too-common oversight because, although iCalendar is strongly analogous to RSS, that analogy isn't yet widely appreciated.

To satisfy Jamie's request, and to demonstrate one way to translate a general-purpose data feed into an iCalendar feed, I wrote a small program to do that translation. Let's explore how it works.

The PDC 2010 OData feed

If you hit the PDC's OData endpoint with your browser, you'll retrieve this Atom service document:


<service xml:base="http://odata.microsoftpdc.com/ODataSchedule.svc/" 

    xmlns:atom="http://www.w3.org/2005/Atom" 

    xmlns:app="http://www.w3.org/2007/app" 

    xmlns="http://www.w3.org/2007/app">

  <workspace>

    <atom:title>Default</atom:title>

    <collection href="ScheduleOfEvents">

      <atom:title>ScheduleOfEvents</atom:title>

    </collection>

    <collection href="Sessions">

      <atom:title>Sessions</atom:title>

    </collection>

    <collection href="Tracks">

      <atom:title>Tracks</atom:title>

    </collection>

    <collection href="TimeSlots">

      <atom:title>TimeSlots</atom:title>

    </collection>

    <collection href="Speakers">

      <atom:title>Speakers</atom:title>

    </collection>

    <collection href="Manifests">

      <atom:title>Manifests</atom:title>

    </collection>

    <collection href="Presenters">

      <atom:title>Presenters</atom:title>

    </collection>

    <collection href="Contents">

      <atom:title>Contents</atom:title>

    </collection>

    <collection href="RelatedSessions">

      <atom:title>RelatedSessions</atom:title>

    </collection>

  </workspace>

</service>

The service document tells you which collections exist, and how to form URLs to access them. If you form the URL http://odata.microso...le.svc/Sessions and hit it with your browser, you'll retrieve an Atom feed with entries like this:


    <content type="application/xml">

      <m:properties>

        <d:SessionState>VOD</d:SessionState>

        <d:Tags>Windows Azure Platform</d:Tags>

        <d:SessionId m:type="Edm.Guid">1b08b109-c959-4470-961b-ebe8840eeb84</d:SessionId>

        <d:TrackId>Cloud Services</d:TrackId>

        <d:TimeSlotId m:type="Edm.Guid">bd676f93-2294-4f76-bf7f-60e355d8577b</d:TimeSlotId>

        <d:Code>CS01</d:Code>

        <d:TwitterHashtag>#azure #platform #pdc2010 #cs01</d:TwitterHashtag>

        <d:ThumbnailUrl>http://az8714.vo.mse...ads/matthew.png</d:ThumbnailUrl>

        <d:ShortUrl>http://bit.ly/9n4t9S</d:ShortUrl>

        <d:Room>McKinley</d:Room>

        <d:StartTime m:type="Edm.Int32">0</d:StartTime>

        <d:ShortTitle>Building High Performance Web Apps with Azure</d:ShortTitle>

        <d:ShortDescription xml:space="preserve">

Windows Azure Platform enables developers to build dynamically

scalable web applications easily. Come and learn how forthcoming new

application services in conjunction with services like the Windows 

</d:ShortDescription>

        <d:FullTitle>Building High Performance Web Applications with the Windows Azure Platform</d:FullTitle>

      </m:properties>

    </content>

To represent this event in iCalendar doesn't require much information: a title, a description, a location, and the times. The mapping from the fields shown here and the properties in an iCalendar feed can go like this:

ODataiCalendar
ShortTitleSUMMARY
ShortDescriptionDESCRIPTION
RoomLOCATION
?DTSTART
?DTEND

We're off to a good start, but where will DTSTART and DTEND come from? Let's check out the TimeSlots collection. It's an Atom feed with entries like this:


  <entry>

    <content type="application/xml">

      <m:properties>

        <d:Duration>01:00:00</d:Duration>

        <d:Id m:type="Edm.Guid">bd676f93-2294-4f76-bf7f-60e355d8577b</d:Id>

        <d:Start m:type="Edm.DateTime">2010-10-29T15:15:00</d:Start>

        <d:End m:type="Edm.DateTime">2010-10-29T16:15:00</d:End>

      </m:properties>

    </content>

  </entry>

Nothing about OData requires the sessions to be modeled this way. The start and end times could have been included in the Sessions table. But since they weren't, we'll access them indirectly by matching the TrackId in the Sessions table to the Id in the TimeSlots table.

Reading the OData collections

OData collections are just Atom feeds, so you can read them using any XML parser. The elements in each entry are optionally typed, according to a convention defined by the OData specification. One way to represent the entries in a feed is as a list of dictionaries, where each dictionary has keys that are field names and values that are objects. Here's one way to convert a feed into that kind of list.


static List<Dictionary<stringobject>> GetODataDicts(byte[] bytes)
    {
      XNamespace ns_odata_metadata = "http://schemas.microsoft.com/ado/2007/08/dataservices/metadata";

      var dicts = new List<Dictionary<stringobject>>();

      var xdoc = XmlUtils.XdocFromXmlBytes(bytes);
  
      IEnumerable<XElement> propbags = from props in xdoc.Descendants(ns_odata_metadata + "properties"select props;

      dicts = UnpackPropBags(propbags);

      return dicts;
    }

// walk and unpack an enumeration of  elements
static List<Dictionary<stringobject>> UnpackPropBags(IEnumerable<XElement> propbags)
    {
      var dicts = new List<Dictionary<stringobject>>();

      foreach (XElement propbag in propbags)
      {
        var dict = new Dictionary<stringobject>();
        IEnumerable<XElement> xprops = from prop in propbag.Descendants() select prop;
        foreach (XElement xprop in xprops)
        {
          object value = xprop.Value;
          var attrs = xprop.Attributes().ToList();
          var type_attr = attrs.Find(a => a.Name.LocalName == "type");
          if ( type_attr != null)
          {
            switch ( type_attr.Value.ToString() )
            {
              case "Edm.DateTime":
                value = Convert.ToDateTime(xprop.Value);
                break;
              case "Edm.Int32":
                value = Convert.ToInt32(xprop.Value);
                break;
              case "Edm.Float":
                value = float.Parse(xprop.Value);
                break;
              case "Edm.Boolean":
                value = Convert.ToBoolean(xprop.Value);
                break;
            }
          }
          dict.Add(xprop.Name.LocalName, value);
        }
        dicts.Add(dict);
      }
      return dicts;
    }

In this C# example the XML parser is the one provided by System.Xml.Linq. The GetODataDicts method creates a System.Xml.Linq.XDocument in the variable xdoc, and then forms a LINQ expression to query the document for elements, which are in the namespace http://schemas.micro...vices/metadata. Then it hands the query to UnpackPropBags, which enumerates the elements and does the following for each:

- Creates an empty dictionary

- Enumerates the properties

- Saves each property's value in an object

- If the property is typed, converts the object to the indicated type

- Adds each property to the dictionary

- Adds the dictionary to the accumulating list

Writing the iCalendar feed

Now we can use GetODataDicts twice, once for sessions and again for timeslots. Then we can walk through the list of session dictionaries, translate each into a corresponding iCalendar object, and finally serialize the calendar as a text file.


      // load sessions
      var sessions_uri = new Uri("http://odata.microsoftpdc.com/ODataSchedule.svc/Sessions");
      var xml_bytes = new WebClient().DownloadData(sessions_uri);
      var session_dicts = GetODataDicts(xml_bytes);

      // load timeslots
      var timeslots_uri = new Uri("http://odata.microsoftpdc.com/ODataSchedule.svc/TimeSlots");
      xml_bytes = new WebClient().DownloadData(timeslots_uri);
      var timeslot_dicts = GetODataDicts(xml_bytes);

      // create calendar object
      var ical = new DDay.iCal.iCalendar();

      // add VTIMEZONE
      var tzid = "Pacific Standard Time";
      var tzinfo = System.TimeZoneInfo.FindSystemTimeZoneById(tzid);
      var timezone = DDay.iCal.iCalTimeZone.FromSystemTimeZone(tzinfo);
      ical.AddChild(timezone);

      foreach (var session_dict in session_dicts)
      {
        var url = "http://microsoftpdc.com";
        var summary = session_dict["ShortTitle"].ToString();
        var description = session_dict["ShortDescription"].ToString();
        var location = session_dict["Room"].ToString();
        var timeslot_id = session_dict["TimeSlotId"]; // find the timeslot
        var timeslot_dict = timeslot_dicts.Find(ts => (string) ts["Id"] == timeslot_id.ToString());
        if (timeslot_dict != null// because test record has id 00000000-0000-0000-0000-000000000000
        {
          var dtstart = (DateTime)timeslot_dict["Start"]; // local time
          var dtend = (DateTime)timeslot_dict["End"];     

          var evt = new DDay.iCal.Event();

          evt.DTStart = new iCalDateTime(dtstart);        // time object with zone
          evt.DTStart.TZID = tzid;                        // "Pacific Standard Time"
          evt.DTEnd = new iCalDateTime(dtend);
          evt.DTEnd.TZID = tzid;
          evt.Summary = summary;
          evt.Url = new Uri(url);
          if (location != null)
            evt.Location = location;
          if (description != null)
            evt.Description = description;
          ical.AddChild(evt);
        }
      }

      var serializer = new DDay.iCal.Serialization.iCalendar.iCalendarSerializer(ical);
      var ics_text = serializer.SerializeToString(ical);
      File.WriteAllText("pdc.ics", ics_text);

In 2010 the PDC was held in Redmond, WA, and the times expressed in the OData feed were local to the Pacific time zone. But the event was globally available and live everywhere. I was following it from my home in the Eastern time zone, for example, so I wanted PDC events starting at 9AM Pacific to show up at 12PM in my calendar. To accomplish that, the program that generates the iCalendar feed has to do two things:

1. Produce a VTIMEZONE component that defines the standard and daylight savings offsets from GMT, and when daylight savings starts and stops. On Windows, you acquire this ruleset by looking up a time zone id (e.g. "Pacific Standard Time") using System.TimeZoneInfo.FindSystemTimeZoneById. DDay.iCal can then apply the ruleset, using DDay.iCal.iCalTimeZone.FromSystemTimeZone, to produce the VTIMEZONE component shown below. On a non-Windows system, using some other iCalendar library, you'd do the same thing -- but in these cases, the ruleset is defined in the Olson (aka Zoneinfo, aka tz) database. If you've never had the pleasure, you should read through this remarkable and entertaining document sometime, it's a classic work of historical scholarship!

2. Relate the local times for events to the specified timezone. Using DDay.iCal, I do that by assigning the same time zone id used in the VTIMEZONE component (i.e., "Pacific Standard Time") to each event's DTStart.TZID and DTEnd.TZID. If you don't do that, the dates and times come out like this:

DTSTART:20101029T090000
DTEND:20101029T100000

If my calendar program is set to Eastern time then I'll see these events at 9AM my time, when I should see them at noon.
If you do assign the time zone id to DTSTART and DTEND, then the dates and times come out like this:

DTSTART;TZID=Pacific Standard Time:20101029T090000
DTEND;TZID=Pacific Standard Time:20101029T100000

Now an event at 9AM Pacific shows up at noon on my Eastern calendar. Here's a picture of my personal calendar and the PDC calendar during the PDC:

Examining the iCalendar feed

Here's a version of the output that I've stripped down to just one event.


BEGIN:VCALENDAR

VERSION:2.0

PRODID:-//ddaysoftware.com//NONSGML DDay.iCal 1.0//EN

BEGIN:VTIMEZONE

TZID:Pacific Standard Time

BEGIN:STANDARD

DTSTART;VALUE=DATE:20090101

RRULE:FREQ=YEARLY;BYDAY=1SU;BYHOUR=2;BYMINUTE=0;BYMONTH=11

TZNAME:Pacific Standard Time

TZOFFSETFROM:-0700

TZOFFSETTO:-0800

END:STANDARD

BEGIN:DAYLIGHT

DTSTART;VALUE=DATE:20090101

RRULE:FREQ=YEARLY;BYDAY=2SU;BYHOUR=2;BYMINUTE=0;BYMONTH=3

TZNAME:Pacific Daylight Time

TZOFFSETFROM:-0800

TZOFFSETTO:-0700

END:DAYLIGHT

END:VTIMEZONE

BEGIN:VEVENT

DESCRIPTION:The Windows Azure Platform is an open and interoperable platfor

 m which supports development using many programming languages and tools.  

 In this session\, you will see how to build large-scale applications in th

 e cloud using Java\, taking advantage of new Windows Azure Platform featur

 es.  You will learn how to build Windows Azure applications using Java wit

 h Eclipse\, Apache Tomcat\, and the Windows Azure SDK for Java.

DTEND;TZID=Pacific Standard Time:20101029T100000

DTSTAMP:20101110T163629

DTSTART;TZID=Pacific Standard Time:20101029T090000

LOCATION:Cascade

SEQUENCE:0

SUMMARY:Open in the Cloud: Windows Azure and Java

UID:f3fbb0bc-4415-4883-ab72-796df7487c35

URL:http://microsoftpdc.com

END:VEVENT

END:VCALENDAR

As I discuss in this week's companion article on the Radar blog, this isn't rocket science. Back in the day, many of us whipped up basic RSS 0.9 feeds "by hand" without using libraries or toolkits. It's feasible to do the same with iCalendar, especially now that there's a validator to help you check your work.



Tags:
0 Subscribe


0 Replies