REST in Practice: How to use Atom for Event-Driven Systems

Atom is an XML-based hypermedia format for representing timestamped lists of web content and metadata such as blog postings and news articles. The following excerpt from REST in Practice shows you how to use Atom to implement an event-driven system.

Atom can be used to implement a staple of enterprise computing: events. Normally with event-driven systems, events are propagated through listeners. Here, however, we plan to publish an ordered list of events that readers can poll to consume events.

Note: We believe Atom is an ideal format for highly scalable event-driven architectures. But as with any web-based system, Atom-based solutions trade scalability for latency, making Atom often inappropriate for very low-latency notifications. However, if we’re building solutions where seconds, or better still, minutes or hours, can pass between events being produced and consumed, publishing Atom feeds works very well.

The Problem

Restbucks’ headquarters chooses which coffees and snacks will be served in its stores. HQ is also responsible for organizing promotions across the regions. It maintains product and promotion information in a centralized product catalog, but a number of other business functions within Restbucks depend on this information, including distribution, local inventory management, point of sale, and order management.

This situation is typical of the integration challenges facing many organizations today: systems that support key business processes need access to data located elsewhere. Such shared data may be required to enable end-to-end processing, or it may be needed in order to provide the organization with a single, consistent view of a business resource.

The benefits of data integration include increased consistency and availability of core data. But to get to this state we often have to overcome the challenges of data redundancy, poor data quality, lack of consistency among multiple sources, and poor availability.

Reference Data

As it has grown, Restbucks has evolved its data and application integration strategy to mirror its business capabilities and processes. This strategy has led to independent services, each of which authoritatively manages the business processes and data belonging to a business unit.

Effectively, Restbucks has decomposed its information technology ecosystem into islands of expertise. The product catalog service, for example, acts as an authoritative source of data and behavior for Restbucks’ product management capabilities.

Data such as product and promotions data is often called reference data. Reference data is the kind of data other applications and services refer to in the course of completing their own tasks.

Sourcing and using reference data are two quite separate concerns. Typically, an application will source a piece of reference data at the point in time it needs to use it. To preserve service autonomy and maintain high availability in a distributed system, however, it is best to maintain separation of concerns by decoupling the activities that own and provide access to reference data from those that consume it. If order management has to query the product catalog for a price for every line item, we’d say the two services were tightly coupled in time. This coupling occurs because the availability of order management is dependent on the availability of the product catalog. By breaking this dependency—separating the sourcing of data from its use—we reduce coupling and increase the availability of the order management service.

Warning: Temporal coupling weakens a solution because it requires numerous independent systems to be running correctly at a specific instant in time. When multiple servers, networks, and software all need to be functioning to support a single business behavior, the chances for failure increase.

To reduce coupling between producers and consumers of reference data, we generally recommend that reference data owners publish copies of their data, which consumers can then cache. Consumers work with their local copy of reference data until it becomes stale. By distributing information this way, services can continue to function even if the network partitions or services become temporarily unavailable. This is exactly how the Web scales.

To solve the coupling problem between the product catalog and its several consumers, Restbucks replicates its product catalog data. Each consumer maintains a local cache of the reference data, which it then updates in response to notifications from the provider. Each consumer can continue to function, albeit with possibly stale data, even if the product catalog becomes unavailable.

To ensure that updates to the product catalog are propagated in a timely manner, Restbucks uses Atom feeds.

Event-Driven Updates

To communicate data changes from the product catalog service to the distribution, inventory, and order management systems, Restbucks has chosen to implement an event-driven architecture. Whenever a new product is introduced, an existing product is changed, or a promotion is created or canceled, the product catalog publishes an event. The systems responsible for distribution, inventory, and order management consume these events and apply the relevant changes to their reference data caches.

Figure 7-1 shows how Restbucks’ product catalog exposes an Atom feed of events. Stores poll this feed at regular intervals to receive updates. When processing a feed, a store first finds the last entry it successfully processed the last time it polled the feed, and then works forward from there.

Figure 7-1. Event-driven architecture using Atom feeds

Restbucks’ underlying business process in this instance isn’t latency-sensitive. Products and promotions don’t change very often, and when an event occurs, it’s OK for stores to find out several minutes later. But while low latency isn’t an issue, guaranteed delivery is: price optimization and campaign management depend on HQ’s product catalog changes definitely being propagated to stores. It’s important, therefore, that we can guarantee that changes reach the stores, and that they are applied by the start of the next business day.

Event-driven systems in general exhibit a high degree of loose coupling. Loose coupling provides failure isolation and allows services and consumers to evolve independently of each other. Restbucks uses polling and caching to loosely couple providers and consumers. This polling solution respects the specific technical and quality-of-service requirements belonging to the challenge at hand (many consumers, guaranteed delivery, but latency-tolerant).

Polling propagates product catalog events in a timely fashion, limited only by the speed with which a store can sustainably poll a service’s feeds. But polling can introduce its own challenges: as stores multiply and polling becomes more frequent, there’s a danger that the product catalog service becomes a bottleneck. To mitigate this, we can introduce caching. Local or intermediary caches help by reducing the workload on the server and masking intermittent failures.

The Anatomy of an Event

An event represents a significant change in the state of a resource at a particular point in time (in the case of the product catalog, the resource is a product or a promotion). An event carries important metadata, including the event type, the date and time it occurred, and the name of the person or system that triggered it. Many events also include a payload, which can contain a snapshot of the state of the associated resource at the time the event was generated, or simply a link to some state located elsewhere, thereby encouraging consumers to GET the latest representation of that resource.

Note: Interestingly, the polling approach inverts the roles and responsibilities normally associated with guaranteed message delivery in a distributed system. Instead of the service or middleware being responsible for guaranteeing delivery of messages, each consumer now becomes responsible for ensuring that it retrieves all relevant information. Since messages are collocated in time-ordered feeds, there’s no chance of a message arriving out of order.

Solution Overview

Restbucks’ product catalog feed is treated as a continuous logical feed. In practice, however, this logical feed consists of a number of physical feeds chained together, much like a linked list. The chain begins with a “working” feed, followed by a series of “archive” feeds. The working feed contains all the events that have occurred between the present moment and a cutoff point in the past. This historical cutoff point is determined by the notification source (the product catalog service in our case). The archives contain all the events that occurred before that cutoff point. The contents of the working feed continue to change until the feed is archived, whereupon the feed becomes immutable and is associated with a single permanent URI.

The product catalog service is responsible for creating this series of feed resources. The contents of each feed represent all the changes that occurred during a particular interval. At any given point in time, only one of these feeds is the working feed. The service creates an entry to represent each event and assigns it to the working feed. When the service determines that the working feed is “complete,” either because a certain period has elapsed since the feed was started or because the number of entries in the feed has reached a predetermined threshold, the service archives the working feed and begins another.

Each feed relates to a specific historical period. This includes the working feed, which always relates to a specific period. The only thing that differentiates the working feed from an archive feed is the open-ended (the “as yet” not determined) nature of the working feed’s period.

In addition to these historical feeds there is one more resource, which we call “the feed of recent events.” Unlike the other feeds, the feed of recent events is not a historical feed; it’s always current. At any given moment, the feed of recent events and the working feed contain the same information, but when the service archives the current working feed and starts a new one, the feed of recent events changes to contain the same data as the new working feed.

As we’ll see, consumers need never know about working or archive feeds. Working and archive feeds are implementation details; as far as consumers are concerned, the notifications feed is the feed of recent events (the current feed). As service designers, however, we’ve found it useful to distinguish between these three types of feeds because of some subtle differences among them, particularly in terms of caching and links.

Using a linked list of feeds, we can maintain a history of everything that has taken place in the product catalog. This allows Restbucks’ stores to navigate the entire history of changes to the catalog if they so wish. Figure 7-2 shows how all these feeds link together, including the feed of recent events, the working feed, and the archive feeds.

Figure 7-2. The notifications feed is a chain of connected feeds

At the Atom level, each feed and entry has its own unique atom:id. A feed’s atom:id remains the same for the lifetime of that feed, irrespective of whether it’s the working feed or it has become an archive feed. This guarantees that stores can identify feeds even if the addressing (URI) scheme radically changes, further ensuring that the solution is loosely coupled and capable of being evolved.

Note: The authors of the Atom format recognized that while feeds and entries require stable identities, the URIs through which they can be addressed often change (due to changes in website property ownership or infrastructure, for example). To put it bluntly, cool URIs never change; URIs change—get over it. Hence the division of responsibilities between atom:id, which provides identification (and in combination with atom:updated identifies the latest version of a document), and a feed or entry’s self link, which provides addressability. The lessons learned by the Atom authors apply more generally, meaning that you should consider providing identities for your domain resources as well as addressable URIs. In the Restbucks example, however, for the sake of brevity we continue to identify products and promotions using only their addressable URIs.

Link Relations

Because we’ve split the notifications feed into a series of linked feeds, we need to help stores navigate them. In other words, we need to both link feeds together and describe how they relate to each other. We do this using Atom’s <atom:link> element.

The <link> element is Atom’s primary hypermedia control. Connecting resources is an important part of building web-friendly systems. Hypermedia controls allow services to connect and clients to access and manipulate resources by sending and receiving resource representations using a uniform set of operations.

To use a hypermedia control successfully, a client must first understand the control’s semantic context. The client must then be able to identify and address the resource with which the control is associated. Finally, it must know which media type to send or what representation formats it can expect to receive when it makes use of the control. These requirements are satisfied by the <atom:link> element’s rel, href, and type attributes, respectively:

A rel attribute’s value describes the link relation or semantic context for the link.
The href attribute value is the URI of the linked resource.
The type attribute describes a linked resource’s likely media type.

Note: The <link> element captures most of the connectedness characteristics we want to include in our resource representations. This fact, coupled with the growing popularity of Atom as a syndication format and the corresponding rise in Atom clients, leads us to suggest making atom:link a common building block of web-friendly distributed systems.

We say that the type attribute value represents the likely media type because this value can always be overridden by the owner of the resource at the end of the link. The Content-Type header in a response is always authoritative, irrespective of any prior indication of the linked resource’s media type. The type attribute remains useful, however, insofar as it allows clients to distinguish between different media type representations of the same resource. Consider a situation in which an entry is linked to JSON and XML representations of a resource. A client interested in only JSON representation would look for links with a type attribute value of application/json.

The Atom specification describes five link relations. Using two of these core link relations, we can now describe the relationships between some of the product catalog feeds, as shown in Table 7-1.

Table 7-1. The core Atom link relations

Link relation	Meaning
`self`	Advertises a link whose `href` identifies a resource equivalent to the current feed or entry.
`via`	Identifies the source for the information in the current feed or entry. Restbucks uses this link to indicate the current source for the feed of recent events.

Table 7-2 describes the remaining three Atom link relations.

Table 7-2. The remaining core Atom link relations

[td]Indicates that the resource at the href is related to the current feed or entry in some way. Restbucks uses this link to correlate each entry (which represents an event) with the domain resource to which the event relates (the related link contains the URI of this domain resource).

Link relation	Meaning[/td]
`alternate`	Indicates that the link connects to an alternative representation of the current feed or entry.
`enclosure`	Indicates that the referenced resource is potentially large in size.
`related`

IANA’s Registry of Link Relations contains a larger list of recognized link relation values.^[75] This list has nearly quadrupled since 2008, and now includes values such as payment (for describing links to resources that accept payments), first, last, previous, next, previous-archive, and next-archive (for navigating paged and archived feeds). For the Restbucks product catalog service, we will use several of these values to help navigate between the feeds that compose the overall set of product notifications.

_{[75] http://www.iana.org/...-relations.html}

Note: Though the link relation values in the Registry are primarily for use in Atom documents, their semantics are in many cases more generally applicable to a wide variety of application protocols. The Registry is a good place to find commonly used shared semantics that we can reuse to build hypermedia consumers and providers.

Polling for Recent Events

Recall that the feed of recent events is the entry point for all consumers of the list of product notifications. The feed is located at the well-known URI http://restbucks.com...s/notifications. The entries in this feed represent the most “recent” events (relative to the point in time when the feed is accessed). But these entries also belong to a specific historical period, which began when the previous working feed was archived. The feed of recent events therefore includes a link (a via link) to the source of the entries for this specific historical period.

Example 7-2 shows a store polling the feed of recent events.

Example 7-2. A store polls the feed of recent events

Request:
GET /product-catalog/notifications HTTP/1.1
Host: restbucks.com

Response:
HTTP/1.1 200 OK
Date: ...
Cache-Control: max-age=3600
Content-Length: ...
Content-Type: application/atom+xml;charset="utf-8"
ETag: "6a0806ca"

<feed xmlns="http://www.w3.org/2005/Atom">

  <id>urn:uuid:be21b6b0-57b4-4029-ada4-09585ee74adc</id>
  <title type="text">Product Catalog Notifications</title>
  <updated>2009-07-05T10:25:00Z</updated>
  <author>
    <name>Product Catalog</name>
  </author>
  <generator uri="http://restbucks.com/products">Product Catalog</generator>
  <link rel="self" href="http://restbucks.com/products/notifications"/>
  <link rel="via" type="application/atom+xml"
    href="http://restbucks.com/products/notifications/2009/7/5"/>
  <link rel="prev-archive"
    href="http://restbucks.com/products/notifications/2009/7/4"/>
  <entry>
    <id>urn:uuid:95506d98-aae9-4d34-a8f4-1ff30bece80c</id>
    <title type="text">product created</title>
    <updated>2009-07-05T10:25:00Z</updated>
    <link rel="self"
      href="http://restbucks.com/products/notifications/95506d98-aae9-4d34-a8f4-
  1ff30bece80c"/>
    <link rel="related" href="http://restbucks.com/products/527"/>
    <category scheme="http://restbucks.com/products/categories/type"
      term="product"/>
    <category scheme="http://restbucks.com/products/categories/status"
      term="new"/>
    <content type="application/vnd.restbucks+xml">
      <product xmlns="http://schemas.restbucks.com/product"
        href="http://restbucks.com/products/527">
        <name>Fairtrade Roma Coffee Beans</name>
        <size>1kg</size>
        <price>10</price>
      </product>
    </content>
  </entry>

  <entry>
    <id>urn:uuid:4c6b6b57-81af-4501-8bbc-12fee2e3cd50</id>
    <title type="text">promotion cancelled</title>
    <updated>2009-07-05T10:15:00Z</updated>
    <link rel="self"
      href="http://restbucks.com/products/notifications/4c6b6b57-81af-4501-8bbc-
  12fee2e3cd50"/>
    <link rel="related" href="http://restbucks.com/promotions/391"/>
    <category scheme="http://restbucks.com/products/categories/type"
      term="promotion"/>
    <category scheme="http://restbucks.com/products/categories/status"
      term="deleted"/>
    <content type="application/vnd.restbucks+xml">
      <promotion xmlns="http://schemas.restbucks.com/promotion"
        href="http://restbucks.com/promotions/391">
        <effective>2009-08-01T00:00:00Z</effective>
        <product type="application/vnd.restbucks+xml"
          href="http://restbucks.com/products/156" />
        <region type="application/vnd.restbucks+xml"
          href="http://restbucks.com/regions/23" />
            </promotion>
    </content>
  </entry>
</feed>

The response here contains two useful HTTP headers: ETag and Cache-Control. The ETag header allows Restbucks’ stores to perform a conditional GET the next time they request the list of recent events from the product catalog service, thereby potentially conserving network bandwidth. The Cache-Control header declares that the response can be cached for up to 3,600 seconds, or one hour.

The decision as to whether to allow the feed of recent events to be cached depends on the behavior of the underlying business resources and the quality-of-service expectations of consumers. In this particular instance, two facts helped Restbucks determine an appropriate caching strategy: products and promotions change infrequently, and consumers can tolerate some delay in finding out about a change. Based on these factors, Restbucks decided the feed of recent events can be cached for at least an hour (and probably longer).

The feed itself contains three <atom:link> elements. The self link contains the URI of the feed requested by the store, which in this case is the feed of recent events. The via link points to the source of entries for the feed of recent events; that is, to the working feed. (Remember, the working feed is a feed associated with a particular historical period. It differs from an archive feed in that it is still changing, and is therefore cacheable for only a short period of time. It differs from the feed of recent events in that at some point it will no longer be current.) The last link, prev-archive, refers to the immediately preceding archive document.^[76] This archive document contains all the events that occurred in the period immediately prior to this one.

_{[76] The prev-archive link relation value is defined in the Feed Paging and Archiving specification: http://tools.ietf.org/html/rfc5005.}

Note: In our example, the Restbucks product catalog service ticks over every day, archiving the current working feed at midnight. Because we use “friendly” URIs for feed links, it looks as if stores can infer the address of an archive feed from the URI structure, but we must emphasize that’s not really the case. Stores should not infer resource semantics based on a URI’s structure. Instead, they should treat each URI as just another opaque address. Stores navigate the archive not by constructing URIs, but by following links based on rel attribute values.

Moving now to the content of the feed, we see that each entry has a self link, indicating that it’s an addressable resource in its own right. Besides being addressable, in our solution every entry is cacheable. This is not always the case with Atom entries, since many Atom feeds contain entries that change over time. But in this particular solution, each entry represents an event that occurs once and never changes. If an underlying product changes twice in quick succession, the product catalog service will create two separate events, which in turn will cause two separate entries to be published into the feed. The service never modifies an existing entry.

Note: The working feed is mutable, limiting its cache friendliness. Every entry, however, is immutable, and therefore cacheable from the moment it is created.

Below the self link are two <atom:category> elements. Atom categories provide a simple means of tagging feeds and entries. Consumers can easily search categorized feeds for entries matching one or more categories. (And by adding feed filters on the server side, we can produce category-specific feeds based on consumer-supplied filter criteria.)

An <atom:category> element must include a term attribute. The value of this term attribute represents the category. Categories can also include two optional attributes: label, which provides a human-readable representation of the category value, and scheme, which identifies the scheme to which a category belongs. Schemes group categories and disambiguate them, much as XML and package namespaces disambiguate elements and classes. This allows entries to be tagged with two categories that have the same terms, but belong to two different schemes.

In Restbucks, we use categories to identify the event type (product or promotion), and its status (new, updated, or cancelled). The last entry in the feed in Example 7-2, for example, indicates that the promotion for the product http://restbucks.com/products/156 has been canceled. Using these categories, Restbucks’ stores can filter specific kinds of events from a feed.

Navigating the Archive

Navigating an individual feed is straightforward. Feeds are ordered by each entry’s <atom:updated> timestamp element, with the most recent entry first.^[77] To process a feed, a Restbucks store steps through the entries looking for the combination of atom:id and atom:updated that belongs to the last entry it successfully processed. Once it has found that entry, it works forward through the feed, applying each entry’s payload to its own local copy of the product catalog data.

_{[77] The value of the feed’s <atom:updated> element matches that of the first entry.}

Atom doesn’t prescribe how a consumer should process the entries in a feed. In our example, Atom entries represent event metadata. This metadata provides a processing context for the event’s business payload. When an Atom processor encounters an <atom:content> element, it delegates control to a media type processor capable of handling the contained product or promotion representation. The client invokes the specialized handler for the content in the knowledge that it is dealing with a representation of state at a particular point in time. We call this ability to hand off from one media type processor to another media type composition.

If the consumer can’t find in the current feed the last entry it successfully processed, it navigates the prev-archive link and looks in the previous archive. It continues to trawl through the archives until either it finds the entry it’s looking for or comes to the end of the oldest archive feed (the oldest archive has no prev-archive link). Example 7-3 shows a consumer retrieving a previous archive.

Example 7-3. The consumer retrieves the previous archive

Request:
GET /product-catalog/notifications/2009/7/4 HTTP/1.1
Host: restbucks.com

Response:
HTTP/1.1 200 OK
Cache-Control: max-age=2592000
Date: ...
Content-Length: ...
Content-Type: application/atom+xml;charset="utf-8"
ETag: "a32d0b30"

<feed xmlns="http://www.w3.org/2005/Atom"
  xmlns:fh="http://purl.org/syndication/history/1.0">

  <id>urn:uuid:be21b6b0-57b4-4029-ada4-09585ee74adc</id>
  <title type="text">Product Catalog Notifications</title>
  <updated>2009-07-04T23:52:00Z</updated>
  <author>
    <name>Product Catalog</name>
  </author>
  <generator uri="http://restbucks.com/products">Product Catalog</generator>
  <fh:archive/>
  <link rel="self" href="http://restbucks.com/products/notifications/2009/7/4"/>
  <link rel="prev-archive"
    href="http://restbucks.com/products/notifications/2009/7/3"/>
  <link rel="next-archive"
    href="http://restbucks.com/products/notifications/2009/7/5"/>

  <!-- Entries omitted for brevity -->

</feed>

The first thing to note about this archive feed is that it contains an <fh:archive> element, which is a simple extension element defined in the Feed Paging and Archiving specification.^[78] The presence of <fh:archive> is a further indication that this archive feed will never change and is therefore safe to cache.

_{[78] http://tools.ietf.org/html/rfc5005}

Following the <fh:archive> element are three <atom:link> elements. As with the feed of recent events, this archive feed contains self and prev-archive links, but it also includes a next-archive link, which links to the feed of events that have occurred in the period immediately following this one—that is, to a feed of more recent events. A store can follow next-archive links all the way up to the current working feed.

Again, just like the feed of recent events, the response contains a Cache-Control header. Whereas the recent feed can only be cached for up to an hour, archive feeds are immutable, meaning they can be cached for up to 2,592,000 seconds, or 30 days.

Caching Feeds

Feeds can be cached locally by each store, as well as by gateway and reverse proxy servers along the response path. Processing an archive is a “there and back again” operation: a consumer follows prev-archive links until it finds an archive containing the last entry it successfully processed, and then works its way back to the head feed—this time following next-archive links. Whenever a consumer dereferences a prev-archive link, its local cache stores a copy of the response. When the consumer next accesses this same resource, most likely as a result of following a next-archive link on the return journey, the cache uses its stored response. Navigating the full extent of an archive is a potentially expensive operation from a network point of view: caching locally helps save valuable network resources when returning to the head feed.

Note: This ability to create or reconstruct a local copy of the product catalog based on the entire archive is a great pattern for bringing a new system online or for supporting crash recovery, and is one of the ways the Restbucks infrastructure scales to thousands or even millions of stores if necessary.

To explore the implications of this strategy in more detail, let’s assume a store goes offline for a period of time—perhaps because of a problem with its local infrastructure. When the store eventually comes back up, it begins to update its local copy of the product catalog by polling the feed of recent events, and then working its way through prev-archive links looking for an archive feed containing the last entry it processed. Figure 7-3 shows the store following prev-archive links and working its way back in time through the archives. At each step, it caches the response locally in accordance with the metadata it receives in the HTTP cache control headers.

Figure 7-3. The consumer works its way back through the archives

At some point, the store finds the last entry it successfully processed. The store can now start working forward in time, applying the contents of each feed entry to its local copy of the product catalog, and following next-archive links whenever it gets to the top of a feed. This forward traversal is shown in Figure 7-4.

Figure 7-4. Working forward through the archives

Every time the store traverses a next-archive link, the response is fetched from the local cache (or an intermediary cache somewhere on the Restbucks network). This happens with every next-archive link—except the last one, when the request once again goes across the network. This last network request happens because the head of the feed is cached (if it is cached at all) against the well-known entry point URI, http://restbucks.com...s/notifications, rather than the working feed URI, which is http://restbucks.com...ations/2009/7/5. Because the store hadn’t accessed the working feed while working back through the archives (it went from the feed of recent events to the first archive feed), it now has to fetch the working feed from the origin server.

This linking and caching strategy trades efficiency for generalization. Generalization comes from our being able to build hypermedia clients that can navigate feeds using standardized prev-archive and next-archive link relations. In other words, there’s no need to build any special logic to prevent unnecessary end-to-end requests: it’s the local caching infrastructure’s job to store and return archive feeds at appropriate points in the feed traversal process. To a consumer, a request that returns a cached response looks the same as a request that goes all the way to the product catalog service.

The overall efficiency of this solution, however, breaks down with the final GET, when the consumer has to make one last network request. Assuming we started with empty caches all the way along the response path, navigating forward and backward through N feeds will cause the product catalog service to handle N+1 requests.

In designing this solution, we’ve assumed Restbucks’ stores won’t navigate the archive quite as often as they access the head feed by itself. If in practice the stores navigated the archives almost every time they polled the feed, we’d change the solution so that every request for the feed of recent events is redirected immediately to the current working feed, as shown in Example 7-4. Such a change would ensure that the head feed is cached against the current working feed’s URI, rather than the entry point URI. An immediate redirect doesn’t cut down on the overall number of wire requests, since the redirect response itself counts as an over-the-wire interaction, but it does reduce the overall volume of data sent in comparison to the first solution.

Example 7-4. Redirecting requests for the feed of recent events to the current working feed

Request:
GET /product-catalog/notifications HTTP/1.1
Host: restbucks.com

Response:
HTTP/1.1 303 See Other
Location: http://restbucks.com/products/notifications/2009/7/5

Request:
GET /product-catalog/notifications/2009/7/5 HTTP/1.1
Host: restbucks.com

Response:
HTTP/1.1 200 OK
Date: ...
Cache-Control: max-age=3600
Content-Length: ...
Content-Type: application/atom+xml;charset="utf-8"
ETag: "6a0806ca"

<feed xmlns="http://www.w3.org/2005/Atom">
  ...
</feed>

Recall that at some point the current working feed will be archived. When this happens, an <fh:archive> element and an <atom:link> element with a rel attribute value of next-archive will be inserted into the feed. The link points to the new current working feed. Responses containing the newly archived feed will include a Cache-Control header whose value allows the now immutable feed to be cached for up to 30 days.

Implementation Considerations

What distinguishes the working feed from an archive feed? As developers, we must distinguish between these feeds because the combination of caching strategy and available links differs. The working feed is cacheable only for short periods of time, whereas archive feeds are cacheable for long periods of time.

Using prev-archive and next-archive links saves us from having to add to each store some specialized knowledge of the product catalog’s URI structure and the periodization rules used to generate archives. Because they depend instead on hypermedia, stores need never go off the rails; they just follow opaque links based on the semantics encoded in link relations. This allows the catalog to vary its URI structure without forcing changes in the consumers. More importantly, it allows the server to vary its feed creation rules based on the flow of events.

During particularly busy periods, for example, the product catalog service may want to archive feeds far more frequently than it does during quiet periods. Instead of archiving at predefined intervals, the service could archive after a certain number of events have occurred. This strategy allows the service to get feeds into a cacheable state very quickly. As far as Restbucks’ stores are concerned, however, nothing changes. Each store still accesses the system through the feed of most recent events, and navigates the archives using prev-archive and next-archive links.

[i]Note:[/i] In low-throughput situations, it’s often acceptable to generate feeds on demand, as and when consumers ask for them. But as usage grows, this approach puts increasing strain on both the application and data access layers of the feed service. In high-throughput scenarios, where lots of consumers make frequent requests for current and archive feeds, we might consider separating the production of feeds from their consumption by clients. Such an approach can use a background process to generate feeds and store them on the filesystem (or in memory), ready to be served rapidly with minimal compute cost when requested by a consumer.

At a feed level, links with link relation values of self, alternate, next-archive, and prev-archive encapsulate the product catalog service’s implementation details—in particular, the service’s archiving strategy and the location of its current and archive feeds. This interlinking helps us both size feeds and tune performance. It also establishes a protocol that allows consumers to navigate the set of feeds and consume the event data held within.

The Restbucks product catalog uses many other Atom elements besides links to create an entire processing context for a list of domain-specific representations. In other words, Restbucks uses Atom to implement hypermedia-driven event handlers. To build this processing context, stores use:

<atom:id> and <atom:updated> to identify the oldest entry requiring processing
Categories to further refine a list of entries to be processed
related links to correlate entries with domain-specific resources
An entry’s <atom:content> element’s type attribute value to determine the processing model to be applied to the enclosed domain-specific representation

Atom helps us separate protocol and processing context from business payload using media type composition. Because the processing context for an event is conveyed solely at the Atom document level, the event-handling protocol itself can be implemented by domain-agnostic client code—that is, by generic Atom clients. The split between event context and business resource state snapshot allows stores to use Atom processors to determine which events to process, and allows domain- or application-specific media type processors to act on an entry’s business payload.

Learn more about this topic from REST in Practice.

REST continues to gain momentum as the best method for building web services, leaving many web architects to consider whether and how to include this approach in their SOA and SOAP-dominated world. In this insightful book, three SOA experts provide a down-to-earth explanation of REST and demonstrate how you can develop simple and elegant distributed hypermedia systems by applying the Web's guiding principles to common enterprise computing problems. You'll learn techniques for implementing specific Web technologies and patterns to solve the needs of a typical company as it grows from modest beginnings to become a global enterprise.