Jump to content

How to explore and automate Twitter's OAuth implementation

+ 1
  JonUdell's Photo
Posted Sep 10 2010 10:02 AM


When Twitter shut down HTTP basic authentication as a means of access to its API, the elmcity service -- which uses that API as described in The power of informal contracts -- had to convert to OAuth. In the current installment of my series on Radar I discuss the trade-offs involved. It's great that Twitter is doing all it can to discourage the password anti-pattern. But OAuth is complicated. That's necessary to support the three-party scenarios for which OAuth was invented. For simpler two-party scenarios like the elmcity/Twitter arrangement, though, it's a challenge to recreate with OAuth what was trivial to do with basic authentication. Now that I've worked through the exercise, I'll share some of the tools and techniques that can help you deal with OAuth and OAuth-like protocols.

Setting up the microscopes

The OAuth protocol is a choreographed dance that involves cryptographic signing, redirection, and callbacks. I guess some people can visualize all that by reading specs and source code, but I'm not one of them. I need to put a working implementation under the microscope and watch it in action.

Actually there are microscopes. One is the Visual Studio debugger, which animates the code that implements the protocol. The other is Fiddler, Eric Lawrence's wildly capable HTTP analyzer, which animates the messages exchanged among the parties. These tools are both commonly called debuggers, but I think of them as visualizers that together enable me to wrap my head around tricky web protocols like OAuth.

Twitter offers two OAuth modes: browser and desktop. In browser mode, after starting the protocol with a request for a temporary token, you specify an endpoint in the cloud where you can receive the HTTP callback that enables your app to request and receive the permanent API access token. Desktop mode works differently. You request a temporary token, call Twitter with that token, receive another token (a PIN), and call Twitter again for the access token. Either mode would have worked for me; I opted for the browser mode.

In order to debug the whole dance, which involved my browser, Twitter, and a local instance of the elmcity service running on my machine in the Azure development fabric, I needed to be able to receive callbacks from Twitter on my local machine. The first order of business, then, was to open up a port on my router and route traffic from my external interface to that machine.

That's a routine thing, so I was surprised when I couldn't make any inbound connections. Then I remembered: This was the first time I'd tried that since switching my Internet service from DSL to Time-Warner's cable system. Was Time-Warner blocking the port I was trying to forward? Yes. Grumbling, I verified that port 8080 was open and forwarded that instead of port 80.

When that still didn't help I had to dig deeper. It turns out that the Azure development fabric, which uses Visual Studio's built-in web server, only listens on the loopback adapter:, aka localhost. So I also had to forward from my private interface to localhost, like so:

ssh -L localhost

Now I could tell Twitter to call me back at port 8080 on my external interface. My router forwarded to port 8080 on my internal interface. And ssh in turn forwarded to the loopback adapter on which the Azure development fabric was listening.

Along the way I learned something else that's worth passing along. Unlike the Azure fabric, the IIS web server can listen on any interface. So I only needed to use port forwarding on the router to be able to receive inbound requests using IIS. But when I used IIS as Visual Studio's web server, instead of the built-in development server, a different problem arose. I could trace inbound requests in the debugger, but I couldn't see the HTTP traffic in Fiddler.

As is so often true, this was a permissions problem. I found the solution in this entry from Mark Needham. In my case, I was running Visual Studio under my own user account. So I went to the IIS manager and created an application pool bound to that identity.

Then I put the test application into that pool:

Now when running the OAuth test app under IIS, both microscopes were active. Visual Studio animated the code, and Fiddler watched the HTTP traffic.

Observing the protocol in action

If you're using C# and need library support for open authentication, Andrew Arnott's DotNetOpenAuth is undoubtedly the gold standard. It's a robust and comprehensive package that knows all about OAuth, OpenID, and Information Cards.

In this case, though, I wanted to start with Hello World. Shannon Whitley's example, based on the foundation kit at oauth.google.com, was a good place to start.

Let's put Shannon's example under the microscopes. I registered it at twitter.com/auth with this callback URL: MY_EXTERNAL_IP_ADDRESS:8080/oAuthSample. With the sample app running and a breakpoint set on the Page_Load method, I entered that address into the browser. Here's the view through both scopes.


Visual Studio

Since there's no token in the request, the app has to initiate the protocol by asking for temporary credentials. Here it's poised to do that:

Here it's forming the request, which has to include a signature based on a key and secret provided by Twitter when the app was registered, a timestamp, a nonce, and the request URL.

Here's where it sends the signed request to Twitter.

In Fiddler we can watch the request go out and the response come back. The response body contains the temporary credentials in this format:


The application needs the oauth_token, which it uses to form the URL that redirects the user back to Twitter in order to deny or allow the application's request to act on behalf of the user.

Here's where the app does the redirect.

In Fiddler we can switch to web view to observe the familiar response.

Over in the browser, I'll click Allow to approve the request. Now we're off to the races. Twitter calls back to the app, the app requests and receives the permanent access token from Twitter, and then, finally, it can make calls to the Twitter API. It would be tedious to walk through the whole sequence at this level of detail. But the point is that when you need to, you can.

Automating the user interaction

As I discuss over on the Radar blog, the elmcity service uses Twitter in an unusual way. OAuth is a three-party protocol. It typically involves a user, an application or service that the user authenticates to directly, and a third-party application or service to which the user delegates agency. The elmcity/Twitter scenario, though, is really just a two-party game. If you're the curator of an elmcity hub, you can associate your hub with a Twitter account. The elmcity service, which exists in Twitterspace as @elmcity_azure, will follow your Twitter account. That enables you to send direct messages to @elmcity_azure. In this scenario there is no third party. The only person who needs to authorize the elmcity Twitter application is me. I'm both the operator of the elmcity service and the person behind the @elmcity_azure Twitter account.

Given that situation, I could have written a standalone app that would only need to be used once to authorize the elmcity service to act on behalf of @elmcity_azure. But the elmcity service is a service, and I like to automate its administration as much as possible. And while in theory the Twitter authorization only needs to happen once, I know that in practice things change and need to be redone. So I decided to empower the elmcity service to authorize itself to Twitter, without requiring me to sit in front of a browser and click an Allow button.

That's doable with a bit of screenscraping. My favorite scraper, Beautiful Soup, is written in Python. The elmcity service includes an instance of IronPython, so I could have used that to run Beautiful Soup, but instead I decided to find out what's available for C#. I found the Html Agility Pack which did the job nicely.

Here's the only code that I wound up adding to Shannon Whitley's example:

public string AutoOAuthLinkGet()
// request temp credentials

var link = ElmcityApp.oauth_twitter.AuthorizationLinkGet();
var initial_response = HttpUtils.FetchUrl(new Uri(link));

// parse the form

HtmlDocument doc = new HtmlDocument();

// parse the tokens
HtmlNode auth_form = doc.DocumentNode.SelectNodes("//form").First();
var authenticity_token_node = 
var authenticity_token = authenticity_token_node.Attributes["value"].Value;
var oauth_token_node = auth_form.SelectNodes("//input[@name='oauth_token']").First();
var oauth_token = oauth_token_node.Attributes["value"].Value;

// compose a request

var post_data_template = 
var auth_uri = OAuthTwitter.AUTHORIZE;
var post_data = String.Format(post_data_template, authenticity_token, 
oauth_token, settings["twitter_account"], settings["twitter_password"]);
var request = (HttpWebRequest)WebRequest.Create(auth_uri);
request.Method = "POST";

// fetch redirect page

var login_response = HttpUtils.DoHttpWebRequest(request,

// parse redirect page

HtmlNode redirect_page = doc.DocumentNode;

// parse callback url

var callback_node = 
var callback_url = callback_node.Attributes["href"].Value;
return callback_url;

It automates a user's interaction with two Twitter responses. The first is the Deny/Allow form:

Because the application isn't logged in to Twitter, the form combines a username-and-password login with the Deny/Allow choice. The code captures the tokens that need to be sent back to Twitter, adds @elmcity_azure's username and password (which it holds securely), and submits the form.

When you do this interactively, Twitter uses Javascript to redirect the browser back to the application's callback URL, adding the token that enables the protocol to continue. Here that won't work. But Twitter doesn't require Javascript. It also provides the callback URL as a clickable link. The elmcity service captures that link and uses it to redirect back to itself. When it answers its own call, it extracts the token and uses it to request a permanent access token from Twitter.

This was a lot of work to get back to how things were before last week, when the elmcity service could authenticate to Twitter with just its Twitter credentials. Whatever you think about the trade-offs involved in the switch to OAuth-only API authentication, it's useful to know how to explore and automate protocols like OAuth. For me, it's tricky stuff. But if I can simultaneously watch the flow of control in your code and the flow of messages over HTTP, it's doable.

0 Subscribe

0 Replies