In the prior post, I described the message flows required for granting a client application write access to a user’s Twitter status. In this post, I’ll cover the additional use case, posting a status message.

The output of a successful authorization of a client application is an oauth_token and an oauth_token_secret. Twitter does not “expire” these items and so the client can re-use them forever.

Typically, clients (applications) will store the oauth_token and oauth_token_secret in a settings file. A .NET application on Windows might put the file in %AppData%, in other words, \users\Someone\AppData\Roaming\Vendor\Appname\Settings.xml.

To post status updates to a user’s Twitter stream, the client application can simply POST to /1/statuses/update.xml or /1/statuses/update.json, including an Authorization header constructed as described previously. In fact there is just one request+response for this use case. This header is formed in exactly the same way as described in the first use-case, except that there are now different required parameters. For regular status updates, the client needs to construct the signature base using:

• all the oauth_ parameters (except the “secret” ones). This will include the oauth_token obtained from the approval dance, but not the oauth_verifier, which is a one-time thing, or the oauth_callback which is used only when requesting client approval.
• the complete status parameter, URL-encoded, which must be inserted into the base in observance of lexicographic sorting order.

Again, the client must normalize the parameters – sort by name, encode the values, and concatenate together with ampersands – and then sign the UTF-8 encoded payload. The added twist here is that the payload to be signed must include named query string parameters. In the case of a status update to Twitter, there is just one query string parameter, ‘status’. That parameter must be included in the signature base, with the value OAuth-percent-encoded as described previously. The result must be inserted into the base in observance of the lexicographic sorting requirement.

The resulting signature base looks something like this:

POST&http%3A%2F%2Fapi.twitter.com%2F1%2Fstatuses%2Fupdate.xml&
    oauth_consumer_key%3DFXJ0DIH50S7ZpXD5HXlalQ%26
    oauth_nonce%3D1j91hk2m%26oauth_signature_method%3DHMAC-SHA1%26
    oauth_timestamp%3D1336764858%26
    oauth_token%3D59152613-HBwngBPTBUIrl7qKJXvBKaM0Ko1FpA2n5Hfmdh4uc%26
    oauth_version%3D1.0%26
    status%3DTwas%2520brillig%252C%2520and%2520the...

Notice that the actual status string is double-URL-encoded. Yes, you must URL-encode it once, and append it to the sorted list of parameters. Then that sorted list gets URL-encoded again, before being concatenated with the method and URL, separated by ampersands. The result, the signature base, is what you see here. To sign, the entire signature base is URL-encoded again, before being UTF-8 encoded and signed. This is the kind of thing that is hard to discern in the scattered Twitter documentation. A simple example like this makes everything clearer, though not exactly clear.

The resulting message with the computed signature looks like this:

POST http://api.twitter.com/1/statuses/update.xml?status=Twas%20bri..  HTTP/1.1
Authorization: OAuth
    oauth_consumer_key="xxxxxxxxxx",
    oauth_nonce="1j91hk2m",
    oauth_signature="YyyTkVQyecxqeGN%2BlgdhsZNkkJw%3D",
    oauth_signature_method="HMAC-SHA1",
    oauth_timestamp="1336764858",
    oauth_token="59152613-PNjdWlAPngxOVa4dCWnLMIzZcUXKmwMoQpQWTkbvD",
    oauth_version="1.0"
Host: api.twitter.com

(That Authorization header is all on one line). The response message, if you are getting XML, looks like this:

<status>
  <created_at>Fri May 11 19:34:12 +0000 2012</created_at>
  <id>201032478964711427</id>
  <text>Twas brillig, and the slithy toves...</text>
  <source>Dino's TweetIt</source>
  <truncated>false</truncated>
  <favorited>false</favorited>
  <in_reply_to_status_id></in_reply_to_status_id>
  <in_reply_to_user_id></in_reply_to_user_id>
  <in_reply_to_screen_name></in_reply_to_screen_name>
  <retweet_count>0</retweet_count>
  <retweeted>false</retweeted>
  <user>
    <id>59152613</id>
    <name>dino chiesa</name>
    <screen_name>dpchiesa</screen_name>
    <location>Seattle</location>
    <description></description>
     <url>http://dinochiesa.net</url>
    <protected>false</protected>
    <followers_count>0</followers_count>
    <profile_background_color>C0DEED</profile_background_color>
    <profile_text_color>333333</profile_text_color>
    <profile_link_color>0084B4</profile_link_color>
    <profile_sidebar_fill_color>DDEEF6</profile_sidebar_fill_color>
    <profile_sidebar_border_color>C0DEED</profile_sidebar_border_color>
    <friends_count>13</friends_count>
    <created_at>Wed Jul 22 15:20:26 +0000 2009</created_at>
    <favourites_count>0</favourites_count>
    <utc_offset>-28800</utc_offset>
    <time_zone>Pacific Time (US &amp; Canada)</time_zone>
    <profile_background_tile>false</profile_background_tile>
    <profile_use_background_image>true</profile_use_background_image>
    <notifications>false</notifications>
    <geo_enabled>false</geo_enabled>
    <verified>false</verified>
    <following>false</following>
    <statuses_count>22</statuses_count>
    <lang>en</lang>
    <contributors_enabled>false</contributors_enabled>
    <follow_request_sent>false</follow_request_sent>
    <listed_count>0</listed_count>
    <show_all_inline_media>false</show_all_inline_media>
    <default_profile>true</default_profile>
    <default_profile_image>true</default_profile_image>
    <is_translator>false</is_translator>
  </user>
  <geo/>
  <coordinates/>
  <place/>
  <contributors/>
</status>

If the client uses the .json suffix on the update URL, then it would get a JSON-formatted message containing the same information.

There may arise a situation in which the user has already approved the client application, but the client does not have access to the oauth_token and oauth_token_secret, and is therefore unable to properly construct the signature for the Authorization header. This might happen when a user gets a new PC or device, or tries to use the client application from a different PC, for example. In that case the client should simply request authorization again, as in Use Case #1, described in the previous post. If this happens, the /oauth/authenticate endpoint can automatically redirect back to your callback without requiring any user interaction or even showing any Twitter UI to the user.

Last time, I covered the basics around Twitter’s use of OAuth.  Now I will look at the protocol, or message flows. Remember, the basic need is to allow a user (resource owner) employing an application (what OAuth calls the client) to post a status message to Twitter (what OAuth calls the server).

There are two use cases: approve the app, and post a status message. Each use case has an associated message flow.

The first time a user tries to employ a client to post a status update to Twitter, the user must explicitly grant access to allow the client to act on its behalf with Twitter. The first step is to request a temporary token, what Twitter calls a “request token”, via a POST to https://api.twitter.com/oauth/request_token like this:

POST https://api.twitter.com/oauth/request_token HTTP/1.1
Authorization: OAuth oauth_callback="oob",
      oauth_consumer_key="XXXXXXXXXXXXXX",
      oauth_nonce="0cv1i19r",
      oauth_signature="mIggVeqh58bsJAMyLsTrgeNq0qo%3D",
      oauth_signature_method="HMAC-SHA1",
      oauth_timestamp="1336759491",
      oauth_version="1.0"
Host: api.twitter.com
Connection: Keep-Alive

In the above message, the Authorization header appears all on one line, with each named oauth parameter beginning with oauth_ separated from the next with a comma-space. It is shown with broken lines only to allow easier reading. That header needs to be specially constructed. According to OAuth, the signature is an HMAC, using SHA1, over a normalized string representation of the relevant pieces of the payload. For this request, that normalized string must include:

1. the HTTP Method
2. a normalized URL
3. a URL-encoded form of the oauth_ parameters (except the “consumer secret” – think of that as a key or password), sorted lexicographically and concatenated with ampersand.

Each of those elements is itself concatenated with ampersands. Think of it like this:

METHOD&URL&PARAMSTRING

Where the intended METHOD is POST or GET (etc), the URL is the URL of course (including any query parameters!), and the PARAMSTRING is a blob of stuff, which itself is the URL-encoded form of something like this:

oauth_callback=oob&oauth_consumer_key=xxxxxxxxxxxxxxx&
    oauth_nonce=0cv1i19r&oauth_signature_method=HMAC-SHA1&
    oauth_timestamp=1336759491&oauth_version=1.0

Once again, that is all one contiguous string – the newlines are there just for visual clarity.  Note the lack of double-quotes here. 

The final result, those three elements concatenated together (remember to url-encode the param string), is known as the signature base.

Now, about the values for those parameters:

• The callback value of ‘oob’  should be used for desktop or mobile applications. For web apps, let’s say a PHP-driven website that will post tweets on the user’s behalf, they need to specify a web URL for the callback. This is a web endpoint that Twitter will redirect to, after the user confirms their authorization decision (either denial or approval).
• The timestamp is simply the number of seconds since January 1, 1970. You can get this from PHP’s time() function, for example. In .NET it’s a little trickier.
• The nonce needs to be a unique string per timestamp, which means you could probably use any monotonically increasing number. Or even a random number. Some people use UUIDs but that’s probably overkill.
• The consumer key is something you get from the Twitter developer dashboard when you register your application.
• The other pieces (signature method, version), for Twitter anyway, are fixed.

That form must then be URL-encoded according to a special approach defined just for OAuth. OAuth specifies percentage hex-encoding (%XX), and rather than defining which characters must be encoded, OAuth declares which characters need no encoding: alpha, digits, dash, dot, underscore, and twiddle (aka tilde). Everything else gets encoded.

The encoded result is then concatenated with the other two elements, and the resulting “signature base” is something like this.

POST&https%3A%2F%2Fapi.twitter.com%2Foauth%2Frequest_token&
    oauth_callback%3Doob%26oauth_consumer_key%3DXXXXXXXXXX%26
    oauth_nonce%3D0cv1i19r%26oauth_signature_method%3DHMAC-SHA1%26
    oauth_timestamp%3D1336759491%26oauth_version%3D1.0

The client app must then compute the HMACSHA1 for this thing, using a key derived as the concatenation of the consumer secret (aka client key) and the token secret, separated by ampersands. In this first message, there is no token secret yet, therefore the key is simply the consumer secret with an ampersand appended. Get the key bytes by UTF-8 encoding that result. Sign the UTF-8 encoding of the URL-encoded signature base with that key, then base64-encode the resulting byte array to get the string value of oauth_signature. Whew! The client needs to embed this signature into the Authorization header of the HTTP request, as shown above.

If the signature and timestamp check out, the server responds with a temporary request token and secret, like this:

HTTP/1.1 200 OK
Date: Fri, 11 May 2012 18:04:46 GMT
Status: 200 OK
Pragma: no-cache
...
Last-Modified: Fri, 11 May 2012 18:04:46 GMT
Content-Length: 147

oauth_token=vXwB2yCZYFAPlr4RcUcjzfIVW6F0b8lPVsAbMe7x8e4
    &oauth_token_secret=T81nWXuM5tUHKm8Kz7Pin8x8k70i2aThfWVuWcyXi0
    &oauth_callback_confirmed=true

Here again, the message is shown on multiple lines, but in response it will actually be all on one line.  This response contains what OAuth calls a temporary token, and what Twitter has called a request token. With this the client must direct the user to grant authorization.  For Twitter, the user does that by opening a web page pointing to https://api.twitter.com/oauth/authorize?oauth_token=xxxx, inserting the proper value of oauth_token.

This pops a web page that looks like so:

When the user clicks the “approve” button, the web page submits the form to https://api.twitter.com/oauth/authorize, Twitter responds with a web page, formatted in HTML, containing an approval PIN. The normal flow is for the web browser to display that form.  The user can then copy that PIN from the browser, and paste it into the client application, to allow the client to complete the message exchange with Twitter required for approval.   If the client itself is controlling the web browser, as with a Windows Forms WebBrowser control, then the client app can extract the PIN programmatically without requiring any explicit cut-n-paste step by the user.

The client then sends another message to Twitter to complete the authorization. Here again, the Authorization header is formatted specially, as described above. In this case the header must include oauth_token and oauth_verifier set to the token and the PIN received in the previous exchange, but it need not include the oauth_callback. Once again, the client must prepare the signature base and sign it, and then embed the resulting signature into the Authorization header. It looks like this:

POST https://api.twitter.com/oauth/access_token HTTP/1.1
Authorization: OAuth
    oauth_consumer_key="xxxxxxxxx",
    oauth_nonce="1ovs611q",
    oauth_signature="24iNJYm4mD4FIyZCM8amT8GPkrg%3D",
    oauth_signature_method="HMAC-SHA1",
    oauth_timestamp="1336764022",
    oauth_token="RtYhup118f01CWCMgbyPvwoOcCHk0fXPeXqlPVRZzM",
    oauth_verifier="0167809",
    oauth_version="1.0"
Host: api.twitter.com

(In an actual message, that Authorization header will be all on one line). The server responds with a regular HTTP response, with message (payload) like this:

oauth_token=59152613-PNjdWlAPngxOVa4dCWnLMIzZcUXKmwMoQpQWTkbvD&
    oauth_token_secret=UV1oFX1wb7lehcH6p6bWbm3N1RcxNUVs3OEGVts4&
    user_id=59152613&
    screen_name=dpchiesa

(All on one line). The oauth_token and oauth_token_secret shown in this response look very similar to the data with the same names received in the prior response. But these are access tokens, not temporary tokens.  The client application now has a token and token secret that it can use for OAuth-protected transactions.

All this is necessary when using an application for the first time, and it has not been authorized yet.

In the next post I’ll describe the message flows that use this token and secret to post a status update to Twitter.

Sending messages to Twitter from any application is straightforward, if you follow the protocol.  The problem is, Twitter does not explicitly define the protocol in any one place.

If you go to the developer center on Twitter’s site, there is a section that deals with how Twitter uses OAuth (which is defined in IETF RFC 5849), but Twitter doesn’t explain exactly how.  Instead, Twitter refers to external sources (one, two) for the definition of the general OAuth protocol, and it expends a bunch of effort explaining why apps cannot use Basic Auth.  Not helpful for an implementer. Twitter’s doc page then points developers to existing third-party OAuth libraries.  There is no documentation that I could find that explicitly describes or demonstrates what I would call the communications protocol: the message flows required to use OAuth to connect to Twitter, for the various scenarios – a web application, a “traditional” desktop app, a mobile app, and so on.

Beyond the communication protocol, there is no firm guidance from Twitter on User Interface models – what an application should look like or do.  There is no firm guidance around the issues surrounding the OAuth consumer keys and secret keys.

For various reasons a developer might want more detailed information.  For one thing, the developer may be developing their own Twitter-friendly library, for various reasons: licensing or other intellectual property concerns, efficiency, control.  If you are developing your own application that uses Twitter, the upshot is: the implementation is the documentation. Twitter provides some reference material, but it is not complete; when you get down to it, you need to try and see. Develop, test, diagnose, see what works. Developers will have to figure it out themselves.

I went through that process with a couple of applications based in C#, and here’s what I learned.  The basics:

1. OAuth isn’t mysterious or mystical. It’s well-defined. Using the protocol with Twitter requires some exploration.
2. OAuth defines 3 parties to the transaction: a server, a client, and a resource owner. The server is the manager of resources; in the case of Twitter, twitter is the server.  The resource owner is the user.  The client is the application that the user employs to connect to the server – for example, something that posts Tweets on behalf of a user. This can be a web application (another server), a desktop application, a mobile phone app, and so on.
3. All OAuth 1.0 clients (and remember, client = application) have an associated “consumer key” and  “consumer secret”.  In later drafts of OAuth 1.0 these were together termed “client credentials”.  When a developer builds an app, he registers it with the OAuth provider – in this case, Twitter – and receives those two opaque strings. As part of the OAuth protocol, the application (aka client) sends the consumer key to Twitter at runtime, encrypted with his consumer secret, in order to identify itself. If Twitter can decrypt the message with the copy of the secret it holds in association to that consumer key, Twitter concludes the client to be authentic.
4. When the client (application) identifies itself to Twitter, Twitter checks to see if the client is authorized to act on behalf of the user or “resource owner”.  If not, Twitter instructs the client to tell the user to approve the client for this interaction, via a web page interaction. In other words, Twitter tells the client, “Have the user visit this web page…” and when the client does so, Twitter asks the user, “Do you approve this app?”.  For this approval, the user (resource owner) must authenticate directly to Twitter with his username and password, and can then reply yes or no.  With an affirmative reply, Twitter marks the client (really, the application, identified by the consumer key it presents to Twitter) as authorized to act on behalf of this particular user.
5. Now that the client is identified and authorized, it can send requests to Twitter to update the resources on behalf of the resource owner. In simple terms: the app can post tweets to the user’s status stream.
6. If the user wants to later de-authorize the client application from acting on its behalf, it can visit Twitter.com to do so.

A Concern about Client Identity

The use of the consumer key or client credential as the basis for application identification means that the claimed identity of desktop applications cannot be trusted.  Here’s why:  OAuth assumes that an application that holds these credentials is the registered application.  Yet most traditional desktop or mobile applications will store this information in the compiled binary, or in some external store. This information will be accessible to other applications, which means the “client identity” can be easily forged, either mistakenly or on purpose. A developer just needs to grab those two items and embed them in a new app, and you’ve got a “stolen application identity” problem.  In short, anyone that uses a desktop application to connect to Twitter has the power to forge the identity of that application.  This is one of a set of concerns around Twitter’s use of OAuth, that have been previously articulated.  I don’t have a good answer for this problem.

In my next post I’ll describe the OAuth 1.0a protocol flows.