Really interesting talk by Alex Stamos, CSO of Yahoo during the AppSec conference.
One takeaway: Network-layer security firewalls operating will not be feasible at 100Gbps network speeds, eg in 3-5 years.
Lots of other observations and views that make a lot of sense to me. This guy gets it.
In part 1 and part 2 of this series, I talked about Drupal REST services, and authenticating, and querying data. Be sure to review those before continuing with this post.
This article talks about how to create or update data on Drupal using REST APIs. It will use the same authentication foundation as described in Part 2.
What kinds of things can you create or update or delete with the Drupal REST API?
Pretty cool. Also, when creating entities, like users, all the normal drupal hooks will run. So that if you programmatically create a new user, and if you have a new-user hook that sends out an email…. then that hook will run and the email address for the newly-created user will get an email sent by Drupal. The API provides a nice way to provision a set of users all at one go, into Drupal, rather than asking each individual user to visit the site and self-register.
There are also special REST endpoints for doing things like resetting passwords or resending the welcome email.
So let’s look at some request payloads !
Request:
curl -i -X PUT \
-H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \
-H X-CSRF-Token:w98sdb9udjiskdjs \
-H Accept:application/json \
-H content-type:application/json \
http://example.com/rest/node/4 \
-d '{
"title": "about multiple themes....",
"body": {
"und": [{
"value": "how to demonstrate multiple themes?. ...",
"summary": "multiple themes?",
"format": "filtered_html",
"safe_value": "themes",
"safe_summary": "themes..."
}]
}
}'
To create a new Forum post (Drupal calls it a Forum topic):
Request:
curl -i -X PUT \
-H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \
-H X-CSRF-Token:w98sdb9udjiskdjs \
-H Accept:application/json \
-H content-type:application/json \
http://example.com/rest/node \
-d '{
"type": "forum",
"title": "test post?",
"language": "und",
"taxonomy_forums": { "und": "1" },
"body": {
"und": [{
"value" : "This is the full text of the forum post",
"summary": "this is a test1",
"format": "full_html"
}]
}
}'
This part…
"taxonomy_forums": { "und": "1" },
…tells which forum to post to. Actually the “parent forum” is a taxonomy term, not a forum container. Nodes carry a taxonomy term on them, to identify which forum they belong to.
If you specify an invalid forum id, you will get this json error response:
406 Unacceptable
...
{
"form_errors": {
"taxonomy_forums][und": "An illegal choice has been detected. Please contact the site administrator.",
"taxonomy_forums": "Select a forum."
}
}
Here’s another “create forum topic” request, to a different forum:
curl -i -X POST \
-H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \
-H X-CSRF-Token:w98sdb9udjiskdjs \
-H Accept:application/json \
-H content-type:application/json \
http://example.com/rest/node \
-d '{
"type": "forum",
"title": "test post #2",
"language": "und",
"taxonomy_forums": { "und": "5" },
"body": {
"und": [{
"value" : "This is a test post. please ignore.",
"summary": "this is a test1",
"format": "full_html"
}]
}
}'
Notice the alternate forum id in that request, as compared to the prior one:
"taxonomy_forums": { "und": "5" }
Step 1: query the vocabulary that corresponds to “forums”:
curl -i -X GET \ -H accept:application/json \ -H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \ 'http://example.com/rest/taxonomy_vocabulary?parameters\[machine_name\]=forums'
Example Response:
[{
"vid": "1",
"name": "Forums",
"machine_name": "forums",
"description": "Forum navigation vocabulary",
"hierarchy": "0",
"module": "forum",
"weight": "-10",
"uri": "http://myserver/rest/taxonomy_vocabulary/1"
}]
The important part is the “vid” – which is the vocabulary ID.
Step 2: Query the terms for that vocabulary. This gives all forum names and IDs.
curl -i -X GET \ -H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \ -H Accept:application/json \ -H content-type:application/json \ 'http://example.com/rest/taxonomy_term?parameters\[vid\]=1'
Example response:
Response:
[{
"tid": "8",
"vid": "1",
"name": "Getting Started",
"description": "",
"format": null,
"weight": "0",
"uuid": "7ff7ce10-0082-46f6-9edd-882410b7c304",
"depth": 0,
"parents": ["0"]
}, {
"tid": "1",
"vid": "1",
"name": "General discussion",
"description": "",
"format": null,
"weight": "1",
"uuid": "dbf914e7-42c2-45f6-b77a-e66a0da72310",
"depth": 0,
"parents": ["0"]
}, {
"tid": "4",
"vid": "1",
"name": "Security and Privacy Issues",
"description": "",
"format": null,
"weight": "2",
"uuid": "7496bfd7-2cb8-4f87-a1e4-f45b1956a01e",
"depth": 0,
"parents": ["0"]
}]
The tid in each array element is what you must use in the “taxonomy_forums”: { “und”: “4” }, … when POSTing a new forum node.
Deleting a node means removing an article, a forum topic (post), a comment, etc.
The request:
curl -i -X DELETE \ -H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \ -H X-CSRF-Token:w98sdb9udjiskdjs \ -H Accept:application/json \ http://example.com/rest/node/8
Example response:
[true]
Weird response, but ok.
By the way, if the cookie and token has timed out, for any of these create, update, or delete calls you may see this response:
["Access denied for user anonymous"].
There is no explicit notice that the cookie has timed out. The remedy is
to re-authenticate and submit the request again.
Deleting a taxonomy term in the taxonomy vocabulary for forums would imply deleting a forum.
curl -i -X DELETE \ -H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \ -H X-CSRF-Token:w98sdb9udjiskdjs \ -H Accept:application/json \ http://dev-wagov1.devportal.apigee.com/rest/taxonomy_term/7
Creating a taxonomy term in the taxonomy vocabulary for forums would imply creating a forum.
curl -i -X POST \
-H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \
-H X-CSRF-Token:w98sdb9udjiskdjs \
-H Accept:application/json \
-H content-type:application/json \
http://dev-wagov1.devportal.apigee.com/rest/taxonomy_term \
-d '{
"vid": "1",
"name": "Another Forum on the site",
"description": "",
"format": null,
"weight": "10"
}'
The UUID and TID for the forum will be generated for you. Unfortunately, the tid will not be returned for you to reference. You need to query to find it. Use the name of the forum you just created:
Request:
curl -i -X GET \ -H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \ -H Accept:application/json \ 'http://example.com/rest/taxonomy_term?parameters\[name\]=Another+Forum+on+the+site'
Example Response:
[{
"tid": "36",
"vid": "1",
"name": "Another Forum on the site",
"description": "",
"format": null,
"weight": "10",
"uuid": "dcbe0118-c160-4556-b0b6-1813241bb851",
"uri": "http://example.com/rest/taxonomy_term/36"
}]
Make sure you use unique names for these taxonomy terms.
curl -i -X POST \
-H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \
-H X-CSRF-Token:w98sdb9udjiskdjs \
-H accept:application/json \
-H content-type:application/json \
http://example.com/rest/user -d '{
"name" : "TestUser1",
"mail" : "Dchiesa+Testuser1@apigee.com",
"pass": "secret123",
"timezone": "America/Los_Angeles",
"field_first_name": {
"und": [{ "value": "Dino"}]
},
"field_last_name": {
"und": [{ "value": "Chiesa"}]
}
}'
Response:
{"uid":"7","uri":"http://example.com/rest/user/7"}
curl -i -X POST \
-H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \
-H X-CSRF-Token:w98sdb9udjiskdjs \
-H accept:application/json \
-H content-type:application/json \
http://example.com/rest/user/7/resend_welcome_email -d '{}'
curl -i -X POST \
-H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \
-H X-CSRF-Token:w98sdb9udjiskdjs \
-H accept:application/json \
-H content-type:application/json \
http://example.com/rest/user/7/password_reset -d '{}'
This shows how to set the user status to 0, in order to de-activate the user.
curl -i -X PUT \
-H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \
-H X-CSRF-Token:w98sdb9udjiskdjs \
-H accept:application/json \
-H content-type:application/json \
http://example.com/rest/user/6 -d '{
"status" : "0"
}'
You could of course update any of the other user attributes as well.
That ought to get you started with creating and updating things in Drupal via the REST Server.
Remember, the basic rules are:
Good luck. Contact me here if these examples are unclear.
Bright-eyed Toyota product planner: “Hey, wouldn’t it be great if we could have the car open automatically, when the owner of the car walks up to it?”
Naive Toyota Engineer: “Yeah! Let’s make a Passive Keyless Entry and Start (PKES) system. The car will call out to the key and will open up if it finds the key.”
Skeptical pointy-haired boss: “Whoa – it sounds like this could be a security problem. What if I’m 20 feet away from my car, around the corner? Will some shady character be able to pull the door handle and gain entry?”
Naive Toyota Engineer: “No worries! We’ll make the radio communication work from only a few feet away. The key will HAVE TO BE right next to the car.”
All: “OK, good plan!”
NYTimes article on an interesting and potentially very expensive vulnerability in systems that allow keyless entry to automobiles. This “innovation” allows you to simply walk up to your car with the key in your pocket, and pull the door handle, to get the car to unlock for you.
How does it work? On the handle pull, the car attempts to contact the key wirelessly, and upon getting a valid response, unlocks. What could possibly go wrong?
Thieves are using simple power amplifiers to extend the distance over which this communication works, so they can open the car in the driveway at night, just by pulling on the handle, while your key is by your bedside. Not only do they gain entry to your vehicle and its contents, they can also start the car, and drive off. The engine won’t stop even if the key goes out of range.
I wonder if Toyota have a software mechanism to turn off this capability. The NYTimes article suggests putting the key in a Faraday cage, like your freezer.
Interesting in-depth analysis of the vulnerability, from 2011, here.
Disclaimer: I don’t drive a Prius. Also I don’t work for Toyota, but I think their vehicles are pretty good, generally. I guess there are other automobile makes that have similar vulnerabilities.
Be sure to start with Part 1 of this series.
To recap: I’ve enabled the Services module in Drupal v7, in order to enable REST calls into Drupal, to do things like:
Clear? The prior post talks about the preparation. This post talks about some of the actual REST calls. Let’s start with Authentication.
These are the steps required to make authenticated calls to Drupal via the Services module:
OK, Let’s look at some example messages.
Request:
curl -i -X POST -H content-type:application/json \ -H Accept:application/json \ http://example.com/rest/user/token
The content-type header is required, even though there is no payload sent with the POST.
Response:
HTTP/1.1 200 OK
Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0
Content-Type: application/json
Etag: "1428629440"
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Last-Modified: Fri, 10 Apr 2015 01:30:40 GMT
Vary: Accept
Content-Length: 55
Accept-Ranges: bytes
Date: Fri, 10 Apr 2015 01:30:51 GMT
Connection: keep-alive
{"token":"woalC7A1sRzpnzDhp8_rtWB1YlXBRalWMSODDX1yfUI"}
That’s a token, surely. I haven’t figured out what I need that token for. It’s worth pointing out that you get a new CSRF token when you login; see below. So I don’t do anything with this token. I never use the call to /rest/user/token .
To do anything interesting, your app needs to login; aka authenticate. After login, your app can invoke regular transactions, using the information returned in that response. Let’s look at the messages.
Request:
curl -i -X POST -H content-type:application/json \
-H Accept:application/json \
http://example.com/rest/user/login \
-d '{
"username" : "YOURUSERNAME",
"password" : "YOURPASSWORD"
}'
Response:
HTTP/1.1 200 OK
Content-Type: application/json
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Last-Modified: Fri, 10 Apr 2015 01:33:35 GMT
Set-Cookie: SESS02caabc123=ShBy6ue5TTabcdefg; expires=Sun, 03-May-2015 05:06:55 GMT; path=/; domain=.example.com; HttpOnly
...
{
"sessid": "ShBy6ue5TTabcdefg",
"session_name": "SESS02caabc123",
"token": "w98sdb9udjiskdjs",
"user": {
"uid": "4",
"name": "YOURUSERNAME",
"mail": "YOUREMAIL@example.com",
"theme": "",
"signature": "",
"signature_format": null,
"created": "1402005877",
"access": "1426280563",
"login": 1426280601,
"status": "1",
"timezone": null,
"language": "",
"picture": "0",
"data": false,
"uuid": "3e1e948e-940e-4a05-bd7a-267c6671c11b",
"roles": {
"2": "authenticated user",
"3": "administrator"
},
"field_first_name": {
"und": [{
"value": "Dino",
"format": null,
"safe_value": "Dino"
}]
},
"field_last_name": {
"und": [{
"value": "Chiesa",
"format": null,
"safe_value": "Chiesa"
}]
},
"metatags": [],
"rdf_mapping": {
"rdftype": ["sioc:UserAccount"],
"name": {
"predicates": ["foaf:name"]
},
"homepage": {
"predicates": ["foaf:page"],
"type": "rel"
}
}
}
}
There are a few data items that are of particular interest.
Briefly, in subsequent calls, your app needs to pass back the cookie specified in the Set-Cookie header. BUT, if you’re coding in Javascript or PHP or C# or Java or whatever, you don’t need to deal with managing cookies, because the cookie value is also contained in the JSON payload. The cookie has the form {SESSIONNAME}={SESSIONID}, and those values are provided right in the JSON. With the response shown above, subsequent GET calls need to specify a header like this:
Cookie: SESS02caabc123=ShBy6ue5TTabcdefg
Subsequent PUT, POST, and DELETE calls need to specify the Cookie as well as the CSRF header, like this:
Cookie: SESS02caabc123=ShBy6ue5TTabcdefg X-CSRF-Token: w98sdb9udjiskdjs
In case it was not obvious: The value of the X-CSRF-Token is the value following the “token” property in the json response. Also: your values for the session name, session id, and token will be different than the ones shown here. Just sayin.
OK, the first thing to do once authenticated: get all the nodes. Here’s the request to do that:
Request:
curl -i -X GET \ -H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \ -H Accept:application/json \ http://example.com/rest/node
The response gives up to “pagesize” elements, which defaults to 20 on my system. You can also append a query parameter ?pagesize=30 for example to increase this. To repeat: you do not need to pass in the X-csrf-token header here for this query. The CSRF token is required for Update operations (POST, PUT, DELETE). Not for GET.
Here’s the response:
[{
"nid": "32",
"vid": "33",
"type": "wquota3",
"language": "und",
"title": "get weather for given WOEID (token)",
"uid": "4",
"status": "1",
"created": "1425419882",
"changed": "1425419904",
"comment": "1",
"promote": "0",
"sticky": "0",
"tnid": "0",
"translate": "0",
"uuid": "9b0b503d-cdd2-410f-9ba6-421804d25d4e",
"uri": "http://example.com/rest/node/32"
}, {
"nid": "33",
"vid": "34",
"type": "wquota3",
"language": "und",
"title": "get weather for given WOEID (key)",
"uid": "4",
"status": "1",
"created": "1425419882",
"changed": "1425419904",
"comment": "1",
"promote": "0",
"sticky": "0",
"tnid": "0",
"translate": "0",
"uuid": "56d233fe-91d4-49e5-aace-59f1c19fbb73",
"uri": "http://example.com/rest/node/33"
}, {
"nid": "31",
"vid": "32",
"type": "cbc",
"language": "und",
"title": "Shorten URL",
"uid": "4",
"status": "0",
"created": "1425419757",
"changed": "1425419757",
"comment": "1",
"promote": "0",
"sticky": "0",
"tnid": "0",
"translate": "0",
"uuid": "8f21a9bc-30e6-4232-adf9-fe705bad6049",
"uri": "http://example.com/rest/node/31"
}
...
]
This is an array, which some people say should never be returned by a REST resource. (Because What if you wanted to add a property to the response? Where would you put it?) But anyway, it works. You don’t get ALL the nodes, you get only a page worth. Also, you don’t get all the details for each node. But you do get the URL for each node, which is your way to get the full details of a node.
What if you want the next page? According to my reading of the scattered Drupal documentation, these are the query parameters accepted for queries on all entity types:
So, to get the next page, just send the same request, but with a query parameter, page=2.
This is easy.
Request:
curl -i -X GET \ -H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \ -H Accept:application/json \ http://example.com/rest/node/75
Response:
HTTP/1.1 200 OK
Content-Type: application/json
...
{
"vid": "76",
"uid": "4",
"title": "Embedding keys securely into the app",
"log": "",
"status": "1",
"comment": "2",
"promote": "0",
"sticky": "0",
"vuuid": "57f3aade-d923-4bb5-8861-1d2c160a9fd5",
"nid": "75",
"type": "forum",
"language": "und",
"created": "1427332570",
"changed": "1427332570",
"tnid": "0",
"translate": "0",
"uuid": "026c029d-5a45-4e10-8aec-ac5e9824a5c5",
"revision_timestamp": "1427332570",
"revision_uid": "4",
"taxonomy_forums": {
"und": [{
"tid": "89"
}]
},
"body": {
"und": [{
"value": "Suppose I have received my key from Healthsparq. Now I would like to embed that key into the app that I'm producing for the mobile device. How can I do this securely, so that undesirables will not be able to find the keys or sniff the key as I use it?",
"summary": "",
"format": "full_html",
"safe_value": "Suppose I have received my key from Healthsparq. Now I would like to embed that key into the app that I'm producing for the mobile device. How can I do this securely, so that undesirables will not be able to find the keys or sniff the key as I use it?
\n",
"safe_summary": ""
}]
},
"metatags": [],
"rdf_mapping": {
"rdftype": ["sioc:Post", "sioct:BoardPost"],
"taxonomy_forums": {
"predicates": ["sioc:has_container"],
"type": "rel"
},
"title": {
"predicates": ["dc:title"]
},
"created": {
"predicates": ["dc:date", "dc:created"],
"datatype": "xsd:dateTime",
"callback": "date_iso8601"
},
"changed": {
"predicates": ["dc:modified"],
"datatype": "xsd:dateTime",
"callback": "date_iso8601"
},
"body": {
"predicates": ["content:encoded"]
},
"uid": {
"predicates": ["sioc:has_creator"],
"type": "rel"
},
"name": {
"predicates": ["foaf:name"]
},
"comment_count": {
"predicates": ["sioc:num_replies"],
"datatype": "xsd:integer"
},
"last_activity": {
"predicates": ["sioc:last_activity_date"],
"datatype": "xsd:dateTime",
"callback": "date_iso8601"
}
},
"cid": "0",
"last_comment_timestamp": "1427332570",
"last_comment_name": null,
"last_comment_uid": "4",
"comment_count": "0",
"name": "DChiesa",
"picture": "0",
"data": null,
"forum_tid": "89",
"path": "http://example.com/content/embedding-keys-securely-app"
}
As you know, in Drupal a node can represent many things. In this case, this node is a forum post. You can see that from the “type”: “forum”, in the response.
Request:
curl -i -X GET \ -H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \ -H Accept:application/json \ 'http://example.com/rest/node?parameters\[type\]=forum'
Request:
curl -i -X GET \ -H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \ -H Accept:application/json \ 'http://example.com/rest/node?parameters\[type\]=faq
Request:
curl -i -X GET \ -H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \ -H Accept:application/json \ 'http://example.com/rest/node?parameters\[type\]=article
The response you get from each of these is the same as you would get from the non-parameterized query (for all nodes). The escaping of the square brackets is necessary only for using curl within bash. If you’re sending this request from an app, you don’t need to backslash-escape the square brackets.
Request:
curl -i -X POST \
-H content-type:application/json \
-H Accept:application/json \
-H Cookie:SESS02caabc123=ShBy6ue5TTabcdefg \
-H X-csrf-token:xxxx \
http://example.com/rest/user/logout -d '{}'
Notes: The value of the cookie header and the X-csrf-token header are obtained from the response to the login call! Also, obviously don’t call Logout until you’re finished making API calls. After the logout call, the Cookie and X-csrf-token will become invalid; discard them.
Response:
HTTP/1.1 200 OK ... [true]
Pretty interesting as a response.
More examples, covering creating things and deleting things, in the next post in this series.
This is Part 1. See also, Part 2 and Part 3.
I’m not an expert on Drupal, but I do have some extensive experience designing and using APIs. (I work for Apigee.)
Recently I’ve been working with Drupal v7, and in particular, getting Drupal to expose a REST interface that would allow me to program against it. I want to write apps that read forum posts, write forum posts, read or post pages, create users, and so on.
Drupal is a server than manages entities, right? This function is primarily exposed via a web UI, but that UI is just a detail. Drupal should be able to expose an API that is similarly capable. Should be!
The bazaar is alive and well with Drupal. It seems that regardless what you want to do with Drupal, there are 13 different ways to do it. And exposing Drupal entities as resources in a RESTful interface, is no different. There are numerous modules designed to help in this task, some of which are complementary to each other, some of which are overlapping, and most of which are poorly documented. Every module has multiple different versions, and every module works with multiple different versions of drupal. So figuring out the best way to proceed, for a Drupal novice like me, is not easy.
Disclaimer: What follows is what I’ve found. If a Drupal expert reads this and thinks, “Dude, you’re way off!” I am very willing to accept constructive suggestions. I’d like to know the best way to proceed. This is what I tried.
I used the Services module. There are other options – restws is one of them. I didn’t have a firm set of criteria for choosing one versus the other, except that I fell into the pit of success more easily with the Services module. It seems to be more popular and has more examples available that I found via Google search.
Services 3.0 is now available. … Note that currently there is no upgrade path for Services 3, and it is not backwards compatible with older implementations of the API. Therefore some existing modules like JSON Server and AMFPHP will not work with it. …
Not that there aren’t problems with it. The lack of backwards compatibility on a programmable interface is a really bad sign (See the blockquote). That reflects poor planning on the part of the designers of that module. And then there is the lack of clear documentation for how to do most things.
The first thing: you need to obtain and activate the Services module. There’s a straightforward guide for doing this. I installed the module, then went to the Admin panel to insure the Rest Server was enabled. A screenshot is below.
Next, you need to create a REST endpoint. To so so, still logged in as Admin, select Structure > Services. Click Add. Then specify rest, REST, and rest. Another screenshot.
That’s it. Your Drupal server is now exposing REST interfaces. You then need to click on “resources” to enable specific access to things like users, nodes, taxonomy, taxonomy terms, and so on. And you’re all set.
Once you have the Rest server enabled, getting an index of the nodes in a Drupal system is probably the most basic thing any programmer will want to do. And beyond that, creating a new node (posting a page or article), creating a user, and so on. For the Services module, there is a nice page that gives examples for doing this sort of basic thing. I’m not really a fan of the layout of that page of documentation; it seems to be all over the place, providing basic REST principles, describing REST testing tools, and then finally giving samples of messages. Those things seem like they all belong on separate, hyperlinked pages. But again, it’s the bazaar, and someone contributed that doc all by himself. If I think it could be better I am welcome to edit that page, I guess.
Here’s one example request from that page:
POST http://services.example.com/rest/user/register
Content-Type: application/json
{
"name":"services_user_1",
"pass":"password",
"mail":"services_user_1@example.com"
}
This is something I can understand. Many of the other doc pages give jQuery example code. Ummmm…..I don’t write in jQuery. Why not just show the messages that need to be sent to the Drupal server? and then let the jQuery people figure out how to type in their ajax methods? ….
The basic examples given there are good but you’ll notice there is nothing there about authentication. Nothing that shows how a developer needs to authenticate to Drupal via the Services module. That ought to be another hyperlinked page, no?
There are multiple steps involved to authenticate:
More detail on all of this in the next post.
“it’s crucial to adopt a four-tier application architecture in which applications are developed and deployed as sets of microservices.”
I love this article from the nginx.com website, courtesy of Hacker News. Very good overview of the “microservices” meme that is currently rippling through the industry. This is stuff we’ve all known – heck, we’ve been doing SOA for 10+ years – but the new term is helping to organize thoughts and understanding about why services as a metaphor is important, why services need to be lightweight, why service contracts (APIs!) need to be loose and forward-compatible, why the development of cooperating services must be done independently.
“It’s becoming increasingly clear that if you keep using development processes and application architectures that worked just fine ten years ago, you simply can’t move fast enough to capture and hold the interest of mobile users who can choose from an ever-growing number of apps.”
Oh yeah! Preach it!
The article discusses Netflix and their adoption of the microservices architecture.
I really respect Netflix as a company that moves quickly and constantly adjusts, seeking optimized architectures to address business problems. All the talking and proselytizing they’re doing around microservices is just the latest reason to really like them. I also really hate Netflix as my children seem to be unable to resist the service for even 15 minutes. Me and Netflix – It’s complicated.
Independent of Netflix and the microservices topic, nginx is also really cool. I found it super easy to configure to accomplish some nifty things in some of my work.
I’m pretty excited about the Swagger editor. But to understand why, you first need to know what Swagger is all about.
Let’s take a step back. As of August 2014, total activity on smartphones and tablets accounted for ~60% of digital media time spent in the U.S. This unabated growth in mobile is driving the growth in enabling technologies: tools for developing apps, managing app communications, measuring app and data usage, analyzing usage and predicting behavior based on that usage. APIs are a key connective technology, allowing innovative mobile apps use APIs to access data and services from companies like Uber or Twitter, or from government bodies like the State of Washington. APIs provide the linkage.
APIs are not solely about mobile apps. They can be used to connect “any app” to “any service”; indeed this website uses code running on the server to invoke the Twitter API to display tweets on the right hand side of this blog. But mobile is the driver. Web is not driving the growth, nor is the Internet-of-Things; not in APIs, nor the growth in any of the other enabling technologies. In the 2000’s it was Web. Tomorrow will be IoT. Today, it is mobile.
Ok, so What is Swagger? Swagger is a way to define and describe APIs. A language for stating exactly what an API offers. The description language is analogous to Interface Definition Languages going back to Sun’s RPC IDL, Corba IDL, DCE IDL, or SOAP’s WSDL. Many of you reading this won’t recognize any of those names; it doesn’t matter. We don’t use most of those technologies any longer, more importantly we don’t utilize the metaphors those technologies imply: function shipping, remote procedure call, or distributed objects. While moving away from the tight coupling of binary protocols and towards technologies like REST and JSON and XML that enable more loosely-coupled interactions, we still recognize that it’s helpful to be able to formally describe programmable interfaces.
OK, so Swagger is at it’s heart, a way to describe a RESTful API. Many of you are Java developers and may be familiar with Swagger Annotations, which allows you to mark up JAX-RPC server application code, which then allows you to generate a Swagger definition from an implementation. Something like doxygen. This is cool, but is sort of a backwards approach. Getting the description of the API from the implementation is analogous to getting the blueprint for a building by taking pictures of the finished framing. Ideally you’d like to go in the other direction – first build the design (or blueprint, if you will) of the API, and then generate the implementation. My friend and colleague Marsh Gardiner discussed the design-first approach last year.
This is what Swagger can do. How does one produce a Swagger document? Well if you’re an old codger like me, you might resort to using a text editor like emacs and its yaml-mode to hand-code the yaml. But a much easier approach is to use The Swagger Editor.
The API Description is basically “a model” of the API. And with that model, one can do all sorts of interesting things: generate a client-side library in one of various languages. Generate a server-side implementation stub. Generate a test harness. Generate documentation. In fact the Swagger project has had a doc-gen capability, named swagger-ui, since the early days of the project.
So what’s the upshot? The result of better enabling tooling around APIs, tooling including Swagger Editor and Swagger UI, as well as an API management layer as provided by Apigee Edge (Disclaimer! I work for Apigee!), means that it is easier for companies to expose capabilities as easy-to-consume APIs, and that it is easier for developers to code against those APIs to build compelling experiences that run on mobile devices. So I’m pretty excited about the new tooling, and I am even more excited about the integration we will soon see between these new modeling tools and the existing API Management tools already available.
Good stuff!
See here.
I like this for several reasons:
ps: My employer, Apigee, is still looking to hire SEs, and other API geeks.
Fortune Magazine is reporting that the US State of Oregon is suing Oracle for $200 million, alleging fraud for Oracle’s role in building the state’s failed health insurance exchange.
Ouch!
You may think of Uber as a disruptive player in the hire-car business. They offer a service similar to taxis, but enabled with an app experience that riders really appreciate. The hailing and pay-by-phone experience is something that I personally love. I love knowing where the driver is, how long it’ll take to arrive, the ease of paying, and so on.
But now Uber is launching an API. They’re aware that disruption is not simply something they dish out to others, but that they are also subject to disruption. They cannot afford to stand still.
Uber has a key asset – a network of drivers available on a for-hire basis. By exposing that asset programmatically via an API, Uber turns itself from simply a taxi service with an app, into a people-and-goods delivery platform upon which other companies can build their businesses. Pharmacies are a key potential partner – think of Prescription delivery services for people with limited mobility. Uber can act as a personalized delivery service for any high-value retail good.
Or think of personal concierge services – what if you could build an app to request a car to any location on behalf of a VIP or a person you’re trying to impress?
Uber have announced 11 partners at the launch of their public API. Obviously the market is interested in building new businesses or expanding existing businesses in partnership with them.
The beautiful thing about APIs is that any business can expose their capabilities programmatically to transform into a “platform”. That means greater reach, more partnerships, lower cost of business development, better return on assets.
Think back: this is exactly the kind of nirvana we in the industry were envisioning, 15 years ago, when we created SOAP and all the WS-* specifications. But we went too far out on the complexity-vs-value spectrum. It turns out the WS-SecureConversation spec was not the key concept that would enable flexible business partnerships. We now realize that simplicity of design and ease of programmability are the keys to unlock the possibilities in this domain. I’m so glad that time has arrived!
What’s the basis for your platform?