Tag: NodeJS

How to Compute an HTTP Signature for Mastodon (and an example in NodeJS)

2023-April-7

I am reading this documentation from Mastodon.

And from it, I understand that Mastodon requires an HTTP Signature, signing at least these headers:
(request-target) host date digest

If your client is written in JavaScript and runs on Nodejs, an example for how to build a signature is given on the npmjs site.

But I believe this example is out of date. It does not use the (request-target) pseudo header. So that’s not gonna work.

So what must you do? Go back to the Mastodon documentation. Unfortunately, that too, is either out of date or confusing. For a post request, the mastodon documentation states that you must first compute the “RSA-SHA256 digest hash of your request’s body”. This is not correct. There is no such thing as “RSA-SHA256 digest”! RSA-SHA256 is not the name of a digest. Message Digests include: SHA1, SHA256, MD5 (old and insecure at this point) and others. According to my reading of the code, Mastodon supports only SHA-256 digests. The documentation should state that you must compute the “SHA256 digest”. (There is no RSA key involved in computing a digest).

Regardless of the digest algorithm you use, the computed digest is a byte array. That brings us to the next question: how to encode that byte array as a string, in order to pass it to Mastodon. Some typical options for encoding are: hex encoding (aka base16 encoding), base64 encoding, or base64-url encoding. The documentation does not state which of those encodings is accepted. Helpfully, the example provided in the documentation shows a digest string that appears to be hex-encoded. Unhelpfully, again according to my reading of the code, Mastodon requires a base64-encoded digest!

With these gaps and misleading things in the documentation, I think it would be impossible for a neophyte to navigate the documentation and successfully implement a client that passes a validatable signature.

  1. produce the POST body
  2. compute the SHA-256 digest of the POST body, including all whitepsace and leading or trailing newlines. Try this online tool to help you verify your work.
  3. Encode that computed digest (which is a byte array) with base64. This should produce a string of about 44 characters.
  4. Set the Digest header to be SHA-256=xxxyyyyy , where xxxyyyy is the base64 encoding of the SHA-256 digest.
  5. Set the http headers for the pending outbound request to include at least host, date, and digest.
  6. compute the signature following the example from the npmjs.com site, with headers of “(request-target) host date digest”, and using the appropriate RSA key pair.

If it were me, I would also include a :created: and an :expires: field in the http signature.

You can play around with HTTP Signatures using this online tool. That tool does not yet support computing a Digest of a POST body, but I’ll look into extending it to do that too.

Let me know in the comments if any of this is not clear.

I posted a working example for Nodejs as a gist on Github.

It depends only on nodejs and the builtin libraries for crypto and URL to compute the hash/digest and signature. It does not actually send a request to Mastodon; that is left for you to do.

nodejs on Google App Engine – forcing HTTPS inbound, via HSTS

2018-June-20

How can I force my nodejs app running on Google App Engine, to always redirect to HTTPS ?

I have a pretty vanilla app that looks like this:

This thing is running in Google App Engine (GAE), and I’d like to make sure it listens only on HTTPS. There are standards like HSTS that can help. How can I use them?

This question and answer on Stackoverflow showed me the way. Basically, just add in a tiny module called yes-https. The new code looks like this:

Redeploying (no change to app.yaml) gets me the always-HTTPS behavior I want. When a client requests my service via http, it receives a 301 redirect pointing to the secure site.

HTTP/1.1 301 Moved Permanently
Date: Wed, 20 Jun 2018 16:27:56 GMT
Transfer-Encoding: chunked
X-Powered-By: Express
Location: https://foo-bar.appspot.com/
Via: 1.1 google

Nice, easy, clear.
Thanks to Justin for this handy module.

medialize/URI.js – why’d you go and get all fancy?

2017-November-8

I have relied on URI.js from medialize for years.

I downloaded it a long time ago, and it just works. It’s handy for parsing and building URIs form within Javascript.
I happen to use nodejs often, but I also use a JavaScript engine that runs in the JVM (via Rhino or Nashorn). So I liked URI.js for its usability across those systems.

Recently I decided to download “the latest and greatest” URI.js, and what I found… did not make me jump for joy.

URI.js is no longer “just downloadable”.

Where before I could just download the raw JS file, URI.js now has a builder that allows me to select which options I wish to include. I get the concept, and it’s a nice idea, but when I de-selected every option, I got a minimized URI.js that I did not want. When I went to the source tree I found a URI.js that included all the require() statements for punycode, Second-Level Domains, and ipv6, all stuff I did not want.

*snif*

I couldn’t figure out how to get it to “just work” in nodejs without all of that, so I had to resort to manually changing the code. Basically I just removed all the require() statements for those unneeded / unwanted modules.

And it works.

It’s possible I’m missing something basic, but for sure, it got more complicated to get the simple solution. Seems like a step backward.

Upgrade your nodejs on Mac OS X to v0.12.6

2015-July-6

Per medium.com, there are critical vulnerabilities in nodejs and iojs. People running nodejs should upgrade to v0.12.6.

On Mac OSX, this takes just a few moments. Thanks to Stackoverflow for the tip.


~/ $ node -v
v0.10.22
~/ $ sudo npm cache clean -f
Password:
npm WARN using --force I sure hope you know what you are doing.
~/ $ sudo npm install -g n
npm http GET https://registry.npmjs.org/n
npm http 200 https://registry.npmjs.org/n
npm http GET https://registry.npmjs.org/n/-/n-1.3.0.tgz
npm http 200 https://registry.npmjs.org/n/-/n-1.3.0.tgz
/usr/local/bin/n -> /usr/local/lib/node_modules/n/bin/n
n@1.3.0 /usr/local/lib/node_modules/n
~/ $ sudo n stable

install : node-v0.12.6
mkdir : /usr/local/n/versions/node/0.12.6
fetch : https://nodejs.org/dist/v0.12.6/node-v0.12.6-darwin-x64.tar.gz
installed : v0.12.6

~/ $ node -v
v0.12.6
~/ $

I don’t really hate NodeJS

2014-June-10

I don’t really hate NodeJS. Yes, a while ago, I said I hate NodeJS. But I didn’t really mean it. I was just suffering from unrealistic expectations. I learned to Let Go of my idea that JavaScript on MacOSX ought to be as easy as JS on Windows.

Since then I have adopted NodeJS pretty strongly. I use it for all sorts of tasks, from tools that automate Apigee Edge (nod to my employer) to API load generation utilities.

I write more code using NodeJS these days, than in anything else.

Recently I had occasion to write a little utility that computes TP99 for a set of API transactions being managed by Apigee Edge.

What it does is retrieve the transaction records logged from the Edge Analytics database, sorts them, and emits a computed TP99 (and TP95, TP90, TP50) to a Carbon server, which is backing Graphite, which then serves up charts of that data. A sample is below.

Sample TP95 chart from Graphite

For this tool, I chose NOT to use NodeJS but instead relied on good-old PERL. I didn’t want asynchrony, and I did want easy file I/O, pattern matching, and sorting. Also I wanted it to be maintainable by old-school sysadmins who no-doubt have not been following the finer points of using Q’s promises in NodeJS. Perl was the obvious choice. The sort required to compute TP99 will run on 60000 records or more, and needs to occur every minute, for all transactions logged during that minute. A cron job running a perl script was perfect for this.

But, recently I wrote another tool… this one automates the provisioning of EC2 instances in AWS, then installs Java and Apigee Edge on them, and configures them into a cluster. I wouldn’t want to do that in perl or Bash. NodeJS was the right tool for that much more complicated job. And of course there are NPM libraries for AWS, and for ssh and scp. Really helpful.

of nodejs and new clothes

2014-June-9

A provocative post by Eric Jiang, entitles “The emperor’s new clothes were built with Node.js”, regarding the undeserved praise being heaped upon NodeJS. While I think he gets his analysis all right, he is still missing the forest for the trees.

Nodejs has grown the way it has grown for the same reason that Visual Basic and PHP grew the way they did: These things work well, and help people get things done quickly and relatively easily. And maybe, people even have fun doing it. The community support has been critical in all three cases.

Sure it’d be nice to have multi-threading like Go and a single set of known and blessed libraries like C# and performance like C. But we don’t have all those things in one package, not yet anyway.

There are weaknesses in NodeJS, just as there are in PHP and in VB. Despite that, JavaScript is effective for many many people, and will continue to be so. NodeJS makes JavaScript much, much more effective with the NPM and the vast set of downloadable modules.

No religion, yo: nodejs is not always the right choice. Of course it isn’t. I like nodejs and use it… daily. but I’m learning Go and am enjoying that as well.

NodeJS documentation is idiotic

2013-August-6

A while back I commented on a rant by someone who said that NodeJS is stupid and if you use it, so are you.
I didn’t agree.

But, NodeJS documentation is stupid. Look, for example, at the doc for the http.request() object and it’s syntactic sugar cousin, http.get(). Let’s suppose you want to do an http get from within a node program. So you look at the doc for http.get, which says: This is like http.request(), except it sets the method to GET and calls req.end() automatically.

I wonder what that callback parameter is? http.get() accepts a callback parameter. Obviously it’s a function, but WHEN Does it get called? The doc doesn’t say.

Hmm, not very much to go on! So now I have to read the entire entry on the http.request() function. The http.get() sugar doesn’t really offer much benefit if I have to read the doc for the lower-level method, now, does it? But guess what! The doc entry for http.request() ALSO does not define the semantics of the callback parameter. It’s just a callback.

I guess if I have to ask, then I don’t need to know.