Node.js is cool, so they say. Everybody’s doing it, so I hear.
I hate it. Node.js has wasted so much of my time, I don’t think I will ever forgive it. But take that with a grain of salt. I don’t use Node.js for its intended purpose, which is server-side Javascript.
What I want is a scripting environment for automation on the local machine. In this particular case, I want to script an FTP session from my bash prompt, I want to script a directory sync. I know I can do this with bash scripting, but I already know Javascript syntax and semantics, so I’d like to use what I know. It’s FTP in this case, but in general I want to be able to script arbitrary things on the local machine.
By the way: I can do this on Windows. I can run Javascript programs from the CMD shell, to script an FTP session, automate filesystem operations, launch applications, and a bunch of other things. It’s really nice. There’s a whole catalog of COM objects that can be scripted, including obscure stuff like the fax system in Windows, or more mainstream things like IE settings (like proxies), or IIS administration. Tons of things.
Of course I could do similar things with Node.js on MacOS, too. The problem is, the Node.js model is designed for server use. Everything is asynchronous. When I retrieve a set of files from an FTP server, and I want to iterate and retrieve each file, I have to do that asynchronously. But I don’t want it to be asynchronous. Writing asynchronous code lets me get really good throughput on a server. Writing asynchronous code for my purpose just obscures the goal of the code. I want to do this:
fileList.forEach(function(item) {
var modTime = ftp.getModTime(item);
if (modTime > lastUpdate) {
ftp.retrieveFile(item);
}
});
But I can’t do that. No sir. No I cannot. I’m using Node.js. And because of that, I need this:
var c = 0, L = flist.length,
checkIfDone = function() {
c++;
if (c == L) { next(); }
};
fileList.forEach(function(item, ix) {
var localPath = (dir == '.') ? item : Path.join(dir, item),
stat, localMtime = ...,
remoteMtime = 0;
// get modification time of the remote file
sync.ftp.raw.mdtm(localPath, function (e, res) {
var d, tm;
if (e) { }
else {
remoteMtime = new Date(res);
if (localMtime == 0 || (remoteMtime > localMtime)) {
// Retrieve the file using async streams
sync.ftp.getGetSocket(localPath, function(e, sock) {
if (e) return console.error(e);
var fd = fs.openSync(localPath, "w+");
// `sock` is a stream. attach events to it.
sock.on("data", function(p) {
fs.writeSync(fd, p, 0, p.length, null);
});
sock.on("close", function(e) {
if (e) return console.error(new Error("error"));
fs.closeSync(fd);
checkIfDone();
});
// The sock stream is paused. Call
// resume() on it to start reading.
sock.resume();
});
}
else {
checkIfDone();
}
}
});
});
Can anyone ELSE see why I’d rather not use an asynchronous-optimized, server-oriented programming environment to script my desktop?
It’s not Javascript that’s the problem here. I can grab the v8 Javascript engine and use it to run JS code. That works. The problem is that there are no JS libraries for v8. There’s no FTP library. There’s no “any” library. The only FTP libraries I’ve found are Node-compliant. To use it, I have to do Node.js things. Likewise for all the other purposes. There’s no package manager for JS libraries, except for Node.js.
I think the solution here is for me to learn to use Bash scripting and use CURL. But that’s a lame solution. It would be nice if MacOS supported Javascript for client-side use, as nicely as Windows does. Because I hate Node.js.
Jameson 2013-January-28
Well, your post sounds like complaining about a spoon when you needed a knife. You might want to take a look at Python – which gives you a cross-platform tool for local scripting and has a lot of batteries included (such as FTP client classes, etc etc). Enjoy.
dpc 2013-February-5
Jameson, I think you’re right. I’m complaining that my spoon is not a knife, an there is no (JavaScript) knife. You’re telling me that there are knives, but they require a different programming language. I think your advice is good and I should convert to python, or bash, or some combination.
Kyle 2013-March-13
Python = Awesome for this.
On Windows, Powershell is very nice and has familiar C-syntax…I’ve done a lot with this.
If you insist on Javascript (I often do)….use Rhino…you write Javascript code and it comes with the entire Java class library so you can pretty much do anything you need to.
Sven Slootweg 2015-October-29
There’s a knife now, though – promises pretty much solve this problem: http://bluebirdjs.com/docs/why-promises.html
Jane 2013-March-2
Im with Jameson.. You need the right tool for the job, node.js was never design to be used in the way you would like to use it.
Either use python, or ruby with rake, which is my favourite way to automate tasks on windows. Whether or not you know pyhton or ruby, if you are already a programmer, then your ftp script should take no more than an hour or two to complete..
ed 2013-March-9
You do realize you can specify synchronous calls right?
dpc 2013-March-13
With FTP? Show me how, please? (link?)
Joshua Tenner 2013-March-23
It’s not the same call, but this is an example.
http://stackoverflow.com/questions/4482686/check-synchronously-if-file-directory-exists-in-node-js
dpc 2013-July-10
Joshua, that applies to the fs module, not to the jsftp module. There is no corresponding http://ftp.getSync() (or similar) in jsftp, as far as I know.
Patrizio Bruno 2013-October-5
You can try with Gnome JS bindings:
https://wiki.gnome.org/Seed
or
https://live.gnome.org/Gjs
in both cases the Gio API gives you access to any supported url types, both ftp:// and file:// are supported. Gjs and Seed are included in every Linux distribution supporting Gnome 3.2 or higher
Mike James 2015-March-7
I disagree, If you don’t like the code you have above, I suggest looking into control flow libraries such as async, Q, rxjs, or even trying out generator functions with node harmony or give iojs try. To give you that flow you’d get with synchronous code.
Mike James 2015-March-7
I am about 3 years late to the party here but for those reading this blog post this may change your mind.
Sven Slootweg 2015-October-29
Q is rather awkward, slow, and messy (it encourages poor and messy practices like defers). async.js somewhat alleviates the callback pyramid problem, but it’s still rather awkward in use, and it’s been pretty much superseded by promises.
Rx is a reasonable option, but may appear overly complex for stuff like that. I’d say that the only realistic option in this case is Bluebird: https://gist.github.com/joepie91/791640557e3e5fd80861
There’s also ES6 promises, but those don’t come with promisify and are a good bit more work for the complex cases.
In 2015, there’s definitely no reason to still be using either Q or async.js. They’ve served their purpose, but have simply been replaced by alternatives that work better.
dpc 2016-January-18
Thanks for the note! I haven’t tried Bluebird but will give it a look.
Lucas Dechow 2013-June-14
I do not agree with you.
I use Node.js for two purposes.
Writing small applications and administrating/create needed files fast with Shell scripts and running Grunt build tasks for different projects.
Node is versatile fast and rather small.
Makes it an easy companion for when you either develop and deploy i prototype of a web application.
It even us so versatile that you can take that web application and actually deploy it for instant use.
It is a server… in JavaScript format…
So it is kind of buckly to do simple tasks, but I believe in convention over style…
But once you get the sheer grip of it and start working on it for projects or create MVC applications with it you will love it for the NPM, for the simple syntax.
I love the routing, and love the style of Node.js
It just makes it easier to develop, deploy and create web applications.
That’s why I love it.
Peter Rust 2013-October-24
I’m about a year late to the party here, but I stumbled on to this post and thought I would mention that most node developers use the async module (it’s the 2nd most popular module, after underscore) for this kind of async iteration of files (instead of the synchronous .forEach) and that you should be able to easily pipe the ftp socket to the file instead of manually handling the data/close events, etc. IOW, using some node idioms would make this clearer and simpler. Admittedly it won’t be quite as clear/simple as the synchronous example — async is always a little more complicated — if you don’t need it, things will be simpler/clearer with Python or Ruby or another synchronous setup.
JB 2013-November-28
You should look into Event Patterns in node. Basically if you subclass the Event Emitter class it will give you the ability of synchronous programming. Doing synchronous tasks can be painful in node but once you get used to it there is no environment that will complete tasks faster. Node is a beast.
Evan Plaice 2013-December-4
You’re doing it wrong…
Try this:
var i = 0;
(function next() {
var item = fileList[i++];
if (!item) { return; }
var modTime = ftp.getModTime(item);
if (modTime > lastUpdate) {
ftp.retrieveFile(item);
}
next();
})();
The code above is basically the node equivalent of a while loop.
If you need help executing async calls in an organized and predictable manner then you should take a look at async.js. The async.series, async.waterfall, and async.parallel methods alone will save you a ton of headaches.
Note: I stumbled on your site because I’m also working on a javascript ftp sync solution. If you’d like to use, contribute, or just see a node ftp syncing in action I highly suggest you check it out.
node-ftpsync
Lex Podgorny 2013-December-28
I am with you on that. I am suffering with async database APIs. Insisting on async-only will limit Nodejs suitable for “small app” and “prototype” purposes only. And here is why: as soon as you try to abstract business logic from application logic (a requirement for enterprise level solutions) you find yourself running nested async methods with callbacks in a loop assuming to do the next task when they are all done, but that requires additional infrastructure that makes your code so obscure that you end up spending 50% of your time just debugging that scaffold. Nightmare.
I do not recommend nodejs if you aim to build anything but a prototype until sync calls to database are enabled. I feel sorry for nodejs project, it started out to be a really good solution and then it got crippled it in the most sorry way by a half-brained architecture.
chasefox 2015-March-21
lol at total bs. the non-blocking model is more performant, making it better positioned for large demand, and obviously enterprise loves node based on widespread adoption.
the event loop was the primary design goal of node. it’s a really good solution because of this. coincidentally node was built using JS because there was a natural fit; the syntax and design of the language did not conflict with the event loop being built on top of it. Best coincidence ever; node benefits greatly from the strength of the JS community.
learn what something is and how to use it before you trash it.
Gavin 2014-March-13
I had massive issues ramping up with NodeJs. Everyone kept banging on about how easy it was and I totally did not find that to be the case. My first problem was the event loop with the “forced” async model, this was very quickly cured by doing more research into why NodeJs was built this way. I came to the conclusion that it scales really well. It can sometimes get lost in translation if you are trying to do simple things where you simply need the result now, not later. The other problem I had was that I came from a background of years of programming in statically typed languages like Java, C# and C++. Stepping into a dynamic language like javascript(which I *thought* I knew) and not really having a good handle on functional programming concepts kind of left me feeling like I was out of my depth. A few years on and with heaps of coffee and persistence I am now one of those twats that says NodeJs is awesome. Don’t give up on NodeJs, asynchronous programming is not going away. It is a mindset that should be embraced with care, understanding and research. C# is moving in that direction too. If you are unlucky like me then your first tendency is to try and synchronise things. This can be a painful affair and you end up wanting to murder your NodeJs programming buddies. My advice is to find a NodeJs meet up group and bounce some idea’s around with guys that know their stuff and perhaps this will shed some light on the situation 🙂
Martin 2015-January-20
This article is completely incompetent and ridiculous. You obviously have no deep knowledge in NodeJS. There are so many people coming from the “old” languages and start spreading bad feedback for NodeJS due to their ignorance and incompetence.
Please, check what Promises are and the async nodeJS library, as well, and read more before posting nonsense.
Gilad 2015-May-24
I was about to reply to the original author when I noticed your response – I completely agree!!!
It’s like saying computers are useless because I wasn’t able to use it during a power failure. Promises, async module and so many other solutions are how you organize your code properly.
It’s easy to trash something when you have no clue about it…
ate 2015-October-5
For any later readers:
– if you want to use node.js for automation/scripting, use grunt!
– use promises anywhere instead of callbacks
Jay Jonah Jameson 2016-January-14
I hate nodejs too. To be fair I don’t know it really at all but I absolutely hate the nested node_modules idea. I’m c++ guy and can get a lot done with a small footprint. Nodejs footprint is huge. Our group has thrust nodejs upon us because it was predicted to be the second coming of Jesus Christ.
On the product we’re making, we have about ten different node applications, each containing nested node modules folders. Each of those are about 50M. So with all the duplicate nodes installed all over my rasberrypi, ih, and then npm, which has more duplicate nodes…. Man what a mess the newfangled node paradigm is! The should only be one node_modules location – the global place.
We have node IDEs, SDKs and cloud apps using node, all on the low-power computer because it was said that nodejs is the cool thing we should be doing. I think rather than making an embedded linux solution, we’re making a node js device for geen youngsters who want to use node red to make a robot.
Personally, I’m not interested in web development, I’m just trying to integrate all things node into the target device and am astounded how much scaffolding and underpinning is required just so entry level programmers can easily make little apps.
All the while I’m wondering if I just wrote the app in c++, it would be smaller, faster and more straightforward.
Node seems way over hyped. It may be fine for quick prototyping and fine for using for what it was designed for if your app format get too complex. But looking down a tree of nested and duplicate node modules cause fear and confusion.
Remember when Ruby was the hot language? I guess node will be like that. It doesn’t appeal to this c++ developer.
Feel free to correct me with any verbiage you wish. Currently though, I hate nodejs… HATE IT (for most embedded solutions).
dpc 2016-January-18
Hi Jay, and thanks for the comment. I’ve “come around” in my opinion on nodejs since I wrote that post. Actually I was frustrated by the lack of support for something that was pretty simple in Windows. But I’ve been warming to nodejs for my regular work, quite a bit. Practice and familiarity breed facility.
But I don’t build embedded systems. Seems to me for embedded systems, Go is the language I would most prefer. It’s a lot like C, but with more objects and a broader community of modules. It’s a lot like C++, but cleaner. If you haven’t tried it, I recommend it. Fast, small, works for embedded systems.
The great thing is the ongoing innovation in languages and platforms.
JB 2016-January-21
At least it’s not Salesforce.
NodejsEnemy 2016-March-8
NODEJS IS A JOKE
I agree and définitly hatte Nodejs.
Too much hack, and unreadable code.
Some says it s cool and I say that is shit.
Want to do real programming whith async capabilities use go or python asyncio which are realy better désigner.
Zvonko 2016-March-31
Hello Dino,
on the side bar I read that you do dotNET.
So check for this file:
C:\Windows\Microsoft.NET\Framework\v4.0.30319\jsc.exe
(or any other dot.net version you have installed)
I make my system batch scripts now only as EXE 😉
Cheerio!
Tom 2016-April-5
This thread is filled with a bunch of people who have no idea how to properly code async patterns in nodeJS. End of.
Mike 2018-January-9
Regardless, learning to use the wrong tool better is not the correct answer. Coding async patterns in nodejs for system scripting is stupid. Learn to use the proper tool for your task or be doomed to low productivity and frustration.