Alexander Morse

Sometimes, you can work with a tool for years and not understand all of its features. And if you happen to be me, some features don't even need to be all that obscure — they might stare you in the face for years before you finally notice them.

npm's support for package-defined executables is one such feature. It makes setting up development environments incredibly easy, and nearly all my own Javascript projects make heavy use of it. It's so transparent and intuitive that you almost don't need to understand how it all works. So I didn't.

(Note: this article assumes a basic knowledge of the npm package management tool for Javascript. If this isn't in your skill set, don't worry — it's barely in mine. If you're interested, you can read this article from FreeCodeCamp for an overview on the major features: What is npm? A Node Package Manager Tutorial for Beginners)

In case it’s not clear what I’m talking about, I mean the ability for npm packages to define their own executables — usually in the form of some CLI interface. We’ll use the package uglify-js as an example. This feature lets us install the package, like so:

npm install uglify-js

...and then invoke the uglifyjs CLI through an npm script, like so:

{
  "name": "my-app",
  "version": "0.0.1",
  "scripts": {
    "uglify": "uglifyjs input.js --compress --mangle"
  }
}

npm run-script uglify
# (Processes input.js and prints it to the console.)

When you execute the above command, it runs just as you'd expect. It works great. You can even write multiple scripts that do different things, and then write a manager-script that calls each of them in sequence, to build yourself a robust task-runner.

The feature is so handy that it’s easy to overlook the important question of why it works at all. Initially you might get the sense that it shouldn’t work — after all, packages are just collections of Javascript files, so where is a CLI coming from? For that matter, executing uglifyjs on the command line won’t work (unless you’ve installed it globally), so it doesn’t look as if anything has been added to our environment’s PATH. How is npm able to use these commands when we seemingly can’t?

Thankfully, we only really need to understand two things: The ‘bin’ field in package files, and some specifics of npm’s run-script command.

All About the "bin" Field

It turns out that creating an executable in npm in embarassingly easy — you can define it with a single field in your package.json file: specifically the "bin" field. Here's an example:

{
  "name": "my-cli-tools",
  "version": "0.0.1",
  "bin": {
    "my-cli": "./bin/my-cli",
    "my-other-cli": "./bin/my-other-cli"
  }
}

That's literally all there is to it. In fact, you'll notice this particular package defines two CLI commands, and you’ll have access to both of them when the package is installed. In other words, if you ever want to know if a package comes with executables, just look in its package.json for the "bin" field.

(Actually, there are a couple other ways these might be defined. If there’s only one command, and that command has the same name as the package, then "bin" is allowed to be set to a single string designating the executable file. Alternatively, a project with a lot of executables might set "directories.bin" to specify an entire directory of commands. But in most cases, "bin" will be set to an object or string.)

By the way, an executable command looks something like this:

#!/usr/bin/env node
// Note - the above shebang line is required to make the command work


const args = process.argv;
console.log(`I'm a CLI, and I was called with the arguments: ${args}`);

Aside from the required shebang line, there’s nothing special about this - it’s just Node. If you want to learn more about building actual CLIs, I found this particular tutorial by Dominik Kundel easy to follow: How to build a CLI with Node.js

How npm Calls Executables

So, we know that packages can define their own executables. But we still can't call them from the command line, so how exactly is npm doing it when we run a package script?

You might have guessed that npm is modifying the path, and you’d be right. Specifically, npm adds a secret folder of scripts any time you use npm run. It’s located at <project_directory>/node_modules/.bin. For example, this is what the folder looks like after we install a package with an executable:

npm install uglify-js
ls node_modules/.bin
# -> uglifyjs*  uglifyjs.cmd  uglifyjs.ps1*

If you examine these files, you’ll notice they’re just helper scripts set up by npm — the executables themselves still live inside their packages — but it’s enough to make them usable when we run something like npm run uglify.

So that's the story. To sum up:

• Packages define executables through the "bin" field in their package.json file.
• npm maintains the node_modules/.bin directory to use when running package scripts.

But you know something? I still can't use a locally-defined package executable on the command line. To do that, I still need to edit the package file, add a script, and then call that script. That's a lot of extra steps. How unfair is that?

Calling Executables with npx

Let's say we want to call a locally-installed package executable, like uglifyjs, directly from the command line.

Right off the bat, there’s the obvious solution — just call the executable directly:

node_modules/.bin/uglifyjs <args>

We could also invoke the ‘real’ executable inside the package itself. Something like:

node node_modules/uglify-js/bin/uglify-js <args>

Both of these work, but you might get the sense that it’s not the best solution. Instead, we can use npx.

npx can be another one of those slightly-arcane tools — it’s easy enough to use, but not always self-explanatory. Worse, most explanations online tend to focus on the more flashy features, like its ability to call executables from packages that aren’t even installed. For our purposes, though, we’ll keep it simple: npx allows us to call locally-installed package executables. Therefore, we simply run:

npx uglifyjs <args>

That's it.

Okay, things do get a little more complicated if uglify-js isn't locally installed. In this case, npx would go looking online for the right command to run, but it wouldn't work since the package name doesn't match the executable. You could help npx out by specifying the package name:

npx -p uglify-js uglifyjs <args>

Beyond this, a full exploration of npx would require its own article.

In Conclusion

This has been a Passable Explanation of npm's package executables. It's not particularly complex, nor immediately useful to anyone who just wants to use the tools provided by the platform, but it's still a foundational feature of the technology, and worth over-thinking at least once.