Nodejs script using Nix-Shell shebangs
19-Nov-2022
OK, this incident is so funny for me, I decided to make another blog posts. Lol.
Before getting to the main part, I have to explain about shebangs and Nix-shell.
If you already knew about this, you can skip to the main part
The shebang
For those who don’t know, shebangs is this little pun we have in the context of shell scripting in Unix-like environment. When you open an executable shell script, most often it will have these character at the beginning of the line:
#!
I don’t know exactly how does it originates, but “shebang” as an English word means some kind of “shady operation” for me. Because I knew the term largely associated with mafia movies, like the sentence “running the whole shebang”.
What makes it a good pun, is that Unix interpreter originally called a “shell”. The program is called “sh” as a short of “shell”. It is usually located at
/bin/sh
#
#
#!
The shebang is important, because it unambiguously tell the program executor to switch to another program to execute the following script. For example, consider the following line at the beginning of a script.
#!/bin/sh
A GNU-Linux distribution might have different shell interpreters, according to the users’ current profile/choice. When I open terminal, it is possible that I run bash as my main shell. When bash reads the first line/character, it reads a
#
!
/bin/sh
/bin/sh
Well perhaps, historically the
sh
sh
Over time, we have many Linux distros and they have their own preferred opinion on where to put binary programs. The shells might be located in
/bin
/usr/bin
/usr/local/bin
env
/usr/bin/env
env
env
PATH
#!/usr/bin/env python
That means
env
python
python
env
python
#!
/usr/bin/env python
It is a very neat and clever tricks of Unix-like systems.
Nix-Shell as shebang interpreter
Nix package manager can install and manage packages without each package conflicting. It solves the issue by installing each programs in a very specific location, only computed by the hash of the program input at build time.
For example, Nix install the
sh
/nix/store/45mbffhkf7wr5av8jwhk7cc4ghh3cwx1-bash-interactive-5.1-p16/bin/sh
45mbffhkf7wr5av8jwhk7cc4ghh3cwx1
But then, how do we write shebang for these packages? The real location of the program might be different for each computer systems. Nix solves this by symlinking the programs and include it in the Nix PATH variable, which is computed when you enter the shell and activate Nix profile. That way, even if you install two different python3 version with the same interpreter name (which is python), it won’t conflict.
If you built a shell script using Nix, you can specify the location of the interpreter. So the shebang may look like this:
#!/nix/store/45mbffhkf7wr5av8jwhk7cc4ghh3cwx1-bash-interactive-5.1-p16/bin/sh
But how does Nix handles scripts that is not built by Nix?
For systems that only uses Nix as package manager, the host system already has basic shell and env program in the standard location. So Nix won’t interfere with that.
For systems that is managed directly by Nix, such as NixOS, it maintain compatibility by symlinking the current profile path to the path expected by other systems. To illustrate, NixOS will symlink
/nix/store/45mbffhkf7wr5av8jwhk7cc4ghh3cwx1-bash-interactive-5.1-p16/bin/sh
/run/current-system/sw/bin/sh
/bin/sh
/usr/bin/env
Now, we get to explain the “next level” part. Hahaha.
Nix has it’s own special shell binary called nix-shell. This shell can act as a standard shell. You know, like sh or bash. Then, it also can evaluate nix expression (duh, of course that’s why it is called nix-shell). Besides that, it can also manipulate nix profile “on the fly” as you execute it.
To explain how mindblowing this is. Suppose that you haven’t installed python in your system. You want to try it out, but you don’t want to install it. With Nix-Shell, you can run it like this:
nix-shell -p python3
Maybe it took some time to download the package at first (then store it in cache). Then the shell prompt that appears has
python3
If you need several packages, you can list it out
nix-shell -p python3 -p kubernetes-helm -p kubectl
This is especially useful when you have certain package version already installed, but you want to shadow it with newer packages to test it out. It saves me in the past when I was testing python2 to python3 migration and helm 2 to helm 3 migration.
It is also possible with Nix-Shell to jump to an interpreter of your choice. Executing this will make you jump into a Lua REPL prompt, instead of regular shell prompt.
nix-shell -p lua --command lua
Now, let’s get back to the “shebang business”. Nix-shell has it’s own special behaviour with a shebang. First, take a look at this example:
#!/usr/bin/env nix-shell
#!nix-shell -p python3 -i python
print("hello from python\n")
The first line is our standard shebang. It calls
env
nix-shell
In short, when you run this file in Nix-capable system, it behaves as if it is a Python script.
It is a polyglot interpreter!
nodeJS in Nix-Shell
Now that you understand how shebang and Nix-Shell works, let’s get back in business.
I made a file like this:
#!/usr/bin/env nix-shell
#!nix-shell -p nodejs -i node
console.log('hello')
The actual content is not “console.log”, but you get idea. I was trying to make a script for this blog, which uses nodeJS. To my surprise, the above script doesn’t work. It threw an error
We can realize immediately that nodejs threw error because it can’t parse
#
#
Then it occured to me that I never used nix-shell with nodejs before. I used to switch from bash to python. In python,
#
So, what if we modify the shebangs to be commented in nodejs? I’m curious and modify it as this, using
/* */
#!/usr/bin/env nix-shell
/*
#!nix-shell -p nodejs -i node
*/
console.log('hello')
And it does work!