Overriding development shell's dependencies using Nix
17-Nov-2022
I wanted to write an article in this blog after I left it for a while. This blog is written using GatsbyJS (a Headless-CMS), so I cloned the repo to my computer.
I immediately run
yarn install
Quick intro to Nix and Nix flakes
I used Nix for my devops works to pin CLI dependencies, so it can be reproducible. But, I’m not really familiar with Javascript stack, like Webpack, Babel, Yarn, etc. So, I never considered to make a Nix recipe for this.
Standard Nix development setup usually requires you to define
default.nix
shell.nix
cd
nix-shell
As of today, Nix has reached version 2.11 which includes experimental support for reproducible build environment called Nix Flakes. You need to enable it first to use it. The usage is quite widespread now. Since Nix Flakes and Nix itself doesn’t tamper each other, generally it is safe to try it out, even if it is labeled “experimental”. The thing about Nix and Nix Flakes is that you can easily rollback if you mess up. Or you can even pin Nix Flakes dependencies. So there is no real risk of making your setup broke.
So, let’s try Nix Flake
Setting up Nix Flake
There is a Nix Flake template provided by NixOS repo. To get it, you can just execute
nix flake init
It looks like this
{
description = "A very basic flake";
outputs = { self, nixpkgs }: {
packages.x86_64-linux.hello = nixpkgs.legacyPackages.x86_64-linux.hello;
packages.x86_64-linux.default = self.packages.x86_64-linux.hello;
};
}
Basically it defines
inputs
outputs
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
};
}
I’m tracking a
nixpkgs-unstable
unstable
Next, I need
node
yarn
outputs
Using Numtide’s Devshell
I will call this Devshell from now on to refer to Numtide’s Devshell config. This is because the original Flake outputs development shell keys are also called
devShell
Anyway. I configured my original flake into:
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
flake-utils.url = "github:numtide/flake-utils";
flake-compat = {
url = "github:edolstra/flake-compat";
flake = false;
};
devshell.url = "github:numtide/devshell";
};
outputs = { self, nixpkgs, flake-utils, devshell, ... }:
flake-utils.lib.eachDefaultSystem (system: {
devShell =
let
pkgs = import nixpkgs {
inherit system;
overlays = [ devshell.overlay ];
};
in
pkgs.devshell.mkShell {
name = "maulana.id";
};
});
}
To explain it bit by bit. I used
flake-utils
x86_64-linux
x86_64-darwin
Since
nodejs
For each system, I defined
devShell
shell.nix
The definition of the
devShell
pkgs.devshell.mkShell
So this is the bare minimum. To test it, run
nix develop
flake.nix
Defining packages to install in development shell
Obviously our own core problem is to have
node
yarn
pkgs.devshell.mkShell {
name = "maulana.id";
packages = with pkgs; [ nodejs yarn ];
}
If I
nix develop
node
yarn
commands
pkgs.devshell.mkShell {
name = "maulana.id";
commands = [
{
name = "yarn";
package = pkgs.yarn;
}
{
name = "node";
package = pkgs.nodejs;
}
];
}
This has the same effect, but when you enter the shell, it will lists all the available commands. Very neat, I must say.
Declaring shell’s environment variables
After getting node and yarn, I ran
yarn install
yarn develop
Oops, what did I do wrong?
I see a significant error message:
error: error:0308010c:digital envelope routines::unsupported
Seems like an openssl thing, based on the error format. A quick google proved me right. I then found this stackoverflow link.
Basically it says that node can’t work with the recent openssl. It suggests to add a
NODE_OPTIONS=--openssl-legacy-provider
pkgs.devshell.mkShell {
name = "maulana.id";
commands = [
{
name = "yarn";
package = pkgs.yarn;
}
{
name = "node";
package = pkgs.nodejs;
}
];
env = [
{
name = "NODE_OPTIONS";
value = "--openssl-legacy-provider";
}
]
}
The error is gone. But I got another error when running
yarn
develop
gatsby-plugin-postcss
I’m super noob in Javascript, so I could not debug it further. I am feeling sad because I originally intend to write an article… But why I’m stuck in this rabbit hole???
Using different version of the same Nix package
I remembered that my site worked when I am using Node v14. So, I decided to change node version. It’s quite easy with Nix. Especially when the nixpkgs repo already prepare a package definition for Node v14 themselves.
Using the new Nix flake based command, I searched the package:
# nixpkgs is the name of the registry
# nodejs is the name of the package
nix search nixpkgs nodejs
Apparently, the package name is
nodejs-14_x
commands = [
{
name = "node";
package = pkgs.nodejs-14_x;
}
]
Rebuilding the flake, and I can see the version is indeed 14, using
node --version
But Gatsby still produce the same error… What else were wrong?
After thinking about it a little. I remembered that
yarn
node
node
node
yarn
To test the idea, I run
yarn node --version
Overriding Nix package attributes
We have arrived to the main focus and the meat of this article.
To fix this, we have to tell Nix yarn package to use Nix node package of version 14.
I go to https://search.nixos.org and searching for yarn. The URL looks like this:
https://search.nixos.org/packages?channel=unstable&from=0&size=50&sort=relevance&type=packages&query=yarn
There is a source button, and I click that, and it will redirect me to the Nix recipe in GitHub: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/tools/yarn/default.nix#L25
This is a Nix function, and at the top is where the parameter is declared. The parameter is an attribute. One of the attribute is
nodejs
{
name = "yarn";
package = pkgs.yarn.override {
nodejs = pkgs.nodejs-14_x;
};
}
After rebuilding the flake,
yarn node --version
But we hit another problem now.
NODE_OPTIONS=--openssl-legacy-provider
Then it occured to me, can we just use OpenSSL v1 instead of OpenSSL v3 (the new one) for node 14?
So we override the attribute again. Repeating the same process, I can see that
nodejs
openssl
nix search nixpkgs openssl
openssl_1_1
My final override for
yarn
{
name = "yarn";
package = pkgs.yarn.override {
nodejs = pkgs.nodejs-14_x.override {
openssl = pkgs.openssl_1_1;
};
};
}
I also used the same method for the
node
{
name = "node";
package = pkgs.nodejs-14_x.override {
openssl = pkgs.openssl_1_1;
};
}
Since both were supposed to use the same nodejs version, I define a new variable to avoid mistake on having to change it in just one place in the future. Basically, we will use DRY principle.
My flake ends up like this:
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
flake-utils.url = "github:numtide/flake-utils";
flake-compat = {
url = "github:edolstra/flake-compat";
flake = false;
};
devshell.url = "github:numtide/devshell";
};
outputs = { self, nixpkgs, flake-utils, devshell, ... }:
flake-utils.lib.eachDefaultSystem (system: {
devShell =
let
pkgs = import nixpkgs {
inherit system;
overlays = [ devshell.overlay ];
};
customNodejs = pkgs.nodejs-14_x.override {
openssl = pkgs.openssl_1_1;
};
in
pkgs.devshell.mkShell {
name = "maulana.id";
commands = [
{
name = "yarn";
package = pkgs.yarn.override {
nodejs = customNodejs;
};
}
{
name = "yarn2nix";
package = pkgs.yarn2nix;
}
{
name = "node";
package = customNodejs;
}
];
packages = [ pkgs.openssl_1_1 ];
env = [];
};
});
}
Everything works now.
Automating shell hooks with direnv (spicing things up)
Since my Nix setup already includes
direnv
nix-direnv
direnv
cd
I created a file called
.envrc
#!/usr/bin/env bash
use flake
This will make direnv look for
flake.nix
You can test it out if you have Nix Flake enabled
Things that made Nix Flake very compelling is that you can use the same flake shared by others.
In this example, I made a Nix flake to describe devShell. I shared it in my github repo. Since it is publicly available, you can test it out.
You can
cd
/tmp/test
nix develop github:lucernae/maulana.id
The syntax is fairly straightforward.
nix develop
github:lucernae/maulana.id
Yes that’s right, it can be from a filesystem, https URL, or special URI like github (because it is very common for people to store their flakes in GitHub).
There are a couple of new Nix command that is quite interesting like
nix build
nix run
These will be stories for another day.
Summary
After a little bit of roundtrip, we finally able to provide a devshell using Nix, Devshell, and Direnv.
I believe the journey and stories to get here is probably useful for new tinkerer when trying out Nix.
Feel free to look at this example flake I used in this blog’s repo
I didn’t dive in further to pin the node dependencies. I’m not familiar with node js, so this is suffice.
I also proceed to do what I wanted to do earlier, which is writing a new article…
But perhaps after this one article :D