Unit Testing Licenses: Monitoring the legality of your node_modules with Jest

Packages from the npm registry are the heart and soul of most larger JavaScript projects. Libraries, frameworks, tooling, etc. - all of our projects stand on the shoulders of giants open source projects. Writing and maintaining all that code in-house would be impossible, even for the biggest companies out there. But there is a price to pay, and supply chain attacks are just one of the risks of using third-party code. In a commercial context, you must always be aware of the legal implications of open source licenses. There are a myriad of different licenses, and they all allow different kinds of use. I will explicitly not go into the details (I am not a lawyer, you should always consult legal professionals for these matters), but it is important to keep track of the licenses you use in your projects. Not every open source license is suitable for every use case, and we have to be aware of this!

• We want to allow our clients (or their legal team) to have an overview of all the licenses we use in their product (often a contractual obligation).
• We want to avoid certain licenses, so we need to prevent developers from unknowingly adding them.
• We want to know if a package has changed its license for some reason, and then check if the new license is still allowed.

Today I want to show how we can address these problems by making licenses part of the test suite. This test suite could be run on every proposed change to the codebase (e.g. as a GitHub action on every Pull Request), so that an unwanted change in licensing will cause our CI to fail early.

Let's say we have a TypeScript project that already has a few unit tests in place. It uses Jest / ts-jest for that, but you can probably adapt the idea for any other test framework. If you're new to Jest, they have an excellent setup guide. ts-jest also has you covered if you want to use TypeScript.

To easily compile the information we want, we are going to use license-checker-rseidelsohn, which gathers the packages from our package file and then extracts license and author information from node_modules. It's then able to generate a list in multiple formats -- in our case, markdown. This package is also handy because it is able to provide a CLI as well as exports for programmatic use. Here's what a CLI run could look like (--direct means: only include direct dependencies, not their transitive dependencies):

→ npx license-checker-rseidelsohn --markdown --direct

We can now write the result to a markdown file (→ npx license-checker-rseidelsohn --markdown > LICENSES.md). If this file is in the repo of our project, GitHub will render pretty HTML for us. For 99% of our projects this will solve use case #1, our client's legal team gets access to that GitHub URL and we're done. To keep the file up to date we can also use a GitHub action like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
name: "🦖 update license-reports"

on:
  push:
    branches:
      - "main"

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: 👀 npm install
      run: npm install
    - name: 🦖 generate report
      run: npx license-checker-rseidelsohn --markdown --direct > LICENSES.md
    - name: ✨ create Pull Request
      uses: peter-evans/create-pull-request@v4
      with:
        commit-message: "chore: update license report"
        title: "chore: update license report"
        body: "# 🦖"
        base: main
        branch: cloud-update-advanced-params
        delete-branch: true

This will open a PR when the LICENSES.md needs to change.

That was easy. Now to the fun part:

Let's put all of the other checks inside a new unit test, let's call it licenses.test.ts (again, we're using TypeScript, which is not necessary for this, if you don't want to). In this test we need to do a few things:

First, let's import the checker and promisify it, so we can use async/await later:

1
2
3
4
import { init } from "license-checker-rseidelsohn";
import { promisify } from "util";

const checkLicenses = promisify(init);

The result from the checker is an object with a packageName@version as its keys and some more information as its values:

1
2
3
4
5
6
7
8
9
10
{
  …
  'react@18.2.0': {
    licenses: 'MIT',
    repository: 'https://github.com/facebook/react',
    path: 'my-project/node_modules/react',
    licenseFile: 'my-project/node_modules/react/LICENSE'
  },
  …
}

Since we don't need all of it, yet the version number is in the key, we should probably tidy it up before we use it. Let's do all of that in beforeAll:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
import { init } from "license-checker-rseidelsohn";
import { promisify } from "util";

const checkLicenses = promisify(init);

type Info = { name: string; licenses: string[] };

let packages: Array<Info>;

beforeAll(async () => {
	const rawResult = await checkLicenses({
		// start at the root of our project
		start: ".",

		// we don't need to check our own license
		excludePackages: "blog-license-check-demo",

		// There's a bug in license-checker-rseidelsohn
		// => https://github.com/RSeidelsohn/license-checker-rseidelsohn/issues/35
		//  🚨 this enables only checking direct dependencies!
		direct: 0 as any,
	});

	packages = Object.entries(rawResult).map(([rawName, result]) => {
		// e.g. @foo/bar@1.2.3 => ["", "foo/bar", "1.2.3"]
		const parts = rawName.split("@");
		const name = parts.slice(0, -1).join("@"); // @foo/bar

		// license-checker supports multiple licenses per package:
		const licenses = result.licenses ?? ["UNLICENSED"];

		return {
			name,
			licenses: Array.isArray(licenses) ? licenses : [licenses],
		};
	});
	// License compilation might take a bit, so we don't want to run into Jest's 5s timeout
}, 30000);

As a first test, let's find out if any current packages use disallowed licenses. As an example, I will only allow ["UNLICENSED", "MIT", "BSD-3-Clause", "Apache-2.0"]. The actual test uses a custom matcher, so we can control the output of failing test messages better. We also need to tell TypeScript what to expect from our custom matcher.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
const LICENSE_ALLOW_LIST = ["UNLICENSED", "MIT", "BSD-3-Clause", "Apache-2.0"];

expect.extend({
	// using a custom matcher for better output when a test fails
	toBeValidLicense: ({ licenses, name }: Info) => {
		// 🚨 this is the important part:
		// pass this test, when every license of the package is in the allow list
		const pass = licenses.every(
			(license) => license && LICENSE_ALLOW_LIST.includes(license)
		);

		return {
			pass,
			// display this message with enough info
			message: () =>
				`"${licenses.join(",")}" is ${
					pass ? "" : "not "
				}an allowed license (dependency: ${name})`,
		};
	},
});

// This could also go in a .d.ts file, I'll keep it here for simplicity
declare global {
	namespace jest {
		interface Matchers<R> {
			toBeValidLicense(): R;
		}
	}
}

Now the first test is very simple:

1
2
3
test("all licenses are allowed", async () => {
	packages.forEach((info) => expect(info).toBeValidLicense());
});

This test will run every package against our custom matcher and fail, if its license(s) is not in the allow list. This solves our use case #2.

Now for our second test, we want to future-proof the set of different licenses in the project in order to detect if a package was added, removed or had its license changed. We can do this using a snapshot test:

1
2
3
4
5
6
test("snapshot package licenses", async () => {
	expect(
		packages.map(({ licenses, name }) => `${name}: ${licenses}`)
	).toMatchSnapshot();
});

For those unaware of what a snapshot test is: when it is executed for the first time, it will save a current snapshot of the result to a file __snapshots__/licenses.test.ts.snap. This file must then be checked into the repository. Everytime the test runs in the future, its result will be compared to that snapshot. If it changed, the test will fail and Jest will tell you.

Alright, let's run our tests: → npm run test

Everything seems good, all our packages honor our LICENSE_ALLOW_LIST and we now have a snapshot file, that we can commit:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`snapshot package licenses 1`] = `
[
  "@types/jest: MIT",
  "@types/node: MIT",
  "@types/react-dom: MIT",
  "@types/react: MIT",
  "@vitejs/plugin-react: MIT",
  "jest: MIT",
  "license-checker-rseidelsohn: BSD-3-Clause",
  "react-dom: MIT",
  "react: MIT",
  "ts-jest: MIT",
  "typescript: Apache-2.0",
  "vite: MIT",
]
`;

Let's see what happens when we add a dependency: → npm install @react-hookz/web → npm run test

From this output we can see, that @react-hookz/web's license is allowed (the first test passed!) but it's not in the snapshot yet. Let's fix that by running npm run test -- -u:

Let's add another dependency: → npm install rimraf → npm run test.:

Uh oh! rimraf uses the ISC license, which is not in our allow list and our first test caught that: "ISC" is not an allowed license (dependency: rimraf). We can now decide if we want to remove this package and use something else or change the allow list of the test.

This should do the trick! All 3 use cases are covered by a few lines of TypeScript and YAML. You can check out a fully working example repo (including GitHub actions) here: https://github.com/peerigon/blog-license-check-demo

If you only want the test file, here you go:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
import { init } from "license-checker-rseidelsohn";
import { promisify } from "util";

const checkLicenses = promisify(init);

type Info = { name: string; licenses: string[] };

let packages: Array<Info>;

beforeAll(async () => {
	const rawResult = await checkLicenses({
		// start at the root of our project
		start: ".",

		// we don't need to check our own license
		excludePackages: "blog-license-check-demo",

		// There's a bug in license-checker-rseidelsohn
		// => https://github.com/RSeidelsohn/license-checker-rseidelsohn/issues/35
		//  🚨 this enables only checking direct dependencies!
		direct: 0 as any,
	});

	packages = Object.entries(rawResult).map(([rawName, result]) => {
		// e.g. @foo/bar@1.2.3 => ["", "foo/bar", "1.2.3"]
		const parts = rawName.split("@");
		const name = parts.slice(0, -1).join("@"); // @foo/bar

		// license-checker supports multiple licenses per package:
		const licenses = result.licenses ?? ["UNLICENSED"];

		return {
			name,
			// license-checker supports multiple licenses per package:
			licenses: Array.isArray(licenses) ? licenses : [licenses],
		};
	});
	// License compilation might take a bit, so we don't want to run into Jest's 5s timeout
}, 30000);

const LICENSE_ALLOW_LIST = ["UNLICENSED", "MIT", "BSD-3-Clause", "Apache-2.0"];

expect.extend({
	// using a custom matcher for better output when a test fails
	toBeValidLicense: ({ licenses, name }: Info) => {
		// 🚨 this is the important part:
		// pass this test, when every license of the package is in the allow list
		const pass = licenses.every(
			(license) => license && LICENSE_ALLOW_LIST.includes(license)
		);

		return {
			pass,
			// display this message with enough info
			message: () =>
				`"${licenses.join(",")}" is ${
					pass ? "" : "not "
				}an allowed license (dependency: ${name})`,
		};
	},
});

// This could also go in a .d.ts file, I'll keep it here for simplicity
declare global {
	namespace jest {
		interface Matchers<R> {
			toBeValidLicense(): R;
		}
	}
}

test("all licenses are allowed", async () => {
	packages.forEach((info) => expect(info).toBeValidLicense());
});

test("snapshot package licenses", async () => {
	expect(
		packages.map(({ licenses, name }) => `${name}: ${licenses}`)
	).toMatchSnapshot();
});