npm-registry-firewall 📦📦🔥

npm registry proxy with on-the-fly filtering

Key Features

Restricts access to remote packages by predicate:
- name
- org
- version range
- license type
- dateRange
- age
- username
- custom filter function
- vulnerability level via builtin npm-registry-firewall/audit plugin
Flexible configuration: use presets, plugins and define as many server/context-path/rules combinations as you need.
Extendable. expressjs-inspired server implementation is under the hood.
Standalone. No clouds, no subscriptions.
Linux / Windows / macOS compatible.
~~Supports Bun Runtime.~~ Not yet.
Has no deps. Literally zero.

Motivation

To mitigate security and legal risks

Open Source is essential for modern software development. According to various estimates, at least 60% of the resulting codebase is composed of open repositories, libraries and packages. And it keeps growing. Synopsys OSSRA 2021 report found that 98% of applications have open source dependencies.

But open does not mean free. The price is the risk that you take:

Availability
Security
Legality / license

Let's consider these problems in the context of the JS universe.

Availability risks

JS packages are distributed in various ways: git repos, cdns and package registries. Regardless of the method, there are only two entry types that are finally resolved by any pkg manager: git-commit pointers and tarball links.

"dependencies": {
  "yaf" : "git://github.com/antongolub/yarn-audit-fix.git#commit-hash",
  "yaf2": "antongolub/yarn-audit-fix",
  "yarn-audit-fix" : "*"
}

yaf2@antongolub/yarn-audit-fix:
  version "9.2.1"
  resolved "https://codeload.github.com/antongolub/yarn-audit-fix/tar.gz/706646bab3b4c7209596080127d90eab9a966be2"
  dependencies:
    "@types/find-cache-dir" "^3.2.1"
    "@types/fs-extra" "^9.0.13"

"node_modules/yaf": {
  "name": "yarn-audit-fix",
  "version": "9.2.1",
  "resolved": "git+ssh://[email protected]/antongolub/yarn-audit-fix.git#706646bab3b4c7209596080127d90eab9a966be2",
  "license": "MIT",

"node_modules/yarn-audit-fix": {
  "version": "9.2.1",
  "resolved": "https://registry.npmjs.org/yarn-audit-fix/-/yarn-audit-fix-9.2.1.tgz",
  "integrity": "sha512-4biFNP4ZLOHboB2cNVuhYyelTFR/twlfmGMQ2TgJgGRORMDM/rQdQqhJdVLuKvfdMLFEPJ832z6Ws5OoCnFcfA==",
  "dependencies": {

So the implementation of mirroring is fundamentally quite simple: we just need to save and expose these assets from an alternative ssh/https entry point. Luckily this has already happened. The main repository for JS code is registry.npmjs.org. And at least 5 public replicas are always available as alternatives:

If this reliability level is not enough, you can easily run one more registry:

Security risks

Any code may not work properly. Due to error or malice. Keep in mind that most OSS licenses exclude any liability for damages. It's also important to always remember that oss code is not verified before being published. These two circumstances sometimes give rise to dangerous incidents like colors.js or node-ipc.

The independent audit process is expensive, time consuming, so only setting a delay before using new pkg version might be effective countermeasure.

Legal risks

License agreement is an attribute of the moment: it can suddenly change and affect the development process (for example, husky-5). Uncontrolled use of new versions may have legal and financial consequences. Therefore, automated license checks should be part of CI/CD pipeline or the registry's own feature.

Implementation notes

The proxy intercepts packuments and tarball requests and applies the specified filters to them:

Removes all forbidden entries from packuments.
Returns 404 if the filtered packument versions are empty.
Returns 404 if registy/[email protected] does not satisfy the policies.
Passes any other requests to the remote registry as is.

Requirements

Node.js >= 14

Install

# npm
npm i npm-registry-firewall

# yarn
yarn add npm-registry-firewall

Usage

CLI

npm-registry-firewall /path/to/config.json

JS API

import {createApp} from 'npm-registry-firewall'

const app = createApp({
  server: {
    host: 'localhost',
    port: 3001,
  },
  firewall: {
    registry: 'https://registry.npmmirror.com',
    rules: [
      {
        policy: 'allow',
        org: '@qiwi'
      },
      {
        policy: 'deny',
        name: '@babel/*,react@^17'  // All @babel-scoped pkgs and react >= 17.0.0
      },
      {
        policy: 'allow',
        filter: ({name, org}) => org === '@types' || name === 'react'  // may be async
      },
      {
        plugin: [['npm-registry-firewall/audit', {
          critical: 'deny',
          moderate: 'warn'
        }]]
      },
    ]
  }
})

await app.start()

TS libdefs

Included

type LetAsync<T> = T | Promise<T>

type TApp = {
  start: () => Promise<void>
  stop: () => Promise<void>
}

type TLogger = typeof console

type TServerConfig = {
  host?: string
  port?: string | number
  base?: string
  healthcheck?: string | null
  metrics?: string | null
  secure?: {
    key: string,
    cert: string
  }
  requestTimeout?: number
  headersTimeout?: number
  keepAliveTimeout?: number
  extend?: string
}

type TPolicy = 'allow' | 'deny' | 'warn'

type TRule = {
  policy?: TPolicy
  name?: string | RegExp | Array<string | RegExp>
  org?: string | RegExp | Array<string | RegExp>
  dateRange?: [string, string]
  age?: number | [number] | [number, number]
  version?: string,
  license?: string | RegExp | Array<string | RegExp>
  username?: string | RegExp | Array<string | RegExp>
  filter?: (entry: Record<string, any>) => LetAsync<boolean | undefined | null>
  extend?: string
  plugin?: TPluginConfig
}

type TPluginConfig = string | [string, any] | TPlugin | [TPlugin, any]

type TCacheConfig = {
  ttl: number
  evictionTimeout?: number
  name?: string
}

type TCacheImpl = {
  add(key: string, value: any, ttl?: number): LetAsync<any>
  has(key: string): LetAsync<boolean>
  get(key: string): LetAsync<any>
  del(key: string): LetAsync<void>
}

type TCacheFactory = {
  (opts: TCacheConfig): TCacheImpl
}

type TFirewallConfig = {
  registry: string
  entrypoint?: string
  token?: string
  base?: string
  rules?: TRule | TRule[]
  cache?: TCacheConfig | TCacheImpl | TCacheFactory
  extend?: string
}

type TConfig = {
  server: TServerConfig | TServerConfig[]
  firewall: TFirewallConfig
  extend?: string
}

type TValidationContext = {
  options: any,
  rule: TRule,
  entry: Record<string, any>
  boundContext: {
    logger: TLogger
    registry: string
    authorization?: string
    entrypoint: string
    name: string
    org?: string
    version?: string
  }
}

type TPlugin = {
  (context: TValidationContext): LetAsync<TPolicy>
}

type TAppOpts = {
  logger?: TLogger
  cache?: TCacheFactory
}

export function createApp(config: string | TConfig | TConfig[], opts?: TAppOpts): Promise<TApp>

type TLoggerOptions = {
  extra?: Record<string, any>,
  formatter?: (logCtx: {level: string, msgChunks: string[], extra: Record<string, any>}) => string
}

export function createLogger(options: TLoggerOptions): TLogger

Config

{
  "server": {
    "host": "localhost",        // Defaults to 127.0.0.1
    "port": 3000,               // 8080 by default
    "secure": {                 // Optional. If declared serves via https
      "cert": "ssl/cert.pem",
      "key": "ssl/key.pem"
    },
    "base": "/",                // Optional. Defaults to '/'
    "healthcheck": "/health",   // Optional. Defaults to '/healthcheck'. Pass null to disable
    "metrics": "/metrics",      // Optional. Uptime, CPU and memory usage. Defaults to '/metrics'. null to disable
    "keepAliveTimeout": 15000,  // Optional. Defaults to 61000
    "headersTimeout": 20000,    // Optional. Defaults to 62000
    "requestTimeout": 10000     // Optional. Defaults to 30000
  },
  "firewall": {
    "registry": "https://registry.npmmirror.com",  // Remote registry
    "token": "NpmToken.*********-e0b2a8e5****",    // Optional bearer token. If empty req.headers.authorization value will be used instead
    "entrypoint": "https://r.qiwi.com/npm",        // Optional. Defaults to `${server.secure ? 'https' : 'http'}://${server.host}:${server.port}${route.base}`
    "base": "/",                // Optional. Defaults to '/'
    "cache": {                  // Optional. Defaults to no-cache (null)
      "ttl": 5,                 // Time to live in minutes. Specifies how long resolved pkg directives will live.
      "evictionTimeout": 1      // Cache invalidation period in minutes. Defaults to cache.ttl.
    },
    "extends": "@qiwi/internal-npm-registry-firewall-rules",  // Optional. Populates the entry with the specified source contents (json/CJS module only)
    "rules": [
      {
        "policy": "allow",
        "org": "@qiwi"
      },
      {
        "policy": "allow",
        "name": ["@babel/*", "@jest/*", "lodash"] // string[] or "comma,separated,list". * works as .+ in regexp
      },
      {
        "policy": "warn",       // `warn` directive works like `allow`, but also logs if someone has requested a tarball matching the rule
        "name": "reqresnext"
      },
      {
        "policy": "deny",
        "extends": "@qiwi/nrf-rule",  // `extends` may be applied at any level, and should return a valid value for the current config section
      },
      {
        "plugin": ["npm-registry-firewall/audit", {"moderate": "warn", "critical": "deny"}]
      },
      {
        "policy": "deny",
        "name": "colors",
        "version": ">= v1.4.0"  // Any semver range: https://github.com/npm/node-semver#ranges
      },
      {
        "policy": "deny",
        "license": "dbad"       // Comma-separated license types or string[]
      },
      {
        "policy": "allow",
        "username": ["sindresorhus", "isaacs"] // Trusted npm authors.
      },
      {
        "policy": "allow",
        "name": "d",
        // `allow` is upper, so it protects `< 1.0.0`-ranged versions that might be omitted on next steps
        "version": "< 1.0.0"
      },
      {
        "policy": "deny",
        // Checks pkg version publish date against the range
        "dateRange": ["2010-01-01T00:00:00.000Z", "2025-01-01T00:00:00.000Z"]
      },
      {
        "policy": "allow",
        "age": 5    // Check the package version is older than 5 days. Like quarantine
      }
    ]
  }
}

Multi-config

// Array at the top level
[
  // Two servers (for example, http and https) share the same preset
  {
    "server": [
      {"port": 3001},
      {"port": 3002, "secure": {"cert": "ssl/cert.pem", "key": "ssl/key.pem" }},
    ],
    "firewall": {
      "registry": "https://registry.yarnpkg.com",
      "rules": {"policy": "deny", "org": "@qiwi"}
    }
  },
  // One server has a pair of separately configured endpoints
  {
    "server": {"port": 3003},
    "firewall": [
      {"base": "/foo", "registry": "https://registry.npmjs.org", "rules": {"policy": "deny", "org": "@qiwi"}},
      {"base": "/bar", "registry": "https://registry.yarnpkg.com", "rules": {"policy": "deny", "org": "@babel"}}
    ]
  }
]

️More config examples

Cache

By default, nrf uses a simple in-memory cache to store patched packuments.

cache: {              // Optional. Defaults to no-cache (null)
  ttl: 5,             // Time to live in minutes. Specifies how long resolved pkg directives will live.
  evictionTimeout: 1, // Cache invalidation period in minutes. Defaults to cache.ttl.
  name: 'unique'      // If and only if you use the same rules for several firewall entrypoints (multi-port proxy)
                      // you can slighly optimise resource consupmtion by sharing the cache. Defaults to `randId()`
}

You can also provide your own implementation instead, for example, to create cassandra-based distributed cache:

import {createApp} from 'npm-registry-firewall'

const cache = {
  add() {}, // each method may be async
  has() {return false},
  get() {},
  del() {}
}

const app = createApp({
  server: {port: 5000},
  firewall: {
    registry: 'https://registry.npmjs.org',
    cache,
    rules: []
  }
})

Or even a cache factory:

const cache = () => {
  // ... init
  return {
    get() {},
    ...
  }
}

Pass null as config.firewall.cache to disable.

Extras

Presets

Introduce your own reusable snippets via extends or preset. This statement can be applied at any config level and should return a valid value for the current section. The specified path will be loaded synchronously through require, so it must be a JSON or CJS module.

const config = {
  // should return `firewall` and `servers`
  extends: '@qiwi/nrf-std-config',
  server: {
    port: 5000,
    extends: '@qiwi/nrf-server-config'
  },
  firewall: {
    // `rules`, `registry`, etc,
    extends: '@qiwi/nrf-firewall-config',
    // NOTE If you redefine `rules` the result will be contatenation of `[...rules, ...extends.rules]`
    rules: [{
      policy: 'deny',
      // `name`, `org`, `filter`, etc
      extends: '@qiwi/nrf-deprecated-pkg-list'
    }, {
      policy: 'allow',
      extends: '@qiwi/nrf-whitelisted-orgs'
    }, {
      extends: '@qiwi/nrf-all-in-one-filter'
    }]
  }
}

For example, extends as a filter:

// '@qiwi/nrf-all-in-one-filter'
module.exports = {
  filter({org, name, time, ...restPkgData}) {
    if (name === 'react') {
      return true
    }

    if (org === '@babel') {
      return false
    }

    if (restPkgData.license === 'dbad') {
      return false
    }
  }
}

Plugins

The plugin API is slightly different from presets:

Async. It's loaded dynamically as a part of rule processing pipeline, so it may be an ESM.
Configurable. Opts may be passed as the 2nd tuple arg.
Composable. There may be more than one per rule.

const rule1 = {
  plugin: ['@qiwi/nrf-plugin']
}

const rule2 = {
  plugin: [
    ['@qiwi/nrf-plugin', {foo: 'bar'}],
    '@qiwi/nrf-another-one'
  ]
}

The plugin interface is an (async) function that accepts TValidationContext and returns policy type value or false as a result:

const plugin = ({
  rule,
  entry,
  options,
  boundContext
}) => entry.name === options.name ? 'deny' : 'allow'