RFC: Native Dependency Patching#862
Conversation
|
Hey @manzoorwanijk, There are a few questions I have:
|
|
Thanks @james-pre - three good questions. 1. Multiple patches for the same package. Punted in v1: see Unresolved Questions and Bikeshedding item 6 ("Stacking patches"). The RFC matches at most one patch per resolved node, and stacking is left as an additive extension (selector value becomes an array). The reason for deferring is that with Practical workaround for now: merge the two diffs into a single 2. Patches needed by dependants / transitive consumers. This is intentionally not supported, and the constraint is by design rather than oversight. From the RFC:
So in your example, if The right tool for "I want consumers of 3. Why not consolidate under Fair point - the current mix ( I'd propose:
Happy to update the RFC to use this shape if there's broader agreement. |
|
The rename is done — |
|
@owlstronaut, while this RFC is still in review, is it fine to create a draft PR of the implementation in the CLI repo for demonstration? |
Absolutely. Love the idea |
…atch-failure carve-out, lockfile-version pin upgrade)
|
Here is the first PR (draft) npm/cli#9439. The PR is feature complete, and the only part left is to support patching for aliases via |
|
Under this proposal, what would be the practical workflow for updating an existing patch that still applies to a new version? With patch-package, it still attempts to apply patches to newer versions (but prints a warning), so you just run With the proposed native patching, I think you'd have to do something like this: npm install <pkg>@new-version --allow-unused-patches
npm patch add <pkg>@new-version
pushd /path/printed/by/npm/patch/add
git apply --ignore-whitespace /path/to/project/patches/<pkg>@old-version.patch
popd
npm patch commit /path/printed/by/npm/patch/add
npm patch rm <pkg>@old-version...and then commit the changes. It would be great if there was a better way like an explicit |
|
|
Added |
…lent partial application
… floor, and align patch-update prose with the implementation
#291) Part of native dependency patching ([npm/rfcs#862](npm/rfcs#862)). This force-excludes the patch files declared in the root package's `patchedDependencies` from the packed file list, even when they are listed in `files`. ## Why `patchedDependencies` maps a dependency selector to a project-local patch file (e.g. `"abbrev@2.0.0": "patches/abbrev@2.0.0.patch"`). Those patches are a property of the project, not something a consumer of the published package applies. Without this, publishing a patched project would ship the patch files — and, once pacote strips the `patchedDependencies` field from the tarball's `package.json`, they would be dangling, unreferenced files. Excluding them keeps the published tarball clean. ## How In `PackWalker.processPackage`, when the walker is the project root and the manifest declares `patchedDependencies`, each declared patch file path is pushed onto the strict (un-overridable) rule set, so it is excluded even if `files` lists it. Design choices: - **Exact files, not directories.** Only the declared patch files are excluded — never their directory. A dedicated `patches/` dir becomes empty and drops out naturally, but a patch that lives in a shared directory (e.g. `src/foo.patch`) does not take the rest of `src/` down with it. - **`--patches-dir` honored for free.** The location is read straight off the `patchedDependencies` values, which already encode wherever the patches were written. - **Root-only.** `patchedDependencies` is root-only state, so the block is gated to the project root and never prunes a bundled dependency's files. - **Path safety.** Absolute paths and paths that escape the package root are skipped (they are never packed anyway). - **Warns** when a `files` entry pulled a patch file in (directly or via its directory), so the override is not silent. ## References Part of - npm/rfcs#862 Related to - npm/cli#9439 - npm/pacote#497
Part of native dependency patching ([npm/rfcs#862](npm/rfcs#862)). When packing a `directory` spec (the `npm publish` / `npm pack` path), this strips a top-level `patchedDependencies` field from the `package.json` written **into the tarball**. ## Why `patchedDependencies` declares project-local patches against installed dependencies. It is honored only in a root manifest, so it is meaningless to consumers of a published package and should never travel through the registry. The published *packument* manifest is already stripped in `libnpmpublish`; this closes the other half — the `package.json` inside the tarball itself — so `npm pack --dry-run` and the published tarball no longer carry the field. It pairs with the npm-packlist change that excludes the patch files themselves; together they guarantee a patched project publishes clean. ## How `DirFetcher` packs the raw on-disk files via `tar.c`, so the tarball's `package.json` is the literal file on disk — there is no manifest seam to edit. The new `#tarOptions()`: 1. Reads the on-disk `package.json` (after `prepare`) via `@npmcli/package-json`. If it has no `patchedDependencies`, returns the existing options unchanged — **non-patched packs are byte-for-byte identical to before**. 2. Otherwise deletes the field and re-serializes preserving the original indent, newline, and key order (the indent/newline symbols `@npmcli/package-json` attaches; `JSON.stringify` ignores them), writes the stripped copy to a temp dir, and removes the temp dir if the write fails. 3. Sets node-tar's `onWriteEntry` to redirect **only** the top-level `package.json` entry's `absolute` at the stripped copy and fix its `stat.size`/`nlink`. `onWriteEntry` runs before the header and the file's hardlink check, so the override is honored; every other file is untouched. 4. The temp dir is removed once the tar source stream emits `end`/`error`, so it outlives content consumption. No behavior change for any package without `patchedDependencies`. ## References Part of - npm/rfcs#862 Related to - npm/cli#9439 - npm/npm-packlist#291
…9439) Implements native dependency patching per [RFC #862](npm/rfcs#862): a first-class way to apply small, local modifications to an installed dependency and have them re-applied automatically on every install, with no external tooling or postinstall scripts. Patches are declared in a new `patchedDependencies` field of the root `package.json`, stored as plain unified diffs under `patches/`, and recorded with a content hash in `package-lock.json`. Because the patch is applied during the install itself, it works for transitive dependencies, across every `install-strategy`, and is **not** disabled by `--ignore-scripts`. ## The `npm patch` command A new command with five subcommands (and a bare `npm patch <pkg>` shorthand for `add`): - **`npm patch add <pkg>[@<version>]`** — extracts a clean copy of the resolved registry tarball into a temp directory outside `node_modules` and prints the path to edit. Ambiguous when multiple versions are installed; the error lists the exact selectors to retry with. - **`npm patch commit <edit-dir>`** — diffs the edited directory against a fresh copy of the original tarball, writes `<patches-dir>/<name>@<version>.patch`, adds the `patchedDependencies` entry, and reifies to apply the patch and record its integrity in the lockfile. `package.json` is excluded from the diff — Arborist resolves the pre-patch manifest, so a patched manifest would change resolution-affecting fields on disk without being honored (silent partial application); `commit` warns when an edit only touches it. - **`npm patch update <pkg>[@<old-version>] [--to <new-version>]`** — rebases an existing patch onto a new version. It reads the target from `--to` or the lockfile, 3-way-merges the existing patch onto the new tarball in a throwaway git repo, and rewrites `package.json` + `package-lock.json` **without touching `node_modules`** (so it works from a failed-install state). On conflict it leaves an edit dir with `<<<<<<<` markers, finalized by `npm patch commit`. Exact selectors are renamed; range/name-only selectors gain a new exact entry and keep the old one while it still wins another installed node. - **`npm patch ls`** — lists registered patches and how many installed nodes each matches (flagging overlapping range selectors that conflict on a node). - **`npm patch rm <pkg>[@<version>]`** — removes the matching entries, deletes the patch file when no other entry references it, and reifies to revert the files. ## Install-time apply pipeline Patch resolution and application live in Arborist so every install path honors them: - **`resolvePatchedDependencies`** resolves the root `patchedDependencies` map against the ideal tree, attaching `node.patched = { path, integrity }` to each matched node. Selector precedence is exact > range-subset > name-only, with ambiguous overlapping ranges surfaced as a hard error. - **reify** applies the diff after extraction and records the patched integrity in the lockfile. `diff.js` forces re-extraction when a node's patch integrity changes, and re-extracts to revert when a previously-patched node loses its selector (`patchRemoved`). - **`install-strategy=linked`** is supported via a content-addressed side-store: the store key is suffixed with the patch identity (`+patch`) so a patched and unpatched copy of the same version coexist without collision. A failed patch under linked strategy is always a hard error (the side-store cannot represent unpatched contents at a patched key without later installs silently trusting it). ## Lockfile Patches require `lockfileVersion: 4` so that older npm clients abort rather than silently installing unpatched code. When any node is patched, npm writes version 4 and **warns** if this upgrades a lower pinned `lockfile-version` (the safety gate cannot be honored otherwise). `npm ci` revalidates each patch's existence and integrity against the lockfile before installing. ## Failure modes By default any patch problem is a hard error that aborts the install: a patch that fails to apply, a registered patch that matches no installed package, a missing patch file, or a patch whose hash does not match the lockfile. Two **CLI-only** relax flags cover one-off cases — `--allow-unused-patches` and `--ignore-patch-failures` — and are rejected in `npm ci` and when set anywhere other than the command line. ## Non-registry dependencies Patches need a stable registry tarball as their baseline, so a dependency reached through a non-registry consumer edge (`file:`, `git:`, `http(s):`) is rejected with `EPATCHNONREGISTRY`, both by `npm patch add` and at install time. The check is edge-based (the consuming spec's type), not node-based, so it does not falsely reject edgeless nodes such as linked-store entries or extraneous installs, which are still registry deps. `npm:` registry aliases are correctly classified as registry deps and are supported by the install engine; the `npm patch add <alias>` ergonomics will land in a fast-follow. ## Publish / pack `patchedDependencies` is stripped from the published **registry manifest** (libnpmpublish) so the field never leaks to the packument. Stripping it from the **tarball's own `package.json`** and excluding the `patches/` directory from the tarball is a coordinated follow-up in `pacote` + `npm-packlist` (those packages own the packed file list and the manifest written into the tarball, neither editable from the CLI) — see Follow-up work. ## Other surfaces - `npm ls` annotates patched dependencies in its output. - New config: `patches-dir`, `edit-dir`, `ignore-existing`, `keep-edit-dir`, plus the two relax flags. - New `npm-patch` man page and nav entry. ## Tests Unit and integration coverage for every subcommand (including `update`'s clean rebase, conflict→commit, and selector-rename/range-fork paths), the apply pipeline, selector matching, linked-strategy apply/removal, lockfile validation, publish stripping, and the relax flags. Arborist and CLI suites pass at 100% coverage. ## Follow-up work A few additive pieces are deliberately deferred — nothing in this PR depends on them. - **Tarball-side strip for publish/pack** — stripping `patchedDependencies` from the tarball's own `package.json` and excluding the `patches/` directory from the published tarball. This can't be done in the CLI: the tarball's file list and manifest come from `pacote` (packs the raw on-disk files) and `npm-packlist`, so it needs coordinated changes there. Raised in the RFC review; the registry-manifest strip in this PR already prevents the field from being honored or appearing in the packument. - **`npm patch add <alias>` ergonomics for `npm:` registry aliases** — the install engine already treats `npm:` aliases as registry dependencies and applies a hand-written `<alias>@<version>` selector correctly today. What remains is the `add`/`commit` convenience: resolving the alias to its real `name@version` tarball as the baseline and keying the written selector on the alias name. Currently `npm patch add <alias>` resolves the alias name as a real package and fails. - **Binary files** — patches are unified text diffs, so binary files (images, wasm, native addons) cannot be patched. This is a limitation of the whole feature (shared with `patch-package`), not a regression; a binary-aware path could be added later. ## References Implements npm/rfcs#862
5 participants
Summary
Adds first-class, install-time patching of installed dependencies to the npm CLI, on parity with
pnpm patch,yarn patch, andbun patch. Introduces a newnpm patchcommand with subcommandsadd/commit/update/ls/rm(withnpm patch <pkg>as shorthand fornpm patch add <pkg>), apatchedDependenciesfield inpackage.json, and apatched.{path,integrity}record inpackage-lock.json(lockfileVersion: 4). Patches apply during Arborist's reify step, uniformly across every supportedinstall-strategy(hoisted,nested,shallow,linked).Why now
The third-party
patch-packageis currently the only path to dependency patching for npm users, and it is structurally limited:--ignore-scripts.patch-packageruns as apostinstallscript. In environments that disable lifecycle scripts — increasingly common in hardened CI and after recent supply-chain incidents like theShai-Huludworm (Sept/Nov 2025) — declared patches simply do not apply, with no error and no warning. Production code can be installed missing fixes that are committed in the project.install-strategy=linked(ds300/patch-package#595).The headline outcome: reproducible, source-controlled dependency hotfixes that survive
--ignore-scriptsand work across every npm install strategy and across workspaces.Relationship to #94
This RFC is a direct response to #94 (closed in 2020 as a footgun). The 2020 proposal was an ad-hoc
npm install --patch foo.patchflag with no manifest record, no lockfile linkage, no transitive-dep support, and no failure-mode story; @isaacs's footgun objection was correct for that shape. This RFC is structured the opposite way — explicit manifest, lockfile-hashed, fail-loud-by-default, version-gated, publish-isolated. A row-by-row response is in the RFC's Prior Art → #94 section.See the RFC for the full design, alternatives considered, implementation plan, tests, and unresolved questions.