Skip to content

fix(npm): discover pnpm virtual-store packages — transitive deps were invisible to apply and scan - #202

Merged
Mikola Lysenko (mikolalysenko) merged 4 commits into
mainfrom
fix/pnpm-virtual-store-discovery
Aug 19, 2026
Merged

fix(npm): discover pnpm virtual-store packages — transitive deps were invisible to apply and scan#202
Mikola Lysenko (mikolalysenko) merged 4 commits into
mainfrom
fix/pnpm-virtual-store-discovery

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Problem

Under pnpm's isolated linker (the default on every pnpm major), a transitive-only dependency lives solely inside node_modules/.pnpm/<entry>/node_modules/<name> — there is no importer-root symlink for it. The npm crawler skipped .pnpm as a hidden directory and never traversed symlinked packages, so:

  • apply reported package_not_installed for packages that were physically installed and runtime-loaded (the classic minimist-behind-mkdirp CVE shape), exiting as if the project were clean;
  • scan never included those packages in the patch-API batch query, so their patches could never surface.

Confirmed empirically on pnpm 7.33.5, 8.15.9, 9.15.9, 10.34.5, 11.22.0, and 12.0.0-rc.7 (2026-08-18 e2e matrix, 18 legs, real corepack installs).

Fix

  • find_by_purls now probes .pnpm store entries (real dirs only; BFS keeps root-install-wins). Store entries whose dir name decodes as name@version are filtered against the pending-target set so large monorepo stores aren't walked wholesale; undecodable names (truncated/hash-suffixed dirs) ride a conservative fallback and stay probeable.
  • crawl_all inventories the virtual store after the root pass; identity re-reads are skipped for already-seen name@version entries while bundled deps inside store entries still walk.
  • One shared store-entry enumerator; the scan helpers are parameterized by a ScanPolicy bit instead of a ~55-line parallel copy.
  • Multi-version installs of one package become individually discoverable (each store entry probes independently; the name+version match keeps them distinct).

The no-symlink-traversal policy is unchanged everywhere else; .pnpm is the one documented exception because the virtual store is the only physical home of transitive deps under the isolated linker.

Verification

  • New hand-built pnpm-farm tests: transitive-only dep, second version of the same package, scoped store entry, decoy entries planted in the hoist dir and hidden children, truncated-name fallback, decoder unit tests (plain / +-escaped scope / v9 peer paren / legacy _ suffix / non-package names → None).
  • New real-pnpm e2e in in_process_alternate_installers.rs: pnpm install of mkdirp@0.5.5 in two sibling projects sharing one hardlink-imported store; offline apply of a hand-staged minimist manifest must patch the canonical .pnpm path (exact bytes + git-sha256), break the hardlink (inode diverges), and leave the sibling + global store byte-pristine.
  • Live re-verified with the release binary on corepack pnpm@10.34.5: before — apply exit 1 package_not_installed, scan batch body [mkdirp] only; after — apply exit 0 applied=1 with marker bytes at the canonical path, store clean, scan batch body includes minimist@1.2.8.
  • Production store-safety suite (e2e_safety_pnpm) re-run green on PATH pnpm 11 (4/4).

🤖 Generated with Claude Code


Note

Medium Risk
Changes core npm package discovery and resolution paths used by apply and scan; behavior is well covered by new unit and e2e tests, but incorrect traversal could still miss packages or patch wrong paths.

Overview
Fixes a gap where pnpm isolated-linker transitive dependencies (only under node_modules/.pnpm/…, often with no importer-root symlink) were skipped because .pnpm was treated like other hidden dirs and store symlinks were not walked correctly. Apply could report package_not_installed; scan could omit those packages from batch discovery.

The npm crawler now defers a .pnpm virtual-store pass after the importer node_modules walk, with a ScanPolicy separating importer trees (record symlinks, defer store) from store-entry trees (real dirs only, skip redundant package.json when already seen). find_by_purls enqueues store entries during BFS, filtering by pending package names via decode_pnpm_store_entry_name (peer suffixes, scoped + escaping, conservative None for truncated/git entries that stay probeable). Shared list_pnpm_store_entries centralizes store layout rules.

Coverage adds decoder unit tests, hand-built pnpm-farm e2e for resolve/scan/filter/bundled deps, and a real pnpm install test (mkdirp → transitive minimist) asserting patch at the canonical store path and CoW (hardlink break) so a sibling project sharing the store stays pristine.

Reviewed by Cursor Bugbot for commit bd4bd19. Configure here.

… invisible to apply and scan

Under pnpm's isolated linker a transitive-only dependency lives solely
inside node_modules/.pnpm/<entry>/node_modules/<name>; the crawler skipped
.pnpm as a hidden dir and never traversed symlinked packages, so apply
reported package_not_installed for packages that were installed and
runtime-loaded, and scan never sent them to the patch API. Confirmed
empirically on pnpm 7, 8, 9, 10, 11, and 12-rc (2026-08-18 matrix).

- find_by_purls: probe .pnpm store entries (real dirs only, root install
  wins via BFS order); entries whose dir name decodes as name@version are
  filtered against pending targets, undecodable names ride a conservative
  fallback so truncated/hashed dirs stay probeable.
- crawl_all: inventory the virtual store after the root pass; identity
  re-reads are skipped for already-seen name@version entries, bundled deps
  inside store entries still walk.
- One shared store-entry enumerator; scan helpers parameterized by a
  ScanPolicy bit instead of a parallel copy.
- Multi-version installs of one package now individually discoverable.

Tests: hand-built pnpm-shaped farm (transitive, multi-version, scoped,
decoys, truncated-name fallback, decoder units) + a real-PATH-pnpm
transitive apply e2e with CoW inode proofs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Fix All in Cursor

Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issues.

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit bd4bd19. Configure here.

Comment thread crates/socket-patch-core/src/crawlers/npm_crawler.rs Outdated
Comment thread crates/socket-patch-cli/tests/in_process_alternate_installers.rs Outdated
… .registry.* stores)

The flat-entry store walk covered pnpm 6+ only. Two earlier layouts
(both confirmed against captured real installs) still hid transitive-only
deps — apply exited 0 claiming success with nothing written:

- pnpm 4/5 (layoutVersion 3): entries nest by registry host —
  .pnpm/<registry-host>/<name>/<version>/node_modules/<name> — so the
  host child has no node_modules of its own and the flat walk found
  nothing behind it.
- pnpm <=3 (layoutVersion <=2): the store is a hidden
  node_modules/.<registry-host> dir (no .pnpm at all), swallowed by the
  hidden-entry skip.

list_pnpm_store_entries now descends a nested host (bounded: depth 3 =
@scope/name/version, 16K-dir fan-out cap, symlinks never traversed) and
synthesizes flat name@version entry names so the pending-name filter and
identity_seen dedup treat nested and flat entries identically;
node_modules/.registry.* dirs are recognized as legacy store roots for
both the resolver and the scan pass (deliberately NOT any-hidden-dir —
that would walk arbitrary tool caches). Root-install-wins ordering
preserved.

Tests: byte-accurate replicas of the captured pnpm-4 and pnpm-3 trees
(decoy entries, hoist dirs, symlink-cycle bait) pinning find_by_purls +
crawl_all for each layout, plus a REAL corepack pnpm@4.14.4 install leg
proving offline apply patches a transitive dep in the genuine nested
layout. Re-verified against the captured matrix trees: apply now exits 0
with the transitive patch applied on both real pnpm 4.14.4 and 3.8.1
projects that previously reported package_not_installed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ed-only targets

Adversarial review of the virtual-store discovery found two fail-opens:

1. Peer-variant duplicates — .pnpm/foo@1.0.0(react@17)/ and
   .pnpm/foo@1.0.0(react@18)/ are two real dirs holding the same
   name@version — resolved to ONE copy per purl: apply exited 0 claiming
   the CVE fixed while dependents through the second instantiation kept
   loading vulnerable bytes (the same single-instance fail-open the
   redirect rewriter eliminates on the lock side). The apply engine now
   fans out: find_pnpm_peer_variant_copies discovers every other physical
   store copy of the primary's name@version (flat + nested layouts,
   canonical-path dedup), the full verify+patch pipeline runs per copy —
   including when the primary is AlreadyPatched, healing pre-fix damage —
   and any copy failing fails the result. The shared write path fans out
   for rollback's restores too; CoW holds per copy. The dispatcher's
   purl-keyed contract is unchanged.
2. The resolver's pending-name store-entry filter hid targets that exist
   ONLY as a bundled dependency inside another package's store entry.
   find_by_purls now runs a second, unfiltered store pass for targets
   still unresolved after the filtered walk (common-case perf intact).

Tests (all RED-verified against the pre-fix code): peer-variant twin
apply+rollback with hardlinked-store inode purity asserted at BOTH
canonical paths; AlreadyPatched-primary heals the lagging twin;
bundled-only target resolves via the fallback. Verified end-to-end on
real corepack pnpm 10.34.5 with a hard-linked crafted twin: both copies
patched, global store pristine, rollback restores all copies.

Known residuals (fail toward patched bytes, never vulnerable ones):
rollback does not fan out deletion of patch-ADDED files, and an
already-original primary early-returns before healing a still-patched
twin.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback: the campaign-added tests (pnpm virtual-store transitive
apply, pnpm4/pnpm3-era legs) called bare .unwrap() on fallible fixture
steps, so a CI failure named no step. All 26 fixture-step unwraps in the
new tests now carry .expect("…") messages in the file's established
style; long-standing tests are untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mikolalysenko
Mikola Lysenko (mikolalysenko) merged commit f5a6bd7 into main Aug 19, 2026
43 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the fix/pnpm-virtual-store-discovery branch August 19, 2026 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants