From 94a6c2eb5d93751394e8fcde6a0610fc0002c1c1 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 10:05:47 -0600 Subject: [PATCH 01/36] Plan one lock model for every lockfile reader Lockfile discovery (vex/discover), the lock inventory (vendor/lock_inventory) and the hosted and vendored writers each read the same project files their own way: two Socket-origin classifiers, three Python precedence rules, second walks of package-lock, yarn, vlt and pnpm locks, a second parse of uv/pylock/script locks, five Cargo.toml dependency-table walks, three readers of Maven's checksums.sha256. The inventory also has no NuGet, JVM, Hatch or Deno models. docs/design/lock-model-unification.md describes the target, one per-project model whose entries carry an Origin (Registry, Hosted, Vendored, Other, Rejected), file selection by install tree (precedence within a tree, union across trees), and the stack of PRs that gets there. scripts/loc-budget.py counts implementation lines (no comments, no cfg(test) items, no test-only modules) for two groups in scripts/loc-budget.json: lock-readers (30,898 lines today) and lock-writers (88,363). The lint-ecosystems job runs it with --warn, which annotates a group over budget but never fails, so bug fixes in these files are not blocked; each PR in the stack lowers the budget as it deletes code. No product code changes. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 6 + docs/design/lock-model-unification.md | 243 +++++++++++++++++++++++ scripts/loc-budget.json | 35 ++++ scripts/loc-budget.py | 271 ++++++++++++++++++++++++++ scripts/tests/test_loc_budget.py | 140 +++++++++++++ 5 files changed, 695 insertions(+) create mode 100644 docs/design/lock-model-unification.md create mode 100644 scripts/loc-budget.json create mode 100755 scripts/loc-budget.py create mode 100644 scripts/tests/test_loc_budget.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a6b071c44..d440fecd6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -208,6 +208,12 @@ jobs: - name: Python — test native installer harnesses run: python3 -B -m unittest discover -s scripts/tests -v + - name: Rust — lock model size budget + # Annotates, never fails: a bug fix elsewhere may add lines to these + # files. The lock-model PRs lower the budget as they delete code + # (docs/design/lock-model-unification.md#size-budget). + run: python3 -B scripts/loc-budget.py --warn + - name: Shell — shellcheck the curl|sh installer # shellcheck is pre-installed on ubuntu-latest. run: shellcheck --shell=sh scripts/install.sh diff --git a/docs/design/lock-model-unification.md b/docs/design/lock-model-unification.md new file mode 100644 index 000000000..9da9c15cb --- /dev/null +++ b/docs/design/lock-model-unification.md @@ -0,0 +1,243 @@ +# One lock model for every reader + +Status: in progress (stacked PRs `lock-model/*`). Owner decisions are dated +2026-11-04. + +Paths below are relative to `crates/socket-patch-core/src/`. + +## Problem + +Three groups of code read the same project files, and each reads them its own +way: + +* **Lockfile discovery** (`vex/discover/`) finds Socket wiring: the hosted + pins and vendored paths in lockfiles and package-manager configs. It is not + only for `vex`. `scan`, `vendor`, the hosted engine, the hosted unwind, + rollout and ledger liveness all call it. +* **The lock inventory** (`vendor/lock_inventory/`) lists every package a + lockfile resolves. `scan` and `get` use it to supplement the installed-tree + crawl, `vendor` uses it to fetch pristine artifacts, `repair` uses it to + recover ledger entries, and ledger liveness uses its every-lock view. +* **The writers** (`patch/redirect/`, `hosted/`, `vendor/`) parse the same + files again to plan their edits. + +`formats/` was started as the shared home ("one model per lockfile format"), +but only pnpm has both an entry view and a wiring view there. A survey of +every format on 2026-11-04 found the duplicated work below. What discovery +adds on top of a walk (the contests between locks, ledger claims and +attestation rules) is real policy and stays. + +| Duplicated work | Where | +|---|---| +| Two Socket-origin classifiers | discovery's `DiscoverCtx::locate` / `hosted_uuid` checks a Socket host allowlist. The inventory's `parse_vendor_path` / hosted-URL-leaf matching accepts any host. `utils/poetry_lock::lock_wiring` uses a third, looser substring test. | +| Three Python precedence rules | the inventory's chain (uv by parse success, then poetry or pdm if they have entries, then Pipfile.lock plus requirements), `formats/governing_locks.rs` (uv, poetry, pdm, Pipfile.lock by presence) and discovery (every lock, plus the cross-lock contest). They disagree on four common layouts. | +| Second walks of one file | `vendor/npm_origin.rs` walks package-lock `packages` again. Yarn writers call `scan_blocks` directly, and the Berry rewriter uses a third model (`formats/yarn/stanzas.rs`). The vlt writers use a separate line grammar. pnpm's `InstalledPackages` and `vendored_npm_uuids`, and the hosted planner's whole-text `grammar::entries`, skip the env-document filter. | +| Second parses of one file | `vex/discover/pypi_locks.rs` parses uv, pylock and script locks itself. The inventory parses them again. `deno.lock` is parsed by `crawlers/deno_crawler.rs` for discovery, the hosted engine and the npm flavor probe. | +| Repeated helper parsers | two Python lock listers, two `-r` include walkers, two PEP 508 direct-reference parsers, five walks of `Cargo.toml` dependency tables, two Rush lock listings, three readers of Maven's `checksums.sha256`, two Gradle Socket-index parsers, two routes to sbt resolution evidence | +| Two file-selection layers | the npm flavor router (`lock_inventory/npm_family.rs`) picks one family and one lock. Discovery reads every lock and sweeps the ignored ones with `recognize_ignored`. | + +The inventory also has gaps. It has no models for NuGet, Maven, Gradle, sbt, +Hatch or Deno, and it reads no `composer.lock` that starts with a BOM, which +discovery reads. + +## Goal + +One per-project lock model. Every consumer reads it, and the end state has +fewer implementation lines than today. `scripts/loc-budget.py` tracks the +lines (see [Size budget](#size-budget)). + +Non-goals: changing what any mode writes, or adding new hosted or vendored +backends. + +## The model + +### Entries + +Every reader yields `LockEntry` records. Each record holds the package +coordinates as the lock spells them, the integrity the lock records, the +resolved location, a source span for writers that splice the file, and an +`Origin`: + +```rust +pub enum Origin { + /// Resolves from a registry the package manager would use anyway. + Registry { resolved: Option }, + /// Routes to a Socket-hosted artifact. + Hosted { uuid: String, url: String }, + /// Routes to a committed `.socket/vendor///` artifact. + Vendored { uuid: String, artifact_rel: String }, + /// git, path, link, workspace, or a URL on another host. + Other(OtherSource), + /// Looks like Socket wiring but fails validation. Keeps the identity + /// the entry would have had, so contests and recognition still see it. + Rejected { would_be: Option, diag: Diag }, +} +``` + +Formats with install semantics that discovery's contests need also carry +flags: bundled, link, the v2 mirror, a `hasShrinkwrap` ancestor, Yarn's +`live`, Berry's checksum cache key, and so on. These are fields of the format's +record, not new side channels. + +One classifier, `formats::origin`, decides `Origin`. It is discovery's +validators moved, not rewritten: + +* the Socket host allowlist and the last-canonical-uuid rule + (`hosted_uuid`); +* `vendor_ref`, `vendor_ref_decorated` and `vendor_uuid_dir`; +* the `socket-patch-` name grammar; +* the validating purl builders. + +The inventory's any-host matching goes away. That is a behavior change only +where a non-allowlisted host serves a URL that looks like Socket's, and it +lands in the install-tree PR with a golden review. + +### Joins + +For JVM and NuGet, origin is not a property of one entry: + +* **Maven:** a pom dependency's `-socket.` version joins one + ``. +* **Gradle:** a lockfile entry joins a row of `.socket/gradle/hosted-index.tsv` + or `.socket/vendor/gradle-index.tsv`. +* **sbt:** pins live in owned files and are confirmed by resolution evidence. +* **NuGet:** `packages.lock.json` entries join `nuget.config`'s + ``. + +So a reader works on a project view, not on a single text. A reader may read +several files and still yield one list of records. The same applies to Cargo, +where `Cargo.toml` `registry =` pins join `.cargo/config.toml` registries, +and to Go, where `go.mod` `replace` directives join `go.sum` lines. + +### Install trees + +File selection is one function over `formats::registry()`: + +```rust +pub struct InstallTree { + /// The files the package manager reads for this tree, governing first. + pub files: Vec, + /// Present files the package manager ignores in favor of `files`. + /// They are swept for Socket identities (discovery rule 11) and never + /// read as wiring. + pub shadowed: Vec, +} +``` + +Rule: **precedence within a tree, union across trees.** + +* npm: `npm-shrinkwrap.json` governs and `package-lock.json` is shadowed. + pnpm: `pnpm-lock.yaml` governs `shrinkwrap.yaml`. Bun: `bun.lock` governs + `bun.lockb`, which `bun_text_lock_drives` already decides. +* **Python (owner decision):** each tool's lock is its own tree, because + which lock is installed depends on which tool runs. uv.lock, poetry.lock, + pdm.lock, Pipfile.lock, each pylock, each PEP 723 script lock, and + `requirements.txt` with its `-r` includes are separate trees. + Discovery's cross-lock contest stays. It is the safety check that withholds + a uv ref when poetry.lock pins the same release unpatched. The inventory + unions the trees, so `scan` sees a patch wired into an exported + `requirements.txt` beside `poetry.lock`. The inventory's precedence chain + and `governing_locks`' presence rule are deleted. The vendor router's + question ("which lock do I write?") becomes a query over the trees. +* **Deno (owner decision: deprecated but supported):** `deno install` reads + `deno.lock` for a `package.json` project's npm dependencies and never reads + the npm-family locks. `deno.lock` and the npm-family lock are one tree with + undecided governance: the files don't say which tool fills + `node_modules`. Today's outcome is kept: a ref in the npm-family lock stays + a ref but is marked unattested. It is now expressed as a tree property, + not the `UnwiredCopy` side channel. +* Rush subspace locks, workspace-member locks and Go workspace members are + trees of their own, as discovery already treats them. + +### Consumers + +* **Inventory**: every entry of every tree, read by the package it resolves. + Socket entries are listed as the package they replace, as today, so a + rescan of a wired checkout still lists it. Then path-safety guards and + `dedup_prefer_integrity`. `inventory_project_every_lock` becomes the + same list without the dedup. +* **Discovery**: the `Hosted`, `Vendored` and `Rejected` entries of each tree + become refs and diagnostics, the shadowed files are swept, and the contests + and ledger claims run as today. The `Registry` and `Other` entries of + other trees replace the `resolved_elsewhere` collection that each extractor + feeds by hand today. +* **Ledger liveness** reads `Origin` directly instead of looking for a uuid + inside the inventory's `resolved` URL. +* **Writers** plan over the same records, using the source spans. + +## JVM and NuGet support (owner decision: in scope) + +What committed files can give the inventory: + +| Ecosystem | Coordinates | Integrity | +|---|---|---| +| NuGet | every `(id, resolved)` per target framework in `packages.lock.json` | `contentHash`, which needs a new `LockIntegrity::NugetContentHash`. For signed packages, and nuget.org signs all of them, it hashes the archive with the signature stripped (`formats/nuget/package.rs::package_content_hash`), not the raw `.nupkg` bytes. | +| Gradle | the resolved GAVs per configuration in `gradle.lockfile` (settings and buildscript locks tagged as build logic) | `verification-metadata.xml`, which has editors but no reader today. `sha256` and `sha1` map to existing variants. `sha512` needs `Sha512Hex`. | +| Maven | the root pom's direct and managed GAVs only. Maven has no lockfile, so transitive dependencies are not in committed files. | `.mvn/checksums/checksums.sha256` lines, normally only for Socket pins | +| sbt | Socket pins only (owned files). No committed file resolves registry entries. | the jar sha256 in the owned file | + +Vendor fetch refuses entries without a verifier, and JVM and NuGet vendoring +take their bytes from caches, registry sidecars or the installed directory +anyway. So the new entries first serve `scan`/`get`, discovery and ledger +liveness. Ledger liveness also needs a `SourceKind` for "mapped to a +non-Socket NuGet source" and the Gradle equivalent, since these entries carry +no URL. + +## Deno + +Deno stays supported while deprecated. It is ported like any other format, so +deprecating it later means deleting one reader and one tree rule. The +`deno.lock` reader moves from `crawlers/deno_crawler.rs` into `formats/` and +serves the crawler, discovery, the hosted engine and the npm flavor probe. +Deprecation messaging is separate from this work. + +## Size budget + +`scripts/loc-budget.py` counts implementation lines: code outside comments, +outside `#[cfg(test)]` items and outside test-only modules. It compares the +groups in `scripts/loc-budget.json` with their maxima. CI's +`lint-ecosystems` job runs it with `--warn`, which annotates a group over +budget but never fails, so another PR's bug fix in one of these files never +fails on it. There are two groups: +`lock-readers` (`formats/`, `vex/discover/`, `vendor/lock_inventory/`, +`gradle/`, the Python lock helpers in `utils/`, `vendor/npm_origin.rs` and +`crawlers/deno_crawler.rs`) and `lock-writers` (`patch/redirect/`, `hosted/` +and the rest of `vendor/`). At 7bde3ba55 they hold 30,898 and 88,363 lines. +Each PR in this stack +lowers the maxima with `--lower` in the same commit. The NuGet and JVM PRs +add code and raise the readers' maximum by hand, saying why. + +## Stack + +Each PR is based on the one before. PRs 1 to 11 must leave +`tests/fixtures/vex-discover-golden/` and the redirect goldens byte for byte +unchanged. Only PR 12 may change them, and its golden diff is reviewed line +by line. + +| PR | Branch | Change | +|---|---|---| +| 0 | `lock-model/0-design` | This document and the size budget | +| 1 | `lock-model/1-origin` | `LockEntry`, `Origin` and `formats::origin` (discovery's classifier, moved). Discovery calls it. The inventory doesn't yet. | +| 2 | `lock-model/2-pnpm` | Pilot. One pnpm reader feeds the inventory, discovery and the hosted planner. Removes `InstalledPackages`, `vendored_npm_uuids`, the whole-text planner walk and the second Rush listing. | +| 3 | `lock-model/3-npm` | package-lock and shrinkwrap. Folds `vendor/npm_origin.rs`'s second walk into the records. | +| 4 | `lock-model/4-yarn` | Classic and Berry. One block model for every reader and writer, `BerryStanzas` included. | +| 5 | `lock-model/5-bun-vlt` | `bun.lock`, `bun.lockb` and vlt. The vlt writers move off their line grammar. | +| 6 | `lock-model/6-deno` | The `deno.lock` reader in `formats/`, for all four of its callers | +| 7 | `lock-model/7-python` | One lister, one parse per lock, one include walker and one PEP 508 parser. `pypi_locks.rs` stops parsing TOML itself. Poetry wiring uses `formats::origin`. Hatch gets inventory entries. | +| 8 | `lock-model/8-cargo-go` | One `Cargo.toml` dependency-table walk. Go `replace` and `go.sum` records. | +| 9 | `lock-model/9-gem-composer` | Gemfile.lock and composer.lock records. The inventory stops dropping BOM-prefixed `composer.lock` files. | +| 10 | `lock-model/10-nuget` | NuGet records with the config join, `NugetContentHash`, and inventory entries | +| 11 | `lock-model/11-jvm` | One `checksums.sha256` reader, one Gradle index parser, a `verification-metadata.xml` reader, then Gradle-lock and root-pom inventory entries and sbt's owned-file entries | +| 12 | `lock-model/12-install-trees` | Install-tree selection, Python trees, Deno governance, the single classifier for the inventory, liveness on `Origin`. Deletes the npm flavor router's lock choice, the Python precedence rules, `resolved_elsewhere` and the `UnwiredCopy` plumbing. | +| 13 | `lock-model/13-cleanup` | Removes the raw-text liveness fallbacks if the model makes them redundant, plus whatever is left of `vendor/lock_inventory/`. Final budget. | + +## Open questions + +* Whether discovery's hosted NuGet refs, which set `Sri("sha512-" + + contentHash)`, are verified anywhere against raw bytes. Socket-served + packages may be unsigned, in which case the two hashes agree. PR 10 checks + this before it adds `NugetContentHash`. +* Whether the raw-text fallbacks (`hosted_wiring_in_files`, + `vendored_wiring_live`) can go entirely. They exist for formats that no + reader reads and for patch servers outside the allowlist. PR 13 decides + from what is left by then. diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json new file mode 100644 index 000000000..28d387299 --- /dev/null +++ b/scripts/loc-budget.json @@ -0,0 +1,35 @@ +{ + "groups": { + "lock-readers": { + "paths": [ + "crates/socket-patch-core/src/formats/", + "crates/socket-patch-core/src/vex/discover/", + "crates/socket-patch-core/src/vendor/lock_inventory/", + "crates/socket-patch-core/src/vendor/npm_origin.rs", + "crates/socket-patch-core/src/gradle/", + "crates/socket-patch-core/src/utils/python_lock.rs", + "crates/socket-patch-core/src/utils/poetry_lock.rs", + "crates/socket-patch-core/src/utils/pdm_lock.rs", + "crates/socket-patch-core/src/utils/requirements.rs", + "crates/socket-patch-core/src/utils/lock_fragments.rs", + "crates/socket-patch-core/src/utils/hatch.rs", + "crates/socket-patch-core/src/utils/pipenv.rs", + "crates/socket-patch-core/src/crawlers/deno_crawler.rs" + ], + "max": 30898 + }, + "lock-writers": { + "paths": [ + "crates/socket-patch-core/src/patch/redirect/", + "crates/socket-patch-core/src/hosted/", + "crates/socket-patch-core/src/vendor/" + ], + "exclude": [ + "crates/socket-patch-core/src/vendor/lock_inventory/", + "crates/socket-patch-core/src/vendor/npm_origin.rs", + "crates/socket-patch-core/src/vendor/test_support" + ], + "max": 88363 + } + } +} diff --git a/scripts/loc-budget.py b/scripts/loc-budget.py new file mode 100755 index 000000000..7ef6795a4 --- /dev/null +++ b/scripts/loc-budget.py @@ -0,0 +1,271 @@ +#!/usr/bin/env python3 +"""Count implementation lines of Rust code and compare them to a budget. + +A line counts when it holds code outside a comment. These never count: + +* blank and comment-only lines (doc comments included); +* items gated by ``#[cfg(test)]`` or ``#[cfg(all(test, ...))]`` (inline + ``mod tests { ... }`` blocks, test-only helpers and ``use`` lines); +* files declared as ``#[cfg(test)] mod name;`` and every module below them; +* files under a ``tests/`` or ``benches/`` directory. + +Budgets live in ``scripts/loc-budget.json``. Each group names path prefixes +(relative to the repo root), optional prefixes to leave out, and the most +implementation lines the group may hold: + + {"groups": {"readers": {"paths": ["crates/x/src/formats/"], + "exclude": ["crates/x/src/formats/old/"], + "max": 123}}} + +Usage: + + scripts/loc-budget.py # print every group's count + scripts/loc-budget.py --check # exit 1 when a group is over budget + scripts/loc-budget.py --warn # annotate groups over budget, exit 0 + scripts/loc-budget.py --lower # rewrite each max down to its count + scripts/loc-budget.py --files GROUP # per-file counts for one group + +``--check`` and ``--warn`` print a GitHub ``::warning::`` annotation per +group over budget; CI uses ``--warn`` so a bug fix that adds lines to these +files never fails on it. ``--lower`` never raises a max: a PR that grows a group raises the +number by hand, where a reviewer sees it. +""" + +from __future__ import annotations + +import argparse +import json +import os +import re +import subprocess +import sys +from dataclasses import dataclass, field +from pathlib import Path + +REPO = Path(__file__).resolve().parent.parent +BUDGET_FILE = REPO / "scripts" / "loc-budget.json" + +CFG_TEST = re.compile(r"#\[cfg\(\s*(?:test|all\(\s*test\b[^\]]*)\)\]") +RAW_STRING = re.compile(r'b?r(#*)"') +CHAR_LITERAL = re.compile(r"'(?:\\(?:x[0-9a-fA-F]{2}|u\{[0-9a-fA-F]+\}|.)|[^\\'])'", re.S) +MOD_DECL = re.compile( + r'(?:#\[path\s*=\s*"([^"]+)"\]\s*)?(?:pub(?:\([^)]*\))?\s+)?mod\s+(\w+)\s*;\s*$' +) + +CODE, COMMENT, STRING = 0, 1, 2 + + +def classify(src: str) -> bytearray: + """Mark every character of Rust source as code, comment or literal.""" + n = len(src) + kind = bytearray(n) + i = 0 + + def ident_before(at: int) -> bool: + return at > 0 and (src[at - 1].isalnum() or src[at - 1] == "_") + + while i < n: + c = src[i] + if src.startswith("//", i): + j = src.find("\n", i) + j = n if j < 0 else j + kind[i:j] = bytes([COMMENT]) * (j - i) + i = j + continue + if src.startswith("/*", i): + depth, j = 0, i + while j < n: + if src.startswith("/*", j): + depth += 1 + j += 2 + elif src.startswith("*/", j): + depth -= 1 + j += 2 + if depth == 0: + break + else: + j += 1 + kind[i:j] = bytes([COMMENT]) * (j - i) + i = j + continue + if c in "rb" and not ident_before(i): + m = RAW_STRING.match(src, i) + if m: + close = '"' + m.group(1) + j = src.find(close, m.end()) + j = n if j < 0 else j + len(close) + kind[i:j] = bytes([STRING]) * (j - i) + i = j + continue + if c == '"' or (src.startswith('b"', i) and not ident_before(i)): + j = i + (2 if c == "b" else 1) + while j < n and src[j] != '"': + j += 2 if src[j] == "\\" else 1 + j = min(j + 1, n) + kind[i:j] = bytes([STRING]) * (j - i) + i = j + continue + if c == "'": + m = CHAR_LITERAL.match(src, i) + if m: + kind[i : m.end()] = bytes([STRING]) * (m.end() - i) + i = m.end() + continue + i += 1 + return kind + + +@dataclass +class FileCount: + code: int + test: int + test_mods: list[tuple[str, str | None]] = field(default_factory=list) + + +def count_source(src: str) -> FileCount: + """Count code lines in one Rust file, splitting off ``cfg(test)`` items.""" + kind = classify(src) + n = len(src) + excluded = bytearray(n) + test_mods: list[tuple[str, str | None]] = [] + for m in CFG_TEST.finditer(src): + start = m.start() + if kind[start] != CODE or excluded[start]: + continue + # The gated item ends at its first top-level `;` or at the brace + # closing its first `{`. + j, depth, end = m.end(), 0, n + while j < n: + if kind[j] == CODE: + ch = src[j] + if ch in "{([": + depth += 1 + elif ch in "})]": + depth -= 1 + if depth == 0 and ch == "}": + end = j + 1 + break + elif ch == ";" and depth == 0: + end = j + 1 + break + j += 1 + item = src[m.end() : end] + decl = MOD_DECL.search(item) + if decl and "{" not in item: + test_mods.append((decl.group(2), decl.group(1))) + excluded[start:end] = b"\x01" * (end - start) + + code_lines: set[int] = set() + test_lines: set[int] = set() + line = 1 + for k, ch in enumerate(src): + if ch == "\n": + line += 1 + elif not ch.isspace() and kind[k] != COMMENT: + (test_lines if excluded[k] else code_lines).add(line) + return FileCount(len(code_lines), len(test_lines - code_lines), test_mods) + + +def resolve_module(declaring: str, name: str, path_attr: str | None) -> str | None: + """The file a ``mod name;`` in ``declaring`` loads, if it exists.""" + d, base = os.path.split(declaring) + owns_dir = base in ("mod.rs", "lib.rs", "main.rs") + sub = d if owns_dir else os.path.join(d, base[:-3]) + if path_attr: + candidates = [os.path.join(sub, path_attr), os.path.join(d, path_attr)] + else: + candidates = [os.path.join(sub, name + ".rs"), os.path.join(sub, name, "mod.rs")] + for c in candidates: + if (REPO / c).is_file(): + return os.path.normpath(c) + return None + + +def rust_files() -> list[str]: + out = subprocess.check_output(["git", "ls-files", "*.rs"], cwd=REPO, text=True) + return [ + f + for f in out.split() + if not re.search(r"(^|/)(tests|benches)/", f) + ] + + +def count_repo() -> dict[str, int]: + """Implementation lines per tracked Rust file (test-only files count 0).""" + files = rust_files() + counts = {f: count_source((REPO / f).read_text(encoding="utf-8", errors="replace")) for f in files} + test_files: set[str] = set() + for f, fc in counts.items(): + for name, path_attr in fc.test_mods: + target = resolve_module(f, name, path_attr) + if target: + test_files.add(target) + # A module below a test-only module is test-only too. + roots = {t[:-3] if not t.endswith("/mod.rs") else os.path.dirname(t) for t in test_files} + for f in files: + if any(f.startswith(r + "/") for r in roots): + test_files.add(f) + return {f: (0 if f in test_files else fc.code) for f, fc in counts.items()} + + +def in_group(path: str, group: dict) -> bool: + return any(path.startswith(p) for p in group["paths"]) and not any( + path.startswith(p) for p in group.get("exclude", []) + ) + + +def group_total(per_file: dict[str, int], group: dict) -> int: + return sum(c for f, c in per_file.items() if in_group(f, group)) + + +def load_budget() -> dict: + return json.loads(BUDGET_FILE.read_text()) + + +def main(argv: list[str] | None = None) -> int: + ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + mode = ap.add_mutually_exclusive_group() + mode.add_argument("--check", action="store_true", help="exit 1 when a group is over budget") + mode.add_argument("--warn", action="store_true", help="annotate groups over budget, exit 0") + mode.add_argument("--lower", action="store_true", help="lower each max to its current count") + mode.add_argument("--files", metavar="GROUP", help="per-file counts for one group") + args = ap.parse_args(argv) + + budget = load_budget() + groups = budget["groups"] + per_file = count_repo() + + if args.files: + group = groups[args.files] + rows = sorted(((c, f) for f, c in per_file.items() if c and in_group(f, group)), reverse=True) + for c, f in rows: + print(f"{c:7d} {f}") + print(f"{sum(c for c, _ in rows):7d} total") + return 0 + + over = [] + for name, g in groups.items(): + total = group_total(per_file, g) + cap = g["max"] + status = "over" if total > cap else "ok" + print(f"{name:24s} {total:8d} / {cap:8d} {status}") + if total > cap: + over.append((name, total, cap)) + if args.lower and total < cap: + g["max"] = total + print(f"{'all implementation':24s} {sum(per_file.values()):8d}") + + if args.lower: + BUDGET_FILE.write_text(json.dumps(budget, indent=2) + "\n") + if (args.check or args.warn) and over: + for name, total, cap in over: + print( + f"::warning title=loc budget::{name} has {total} implementation lines, " + f"over its budget of {cap} (scripts/loc-budget.json)" + ) + return 1 if args.check else 0 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/tests/test_loc_budget.py b/scripts/tests/test_loc_budget.py new file mode 100644 index 000000000..f3156b1aa --- /dev/null +++ b/scripts/tests/test_loc_budget.py @@ -0,0 +1,140 @@ +"""Tests for scripts/loc-budget.py's Rust line counter.""" + +import importlib.util +import json +import sys +import unittest +from pathlib import Path + +SCRIPT = Path(__file__).resolve().parent.parent / "loc-budget.py" +spec = importlib.util.spec_from_file_location("loc_budget", SCRIPT) +loc_budget = importlib.util.module_from_spec(spec) +# dataclasses resolve annotations through sys.modules. +sys.modules["loc_budget"] = loc_budget +spec.loader.exec_module(loc_budget) + + +def count(src: str): + return loc_budget.count_source(src) + + +class CountSourceTest(unittest.TestCase): + def test_blank_and_comment_lines_do_not_count(self): + src = "//! module doc\n\n/// item doc\nfn a() {}\n/* block\n comment */\n" + self.assertEqual(count(src).code, 1) + + def test_nested_block_comment(self): + src = "/* outer /* inner */ still comment */\nfn a() {}\n" + self.assertEqual(count(src).code, 1) + + def test_cfg_test_module_is_excluded(self): + src = ( + "fn a() {}\n" + "#[cfg(test)]\n" + "mod tests {\n" + " #[test]\n" + " fn t() { assert!(true); }\n" + "}\n" + "fn b() {}\n" + ) + fc = count(src) + self.assertEqual(fc.code, 2) + self.assertEqual(fc.test, 5) + + def test_cfg_all_test_is_excluded_but_cfg_any_is_not(self): + src = ( + "#[cfg(all(test, unix))]\nfn only_test() {}\n" + "#[cfg(any(test, feature = \"x\"))]\nfn maybe_shipped() {}\n" + ) + self.assertEqual(count(src).code, 2) + + def test_braces_inside_strings_and_chars_do_not_end_the_item(self): + src = ( + "#[cfg(test)]\n" + "mod tests {\n" + ' const S: &str = "}";\n' + " const C: char = '}';\n" + ' const R: &str = r#"}"#;\n' + " // }\n" + "}\n" + "fn shipped() {}\n" + ) + self.assertEqual(count(src).code, 1) + + def test_lifetimes_are_not_char_literals(self): + src = ( + "#[cfg(test)]\n" + "fn helper<'a>(s: &'a str) -> &'a str { s }\n" + "fn shipped<'b>(s: &'b str) -> &'b str { s }\n" + ) + self.assertEqual(count(src).code, 1) + + def test_cfg_test_inside_a_string_is_ignored(self): + src = 'const DOC: &str = "#[cfg(test)]";\nfn a() {}\n' + self.assertEqual(count(src).code, 2) + + def test_test_module_declarations_are_reported(self): + src = ( + "#[cfg(test)]\nmod tests;\n" + '#[cfg(test)]\n#[path = "x_tests.rs"]\nmod x_tests;\n' + "mod shipped;\n" + ) + fc = count(src) + self.assertEqual(fc.test_mods, [("tests", None), ("x_tests", "x_tests.rs")]) + self.assertEqual(fc.code, 1) + + +class ModeTest(unittest.TestCase): + """--check fails over budget, --warn only annotates.""" + + def setUp(self): + self.saved = (loc_budget.load_budget, loc_budget.count_repo) + loc_budget.load_budget = lambda: { + "groups": { + "g": {"paths": ["a/"], "exclude": ["a/skip/"], "max": 10}, + } + } + loc_budget.count_repo = lambda: {"a/x.rs": 8, "a/y.rs": 5, "a/skip/z.rs": 100, "b/w.rs": 1} + + def tearDown(self): + loc_budget.load_budget, loc_budget.count_repo = self.saved + + def run_main(self, *argv): + import contextlib + import io + + out = io.StringIO() + with contextlib.redirect_stdout(out): + code = loc_budget.main(list(argv)) + return code, out.getvalue() + + def test_exclude_and_over_budget(self): + code, out = self.run_main("--check") + self.assertEqual(code, 1) + self.assertIn("g 13 / 10 over", out) + self.assertIn("::warning title=loc budget::g has 13", out) + + def test_warn_annotates_but_passes(self): + code, out = self.run_main("--warn") + self.assertEqual(code, 0) + self.assertIn("::warning title=loc budget::g has 13", out) + + def test_report_mode_never_fails(self): + code, out = self.run_main() + self.assertEqual(code, 0) + self.assertNotIn("::warning", out) + + +class BudgetFileTest(unittest.TestCase): + def test_budget_file_is_well_formed(self): + budget = json.loads(loc_budget.BUDGET_FILE.read_text()) + self.assertTrue(budget["groups"]) + for name, group in budget["groups"].items(): + self.assertIsInstance(group["max"], int, name) + self.assertTrue(group["paths"], name) + for p in group["paths"]: + self.assertTrue((loc_budget.REPO / p).exists(), f"{name}: {p} does not exist") + + +if __name__ == "__main__": + unittest.main() From a08a113053e6e93841c203bd48ecda9796e7080d Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 10:13:50 -0600 Subject: [PATCH 02/36] Keep the origin classifier out of formats/ formats/ models are guarded to stay free of I/O and of the host policy (formats/mod.rs architecture_tests bans hosted_patch_uuid there), so the classifier is a top-level `origin` module and a project layer applies it. Spell out the three layers and where each part of the classifier moves from. Co-Authored-By: Claude Opus 5.5 --- docs/design/lock-model-unification.md | 38 ++++++++++++++++++++------- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/docs/design/lock-model-unification.md b/docs/design/lock-model-unification.md index 9da9c15cb..19084b71c 100644 --- a/docs/design/lock-model-unification.md +++ b/docs/design/lock-model-unification.md @@ -51,6 +51,19 @@ backends. ## The model +### Layers + +1. **`formats/`: pure models.** Text in, records out. Each record carries + the raw location strings the lock spells (resolved URL, tarball, path, + source). The existing architecture test keeps these models free of I/O + and of the host policy, and that rule stays. +2. **`origin`: the one classifier.** A top-level module, not part of + `formats/`, because it applies the host policy. See [Origin](#origin). +3. **The project layer.** Reads files through `ProjectView`, picks install + trees, joins files where a format needs it, and turns each record's raw + location into an `Origin`. Discovery, the inventory and the writers all + consume it. + ### Entries Every reader yields `LockEntry` records. Each record holds the package @@ -79,14 +92,21 @@ flags: bundled, link, the v2 mirror, a `hasShrinkwrap` ancestor, Yarn's `live`, Berry's checksum cache key, and so on. These are fields of the format's record, not new side channels. -One classifier, `formats::origin`, decides `Origin`. It is discovery's -validators moved, not rewritten: +### Origin + +One classifier, the `origin` module, decides `Origin`. It is moved, not +rewritten, from two places: + +* from `patch::redirect`: `hosted_patch_uuid` (Socket's patch server or a + configured origin, the last canonical-uuid path segment) and + `SOCKET_PATCH_SERVER_HOST`. The hosted unwind already calls these + directly; +* from discovery: `OriginPolicy::hosted_uuid` (the same rule with a memoized + fast path for plain registry URLs), `vendor_ref`, `vendor_ref_decorated` + and `vendor_uuid_dir`, `locate`, `Wired`, and the `socket-patch-` + name grammar. -* the Socket host allowlist and the last-canonical-uuid rule - (`hosted_uuid`); -* `vendor_ref`, `vendor_ref_decorated` and `vendor_uuid_dir`; -* the `socket-patch-` name grammar; -* the validating purl builders. +The validating purl builders already live in `utils::purl`. The inventory's any-host matching goes away. That is a behavior change only where a non-allowlisted host serves a URL that looks like Socket's, and it @@ -217,13 +237,13 @@ by line. | PR | Branch | Change | |---|---|---| | 0 | `lock-model/0-design` | This document and the size budget | -| 1 | `lock-model/1-origin` | `LockEntry`, `Origin` and `formats::origin` (discovery's classifier, moved). Discovery calls it. The inventory doesn't yet. | +| 1 | `lock-model/1-origin` | The `origin` module: discovery's classifier and the redirect's hosted-URL rule, moved verbatim. Discovery and the redirect re-export it. | | 2 | `lock-model/2-pnpm` | Pilot. One pnpm reader feeds the inventory, discovery and the hosted planner. Removes `InstalledPackages`, `vendored_npm_uuids`, the whole-text planner walk and the second Rush listing. | | 3 | `lock-model/3-npm` | package-lock and shrinkwrap. Folds `vendor/npm_origin.rs`'s second walk into the records. | | 4 | `lock-model/4-yarn` | Classic and Berry. One block model for every reader and writer, `BerryStanzas` included. | | 5 | `lock-model/5-bun-vlt` | `bun.lock`, `bun.lockb` and vlt. The vlt writers move off their line grammar. | | 6 | `lock-model/6-deno` | The `deno.lock` reader in `formats/`, for all four of its callers | -| 7 | `lock-model/7-python` | One lister, one parse per lock, one include walker and one PEP 508 parser. `pypi_locks.rs` stops parsing TOML itself. Poetry wiring uses `formats::origin`. Hatch gets inventory entries. | +| 7 | `lock-model/7-python` | One lister, one parse per lock, one include walker and one PEP 508 parser. `pypi_locks.rs` stops parsing TOML itself. Poetry wiring uses `origin`. Hatch gets inventory entries. | | 8 | `lock-model/8-cargo-go` | One `Cargo.toml` dependency-table walk. Go `replace` and `go.sum` records. | | 9 | `lock-model/9-gem-composer` | Gemfile.lock and composer.lock records. The inventory stops dropping BOM-prefixed `composer.lock` files. | | 10 | `lock-model/10-nuget` | NuGet records with the config join, `NugetContentHash`, and inventory entries | From 0006f141d5a847ca488827ae136f99ed155f4609 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 10:22:16 -0600 Subject: [PATCH 03/36] Make one set of behaviors the goal, hosted semantics first Owner decisions: inconsistencies between discovery, the inventory and the two modes are bugs to remove, not behavior to preserve; hosted and vendored share one core and differ only in where a rewired entry points; when in doubt the hosted mode's behavior wins, and every such call is listed in the doc so it can be overruled. Behavior changes are allowed per PR, listed, pinned by tests and shown as golden diffs. First decision recorded: pnpm 11's env-lockfile document counts as its own install tree, as the hosted planner already treats it. Co-Authored-By: Claude Opus 5.5 --- docs/design/lock-model-unification.md | 53 +++++++++++++++++++++------ 1 file changed, 41 insertions(+), 12 deletions(-) diff --git a/docs/design/lock-model-unification.md b/docs/design/lock-model-unification.md index 19084b71c..742d272fe 100644 --- a/docs/design/lock-model-unification.md +++ b/docs/design/lock-model-unification.md @@ -42,12 +42,36 @@ discovery reads. ## Goal -One per-project lock model. Every consumer reads it, and the end state has -fewer implementation lines than today. `scripts/loc-budget.py` tracks the -lines (see [Size budget](#size-budget)). - -Non-goals: changing what any mode writes, or adding new hosted or vendored -backends. +One per-project lock model and one set of behaviors. Every consumer reads +the model, and the end state has fewer implementation lines than today. +`scripts/loc-budget.py` tracks the lines (see [Size budget](#size-budget)). + +## Principles (owner decisions, 2026-11-04) + +1. **One set of behaviors.** Discovery (VEX), the inventory, hosted mode and + vendored mode answer each question the same way: which files count, + which entries are installed, what is Socket wiring, what a malformed + entry means. An inconsistency between them is a bug to remove, not + behavior to preserve. +2. **Hosted and vendored share the core.** The two modes differ only in where + a rewired entry points (a Socket URL or a committed `.socket/vendor/` + path). Everything else, from reading the lock to choosing entries and + refusing unsafe states, is one implementation. +3. **When in doubt, hosted semantics are correct.** Where the modes or + readers disagree and the right answer isn't clear, the hosted mode's + current behavior wins. Every such call is listed under + [Decisions](#decisions-under-the-hosted-rule), so it can be overruled. +4. **Behavior changes are allowed, and visible.** A PR that aligns behavior + lists each change in its description, adds a test that pins the new + behavior, and shows its golden diffs. + +Non-goal: adding new hosted or vendored backends. + +## Decisions under the hosted rule + +| Question | Readers before | Decision | +|---|---|---| +| Is pnpm 11's env-lockfile document (the first YAML document, holding `configDependencies` / `packageManagerDependencies`) part of the project? | The hosted planner and the v9 vendored in-use check read it. `PnpmLock` (inventory and discovery), `InstalledPackages` and the legacy vendored check skip it (#1949). | It counts. pnpm installs config dependencies (into its config store, not `node_modules`), and the hosted planner patches them. The document becomes its own install tree. | ## The model @@ -229,16 +253,21 @@ add code and raise the readers' maximum by hand, saying why. ## Stack -Each PR is based on the one before. PRs 1 to 11 must leave -`tests/fixtures/vex-discover-golden/` and the redirect goldens byte for byte -unchanged. Only PR 12 may change them, and its golden diff is reviewed line -by line. +Each PR is based on the one before. A PR that changes behavior follows +principle 4: it lists every change, pins it with a test and shows its golden +diffs (`tests/fixtures/vex-discover-golden/`, `tests/fixtures/redirect/`). + +Each format PR does two things for its format: one reader model that +discovery, the inventory and both modes consume, with every divergence +resolved under the principles; and, where the hosted and vendored planners +duplicate each other, one planner with the target as its only parameter. +A format whose planner merge is large splits into two PRs, readers first. | PR | Branch | Change | |---|---|---| | 0 | `lock-model/0-design` | This document and the size budget | | 1 | `lock-model/1-origin` | The `origin` module: discovery's classifier and the redirect's hosted-URL rule, moved verbatim. Discovery and the redirect re-export it. | -| 2 | `lock-model/2-pnpm` | Pilot. One pnpm reader feeds the inventory, discovery and the hosted planner. Removes `InstalledPackages`, `vendored_npm_uuids`, the whole-text planner walk and the second Rush listing. | +| 2 | `lock-model/2-pnpm` | Pilot. One pnpm entry walk for the inventory, discovery, the store crawler, the hosted planner and the vendored in-use check. The env-lockfile document becomes its own tree (see Decisions). | | 3 | `lock-model/3-npm` | package-lock and shrinkwrap. Folds `vendor/npm_origin.rs`'s second walk into the records. | | 4 | `lock-model/4-yarn` | Classic and Berry. One block model for every reader and writer, `BerryStanzas` included. | | 5 | `lock-model/5-bun-vlt` | `bun.lock`, `bun.lockb` and vlt. The vlt writers move off their line grammar. | @@ -248,7 +277,7 @@ by line. | 9 | `lock-model/9-gem-composer` | Gemfile.lock and composer.lock records. The inventory stops dropping BOM-prefixed `composer.lock` files. | | 10 | `lock-model/10-nuget` | NuGet records with the config join, `NugetContentHash`, and inventory entries | | 11 | `lock-model/11-jvm` | One `checksums.sha256` reader, one Gradle index parser, a `verification-metadata.xml` reader, then Gradle-lock and root-pom inventory entries and sbt's owned-file entries | -| 12 | `lock-model/12-install-trees` | Install-tree selection, Python trees, Deno governance, the single classifier for the inventory, liveness on `Origin`. Deletes the npm flavor router's lock choice, the Python precedence rules, `resolved_elsewhere` and the `UnwiredCopy` plumbing. | +| 12 | `lock-model/12-install-trees` | Install-tree selection across formats, Python trees, Deno governance, the single classifier for the inventory, liveness on `Origin`. Deletes the npm flavor router's lock choice, the Python precedence rules, `resolved_elsewhere` and the `UnwiredCopy` plumbing. | | 13 | `lock-model/13-cleanup` | Removes the raw-text liveness fallbacks if the model makes them redundant, plus whatever is left of `vendor/lock_inventory/`. Final budget. | ## Open questions From 950f9fe8128806679a1fcb600d2475dc385c1411 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 10:34:01 -0600 Subject: [PATCH 04/36] Catalog where readers and modes disagree docs/design/lock-model-divergences.md lists every question on which discovery, the inventory, hosted and vendored mode answer differently today, across all ecosystems, with the resolution under the principles and flags for false attestations and target-specific differences. Owner decisions recorded: the modes are ground truth over the readers; vendored mode's Maven profile and reactor support and its scala-cli wiring are back-ported to hosted as the planners merge (the JVM PR splits into readers, Maven planner, and Gradle/sbt/scala-cli). One call is pending: whether hosted keeps rewriting a shrinkwrap.yaml that pnpm never reads. Corrects the install-tree section: npm-shrinkwrap.json does not shadow package-lock.json; every consumer but the inventory already reads both. Co-Authored-By: Claude Opus 5.5 --- docs/design/lock-model-divergences.md | 136 ++++++++++++++++++++++++++ docs/design/lock-model-unification.md | 29 ++++-- 2 files changed, 157 insertions(+), 8 deletions(-) create mode 100644 docs/design/lock-model-divergences.md diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md new file mode 100644 index 000000000..1fb560ff6 --- /dev/null +++ b/docs/design/lock-model-divergences.md @@ -0,0 +1,136 @@ +# Where the readers and modes disagree + +Companion to [lock-model-unification.md](lock-model-unification.md). Each +row is a question that discovery (VEX), the lock inventory, hosted mode and +vendored mode answer differently today, with its resolution under that +document's [principles](lock-model-unification.md#principles-owner-decisions-2026-11-04). +Paths are relative to `crates/socket-patch-core/src/`. Line numbers are from +the survey of 2026-11-04 and will drift. + +Flags: + +* **false attestation**: a reader attests a copy the install may not use + today. Fixing these is a correctness priority. +* **needs owner**: following hosted would be unsafe or would delete a + working feature. Listed under "Calls that go against hosted" for the + owner to decide. +* **target-specific**: the difference comes from where the entry points + (a Socket URL or a committed path) and stays. + +## Calls that go against hosted (owner decided 2026-11-04) + +Owner: the false Maven attestations get fixed, the modes are ground truth +over the readers, and the vendored mode's profile, reactor and scala-cli +support is back-ported so hosted gains it as the planners merge. P3 and P4 +follow from the install-tree decision. + +| # | Question | Hosted today | Decision | +|---|---|---|---| +| J1 | Does a base-version Maven dependency declared in a `` block block attestation? | Warns, still pins the top level and counts the package as redirected (`patch/redirect/mod.rs` ~8744) | Vendored's rule: any profile declaration at the base version makes the ref unattested (fixed now in discovery), and the merged planner rewrites profile declarations in both modes. | +| J3 | Which Maven poms count? | Refuses a reactor root (~8572) | The reactor reader becomes the shared core, so hosted gains reactor support. | +| J12 | scala-cli | Guidance only (`patch/redirect/scala_guidance.rs`) | Back-port: hosted wires scala-cli through the merged planner (a same-GAV Socket repository instead of a suffixed version). | +| P3 | Is a shadowed `pdm.lock` rewritten? | Only when it governs (`pdm_drives`, ~564) | Rewrite it like every other tree, per the install-tree decision. | +| P4 | Are `-r` includes of `requirements.txt` rewritten? | Root file only (`patch/redirect/requirements.rs` ~208) | Follow includes, per the install-tree decision. | +| N11 | Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` rewritten? | Yes; its all-or-nothing gate covers both files | **Pending.** Proposed: treat it as shadowed (swept for stale identities, never written). pnpm never reads it, and an unparseable debris key in it can refuse the dependency in every lock. | + +## JVM + +| # | Question | Today | Resolution | Flag | +|---|---|---|---|---| +| J2 | Does an executable classifier variant (`tests`, native) of the patched package block attestation? | Hosted skips the dependency. Discovery marks it unattested. Vendored only warns, then `entry_wired_checked` attests (`vendor/jvm/maven_reactor.rs` ~1884, `vendor/jvm/apply.rs` ~1042). | Hosted | false attestation | +| J4 | How are `${prop}` versions resolved, and does `.mvn/maven.config` win? | Discovery: one level of pom properties, no `-D` user properties. Hosted refuses `${…}`. Vendored: user properties, parent chain, profiles, depth 16. | One interpolator (vendored's). Hosted's refusal stays a writer policy. | false attestation | +| J5 | Do Gradle lockfiles gate liveness? | Hosted and discovery check every lock. Vendored reads none (`vendor/jvm/gradle.rs` ~1227). | Hosted | false attestation | +| J6 | Is sbt resolution evidence required before attesting? | Hosted: no integrity without verified evidence. Vendored: `wired_checked` only checks the owned-file row; discovery's `vendored_check` diagnostics are never consumed. | Hosted. One evidence route (`ResolutionDoc` + `gate::check_new`). | false attestation | +| J7 | Another version of a mapped NuGet id is locked | Both writers refuse. Discovery emits a ref per locked version, keyed by raw text, not `normalize_nuget_version`. | Hosted: a diagnostic, no ref | false attestation | +| J8 | What counts as "wired" for vendored Maven liveness and GC? | Vendored: substring match on masked text, so a suffix in ``, a profile or `` counts. Discovery and hosted use the scoped `PomScope`. | Scoped reader | | +| J9 | `checksums.sha256` and the `TRUSTED_CHECKSUMS_ON` switch | Four readers with different splitting and switch tests. Hosted's merge silently drops lines it can't split on two spaces. | One reader with the hosted grammar. The merge keeps lines it can't parse. | | +| J10 | Gradle Socket index rows | Two parsers. Hosted validates the uuid, URL and hashes; vendored accepts any token and uses a raw `split('\t')`. Vendored checks only root settings; hosted checks every build. | One row grammar, hosted's validation | | +| J11 | `verification-metadata.xml` | Two editors, no reader | One editor; a reader for the inventory | | +| J13 | NuGet config scope | Writers read inherited parent configs; discovery doesn't. Discovery refuses a disabled Socket source; the writers don't check ``. | Shared inheritance read. The writers adopt the disabled-source refusal. | | +| J14 | NuGet lock enumeration | In-memory views fall back to the root lock. The unwind bypasses the snapshot view (`governed_locks_on_disk`). | One enumeration over `ProjectView` | | +| J15 | JVM liveness on a read error | `entry_wired` is `unwrap_or(true)` for `scan`/`vendor`; `vex` uses the checked variant | Fail closed everywhere | false attestation | + +## Python + +| # | Question | Today | Resolution | Flag | +|---|---|---|---|---| +| P1 | Which files count? | Hosted rewrites every file except a non-governing `pdm.lock`. Vendored picks one winner and warns that the rest stay unpatched. The inventory uses a chain. Discovery reads every lock with the cross-lock contest. `formats/governing_locks.rs` still gates four callers. | One tree per tool lock (owner decision). Vendored wires every tree. `governing_locks` goes. | | +| P5 | Does the inventory list Socket entries? | It keeps hosted entries (with the patched artifact's URL and hash) but drops vendored uv path sources | Keep both, read as the package they replace. Check that vendor's pristine fetch and `repair` never take patched bytes from a hosted entry. | | +| P6 | What counts as Socket wiring? | Discovery and the unwind use the host allowlist. The inventory accepts any host. Ledger liveness uses `resolved.contains(uuid)`. `poetry_lock::lock_wiring` uses `contains("/patch/pypi/")`. | The `origin` allowlist everywhere | false attestation (liveness substring) | +| P7 | uv workspace roots | Hosted refuses members only. Vendored refuses roots too. | Vendored wires roots, after a test that a member's own `[tool.uv.sources]` can't override the root | | +| P8 | `uv.lock` without `pyproject.toml` | Hosted edits the lock; discovery attests it. Vendored refuses. | Vendored proceeds | | +| P9 | uv 0.2 `[[distribution]]` locks | Hosted rewrites. Vendored refuses (no portable path). The unwind refuses to restore them. | Vendored's refusal is target-specific and stays. The unwind must restore what hosted writes. | target-specific | +| P13 | PDM `lock_version` | Writers use an allowlist; discovery a denylist; the inventory checks nothing | The allowlist everywhere | | +| P14 | `pipfile-spec` | Writers require 6; discovery and the inventory accept any | 6 everywhere | | + +## Cargo, Go, Ruby, Composer + +| # | Question | Today | Resolution | Flag | +|---|---|---|---|---| +| G1 | Which `go.work` and members count? | Discovery: a root `go.work` and in-root members only, ignoring an ancestor `go.work`, `GOWORK` and `-modfile`. Hosted and vendored resolve the workspace go would use (`vendor/go_workspace.rs`). | `go_workspace::load` everywhere | false attestation | +| G16 | Go inventory | Root `go.sum` only, lenient parse, lists `patch.socket.dev/gopatch/` lines as packages | Discovery's model and workspace | | +| C2 | Which manifests decide whether a Cargo lock entry is used? | Hosted pins root and member manifests; vendored reads members. Discovery reads only the root `Cargo.toml`. | Members everywhere | false attestation | +| C11 | `cargo vendor` / local-registry source replacement | Hosted refuses; vendored and discovery ignore it | Stays | target-specific | +| R10 | `gems.rb` / `gems.locked` | Hosted wires the pair bundler loads. Vendored refuses any `gems.rb`. | Vendored supports it | | +| R15 | Malformed Gemfile.lock | Discovery reports it unparseable; the inventory lists what parsed | Discovery's rule everywhere | | +| X12 | `composer.lock` with a BOM | Discovery strips it. The inventory drops the file. Vendored refuses to plan, errors on revert, and reports nothing stranded. | Strip one BOM everywhere. Check that GC never trusts the empty "stranded" answer. | | + +## npm family + +| # | Question | Today | Resolution | Flag | +|---|---|---|---|---| +| N1 | pnpm `gitBranchLockfile` with a branch lock present: is the root lock the install tree? | Hosted withholds the root locks and refuses (`hosted/engine.rs` ~930). Vendored refuses. Discovery and the inventory never check the setting. | Hosted | false attestation | +| N2 | npm user patches (#711) and `bun patch` (#367) | Hosted skips that one dependency and records it as refused. Vendored refuses the whole npm v4 lock, or the one bun dependency. Discovery never reads `patchedDependencies` / `"patched"` and attests a pre-#711 hosted pin. | Hosted's per-dependency skip, in discovery too | false attestation | +| N3 | A workspace member's stray npm/bun/vlt lock | Discovery treats it as absent. Hosted and vendored refuse. The inventory reads it as the project's lock. | The inventory ignores it | | +| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by content, not position | | +| N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere | revert could rewrite another project's wiring | +| N6 | `npm-shrinkwrap.json` and `package-lock.json` | Hosted, vendored and discovery read both, with a contest. The inventory reads the shrinkwrap only. | Both, everywhere | | +| N7 | npm lockfileVersion | Hosted rewrites v1 and v4. Vendored accepts v2/v3 only. The inventory ignores v1. Discovery reads v1 and v2+. | Hosted, once vendored `file:` wiring under npm 6 and npm 12.1 v4 has fixtures | | +| N8 | npm `file:` directory link identity (#1481) | Hosted and discovery give a versionless link its source directory's `name@version`. Vendored matches the entry's own version, wiring the registry copies with no warning. | Hosted | silent vendored success | +| N9 | yarn classic blocks reached only through an alias | Hosted skips them. Vendored rewrites them. Discovery counts an alias block as an install of the real package. | Back-port: hosted rewrites alias blocks (vendored supports what hosted lacks) | | +| N10 | Locks of several npm-family package managers in one project | Hosted rewrites every family; discovery reads every family with the contest. The inventory and vendored pick one family (vlt > bun > pnpm > yarn > npm). | Union (separate trees) | | +| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | Pending owner decision (see above) | needs owner | +| N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. | target-specific (vendored) | +| N13 | Plug'n'Play | Vendored refuses. The hosted berry preflight has no PnP gate. The inventory returns an error, so `scan` sees nothing in a project hosted pinned. | The inventory reads PnP locks. Vendored keeps its refusal. | target-specific (vendored) | +| N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery | target-specific | +| N15 | CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml` | Vendored refuses; the rest handle it | Handle it everywhere | | +| N16 | Lock versions | pnpm: vendored accepts only 5.4/6.0/9.0; hosted any but early shrinkwrap. vlt: vendored refuses a lock with no version; hosted rewrites and warns. | Hosted | | +| N17 | Bun workspaces, yarn classic offline mirror | Refused differently by the two modes | Stays | target-specific | +| N18 | An unparseable npm lock | Hosted warns and skips. Vendored refuses the primary lock. Discovery diagnoses. The inventory returns nothing silently. | Discovery's diagnostic, surfaced by every consumer | | +| N19 | yarn.lock with a stray top-level line | Discovery rejects the file; hosted and the inventory scan its blocks | Discovery's rule | | +| N20 | Yarn's last-block-wins (`live`) | Discovery respects it; the inventory and both hosted yarn rewriters don't | Respect it everywhere | | +| N21 | Accepted npm-family integrity | The inventory records any string as SRI; discovery requires an SRI pin (`sha1-` included) | Discovery's check | | + +## Fixes for false attestations on main + +These are fixed directly against `main`, ahead of the stack, one PR each. +The stack rebases over them. + +| # | Fix | +|---|---| +| J1, J4 | Maven discovery: a profile declaration or a `.mvn/maven.config` user property that resolves the base version leaves the ref unattested | +| J2, J8, J15 | Vendored Maven liveness: executable classifiers block, "wired" is read from scoped declarations, read errors fail closed | +| J5 | Vendored Gradle liveness reads the lock files | +| J6 | Vendored sbt attestation requires resolution evidence | +| J7 | NuGet discovery: no ref for a version the writers refuse | +| N1, N2, G1, C2, P6 | Next batch: pnpm `gitBranchLockfile`, npm/bun user patches, Go workspaces, Cargo members, the liveness substring match | + +## Planner merges (hosted and vendored) + +Estimated implementation lines removable by giving each format one planner +whose only parameter is the target: + +| Format | Today | Estimate | +|---|---|---| +| uv.lock | Vendored's text surgery in `vendor/pypi_uv.rs` duplicates hosted's `PythonLockSession` | 1,000–1,400 | +| NuGet | `add_nuget_source` / `rewrite_nuget` against `vendor/nuget_feed.rs`'s config edit, which says it mirrors it | ~400 | +| Gradle | Build discovery, apply-line helpers, index parse and merge, owned script, verification edits | 500–700 | +| Maven | Hosted is a root-only subset of the reactor planner. Four pom parsers collapse to one (`maven_reactor::Doc` survives). | largest; split readers first | +| sbt | Build-source refusals and per-pin decisions; one evidence route | 150–250 | +| Python router | `detect_pypi_flavor` and hosted's fixed chain become one query over trees | ~300 | +| requirements, Pipenv, Poetry, PDM | Shared lexers or rewriters already; preflight and matchers duplicated | a few hundred | +| Cargo | Dependency-table walk, member enumeration, CRLF handling | a few hundred | +| Go | Version-selected and in-graph checks | ~100 | +| Gem | Pair selection and spec-block moves | a few hundred | +| Composer | Regex splice against JSON re-serialization; one locator and one splice | ~300 | +| Unwind | The hosted `upstream/*` modules mirror the vendored unplans and become one unplan per format | follows the planner merges | diff --git a/docs/design/lock-model-unification.md b/docs/design/lock-model-unification.md index 742d272fe..f70540e55 100644 --- a/docs/design/lock-model-unification.md +++ b/docs/design/lock-model-unification.md @@ -57,10 +57,15 @@ the model, and the end state has fewer implementation lines than today. a rewired entry points (a Socket URL or a committed `.socket/vendor/` path). Everything else, from reading the lock to choosing entries and refusing unsafe states, is one implementation. -3. **When in doubt, hosted semantics are correct.** Where the modes or - readers disagree and the right answer isn't clear, the hosted mode's - current behavior wins. Every such call is listed under - [Decisions](#decisions-under-the-hosted-rule), so it can be overruled. +3. **When in doubt, the modes are ground truth.** Where a reader (discovery, + the inventory) disagrees with what hosted or vendored mode writes, the + mode wins. Between the two modes, hosted's behavior wins, except where + vendored supports something hosted lacks: then the support is + back-ported to hosted as the planners merge (Maven profiles and reactors, + scala-cli). Every call is listed under + [Decisions](#decisions-under-the-hosted-rule) and in + [lock-model-divergences.md](lock-model-divergences.md), so it can be + overruled. 4. **Behavior changes are allowed, and visible.** A PR that aligns behavior lists each change in its description, adds a test that pins the new behavior, and shows its golden diffs. @@ -170,9 +175,15 @@ pub struct InstallTree { Rule: **precedence within a tree, union across trees.** -* npm: `npm-shrinkwrap.json` governs and `package-lock.json` is shadowed. - pnpm: `pnpm-lock.yaml` governs `shrinkwrap.yaml`. Bun: `bun.lock` governs - `bun.lockb`, which `bun_text_lock_drives` already decides. +* npm: `npm-shrinkwrap.json` and `package-lock.json` are both read, as + hosted, vendored and discovery already do, with discovery's contest + between the two. The inventory, which reads only the shrinkwrap today, + follows. pnpm: `pnpm-lock.yaml` governs `shrinkwrap.yaml`. Bun: `bun.lock` + governs `bun.lockb`, which `bun_text_lock_drives` already decides. +* npm-family locks of different package managers in one project (for + example `pnpm-lock.yaml` beside `yarn.lock`) are separate trees, as hosted + and discovery treat them. The inventory and vendored mode stop picking one + family. * **Python (owner decision):** each tool's lock is its own tree, because which lock is installed depends on which tool runs. uv.lock, poetry.lock, pdm.lock, Pipfile.lock, each pylock, each PEP 723 script lock, and @@ -276,7 +287,9 @@ A format whose planner merge is large splits into two PRs, readers first. | 8 | `lock-model/8-cargo-go` | One `Cargo.toml` dependency-table walk. Go `replace` and `go.sum` records. | | 9 | `lock-model/9-gem-composer` | Gemfile.lock and composer.lock records. The inventory stops dropping BOM-prefixed `composer.lock` files. | | 10 | `lock-model/10-nuget` | NuGet records with the config join, `NugetContentHash`, and inventory entries | -| 11 | `lock-model/11-jvm` | One `checksums.sha256` reader, one Gradle index parser, a `verification-metadata.xml` reader, then Gradle-lock and root-pom inventory entries and sbt's owned-file entries | +| 11a | `lock-model/11a-jvm-readers` | One pom model (`maven_reactor::Doc`), one interpolator, one `checksums.sha256` reader, one Gradle index grammar, a `verification-metadata.xml` reader, then Gradle-lock and pom inventory entries and sbt's owned-file entries | +| 11b | `lock-model/11b-maven-planner` | One Maven planner for both modes. Hosted gains reactor and profile support. | +| 11c | `lock-model/11c-gradle-sbt-scala` | One Gradle and one sbt planner, one evidence route. Hosted gains scala-cli wiring. | | 12 | `lock-model/12-install-trees` | Install-tree selection across formats, Python trees, Deno governance, the single classifier for the inventory, liveness on `Origin`. Deletes the npm flavor router's lock choice, the Python precedence rules, `resolved_elsewhere` and the `UnwiredCopy` plumbing. | | 13 | `lock-model/13-cleanup` | Removes the raw-text liveness fallbacks if the model makes them redundant, plus whatever is left of `vendor/lock_inventory/`. Final budget. | From a396590bc69beada3eb132c5398ea0d33fe6b8d7 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 11:18:35 -0600 Subject: [PATCH 05/36] Record user-patch and repo-root decisions Owner decisions: a dependency the user patched themselves is never patched or attested by default, in every mode and package manager, unless the user explicitly overrides; and no reader or writer reads project files outside the git repository root. Co-Authored-By: Claude Opus 5.5 --- docs/design/lock-model-unification.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/design/lock-model-unification.md b/docs/design/lock-model-unification.md index f70540e55..847dd1492 100644 --- a/docs/design/lock-model-unification.md +++ b/docs/design/lock-model-unification.md @@ -77,6 +77,8 @@ Non-goal: adding new hosted or vendored backends. | Question | Readers before | Decision | |---|---|---| | Is pnpm 11's env-lockfile document (the first YAML document, holding `configDependencies` / `packageManagerDependencies`) part of the project? | The hosted planner and the v9 vendored in-use check read it. `PnpmLock` (inventory and discovery), `InstalledPackages` and the legacy vendored check skip it (#1949). | It counts. pnpm installs config dependencies (into its config store, not `node_modules`), and the hosted planner patches them. The document becomes its own install tree. | +| Does Socket patch a dependency the user patched themselves (npm `patchedDependencies` / `"patched"`, `bun patch`, pnpm `patchedDependencies`, yarn berry `patch:`, vlt patches)? | Hosted skips npm and bun user patches (#711, #367). Vendored refuses them. Discovery attested them until #2010. pnpm, yarn and vlt user patches are not detected anywhere. | **Owner decision:** never, by default, in every mode and every package manager. A Socket pin under a user patch is not attested. Only an explicit user override patches it. | +| May a reader or writer read project files outside the git repository root? | Several do: vendored paths spelled `../.socket/vendor/…`, an ancestor `go.work`, out-of-root `-r` includes, parent NuGet configs. | **Owner decision:** no. Every project-file read is bounded to the git repository root (the project root when there is no repository). Machine-level tool configuration (for example `GOENV`, the user `NuGet.Config`) is listed separately for review. | ## The model From 134d40eb593e88ce25770b47215589fcd06e40df Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 10:14:33 -0600 Subject: [PATCH 06/36] Move the Socket origin classifier into one module Discovery's classifier (the memoized hosted_uuid fast path, vendor_ref and its variants, locate, Wired, the socket-patch- name grammar) and the redirect's hosted URL rule (hosted_patch_uuid, hosted_patch_url_uuids, SOCKET_PATCH_SERVER_HOST) now live in crates/socket-patch-core/src/origin.rs, with their tests. The code moves verbatim. DiscoverCtx holds an OriginPolicy (the per-run origin allowlist and its memoized host set) and delegates hosted_uuid and locate to it. vex::discover and patch::redirect re-export the moved items, so no caller changes. origin.rs sits outside formats/ on purpose: format models are guarded to apply no host policy (formats/mod.rs architecture_tests). Budget: lock-readers 30,898 -> 30,976 (49 lines moved in from patch/redirect, ~30 lines of OriginPolicy and re-export glue); lock-writers 88,363 -> 88,314. No behavior change. Co-Authored-By: Claude Opus 5.5 --- crates/socket-patch-core/src/lib.rs | 1 + crates/socket-patch-core/src/origin.rs | 849 ++++++++++++++++++ .../src/patch/redirect/mod.rs | 214 +---- .../socket-patch-core/src/vex/discover/mod.rs | 605 +------------ scripts/loc-budget.json | 5 +- 5 files changed, 878 insertions(+), 796 deletions(-) create mode 100644 crates/socket-patch-core/src/origin.rs diff --git a/crates/socket-patch-core/src/lib.rs b/crates/socket-patch-core/src/lib.rs index c0a9cacb0..a8e08158b 100644 --- a/crates/socket-patch-core/src/lib.rs +++ b/crates/socket-patch-core/src/lib.rs @@ -7,6 +7,7 @@ pub mod hash; pub mod hosted; pub mod ledgers; pub mod manifest; +pub mod origin; pub mod patch; pub mod policy; pub mod rollout; diff --git a/crates/socket-patch-core/src/origin.rs b/crates/socket-patch-core/src/origin.rs new file mode 100644 index 000000000..27714f2d4 --- /dev/null +++ b/crates/socket-patch-core/src/origin.rs @@ -0,0 +1,849 @@ +//! Socket origin: whether a string a lockfile or package-manager config +//! records routes a package to a Socket patch, and to which one. +//! +//! This is the one classifier every reader applies: +//! +//! * **hosted** — [`hosted_patch_uuid`]: an `http(s)` url on Socket's patch +//! server or on an operator-configured origin, the patch uuid its last +//! canonical-uuid path segment. [`OriginPolicy::hosted_uuid`] answers the +//! same question with a memoized fast path for the plain registry urls +//! every lock is full of; +//! * **vendored** — [`vendor_ref`] / [`vendor_ref_decorated`] / +//! [`vendor_uuid_dir`]: a path that STARTS at this project's +//! `.socket/vendor///`, its leaf traversal-safe; +//! * **names** — [`socket_patch_name_uuid`]: the `socket-patch-` +//! registry, repository and source names. +//! +//! [`OriginPolicy::locate`] computes the hosted and vendored answers for one +//! string at once. +//! +//! The format models ([`crate::formats`]) stay pure: they hand out the raw +//! location strings and never apply the host policy (their architecture +//! test bans it). The readers that apply a model to a project apply this +//! module. See `docs/design/lock-model-unification.md`. + +use std::collections::BTreeSet; +use std::sync::OnceLock; + +use crate::patch::path_safety::{is_canonical_uuid, is_safe_multi_segment}; +use crate::utils::purl_key::canonical_base_purl; +use crate::vendor::path::{leaf_to_purl, ECOSYSTEM_DIRS, VENDOR_DIR}; + +// ── hosted urls ────────────────────────────────────────────────────────── + +/// Public host of Socket's patch server: the origin every production hosted +/// artifact / registry URL is served from (`https://patch.socket.dev/patch/…`, +/// `…/patch-registry/…`), and the root of the Go module namespace +/// [`crate::vendor::go_mod_edit::HOSTED_GO_MODULE_PREFIX`]. +pub const SOCKET_PATCH_SERVER_HOST: &str = "patch.socket.dev"; + +/// The Socket patch uuid a lockfile-recorded HOSTED reference names, or +/// `None` when `url` is not a Socket-hosted patch URL — the inverse of the +/// rewriters, used by `vex`'s manifest-less lockfile discovery. +/// +/// Recognition is deliberately strict, because the answer decides whether a +/// committed (tamper-able) lockfile line becomes an attestation input: +/// +/// * the ORIGIN must be Socket's patch server (`https://` + +/// [`SOCKET_PATCH_SERVER_HOST`]) or one of `extra_origins` — the +/// operator's `--patch-server-url` deployment, compared on scheme + host + +/// port. A uuid inside any other host's URL is a user's own dependency +/// source, never a patch reference; +/// * no userinfo — a Socket-written URL never carries credentials; +/// * the uuid is the LAST path segment passing the canonical-uuid grammar: +/// hosted URLs carry the grant token in the level before the uuid +/// (`…/patch/npm/////`, +/// `…/patch-registry////…`), and grant tokens may +/// themselves be uuid-shaped, so "the first uuid" would elect the token. +/// +/// Every spelling the lock formats record the same URL in is accepted: a +/// `#fragment` (yarn classic `#`, pip `#sha256=`) and a `?query` are +/// ignored; `\/`-escaped slashes (older composer locks) are unescaped; a +/// wholly percent-encoded URL (yarn berry's `__archiveUrl=` binding) is +/// decoded; a cargo source-kind prefix (`sparse+`, `registry+`) is dropped. +/// Path segments are percent-decoded AFTER splitting, so an encoded `/` +/// can never manufacture a segment. +pub fn hosted_patch_uuid(url: &str, extra_origins: &[String]) -> Option { + hosted_patch_url_uuids(url, extra_origins)?.pop() +} + +/// EVERY canonical-uuid path segment of a Socket-HOSTED url, in path order +/// (the grant token first when it is uuid-shaped, the patch uuid last), or +/// `None` when `url` is not on an accepted origin — [`hosted_patch_uuid`]'s +/// exact acceptance rules, without electing one segment. `vex` discovery +/// uses it to RECOGNIZE every Socket identity a lockfile mentions, including +/// the malformed or rejected shapes whose "last uuid" is not a patch. +pub fn hosted_patch_url_uuids(url: &str, extra_origins: &[String]) -> Option> { + use crate::patch::path_safety::is_canonical_uuid; + use crate::utils::purl::percent_decode_purl_component; + + let unescaped = url.trim().replace("\\/", "/"); + let lower = unescaped.to_ascii_lowercase(); + let decoded = if lower.starts_with("https%3a%2f%2f") || lower.starts_with("http%3a%2f%2f") { + percent_decode_purl_component(&unescaped).into_owned() + } else { + unescaped + }; + // `sparse+https://…` / `registry+https://…` (Cargo.lock `source`). + let (scheme, _) = decoded.split_once("://")?; + let text = match scheme.rsplit_once('+') { + Some((kind, _)) if !kind.is_empty() && kind.bytes().all(|b| b.is_ascii_alphabetic()) => { + &decoded[kind.len() + 1..] + } + _ => decoded.as_str(), + }; + let parsed = reqwest::Url::parse(text).ok()?; + if !matches!(parsed.scheme(), "https" | "http") + || !parsed.username().is_empty() + || parsed.password().is_some() + { + return None; + } + let socket_host = parsed.scheme() == "https" + && parsed.host_str() == Some(SOCKET_PATCH_SERVER_HOST) + && parsed.port_or_known_default() == Some(443); + let configured = extra_origins.iter().any(|origin| { + reqwest::Url::parse(origin.trim()).is_ok_and(|o| { + o.scheme() == parsed.scheme() + && o.host_str().is_some() + && o.host_str() == parsed.host_str() + && o.port_or_known_default() == parsed.port_or_known_default() + }) + }); + if !socket_host && !configured { + return None; + } + Some( + parsed + .path_segments()? + .map(|segment| percent_decode_purl_component(segment).into_owned()) + .filter(|segment| is_canonical_uuid(segment)) + .collect(), + ) +} + +/// The host of `url` when it is a PLAIN `http(s)` url — printable ASCII +/// only, no `%`, `\\` or userinfo, a DNS name of letters, digits, `.` and +/// `-` whose last label starts with a letter — lowercased, which is then +/// exactly what [`hosted_patch_url_uuids`]'s url +/// parse would call its host (no decoding, IDNA mapping or IP-address +/// reading applies to such a name). `None` for anything else, which takes +/// the full parse. +fn plain_url_domain(url: &str) -> Option { + let url = url.trim(); + if !url + .bytes() + .all(|b| b.is_ascii_graphic() && b != b'%' && b != b'\\') + { + return None; + } + let (scheme, rest) = url.split_once("://")?; + // `sparse+https://…` / `registry+https://…`, as the parse strips it. + let scheme = match scheme.rsplit_once('+') { + Some((kind, scheme)) + if !kind.is_empty() && kind.bytes().all(|b| b.is_ascii_alphabetic()) => + { + scheme + } + _ => scheme, + }; + if !scheme.eq_ignore_ascii_case("https") && !scheme.eq_ignore_ascii_case("http") { + return None; + } + let authority = rest.split(['/', '?', '#']).next()?; + let host = match authority.rsplit_once(':') { + Some((host, port)) if port.bytes().all(|b| b.is_ascii_digit()) => host, + Some(_) => return None, + None => authority, + }; + let last = host.strip_suffix('.').unwrap_or(host).rsplit('.').next()?; + let plain = !host.is_empty() + && !rest.starts_with('/') + && host + .bytes() + .all(|b| b.is_ascii_alphanumeric() || b == b'.' || b == b'-') + && last.starts_with(|c: char| c.is_ascii_alphabetic()) + && !host + .split('.') + .any(|label| label.len() >= 4 && label[..4].eq_ignore_ascii_case("xn--")); + plain.then(|| host.to_ascii_lowercase()) +} + +/// The host policy one run classifies under: Socket's patch server plus the +/// operator's `--patch-server-url` origins. +#[derive(Debug)] +pub(crate) struct OriginPolicy<'a> { + origins: &'a [String], + /// The hosts a Socket-hosted url can name (see [`Self::hosted_uuid`]). + hosts: OnceLock>, +} + +impl<'a> OriginPolicy<'a> { + pub(crate) fn new(origins: &'a [String]) -> Self { + OriginPolicy { + origins, + hosts: OnceLock::new(), + } + } + + /// The configured extra origins. + pub(crate) fn origins(&self) -> &'a [String] { + self.origins + } + + /// The patch uuid of a Socket-HOSTED url, or `None` for anything else + /// (see [`hosted_patch_uuid`] for the accepted + /// spellings and the host allowlist). + /// + /// A plain url ([`plain_url_domain`]) on any other host is answered + /// without parsing it: every lock entry's registry url comes through + /// here. + pub(crate) fn hosted_uuid(&self, url: &str) -> Option { + if let Some(host) = plain_url_domain(url) { + let hosts = self.hosts.get_or_init(|| { + let mut hosts = BTreeSet::from([SOCKET_PATCH_SERVER_HOST.to_string()]); + hosts.extend(self.origins.iter().filter_map(|o| { + reqwest::Url::parse(o.trim()) + .ok()? + .host_str() + .map(str::to_string) + })); + hosts + }); + if !hosts.contains(&host) { + return None; + } + } + hosted_patch_uuid(url, self.origins) + } + + /// Classify `s` (see [`Located`]). + pub(crate) fn locate(&self, s: &str, opts: LocateOpts) -> Located { + let vendored = match opts.grammar { + LeafGrammar::Literal => vendor_ref(s), + LeafGrammar::Decorated => vendor_ref_decorated(s), + }; + let decorated_leaf = opts.decorated_check + && opts.grammar == LeafGrammar::Literal + && is_decorated_vendor_path(s); + Located { + vendored, + hosted: self.hosted_uuid(s), + decorated_leaf, + } + } +} + +// ── names ──────────────────────────────────────────────────────────────── + +/// The uuid of a Socket-owned registry / repository / source NAME: +/// `socket-patch-` (cargo `[registries.*]` + Cargo.toml `registry =`, +/// maven ``, nuget ``) or, with +/// `vendored = true`, `socket-patch-vendor-` (maven's vendored repo +/// id). Exact grammar only — a suffix or a non-canonical uuid is not ours. +pub fn socket_patch_name_uuid(name: &str, vendored: bool) -> Option { + crate::patch::redirect::generation::pin_name_uuid(name.trim(), vendored).map(str::to_string) +} + +// ── vendored paths ─────────────────────────────────────────────────────── + +/// A vendored artifact reference recovered from a wiring string. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct VendorRef { + /// Vendor ecosystem dir (`npm`, `cargo`, `pypi`, …). + pub eco: String, + pub uuid: String, + /// The natural leaf, lock-format suffixes stripped ([`strip_leaf_suffix`]), + /// forward-slashed, no trailing slash. + pub leaf: String, + /// `.socket/vendor///` — the root-relative artifact path. + pub artifact_rel: String, +} + +/// Recover a vendored artifact reference from a lockfile/config string that +/// points into THIS project's `.socket/vendor/`: `file:` / `./` / bare +/// spellings, backslashed Windows spellings, and `\/`-escaped JSON text. +/// +/// Stricter than [`crate::vendor::parse_vendor_path`] (which finds the anchor +/// anywhere, for the orphan sweep): the anchor must START the path. A +/// `../.socket/vendor/…` or absolute spelling points the package manager at +/// an artifact OUTSIDE the project root, and verifying the root's same-named +/// artifact instead would attest bytes the build never consumes. Callers +/// whose format legitimately prefixes the project root (hatch +/// `{root:uri}/`, maven `file://${project.basedir}/`, pnpm-legacy absolute +/// importer paths) strip that exact prefix first. +/// +/// The leaf must be a traversal-safe relative path, taken LITERALLY: npm, +/// pnpm, bun, composer, bundler, go, cargo, maven, nuget and every pypi +/// lock read the string as a file path, so a leaf carrying `#` or `::` is +/// rejected (`None`) rather than cut — `…/foo-1.0.0.tgz#evil.tgz` is a file +/// of that literal name, which npm installs while the cut leaf would verify +/// the good `foo-1.0.0.tgz` beside it. The formats whose grammar DEFINES +/// those decorations use [`vendor_ref_decorated`] instead. +pub fn vendor_ref(s: &str) -> Option { + vendor_ref_with(s, false) +} + +/// [`vendor_ref`] for the formats that decorate a vendored path: yarn +/// classic `#`, berry `#./…::hash=…` fragments and `::locator=…` +/// bindings, and url-form pip / Hatch references (`file:…#sha256=…`, +/// `{root:uri}/…#sha256=…`). The leaf keeps only its natural name — the +/// decorations ([`strip_leaf_suffix`]) are cut before the literal-path +/// checks. +pub fn vendor_ref_decorated(s: &str) -> Option { + vendor_ref_with(s, true) +} + +fn vendor_ref_with(s: &str, decorated: bool) -> Option { + let rest = root_anchored_vendor_rest(s)?; + let mut it = rest.splitn(3, '/'); + let eco = it.next()?; + let uuid = it.next()?; + let leaf_raw = it.next()?; + if !ECOSYSTEM_DIRS.contains(&eco) || !is_canonical_uuid(uuid) { + return None; + } + let leaf = if decorated { + strip_leaf_suffix(leaf_raw) + } else if leaf_raw.contains('#') || leaf_raw.contains("::") { + return None; + } else { + leaf_raw + }; + let leaf = leaf.trim_end_matches('/'); + if leaf.is_empty() || !is_safe_multi_segment(leaf) { + return None; + } + Some(VendorRef { + eco: eco.to_string(), + uuid: uuid.to_string(), + leaf: leaf.to_string(), + artifact_rel: format!("{VENDOR_DIR}/{eco}/{uuid}/{leaf}"), + }) +} + +/// `(eco, uuid)` for a root-anchored string that ends AT the uuid dir +/// (`.socket/vendor/nuget/`, a maven repository url's path) — the +/// shapes [`vendor_ref`] rejects for having no leaf. +pub fn vendor_uuid_dir(s: &str) -> Option<(String, String)> { + let rest = root_anchored_vendor_rest(s)?; + let (eco, uuid) = rest.trim_end_matches('/').split_once('/')?; + if !ECOSYSTEM_DIRS.contains(&eco) || !is_canonical_uuid(uuid) { + return None; + } + Some((eco.to_string(), uuid.to_string())) +} + +/// Whether `s` is a vendored artifact path [`vendor_ref`] rejects ONLY for +/// a `#` / `::` in its leaf (what [`vendor_ref_decorated`] would have cut): +/// to a literal-path format that names another file, which extractors +/// diagnose rather than skip silently. +pub(crate) fn is_decorated_vendor_path(s: &str) -> bool { + vendor_ref(s).is_none() && vendor_ref_decorated(s).is_some() +} + +/// Everything after a LEADING `.socket/vendor/` anchor (after `file:`, one +/// `./`, backslash and `\/` normalization), or `None`. +fn root_anchored_vendor_rest(s: &str) -> Option { + let norm = s.trim().replace("\\/", "/").replace('\\', "/"); + let norm = norm.strip_prefix("file:").unwrap_or(&norm); + let norm = norm.strip_prefix("./").unwrap_or(norm); + norm.strip_prefix(&format!("{VENDOR_DIR}/")) + .map(str::to_string) +} + +/// Cut the lock-format decorations off a vendored leaf: everything from the +/// first `#` (yarn classic `#`, berry `#./::hash=…`, pip/hatch +/// `#sha256=…`) or `::` (berry `::locator=…`). +pub fn strip_leaf_suffix(leaf: &str) -> &str { + let cut = [leaf.find('#'), leaf.find("::")] + .into_iter() + .flatten() + .min() + .unwrap_or(leaf.len()); + &leaf[..cut] +} + +/// The canonical base purl a vendored leaf names (see +/// [`crate::vendor::path`]'s leaf table), suffixes stripped and pypi names +/// canonicalized. `None` when the leaf is not a recognizable artifact name. +pub fn vendored_leaf_purl(eco: &str, leaf: &str) -> Option { + leaf_to_purl(eco, strip_leaf_suffix(leaf).trim_end_matches('/')) + .map(|purl| canonical_base_purl(&purl)) +} + +/// How a lock format spells a vendored leaf: literally ([`vendor_ref`]) or +/// with `#` / `::` decorations after it ([`vendor_ref_decorated`]). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum LeafGrammar { + /// The string is a file path: a `#` / `::` belongs to the leaf. + Literal, + /// The format defines `#` / `::` decorations after the leaf (yarn + /// classic / berry, url-form pip and Hatch references). + Decorated, +} + +/// What [`OriginPolicy::locate`] computes for one wiring string. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) struct LocateOpts { + pub(crate) grammar: LeafGrammar, + /// [`LeafGrammar::Literal`] only: also report a vendored path whose leaf + /// carries a decoration ([`is_decorated_vendor_path`]) — the npm, pnpm + /// and bun extractors diagnose it; composer and the pypi readers do not + /// check it. + pub(crate) decorated_check: bool, +} + +impl LocateOpts { + /// A literal path, decorated leaves reported (npm, pnpm, bun). + pub(crate) const LITERAL_CHECKED: Self = LocateOpts { + grammar: LeafGrammar::Literal, + decorated_check: true, + }; + /// A literal path, decorated leaves not checked (composer, pypi locks, + /// bare pip / Hatch paths). + pub(crate) const LITERAL: Self = LocateOpts { + grammar: LeafGrammar::Literal, + decorated_check: false, + }; + /// A decorated spelling (yarn, url-form pip / Hatch references). + pub(crate) const DECORATED: Self = LocateOpts { + grammar: LeafGrammar::Decorated, + decorated_check: false, + }; +} + +/// Where one lockfile / config string points — every identity decision an +/// extractor takes from it, computed once. Each extractor keeps its own +/// decision ORDER over the fields. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub(crate) struct Located { + /// The root-anchored vendored artifact the string names, read with the + /// requested [`LeafGrammar`]. + pub(crate) vendored: Option, + /// The patch uuid of a Socket-hosted url ([`OriginPolicy::hosted_uuid`]), + /// computed eagerly: it needs an http(s) url on an allowed origin while + /// `vendored` needs the string to START at `.socket/vendor/`, so at most + /// one of the two is ever set. + pub(crate) hosted: Option, + /// [`is_decorated_vendor_path`] when [`LocateOpts::decorated_check`] is + /// set (Literal grammar only), else `false`. + pub(crate) decorated_leaf: bool, +} + +/// The Socket wiring a lock location carries. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) enum Wired { + Hosted(String), + Vendored(VendorRef), +} + +impl Wired { + pub(crate) fn uuid(&self) -> &str { + match self { + Wired::Hosted(uuid) => uuid, + Wired::Vendored(vref) => &vref.uuid, + } + } + + /// The same patch artifact: the same hosted uuid, or the same vendored + /// artifact path. + pub(crate) fn same_artifact(&self, other: &Wired) -> bool { + match (self, other) { + (Wired::Hosted(a), Wired::Hosted(b)) => a == b, + (Wired::Vendored(a), Wired::Vendored(b)) => a.artifact_rel == b.artifact_rel, + _ => false, + } + } +} + +/// Whether `s` points into SOME `.socket/vendor/` tree, any anchoring +/// (backslashes folded) — a Socket-shaped string whose rejection is worth a +/// diagnostic (cargo `[patch]` paths, gem remotes, composer path dists, +/// maven repository urls). +pub(crate) fn names_vendor_dir(s: &str) -> bool { + s.replace('\\', "/").contains(&format!("{VENDOR_DIR}/")) +} + +/// [`names_vendor_dir`] that also folds JSON / TOML `\/` escapes (the pypi +/// readers). +pub(crate) fn mentions_vendor_dir(s: &str) -> bool { + s.replace("\\/", "/") + .replace('\\', "/") + .contains(&format!("{VENDOR_DIR}/")) +} + +/// Whether `s` STARTS at this project's `.socket/vendor/` after the `file:` +/// / `./` spellings [`vendor_ref_decorated`] accepts (yarn's rule; `\/` is +/// not folded). +pub(crate) fn root_anchored_spelling(s: &str) -> bool { + let norm = s.trim().replace('\\', "/"); + let norm = norm.strip_prefix("file:").unwrap_or(&norm); + let norm = norm.strip_prefix("./").unwrap_or(norm); + norm.starts_with(&format!("{VENDOR_DIR}/")) +} + +/// Whether vendored artifact `vref` is the npm tarball the vendor backends +/// write for `purl` (`[@scope/]-.tgz` in the npm vendor dir) +/// — the leaf rule the npm, pnpm and yarn extractors share. +pub(crate) fn npm_vendored_tarball_names(vref: &VendorRef, purl: &str) -> bool { + vref.eco == "npm" && vendored_leaf_purl("npm", &vref.leaf).as_deref() == Some(purl) +} + +#[cfg(test)] +mod hosted_patch_uuid_tests { + //! `hosted_patch_uuid` is the trust gate between a committed lockfile + //! line and a VEX attestation input: pin the accepted spellings AND the + //! rejections (foreign hosts, credentials, non-canonical tokens). + use super::*; + + const UUID: &str = "7c8d9e0f-1a2b-4a1b-8c2d-3e4f5a6b7c8d"; + /// A uuid-SHAPED grant token: the fixtures use them, and production + /// tokens are not guaranteed otherwise — the LAST uuid segment must win. + const TOKEN: &str = "11111111-2222-4333-8444-555555555555"; + + fn none() -> Vec { + Vec::new() + } + + #[test] + fn artifact_and_registry_shapes_yield_the_patch_uuid_not_the_token() { + for url in [ + format!("https://patch.socket.dev/patch/npm/left-pad/1.3.0/{TOKEN}/{UUID}/left-pad-1.3.0.tgz"), + format!("https://patch.socket.dev/patch/npm/{TOKEN}/{UUID}/left-pad-1.3.0.tgz"), + format!("https://patch.socket.dev/patch-registry/gem/{TOKEN}/{UUID}/"), + format!("https://patch.socket.dev/patch-registry/gem/{TOKEN}/{UUID}/gems/rack-2.2.3.gem"), + format!("https://patch.socket.dev/patch-registry/maven/{TOKEN}/{UUID}/maven2"), + format!("https://patch.socket.dev/patch-registry/nuget/{TOKEN}/{UUID}/index.json"), + format!("sparse+https://patch.socket.dev/patch-registry/cargo/{TOKEN}/{UUID}/index/"), + format!("registry+https://patch.socket.dev/patch-registry/cargo/{TOKEN}/{UUID}/index/"), + ] { + assert_eq!( + hosted_patch_uuid(&url, &none()).as_deref(), + Some(UUID), + "{url}" + ); + } + } + + #[test] + fn lock_format_spellings_are_normalized() { + let url = format!("https://patch.socket.dev/patch/npm/{TOKEN}/{UUID}/left-pad-1.3.0.tgz"); + // yarn classic `#`, pip/hatch `#sha256=`, a stray query. + for spelled in [ + format!("{url}#0123456789abcdef0123456789abcdef01234567"), + format!("{url}#sha256=abc"), + format!("{url}?x=1"), + // composer's `\/`-escaped slashes. + url.replace('/', "\\/"), + // yarn berry's percent-encoded `__archiveUrl=` binding value. + url.replace(':', "%3A").replace('/', "%2F"), + format!(" {url} "), + ] { + assert_eq!( + hosted_patch_uuid(&spelled, &none()).as_deref(), + Some(UUID), + "{spelled}" + ); + } + } + + #[test] + fn foreign_hosts_credentials_and_plain_http_are_refused() { + for url in [ + format!("https://registry.npmjs.org/{TOKEN}/{UUID}/x.tgz"), + format!("https://patch.socket.dev.evil.example/{TOKEN}/{UUID}/x.tgz"), + format!("https://evil.example/patch.socket.dev/{TOKEN}/{UUID}/x.tgz"), + format!("http://patch.socket.dev/patch/npm/{TOKEN}/{UUID}/x.tgz"), + format!("https://patch.socket.dev:8443/patch/npm/{TOKEN}/{UUID}/x.tgz"), + format!("https://user:pw@patch.socket.dev/patch/npm/{TOKEN}/{UUID}/x.tgz"), + format!("git+ssh://patch.socket.dev/{UUID}"), + format!("file:.socket/vendor/npm/{UUID}/x.tgz"), + format!("patch.socket.dev/gopatch/{UUID}"), + ] { + assert_eq!(hosted_patch_uuid(&url, &none()), None, "{url}"); + } + } + + #[test] + fn non_canonical_segments_are_not_patch_uuids() { + for url in [ + // Placeholder tokens (fixtures use `uuid`, `tok`, `some-uuid`). + "https://patch.socket.dev/patch/npm/tok/uuid/x.tgz".to_string(), + // Uppercase is not the canonical grammar. + format!( + "https://patch.socket.dev/patch/npm/tok/{}/x.tgz", + UUID.to_ascii_uppercase() + ), + // A uuid only in the query / fragment is not a path level. + format!("https://patch.socket.dev/patch/npm/x.tgz?u={UUID}"), + format!("https://patch.socket.dev/patch/npm/x.tgz#{UUID}"), + // An encoded `/` cannot split a segment into a uuid. + format!("https://patch.socket.dev/patch/npm/tok%2F{UUID}/x.tgz"), + ] { + assert_eq!(hosted_patch_uuid(&url, &none()), None, "{url}"); + } + } + + #[test] + fn configured_patch_server_origin_is_accepted_exactly() { + let origins = vec!["http://127.0.0.1:4545/some/base".to_string()]; + let url = format!("http://127.0.0.1:4545/patch/npm/{TOKEN}/{UUID}/x.tgz"); + assert_eq!(hosted_patch_uuid(&url, &origins).as_deref(), Some(UUID)); + // Same host, other port / scheme: a different origin. + for other in [ + format!("http://127.0.0.1:4546/patch/npm/{TOKEN}/{UUID}/x.tgz"), + format!("https://127.0.0.1:4545/patch/npm/{TOKEN}/{UUID}/x.tgz"), + ] { + assert_eq!(hosted_patch_uuid(&other, &origins), None, "{other}"); + } + // The default host stays accepted alongside the override, and a + // malformed override is ignored rather than widening the allowlist. + let default = format!("https://patch.socket.dev/patch/npm/{TOKEN}/{UUID}/x.tgz"); + assert_eq!(hosted_patch_uuid(&default, &origins).as_deref(), Some(UUID)); + assert_eq!(hosted_patch_uuid(&url, &["not a url".to_string()]), None); + } + + /// The Go hosted namespace lives on the same host the URL allowlist + /// pins — the two spellings of "Socket's patch server" cannot drift. + #[test] + fn go_module_namespace_is_on_the_patch_server_host() { + assert!(crate::vendor::go_mod_edit::HOSTED_GO_MODULE_PREFIX + .starts_with(&format!("{SOCKET_PATCH_SERVER_HOST}/"))); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const UUID: &str = "7c8d9e0f-1a2b-4a1b-8c2d-3e4f5a6b7c8d"; + /// A uuid-SHAPED grant token. + const TOKEN: &str = "11111111-2222-4333-8444-555555555555"; + + #[test] + fn vendor_ref_strips_lock_suffixes_and_requires_a_root_anchor() { + let a = UUID; + for spelled in [ + format!("file:.socket/vendor/npm/{a}/left-pad-1.3.0.tgz"), + format!("file:./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz"), + format!("./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz"), + format!(".socket/vendor/npm/{a}/left-pad-1.3.0.tgz"), + format!(".socket\\vendor\\npm\\{a}\\left-pad-1.3.0.tgz"), + format!(".socket\\/vendor\\/npm\\/{a}\\/left-pad-1.3.0.tgz"), + // yarn classic `#`, berry fragment + locator, pip `#sha256=`. + format!("file:./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz#0123abcd"), + format!( + "./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz#./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz::hash=abc123&locator=app%40workspace%3A." + ), + format!("./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz::locator=app%40workspace%3A."), + ] { + let decorated = spelled.contains('#') || spelled.contains("::"); + // The literal-path reader refuses a decorated leaf outright: to + // npm / pnpm / bun / … it names a DIFFERENT file. + assert_eq!(vendor_ref(&spelled).is_none(), decorated, "{spelled}"); + let v = vendor_ref_decorated(&spelled).unwrap_or_else(|| panic!("{spelled}")); + assert_eq!(v.eco, "npm", "{spelled}"); + assert_eq!(v.uuid, a, "{spelled}"); + assert_eq!(v.leaf, "left-pad-1.3.0.tgz", "{spelled}"); + assert_eq!( + v.artifact_rel, + format!(".socket/vendor/npm/{a}/left-pad-1.3.0.tgz"), + "{spelled}" + ); + } + // Dir-shaped leaves keep their nesting, trailing slash dropped. + let v = vendor_ref(&format!( + "./.socket/vendor/golang/{a}/github.com/foo/bar@v1.2.3/" + )) + .unwrap(); + assert_eq!(v.leaf, "github.com/foo/bar@v1.2.3"); + } + + #[test] + fn vendor_ref_rejects_escapes_foreign_anchors_and_bad_uuids() { + let a = UUID; + for bad in [ + // The package manager would read OUTSIDE the project root. + format!("../.socket/vendor/npm/{a}/x-1.0.0.tgz"), + format!("file:../.socket/vendor/npm/{a}/x-1.0.0.tgz"), + format!("/abs/project/.socket/vendor/npm/{a}/x-1.0.0.tgz"), + format!("sub/.socket/vendor/npm/{a}/x-1.0.0.tgz"), + // Traversal inside the leaf. + format!(".socket/vendor/npm/{a}/../../../etc/passwd"), + format!(".socket/vendor/npm/{a}/a//b.tgz"), + // No leaf / uuid-dir only / unknown eco / non-canonical uuid. + format!(".socket/vendor/npm/{a}"), + format!(".socket/vendor/npm/{a}/"), + format!(".socket/vendor/jsr/{a}/x-1.0.0.tgz"), + format!(".socket/vendor/npm/{}/x-1.0.0.tgz", a.to_ascii_uppercase()), + ".socket/vendor/npm/not-a-uuid/x-1.0.0.tgz".to_string(), + "https://registry.npmjs.org/x/-/x-1.0.0.tgz".to_string(), + ] { + assert_eq!(vendor_ref(&bad), None, "{bad}"); + } + } + + #[test] + fn vendor_uuid_dir_accepts_only_the_bare_uuid_dir() { + let a = UUID; + assert_eq!( + vendor_uuid_dir(&format!(".socket/vendor/nuget/{a}")), + Some(("nuget".to_string(), a.to_string())) + ); + assert_eq!( + vendor_uuid_dir(&format!("./.socket/vendor/maven/{a}/")), + Some(("maven".to_string(), a.to_string())) + ); + assert_eq!( + vendor_uuid_dir(&format!(".socket/vendor/nuget/{a}/x.nupkg")), + None + ); + assert_eq!( + vendor_uuid_dir(&format!("../.socket/vendor/nuget/{a}")), + None + ); + assert_eq!(vendor_uuid_dir(".socket/vendor/nuget/nope"), None); + } + + #[test] + fn vendored_leaf_purl_strips_and_canonicalizes() { + assert_eq!( + vendored_leaf_purl("npm", "@scope/pkg-1.2.3.tgz#deadbeef").as_deref(), + Some("pkg:npm/@scope/pkg@1.2.3") + ); + assert_eq!( + vendored_leaf_purl( + "pypi", + "Python_Dateutil-2.8.2-py2.py3-none-any.whl#sha256=ab" + ) + .as_deref(), + Some("pkg:pypi/python-dateutil@2.8.2") + ); + assert_eq!( + vendored_leaf_purl("cargo", "serde-1.0.0/").as_deref(), + Some("pkg:cargo/serde@1.0.0") + ); + assert_eq!(vendored_leaf_purl("npm", "not-a-tarball"), None); + } + + #[test] + fn socket_patch_names_are_exact() { + assert_eq!( + socket_patch_name_uuid(&format!("socket-patch-{UUID}"), false).as_deref(), + Some(UUID) + ); + assert_eq!( + socket_patch_name_uuid(&format!("socket-patch-vendor-{UUID}"), true).as_deref(), + Some(UUID) + ); + // The hosted grammar never matches the vendored id and vice versa. + assert_eq!( + socket_patch_name_uuid(&format!("socket-patch-vendor-{UUID}"), false), + None + ); + assert_eq!( + socket_patch_name_uuid(&format!("socket-patch-{UUID}"), true), + None + ); + assert_eq!( + socket_patch_name_uuid(&format!("socket-patch-{UUID}-x"), false), + None + ); + assert_eq!(socket_patch_name_uuid("socket-patch-nope", false), None); + } + + /// [`OriginPolicy::hosted_uuid`]'s plain-url shortcut answers exactly + /// what the full parse does, for plain urls on other hosts and for + /// every spelling it must leave to the parse. + #[test] + fn hosted_uuid_shortcut_agrees_with_the_full_parse() { + let (a, t) = (UUID, TOKEN); + let path = format!("/patch/npm/x/1.0.0/{t}/{a}/x-1.0.0.tgz"); + let hosts = [ + "patch.socket.dev", + "PATCH.Socket.DEV", + "patch.socket.dev.", + "patch.socket.dev:443", + "patch.socket.dev:8443", + "patch.socket.dev:", + "registry.npmjs.org", + "registry.npmjs.org:443", + "127.0.0.1:4545", + "127.0.0.1", + "0x7f.0.0.1:4545", + "127.1:4545", + "[::1]:4545", + "staging.example.com:8443", + "Staging.Example.COM:8443", + "staging.example.com", + "xn--stging-bua.example.com", + "user@patch.socket.dev", + "user:pw@registry.npmjs.org", + "patch%2esocket.dev", + "patch.soc\tket.dev", + "patch.socket.dev\\", + "foo.123", + "foo.0x1f", + "", + ]; + let urls: Vec = hosts + .iter() + .flat_map(|host| { + [ + format!("https://{host}{path}"), + format!("http://{host}{path}"), + format!("HTTPS://{host}{path}"), + format!("sparse+https://{host}{path}"), + format!("git+https://{host}{path}"), + format!("https:///{host}{path}"), + format!("https://{host}?q={path}"), + format!(" https://{host}{path} "), + format!("https://{host}"), + ] + }) + .chain([ + format!("https:\\/\\/patch.socket.dev{}", path.replace('/', "\\/")), + format!("https%3A%2F%2Fpatch.socket.dev{}", path.replace('/', "%2F")), + "file:x.tgz".to_string(), + "npm:x@1.0.0".to_string(), + "https://".to_string(), + ]) + .collect(); + for origins in [ + vec![], + vec!["http://127.0.0.1:4545".to_string()], + vec![ + "https://staging.example.com:8443".to_string(), + "http://[::1]:4545".to_string(), + "https://xn--stging-bua.example.com".to_string(), + ], + ] { + let ctx = OriginPolicy::new(&origins); + for url in &urls { + assert_eq!( + ctx.hosted_uuid(url), + hosted_patch_uuid(url, &origins), + "{url} under {origins:?}" + ); + } + assert_eq!( + ctx.hosted_uuid(&format!("https://patch.socket.dev{path}")) + .as_deref(), + Some(a) + ); + } + assert_eq!( + plain_url_domain("https://Registry.NPMJS.org/x"), + Some("registry.npmjs.org".into()) + ); + for not_plain in [ + "https://127.0.0.1/x", + "https://a%2eb.com/x", + "https://u@a.com/x", + "https://xn--a.com/x", + ] { + assert_eq!(plain_url_domain(not_plain), None, "{not_plain}"); + } + } +} diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 0c778ac74..706b6eb21 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -7606,96 +7606,7 @@ pub fn grant_token_path_segment(url: &str, patch_uuid: &str) -> Option { (!token.is_empty()).then(|| token.to_string()) } -/// Public host of Socket's patch server: the origin every production hosted -/// artifact / registry URL is served from (`https://patch.socket.dev/patch/…`, -/// `…/patch-registry/…`), and the root of the Go module namespace -/// [`crate::vendor::go_mod_edit::HOSTED_GO_MODULE_PREFIX`]. -pub const SOCKET_PATCH_SERVER_HOST: &str = "patch.socket.dev"; - -/// The Socket patch uuid a lockfile-recorded HOSTED reference names, or -/// `None` when `url` is not a Socket-hosted patch URL — the inverse of the -/// rewriters, used by `vex`'s manifest-less lockfile discovery. -/// -/// Recognition is deliberately strict, because the answer decides whether a -/// committed (tamper-able) lockfile line becomes an attestation input: -/// -/// * the ORIGIN must be Socket's patch server (`https://` + -/// [`SOCKET_PATCH_SERVER_HOST`]) or one of `extra_origins` — the -/// operator's `--patch-server-url` deployment, compared on scheme + host + -/// port. A uuid inside any other host's URL is a user's own dependency -/// source, never a patch reference; -/// * no userinfo — a Socket-written URL never carries credentials; -/// * the uuid is the LAST path segment passing the canonical-uuid grammar: -/// hosted URLs carry the grant token in the level before the uuid -/// (`…/patch/npm/////`, -/// `…/patch-registry////…`), and grant tokens may -/// themselves be uuid-shaped, so "the first uuid" would elect the token. -/// -/// Every spelling the lock formats record the same URL in is accepted: a -/// `#fragment` (yarn classic `#`, pip `#sha256=`) and a `?query` are -/// ignored; `\/`-escaped slashes (older composer locks) are unescaped; a -/// wholly percent-encoded URL (yarn berry's `__archiveUrl=` binding) is -/// decoded; a cargo source-kind prefix (`sparse+`, `registry+`) is dropped. -/// Path segments are percent-decoded AFTER splitting, so an encoded `/` -/// can never manufacture a segment. -pub fn hosted_patch_uuid(url: &str, extra_origins: &[String]) -> Option { - hosted_patch_url_uuids(url, extra_origins)?.pop() -} - -/// EVERY canonical-uuid path segment of a Socket-HOSTED url, in path order -/// (the grant token first when it is uuid-shaped, the patch uuid last), or -/// `None` when `url` is not on an accepted origin — [`hosted_patch_uuid`]'s -/// exact acceptance rules, without electing one segment. `vex` discovery -/// uses it to RECOGNIZE every Socket identity a lockfile mentions, including -/// the malformed or rejected shapes whose "last uuid" is not a patch. -pub fn hosted_patch_url_uuids(url: &str, extra_origins: &[String]) -> Option> { - use crate::patch::path_safety::is_canonical_uuid; - use crate::utils::purl::percent_decode_purl_component; - - let unescaped = url.trim().replace("\\/", "/"); - let lower = unescaped.to_ascii_lowercase(); - let decoded = if lower.starts_with("https%3a%2f%2f") || lower.starts_with("http%3a%2f%2f") { - percent_decode_purl_component(&unescaped).into_owned() - } else { - unescaped - }; - // `sparse+https://…` / `registry+https://…` (Cargo.lock `source`). - let (scheme, _) = decoded.split_once("://")?; - let text = match scheme.rsplit_once('+') { - Some((kind, _)) if !kind.is_empty() && kind.bytes().all(|b| b.is_ascii_alphabetic()) => { - &decoded[kind.len() + 1..] - } - _ => decoded.as_str(), - }; - let parsed = reqwest::Url::parse(text).ok()?; - if !matches!(parsed.scheme(), "https" | "http") - || !parsed.username().is_empty() - || parsed.password().is_some() - { - return None; - } - let socket_host = parsed.scheme() == "https" - && parsed.host_str() == Some(SOCKET_PATCH_SERVER_HOST) - && parsed.port_or_known_default() == Some(443); - let configured = extra_origins.iter().any(|origin| { - reqwest::Url::parse(origin.trim()).is_ok_and(|o| { - o.scheme() == parsed.scheme() - && o.host_str().is_some() - && o.host_str() == parsed.host_str() - && o.port_or_known_default() == parsed.port_or_known_default() - }) - }); - if !socket_host && !configured { - return None; - } - Some( - parsed - .path_segments()? - .map(|segment| percent_decode_purl_component(segment).into_owned()) - .filter(|segment| is_canonical_uuid(segment)) - .collect(), - ) -} +pub use crate::origin::{hosted_patch_url_uuids, hosted_patch_uuid, SOCKET_PATCH_SERVER_HOST}; /// A dep's Socket index URL as a regex source with the per-request rotating /// segments (grant token, patch uuid) wildcarded — an exact-URL pattern @@ -29658,129 +29569,6 @@ mod hatch_tests { } } -#[cfg(test)] -mod hosted_patch_uuid_tests { - //! `hosted_patch_uuid` is the trust gate between a committed lockfile - //! line and a VEX attestation input: pin the accepted spellings AND the - //! rejections (foreign hosts, credentials, non-canonical tokens). - use super::*; - - const UUID: &str = "7c8d9e0f-1a2b-4a1b-8c2d-3e4f5a6b7c8d"; - /// A uuid-SHAPED grant token: the fixtures use them, and production - /// tokens are not guaranteed otherwise — the LAST uuid segment must win. - const TOKEN: &str = "11111111-2222-4333-8444-555555555555"; - - fn none() -> Vec { - Vec::new() - } - - #[test] - fn artifact_and_registry_shapes_yield_the_patch_uuid_not_the_token() { - for url in [ - format!("https://patch.socket.dev/patch/npm/left-pad/1.3.0/{TOKEN}/{UUID}/left-pad-1.3.0.tgz"), - format!("https://patch.socket.dev/patch/npm/{TOKEN}/{UUID}/left-pad-1.3.0.tgz"), - format!("https://patch.socket.dev/patch-registry/gem/{TOKEN}/{UUID}/"), - format!("https://patch.socket.dev/patch-registry/gem/{TOKEN}/{UUID}/gems/rack-2.2.3.gem"), - format!("https://patch.socket.dev/patch-registry/maven/{TOKEN}/{UUID}/maven2"), - format!("https://patch.socket.dev/patch-registry/nuget/{TOKEN}/{UUID}/index.json"), - format!("sparse+https://patch.socket.dev/patch-registry/cargo/{TOKEN}/{UUID}/index/"), - format!("registry+https://patch.socket.dev/patch-registry/cargo/{TOKEN}/{UUID}/index/"), - ] { - assert_eq!( - hosted_patch_uuid(&url, &none()).as_deref(), - Some(UUID), - "{url}" - ); - } - } - - #[test] - fn lock_format_spellings_are_normalized() { - let url = format!("https://patch.socket.dev/patch/npm/{TOKEN}/{UUID}/left-pad-1.3.0.tgz"); - // yarn classic `#`, pip/hatch `#sha256=`, a stray query. - for spelled in [ - format!("{url}#0123456789abcdef0123456789abcdef01234567"), - format!("{url}#sha256=abc"), - format!("{url}?x=1"), - // composer's `\/`-escaped slashes. - url.replace('/', "\\/"), - // yarn berry's percent-encoded `__archiveUrl=` binding value. - url.replace(':', "%3A").replace('/', "%2F"), - format!(" {url} "), - ] { - assert_eq!( - hosted_patch_uuid(&spelled, &none()).as_deref(), - Some(UUID), - "{spelled}" - ); - } - } - - #[test] - fn foreign_hosts_credentials_and_plain_http_are_refused() { - for url in [ - format!("https://registry.npmjs.org/{TOKEN}/{UUID}/x.tgz"), - format!("https://patch.socket.dev.evil.example/{TOKEN}/{UUID}/x.tgz"), - format!("https://evil.example/patch.socket.dev/{TOKEN}/{UUID}/x.tgz"), - format!("http://patch.socket.dev/patch/npm/{TOKEN}/{UUID}/x.tgz"), - format!("https://patch.socket.dev:8443/patch/npm/{TOKEN}/{UUID}/x.tgz"), - format!("https://user:pw@patch.socket.dev/patch/npm/{TOKEN}/{UUID}/x.tgz"), - format!("git+ssh://patch.socket.dev/{UUID}"), - format!("file:.socket/vendor/npm/{UUID}/x.tgz"), - format!("patch.socket.dev/gopatch/{UUID}"), - ] { - assert_eq!(hosted_patch_uuid(&url, &none()), None, "{url}"); - } - } - - #[test] - fn non_canonical_segments_are_not_patch_uuids() { - for url in [ - // Placeholder tokens (fixtures use `uuid`, `tok`, `some-uuid`). - "https://patch.socket.dev/patch/npm/tok/uuid/x.tgz".to_string(), - // Uppercase is not the canonical grammar. - format!( - "https://patch.socket.dev/patch/npm/tok/{}/x.tgz", - UUID.to_ascii_uppercase() - ), - // A uuid only in the query / fragment is not a path level. - format!("https://patch.socket.dev/patch/npm/x.tgz?u={UUID}"), - format!("https://patch.socket.dev/patch/npm/x.tgz#{UUID}"), - // An encoded `/` cannot split a segment into a uuid. - format!("https://patch.socket.dev/patch/npm/tok%2F{UUID}/x.tgz"), - ] { - assert_eq!(hosted_patch_uuid(&url, &none()), None, "{url}"); - } - } - - #[test] - fn configured_patch_server_origin_is_accepted_exactly() { - let origins = vec!["http://127.0.0.1:4545/some/base".to_string()]; - let url = format!("http://127.0.0.1:4545/patch/npm/{TOKEN}/{UUID}/x.tgz"); - assert_eq!(hosted_patch_uuid(&url, &origins).as_deref(), Some(UUID)); - // Same host, other port / scheme: a different origin. - for other in [ - format!("http://127.0.0.1:4546/patch/npm/{TOKEN}/{UUID}/x.tgz"), - format!("https://127.0.0.1:4545/patch/npm/{TOKEN}/{UUID}/x.tgz"), - ] { - assert_eq!(hosted_patch_uuid(&other, &origins), None, "{other}"); - } - // The default host stays accepted alongside the override, and a - // malformed override is ignored rather than widening the allowlist. - let default = format!("https://patch.socket.dev/patch/npm/{TOKEN}/{UUID}/x.tgz"); - assert_eq!(hosted_patch_uuid(&default, &origins).as_deref(), Some(UUID)); - assert_eq!(hosted_patch_uuid(&url, &["not a url".to_string()]), None); - } - - /// The Go hosted namespace lives on the same host the URL allowlist - /// pins — the two spellings of "Socket's patch server" cannot drift. - #[test] - fn go_module_namespace_is_on_the_patch_server_host() { - assert!(crate::vendor::go_mod_edit::HOSTED_GO_MODULE_PREFIX - .starts_with(&format!("{SOCKET_PATCH_SERVER_HOST}/"))); - } -} - /// #742 / #650: a superseding patch uuid for a package an earlier hosted /// scan already wired re-pins socket-patch's own source, end to end through /// the redirect planner, instead of warning "revert it" and leaving the old diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 815d4644d..5c08758c3 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -187,16 +187,25 @@ use std::path::{Path, PathBuf}; use std::sync::Mutex; use crate::crawlers::Ecosystem; -use crate::patch::path_safety::{is_canonical_uuid, is_safe_multi_segment}; +use crate::patch::path_safety::is_canonical_uuid; use crate::utils::purl_key::canonical_base_purl; use crate::utils::purl_key::PurlKey; use crate::vendor::go_mod_edit::HOSTED_GO_MODULE_PREFIX; use crate::vendor::lock_inventory::{ inventory_project_every_lock, lookup, LockIntegrity, LockfileEntry, SourceKind, }; -use crate::vendor::path::{ecosystem_dir_for_purl, leaf_to_purl, ECOSYSTEM_DIRS, VENDOR_DIR}; +use crate::vendor::path::{ecosystem_dir_for_purl, ECOSYSTEM_DIRS, VENDOR_DIR}; use crate::vendor::state::VendorEntry; +pub(crate) use crate::origin::{ + mentions_vendor_dir, names_vendor_dir, npm_vendored_tarball_names, root_anchored_spelling, + socket_patch_name_uuid, LocateOpts, Located, OriginPolicy, Wired, +}; +pub use crate::origin::{ + strip_leaf_suffix, vendor_ref, vendor_ref_decorated, vendor_uuid_dir, vendored_leaf_purl, + VendorRef, +}; + pub(crate) mod bun; pub(crate) mod cargo; pub(crate) mod composer; @@ -1569,10 +1578,8 @@ async fn ignore_member_stray_locks(ctx: &mut DiscoverCtx<'_>, out: &mut Discover }; for rel in &stray.ignored { let mentions = match ctx.view.read_bytes(rel).await { - Ok(bytes) => { - !socket_identities(&String::from_utf8_lossy(&bytes), ctx.patch_server_origins) - .is_empty() - } + Ok(bytes) => !socket_identities(&String::from_utf8_lossy(&bytes), ctx.origin.origins()) + .is_empty(), Err(_) => false, }; ctx.recognize_ignored(rel).await; @@ -1604,7 +1611,8 @@ pub(crate) struct DiscoverCtx<'a> { /// Where the guarded reads read from: the root on disk, a per-run /// snapshot of it, or an in-memory project. pub(crate) view: crate::vendor::lock_inventory::ProjectView<'a>, - patch_server_origins: &'a [String], + /// The host policy [`Self::hosted_uuid`] and [`Self::locate`] classify under. + origin: OriginPolicy<'a>, /// Root-relative lock files the package manager never reads here (a /// workspace member's stray lock, see [`ignore_member_stray_locks`]): /// the guarded reads see them as absent. @@ -1614,8 +1622,6 @@ pub(crate) struct DiscoverCtx<'a> { /// scratch `Discovery` (a file parsed only to explain it) still counts. /// A `Mutex` keeps the ctx `Sync` across the extractors' `.await`s. recognized: Mutex>, - /// The hosts a Socket-hosted url can name (see [`DiscoverCtx::hosted_uuid`]). - hosted_hosts: std::sync::OnceLock>, /// The ecosystem the running extractor reads for (set by /// [`discover_with_ctx`] between extractors), tagging [`Self::read`]. ecosystem: &'static str, @@ -1637,10 +1643,9 @@ impl<'a> DiscoverCtx<'a> { ) -> Self { DiscoverCtx { view, - patch_server_origins, + origin: OriginPolicy::new(patch_server_origins), ignored: Vec::new(), recognized: Mutex::new(BTreeSet::new()), - hosted_hosts: std::sync::OnceLock::new(), ecosystem: "", read: Mutex::new(BTreeSet::new()), } @@ -1686,7 +1691,7 @@ impl<'a> DiscoverCtx<'a> { /// Record every Socket identity `text` (the content of root-relative /// `rel`) mentions — see [`socket_identities`]. fn recognize_text(&self, rel: &str, text: &str) { - let found = socket_identities(text, self.patch_server_origins); + let found = socket_identities(text, self.origin.origins()); if found.is_empty() { return; } @@ -1747,31 +1752,14 @@ impl<'a> DiscoverCtx<'a> { } } - /// The patch uuid of a Socket-HOSTED url, or `None` for anything else - /// (see [`crate::patch::redirect::hosted_patch_uuid`] for the accepted - /// spellings and the host allowlist). - /// - /// A plain url ([`plain_url_domain`]) on any other host is answered - /// without parsing it: every lock entry's registry url comes through - /// here. + /// [`OriginPolicy::hosted_uuid`] under this run's origins. pub(crate) fn hosted_uuid(&self, url: &str) -> Option { - if let Some(host) = plain_url_domain(url) { - let hosts = self.hosted_hosts.get_or_init(|| { - let mut hosts = - BTreeSet::from([crate::patch::redirect::SOCKET_PATCH_SERVER_HOST.to_string()]); - hosts.extend(self.patch_server_origins.iter().filter_map(|o| { - reqwest::Url::parse(o.trim()) - .ok()? - .host_str() - .map(str::to_string) - })); - hosts - }); - if !hosts.contains(&host) { - return None; - } - } - crate::patch::redirect::hosted_patch_uuid(url, self.patch_server_origins) + self.origin.hosted_uuid(url) + } + + /// [`OriginPolicy::locate`] under this run's origins. + pub(crate) fn locate(&self, s: &str, opts: LocateOpts) -> Located { + self.origin.locate(s, opts) } /// Whether `rel` exists (lstat — a dangling symlink still "exists", the @@ -1902,62 +1890,6 @@ impl<'a> DiscoverCtx<'a> { // ── identity helpers ───────────────────────────────────────────────────── -/// The host of `url` when it is a PLAIN `http(s)` url — printable ASCII -/// only, no `%`, `\\` or userinfo, a DNS name of letters, digits, `.` and -/// `-` whose last label starts with a letter — lowercased, which is then -/// exactly what [`crate::patch::redirect::hosted_patch_url_uuids`]'s url -/// parse would call its host (no decoding, IDNA mapping or IP-address -/// reading applies to such a name). `None` for anything else, which takes -/// the full parse. -fn plain_url_domain(url: &str) -> Option { - let url = url.trim(); - if !url - .bytes() - .all(|b| b.is_ascii_graphic() && b != b'%' && b != b'\\') - { - return None; - } - let (scheme, rest) = url.split_once("://")?; - // `sparse+https://…` / `registry+https://…`, as the parse strips it. - let scheme = match scheme.rsplit_once('+') { - Some((kind, scheme)) - if !kind.is_empty() && kind.bytes().all(|b| b.is_ascii_alphabetic()) => - { - scheme - } - _ => scheme, - }; - if !scheme.eq_ignore_ascii_case("https") && !scheme.eq_ignore_ascii_case("http") { - return None; - } - let authority = rest.split(['/', '?', '#']).next()?; - let host = match authority.rsplit_once(':') { - Some((host, port)) if port.bytes().all(|b| b.is_ascii_digit()) => host, - Some(_) => return None, - None => authority, - }; - let last = host.strip_suffix('.').unwrap_or(host).rsplit('.').next()?; - let plain = !host.is_empty() - && !rest.starts_with('/') - && host - .bytes() - .all(|b| b.is_ascii_alphanumeric() || b == b'.' || b == b'-') - && last.starts_with(|c: char| c.is_ascii_alphabetic()) - && !host - .split('.') - .any(|label| label.len() >= 4 && label[..4].eq_ignore_ascii_case("xn--")); - plain.then(|| host.to_ascii_lowercase()) -} - -/// The uuid of a Socket-owned registry / repository / source NAME: -/// `socket-patch-` (cargo `[registries.*]` + Cargo.toml `registry =`, -/// maven ``, nuget ``) or, with -/// `vendored = true`, `socket-patch-vendor-` (maven's vendored repo -/// id). Exact grammar only — a suffix or a non-canonical uuid is not ours. -pub(crate) fn socket_patch_name_uuid(name: &str, vendored: bool) -> Option { - crate::patch::redirect::generation::pin_name_uuid(name.trim(), vendored).map(str::to_string) -} - /// Every Socket patch identity `text` MENTIONS, by wiring mode — the sweep /// behind rule 11, run over the content of every file discovery reads. It is /// deliberately a superset of what any extractor accepts (it must recognize @@ -2256,269 +2188,6 @@ fn token_end(bytes: &[u8], from: usize, extra: &[u8]) -> usize { end } -/// A vendored artifact reference recovered from a wiring string. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct VendorRef { - /// Vendor ecosystem dir (`npm`, `cargo`, `pypi`, …). - pub eco: String, - pub uuid: String, - /// The natural leaf, lock-format suffixes stripped ([`strip_leaf_suffix`]), - /// forward-slashed, no trailing slash. - pub leaf: String, - /// `.socket/vendor///` — the root-relative artifact path. - pub artifact_rel: String, -} - -/// Recover a vendored artifact reference from a lockfile/config string that -/// points into THIS project's `.socket/vendor/`: `file:` / `./` / bare -/// spellings, backslashed Windows spellings, and `\/`-escaped JSON text. -/// -/// Stricter than [`crate::vendor::parse_vendor_path`] (which finds the anchor -/// anywhere, for the orphan sweep): the anchor must START the path. A -/// `../.socket/vendor/…` or absolute spelling points the package manager at -/// an artifact OUTSIDE the project root, and verifying the root's same-named -/// artifact instead would attest bytes the build never consumes. Callers -/// whose format legitimately prefixes the project root (hatch -/// `{root:uri}/`, maven `file://${project.basedir}/`, pnpm-legacy absolute -/// importer paths) strip that exact prefix first. -/// -/// The leaf must be a traversal-safe relative path, taken LITERALLY: npm, -/// pnpm, bun, composer, bundler, go, cargo, maven, nuget and every pypi -/// lock read the string as a file path, so a leaf carrying `#` or `::` is -/// rejected (`None`) rather than cut — `…/foo-1.0.0.tgz#evil.tgz` is a file -/// of that literal name, which npm installs while the cut leaf would verify -/// the good `foo-1.0.0.tgz` beside it. The formats whose grammar DEFINES -/// those decorations use [`vendor_ref_decorated`] instead. -pub fn vendor_ref(s: &str) -> Option { - vendor_ref_with(s, false) -} - -/// [`vendor_ref`] for the formats that decorate a vendored path: yarn -/// classic `#`, berry `#./…::hash=…` fragments and `::locator=…` -/// bindings, and url-form pip / Hatch references (`file:…#sha256=…`, -/// `{root:uri}/…#sha256=…`). The leaf keeps only its natural name — the -/// decorations ([`strip_leaf_suffix`]) are cut before the literal-path -/// checks. -pub fn vendor_ref_decorated(s: &str) -> Option { - vendor_ref_with(s, true) -} - -fn vendor_ref_with(s: &str, decorated: bool) -> Option { - let rest = root_anchored_vendor_rest(s)?; - let mut it = rest.splitn(3, '/'); - let eco = it.next()?; - let uuid = it.next()?; - let leaf_raw = it.next()?; - if !ECOSYSTEM_DIRS.contains(&eco) || !is_canonical_uuid(uuid) { - return None; - } - let leaf = if decorated { - strip_leaf_suffix(leaf_raw) - } else if leaf_raw.contains('#') || leaf_raw.contains("::") { - return None; - } else { - leaf_raw - }; - let leaf = leaf.trim_end_matches('/'); - if leaf.is_empty() || !is_safe_multi_segment(leaf) { - return None; - } - Some(VendorRef { - eco: eco.to_string(), - uuid: uuid.to_string(), - leaf: leaf.to_string(), - artifact_rel: format!("{VENDOR_DIR}/{eco}/{uuid}/{leaf}"), - }) -} - -/// `(eco, uuid)` for a root-anchored string that ends AT the uuid dir -/// (`.socket/vendor/nuget/`, a maven repository url's path) — the -/// shapes [`vendor_ref`] rejects for having no leaf. -pub fn vendor_uuid_dir(s: &str) -> Option<(String, String)> { - let rest = root_anchored_vendor_rest(s)?; - let (eco, uuid) = rest.trim_end_matches('/').split_once('/')?; - if !ECOSYSTEM_DIRS.contains(&eco) || !is_canonical_uuid(uuid) { - return None; - } - Some((eco.to_string(), uuid.to_string())) -} - -/// Whether `s` is a vendored artifact path [`vendor_ref`] rejects ONLY for -/// a `#` / `::` in its leaf (what [`vendor_ref_decorated`] would have cut): -/// to a literal-path format that names another file, which extractors -/// diagnose rather than skip silently. -pub(crate) fn is_decorated_vendor_path(s: &str) -> bool { - vendor_ref(s).is_none() && vendor_ref_decorated(s).is_some() -} - -/// Everything after a LEADING `.socket/vendor/` anchor (after `file:`, one -/// `./`, backslash and `\/` normalization), or `None`. -fn root_anchored_vendor_rest(s: &str) -> Option { - let norm = s.trim().replace("\\/", "/").replace('\\', "/"); - let norm = norm.strip_prefix("file:").unwrap_or(&norm); - let norm = norm.strip_prefix("./").unwrap_or(norm); - norm.strip_prefix(&format!("{VENDOR_DIR}/")) - .map(str::to_string) -} - -/// Cut the lock-format decorations off a vendored leaf: everything from the -/// first `#` (yarn classic `#`, berry `#./::hash=…`, pip/hatch -/// `#sha256=…`) or `::` (berry `::locator=…`). -pub fn strip_leaf_suffix(leaf: &str) -> &str { - let cut = [leaf.find('#'), leaf.find("::")] - .into_iter() - .flatten() - .min() - .unwrap_or(leaf.len()); - &leaf[..cut] -} - -/// The canonical base purl a vendored leaf names (see -/// [`crate::vendor::path`]'s leaf table), suffixes stripped and pypi names -/// canonicalized. `None` when the leaf is not a recognizable artifact name. -pub fn vendored_leaf_purl(eco: &str, leaf: &str) -> Option { - leaf_to_purl(eco, strip_leaf_suffix(leaf).trim_end_matches('/')) - .map(|purl| canonical_base_purl(&purl)) -} - -// ── shared extractor helpers ───────────────────────────────────────────── - -/// How a lock format spells a vendored leaf: literally ([`vendor_ref`]) or -/// with `#` / `::` decorations after it ([`vendor_ref_decorated`]). -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub(crate) enum LeafGrammar { - /// The string is a file path: a `#` / `::` belongs to the leaf. - Literal, - /// The format defines `#` / `::` decorations after the leaf (yarn - /// classic / berry, url-form pip and Hatch references). - Decorated, -} - -/// What [`DiscoverCtx::locate`] computes for one wiring string. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub(crate) struct LocateOpts { - pub(crate) grammar: LeafGrammar, - /// [`LeafGrammar::Literal`] only: also report a vendored path whose leaf - /// carries a decoration ([`is_decorated_vendor_path`]) — the npm, pnpm - /// and bun extractors diagnose it; composer and the pypi readers do not - /// check it. - pub(crate) decorated_check: bool, -} - -impl LocateOpts { - /// A literal path, decorated leaves reported (npm, pnpm, bun). - pub(crate) const LITERAL_CHECKED: Self = LocateOpts { - grammar: LeafGrammar::Literal, - decorated_check: true, - }; - /// A literal path, decorated leaves not checked (composer, pypi locks, - /// bare pip / Hatch paths). - pub(crate) const LITERAL: Self = LocateOpts { - grammar: LeafGrammar::Literal, - decorated_check: false, - }; - /// A decorated spelling (yarn, url-form pip / Hatch references). - pub(crate) const DECORATED: Self = LocateOpts { - grammar: LeafGrammar::Decorated, - decorated_check: false, - }; -} - -/// Where one lockfile / config string points — every identity decision an -/// extractor takes from it, computed once. Each extractor keeps its own -/// decision ORDER over the fields. -#[derive(Debug, Clone, Default, PartialEq, Eq)] -pub(crate) struct Located { - /// The root-anchored vendored artifact the string names, read with the - /// requested [`LeafGrammar`]. - pub(crate) vendored: Option, - /// The patch uuid of a Socket-hosted url ([`DiscoverCtx::hosted_uuid`]), - /// computed eagerly: it needs an http(s) url on an allowed origin while - /// `vendored` needs the string to START at `.socket/vendor/`, so at most - /// one of the two is ever set. - pub(crate) hosted: Option, - /// [`is_decorated_vendor_path`] when [`LocateOpts::decorated_check`] is - /// set (Literal grammar only), else `false`. - pub(crate) decorated_leaf: bool, -} - -impl DiscoverCtx<'_> { - /// Classify `s` (see [`Located`]). - pub(crate) fn locate(&self, s: &str, opts: LocateOpts) -> Located { - let vendored = match opts.grammar { - LeafGrammar::Literal => vendor_ref(s), - LeafGrammar::Decorated => vendor_ref_decorated(s), - }; - let decorated_leaf = opts.decorated_check - && opts.grammar == LeafGrammar::Literal - && is_decorated_vendor_path(s); - Located { - vendored, - hosted: self.hosted_uuid(s), - decorated_leaf, - } - } -} - -/// The Socket wiring a lock location carries. -#[derive(Debug, Clone, PartialEq, Eq)] -pub(crate) enum Wired { - Hosted(String), - Vendored(VendorRef), -} - -impl Wired { - pub(crate) fn uuid(&self) -> &str { - match self { - Wired::Hosted(uuid) => uuid, - Wired::Vendored(vref) => &vref.uuid, - } - } - - /// The same patch artifact: the same hosted uuid, or the same vendored - /// artifact path. - pub(crate) fn same_artifact(&self, other: &Wired) -> bool { - match (self, other) { - (Wired::Hosted(a), Wired::Hosted(b)) => a == b, - (Wired::Vendored(a), Wired::Vendored(b)) => a.artifact_rel == b.artifact_rel, - _ => false, - } - } -} - -/// Whether `s` points into SOME `.socket/vendor/` tree, any anchoring -/// (backslashes folded) — a Socket-shaped string whose rejection is worth a -/// diagnostic (cargo `[patch]` paths, gem remotes, composer path dists, -/// maven repository urls). -pub(crate) fn names_vendor_dir(s: &str) -> bool { - s.replace('\\', "/").contains(&format!("{VENDOR_DIR}/")) -} - -/// [`names_vendor_dir`] that also folds JSON / TOML `\/` escapes (the pypi -/// readers). -pub(crate) fn mentions_vendor_dir(s: &str) -> bool { - s.replace("\\/", "/") - .replace('\\', "/") - .contains(&format!("{VENDOR_DIR}/")) -} - -/// Whether `s` STARTS at this project's `.socket/vendor/` after the `file:` -/// / `./` spellings [`vendor_ref_decorated`] accepts (yarn's rule; `\/` is -/// not folded). -pub(crate) fn root_anchored_spelling(s: &str) -> bool { - let norm = s.trim().replace('\\', "/"); - let norm = norm.strip_prefix("file:").unwrap_or(&norm); - let norm = norm.strip_prefix("./").unwrap_or(norm); - norm.starts_with(&format!("{VENDOR_DIR}/")) -} - -/// Whether vendored artifact `vref` is the npm tarball the vendor backends -/// write for `purl` (`[@scope/]-.tgz` in the npm vendor dir) -/// — the leaf rule the npm, pnpm and yarn extractors share. -pub(crate) fn npm_vendored_tarball_names(vref: &VendorRef, purl: &str) -> bool { - vref.eco == "npm" && vendored_leaf_purl("npm", &vref.leaf).as_deref() == Some(purl) -} - /// Parse a JSON document read from root-relative `file`; `Err` carries the /// diagnostic detail `" is not valid JSON: "`. No BOM handling: /// the callers that tolerate one strip it first. @@ -3625,138 +3294,6 @@ mod tests { assert_eq!(discovery.vendor_entry_in_use(root, &entry).await, None); } - #[test] - fn vendor_ref_strips_lock_suffixes_and_requires_a_root_anchor() { - let a = UUID_A; - for spelled in [ - format!("file:.socket/vendor/npm/{a}/left-pad-1.3.0.tgz"), - format!("file:./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz"), - format!("./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz"), - format!(".socket/vendor/npm/{a}/left-pad-1.3.0.tgz"), - format!(".socket\\vendor\\npm\\{a}\\left-pad-1.3.0.tgz"), - format!(".socket\\/vendor\\/npm\\/{a}\\/left-pad-1.3.0.tgz"), - // yarn classic `#`, berry fragment + locator, pip `#sha256=`. - format!("file:./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz#0123abcd"), - format!( - "./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz#./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz::hash=abc123&locator=app%40workspace%3A." - ), - format!("./.socket/vendor/npm/{a}/left-pad-1.3.0.tgz::locator=app%40workspace%3A."), - ] { - let decorated = spelled.contains('#') || spelled.contains("::"); - // The literal-path reader refuses a decorated leaf outright: to - // npm / pnpm / bun / … it names a DIFFERENT file. - assert_eq!(vendor_ref(&spelled).is_none(), decorated, "{spelled}"); - let v = vendor_ref_decorated(&spelled).unwrap_or_else(|| panic!("{spelled}")); - assert_eq!(v.eco, "npm", "{spelled}"); - assert_eq!(v.uuid, a, "{spelled}"); - assert_eq!(v.leaf, "left-pad-1.3.0.tgz", "{spelled}"); - assert_eq!( - v.artifact_rel, - format!(".socket/vendor/npm/{a}/left-pad-1.3.0.tgz"), - "{spelled}" - ); - } - // Dir-shaped leaves keep their nesting, trailing slash dropped. - let v = vendor_ref(&format!( - "./.socket/vendor/golang/{a}/github.com/foo/bar@v1.2.3/" - )) - .unwrap(); - assert_eq!(v.leaf, "github.com/foo/bar@v1.2.3"); - } - - #[test] - fn vendor_ref_rejects_escapes_foreign_anchors_and_bad_uuids() { - let a = UUID_A; - for bad in [ - // The package manager would read OUTSIDE the project root. - format!("../.socket/vendor/npm/{a}/x-1.0.0.tgz"), - format!("file:../.socket/vendor/npm/{a}/x-1.0.0.tgz"), - format!("/abs/project/.socket/vendor/npm/{a}/x-1.0.0.tgz"), - format!("sub/.socket/vendor/npm/{a}/x-1.0.0.tgz"), - // Traversal inside the leaf. - format!(".socket/vendor/npm/{a}/../../../etc/passwd"), - format!(".socket/vendor/npm/{a}/a//b.tgz"), - // No leaf / uuid-dir only / unknown eco / non-canonical uuid. - format!(".socket/vendor/npm/{a}"), - format!(".socket/vendor/npm/{a}/"), - format!(".socket/vendor/jsr/{a}/x-1.0.0.tgz"), - format!(".socket/vendor/npm/{}/x-1.0.0.tgz", a.to_ascii_uppercase()), - ".socket/vendor/npm/not-a-uuid/x-1.0.0.tgz".to_string(), - "https://registry.npmjs.org/x/-/x-1.0.0.tgz".to_string(), - ] { - assert_eq!(vendor_ref(&bad), None, "{bad}"); - } - } - - #[test] - fn vendor_uuid_dir_accepts_only_the_bare_uuid_dir() { - let a = UUID_A; - assert_eq!( - vendor_uuid_dir(&format!(".socket/vendor/nuget/{a}")), - Some(("nuget".to_string(), a.to_string())) - ); - assert_eq!( - vendor_uuid_dir(&format!("./.socket/vendor/maven/{a}/")), - Some(("maven".to_string(), a.to_string())) - ); - assert_eq!( - vendor_uuid_dir(&format!(".socket/vendor/nuget/{a}/x.nupkg")), - None - ); - assert_eq!( - vendor_uuid_dir(&format!("../.socket/vendor/nuget/{a}")), - None - ); - assert_eq!(vendor_uuid_dir(".socket/vendor/nuget/nope"), None); - } - - #[test] - fn vendored_leaf_purl_strips_and_canonicalizes() { - assert_eq!( - vendored_leaf_purl("npm", "@scope/pkg-1.2.3.tgz#deadbeef").as_deref(), - Some("pkg:npm/@scope/pkg@1.2.3") - ); - assert_eq!( - vendored_leaf_purl( - "pypi", - "Python_Dateutil-2.8.2-py2.py3-none-any.whl#sha256=ab" - ) - .as_deref(), - Some("pkg:pypi/python-dateutil@2.8.2") - ); - assert_eq!( - vendored_leaf_purl("cargo", "serde-1.0.0/").as_deref(), - Some("pkg:cargo/serde@1.0.0") - ); - assert_eq!(vendored_leaf_purl("npm", "not-a-tarball"), None); - } - - #[test] - fn socket_patch_names_are_exact() { - assert_eq!( - socket_patch_name_uuid(&format!("socket-patch-{UUID_A}"), false).as_deref(), - Some(UUID_A) - ); - assert_eq!( - socket_patch_name_uuid(&format!("socket-patch-vendor-{UUID_A}"), true).as_deref(), - Some(UUID_A) - ); - // The hosted grammar never matches the vendored id and vice versa. - assert_eq!( - socket_patch_name_uuid(&format!("socket-patch-vendor-{UUID_A}"), false), - None - ); - assert_eq!( - socket_patch_name_uuid(&format!("socket-patch-{UUID_A}"), true), - None - ); - assert_eq!( - socket_patch_name_uuid(&format!("socket-patch-{UUID_A}-x"), false), - None - ); - assert_eq!(socket_patch_name_uuid("socket-patch-nope", false), None); - } - #[test] fn push_validates_canonicalizes_and_dedupes() { let mut out = Discovery::default(); @@ -4769,100 +4306,6 @@ mod tests { ); } - /// [`DiscoverCtx::hosted_uuid`]'s plain-url shortcut answers exactly - /// what the full parse does, for plain urls on other hosts and for - /// every spelling it must leave to the parse. - #[test] - fn hosted_uuid_shortcut_agrees_with_the_full_parse() { - let (a, t) = (UUID_A, TOKEN); - let path = format!("/patch/npm/x/1.0.0/{t}/{a}/x-1.0.0.tgz"); - let hosts = [ - "patch.socket.dev", - "PATCH.Socket.DEV", - "patch.socket.dev.", - "patch.socket.dev:443", - "patch.socket.dev:8443", - "patch.socket.dev:", - "registry.npmjs.org", - "registry.npmjs.org:443", - "127.0.0.1:4545", - "127.0.0.1", - "0x7f.0.0.1:4545", - "127.1:4545", - "[::1]:4545", - "staging.example.com:8443", - "Staging.Example.COM:8443", - "staging.example.com", - "xn--stging-bua.example.com", - "user@patch.socket.dev", - "user:pw@registry.npmjs.org", - "patch%2esocket.dev", - "patch.soc\tket.dev", - "patch.socket.dev\\", - "foo.123", - "foo.0x1f", - "", - ]; - let urls: Vec = hosts - .iter() - .flat_map(|host| { - [ - format!("https://{host}{path}"), - format!("http://{host}{path}"), - format!("HTTPS://{host}{path}"), - format!("sparse+https://{host}{path}"), - format!("git+https://{host}{path}"), - format!("https:///{host}{path}"), - format!("https://{host}?q={path}"), - format!(" https://{host}{path} "), - format!("https://{host}"), - ] - }) - .chain([ - format!("https:\\/\\/patch.socket.dev{}", path.replace('/', "\\/")), - format!("https%3A%2F%2Fpatch.socket.dev{}", path.replace('/', "%2F")), - "file:x.tgz".to_string(), - "npm:x@1.0.0".to_string(), - "https://".to_string(), - ]) - .collect(); - for origins in [ - vec![], - vec!["http://127.0.0.1:4545".to_string()], - vec![ - "https://staging.example.com:8443".to_string(), - "http://[::1]:4545".to_string(), - "https://xn--stging-bua.example.com".to_string(), - ], - ] { - let ctx = DiscoverCtx::with_origins(Path::new("/nonexistent"), &origins); - for url in &urls { - assert_eq!( - ctx.hosted_uuid(url), - crate::patch::redirect::hosted_patch_uuid(url, &origins), - "{url} under {origins:?}" - ); - } - assert_eq!( - ctx.hosted_uuid(&format!("https://patch.socket.dev{path}")) - .as_deref(), - Some(a) - ); - } - assert_eq!( - plain_url_domain("https://Registry.NPMJS.org/x"), - Some("registry.npmjs.org".into()) - ); - for not_plain in [ - "https://127.0.0.1/x", - "https://a%2eb.com/x", - "https://u@a.com/x", - "https://xn--a.com/x", - ] { - assert_eq!(plain_url_domain(not_plain), None, "{not_plain}"); - } - } - /// The claim API: a recognized uuid is decided by the refs alone (live /// only for the package — and, vendored, the artifact — a ref wires), /// while an unrecognized one is left to the caller (`None`). diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json index 28d387299..efdb902f1 100644 --- a/scripts/loc-budget.json +++ b/scripts/loc-budget.json @@ -2,6 +2,7 @@ "groups": { "lock-readers": { "paths": [ + "crates/socket-patch-core/src/origin.rs", "crates/socket-patch-core/src/formats/", "crates/socket-patch-core/src/vex/discover/", "crates/socket-patch-core/src/vendor/lock_inventory/", @@ -16,7 +17,7 @@ "crates/socket-patch-core/src/utils/pipenv.rs", "crates/socket-patch-core/src/crawlers/deno_crawler.rs" ], - "max": 30898 + "max": 30976 }, "lock-writers": { "paths": [ @@ -29,7 +30,7 @@ "crates/socket-patch-core/src/vendor/npm_origin.rs", "crates/socket-patch-core/src/vendor/test_support" ], - "max": 88363 + "max": 88314 } } } From b69623501989872efdb57c8b9b514caf85b2a049 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 10:49:22 -0600 Subject: [PATCH 07/36] Read every pnpm lock document, the env lockfile as its own tree PnpmLock is now the one pnpm entry walk, over every document of the lock. Each entry is tagged with its document (PnpmDocument::Project or Env), and documents are told apart by content: in a multi-document lock, one whose importers carry only configDependencies / packageManagerDependencies is pnpm 11+'s env lockfile, every other document is a project one. grammar::main_document (the text after the last `---`) is gone, so a crafted lock can no longer pick its own "project" and hide the rest (lock-model divergence N4). Behavior changes (owner decision: the env document counts and is its own install tree): - The inventory (PnpmLock::entries), PnpmLock::resolves, the legacy vendored in-use check (vendored_in_use) and InstalledPackages read the env document too, as the hosted planner and the v9 in-use check already did. env_document_is_not_in_the_inventory pinned #1949's skip; it becomes env_document_is_read_as_its_own_tree. - Discovery emits refs for Socket wiring in the env document. Its records are keyed ` (env lockfile)` while the contests run, so the project document's registry entries, file: copies and bundled copies never contest its refs (and the reverse), its wiring does not exempt the project document from contesting another lock, and it is skipped by the cross-lock contest both ways (config dependencies install apart from every node_modules). finalize maps the key back to the lock path. Pinned by pnpm_env_document_is_its_own_install_tree. - A lock with several non-env documents reads all of them as project documents (before: only the last). Pinned by documents_are_identified_by_content_not_position. InstalledPackages::from_lock_text is rebuilt on PnpmLock (keeping its last-wins name:/version: rule and its None cases). pnpm writes the current lockfile (node_modules/.pnpm/lock.yaml) as the project lock alone and installs config dependencies outside the virtual store, so an env document is not expected there; if one appears its entries keep their store entries, the side that never hides an installed package. The vendored set is computed lazily (OnceLock), so the crawler does not pay for it. lock_text_has_member_importers, pnpm_settings_major and the hosted unwind's tarball evidence read documents by content too (the latter through PnpmLock, project documents only, as before). Co-Authored-By: Claude Opus 5.5 --- .../src/formats/pnpm/grammar.rs | 89 +++++- .../src/formats/pnpm/lines.rs | 3 +- .../socket-patch-core/src/formats/pnpm/mod.rs | 256 ++++++++++++------ .../src/formats/pnpm/workspace.rs | 12 +- .../src/patch/redirect/upstream/npm.rs | 13 +- .../src/vendor/lock_inventory/tests.rs | 3 +- .../socket-patch-core/src/vex/discover/mod.rs | 85 +++++- .../socket-patch-core/src/vex/discover/npm.rs | 161 ++++++++++- 8 files changed, 513 insertions(+), 109 deletions(-) diff --git a/crates/socket-patch-core/src/formats/pnpm/grammar.rs b/crates/socket-patch-core/src/formats/pnpm/grammar.rs index 6030b57bf..10a7a57fc 100644 --- a/crates/socket-patch-core/src/formats/pnpm/grammar.rs +++ b/crates/socket-patch-core/src/formats/pnpm/grammar.rs @@ -9,6 +9,8 @@ use std::ops::Range; use crate::formats::text::strip_bom; +use super::lines::ENV_DOC_IMPORTER_FIELDS; + /// Early pnpm 1 writes shrinkwrapVersion 3 without a minor version and /// unconditionally drops registry tarball URLs on install. Its frozen flag /// cannot preserve this redirect (verified with pnpm 1.0.0). @@ -88,24 +90,93 @@ pub(crate) fn is_pnpm_lock_text(text: &str) -> bool { .any(|line| line.starts_with("lockfileVersion:") || line.starts_with("shrinkwrapVersion:")) } -/// The main document of a pnpm lock: the text after its last column-0 -/// `---` marker, else all of it. pnpm 11+ writes the env lockfile -/// (`configDependencies` / `packageManagerDependencies`, whose resolutions -/// never carry `tarball:`) as a first document ahead of the project lock. -pub(crate) fn main_document(content: &str) -> &str { +/// Which document of a pnpm lock a span belongs to ([`documents`]). +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum PnpmDocument { + /// The project lock: what `pnpm install` links into `node_modules`. + Project, + /// pnpm 11+'s env lockfile: the `configDependencies` / + /// `packageManagerDependencies` pnpm installs into its config store + /// and its own package-manager install, not into `node_modules`. Its + /// own install tree (docs/design/lock-model-unification.md, Decisions). + Env, +} + +/// The documents of a pnpm lock, in file order: the byte range of each +/// (separator lines excluded, a leading BOM inside the first) and what it +/// is. A text without a column-0 `---` line is one project document. A +/// blank document (the empty span before a leading `---`) is dropped. +/// +/// pnpm >= 11 writes the env lockfile as a first document ahead of the +/// project lock. A document is identified by its CONTENT, not by its +/// position: in a lock of two or more documents, one whose `importers:` +/// carry only [`ENV_DOC_IMPORTER_FIELDS`] (at least one) is +/// [`PnpmDocument::Env`]; every other document is +/// [`PnpmDocument::Project`]. So a crafted lock cannot hide entries from a +/// reader by adding documents: each one is read, as the tree its content +/// says it is. CRLF reads like LF. +pub(crate) fn documents(content: &str) -> Vec<(Range, PnpmDocument)> { + let mut spans = Vec::new(); let mut start = 0; let mut offset = 0; - for line in content.split_inclusive('\n') { - offset += line.len(); + for (i, line) in content.split_inclusive('\n').enumerate() { let text = line.trim_end_matches(['\r', '\n']); + let text = if i == 0 { strip_bom(text) } else { text }; if text .strip_prefix("---") .is_some_and(|rest| rest.is_empty() || rest.starts_with([' ', '\t'])) { - start = offset; + spans.push(start..offset); + start = offset + line.len(); + } + offset += line.len(); + } + spans.push(start..content.len()); + spans.retain(|span| { + strip_bom(&content[span.clone()]).lines().any(|line| { + let line = line.trim(); + !line.is_empty() && !line.starts_with('#') + }) + }); + if spans.is_empty() { + return vec![(0..content.len(), PnpmDocument::Project)]; + } + let several = spans.len() > 1; + spans + .into_iter() + .map(|span| { + let kind = if several && is_env_document(&content[span.clone()]) { + PnpmDocument::Env + } else { + PnpmDocument::Project + }; + (span, kind) + }) + .collect() +} + +/// Whether one document's `importers:` carry only the env lockfile's +/// fields ([`ENV_DOC_IMPORTER_FIELDS`]), at least one of them. +fn is_env_document(doc: &str) -> bool { + let mut in_importers = false; + let mut env_fields = 0usize; + for line in strip_bom(doc).split('\n') { + let line = line.strip_suffix('\r').unwrap_or(line); + if !line.is_empty() && !line.starts_with([' ', '#']) { + in_importers = line == "importers:"; + continue; + } + if !in_importers { + continue; + } + if let Some((field, _, _)) = super::lines::parse_key_line(line, 4) { + if !ENV_DOC_IMPORTER_FIELDS.contains(&field) { + return false; + } + env_fields += 1; } } - &content[start..] + env_fields > 0 } /// The value of the entry-level (four-space) `field:` line of a packages diff --git a/crates/socket-patch-core/src/formats/pnpm/lines.rs b/crates/socket-patch-core/src/formats/pnpm/lines.rs index 0badfcb72..3899f56ea 100644 --- a/crates/socket-patch-core/src/formats/pnpm/lines.rs +++ b/crates/socket-patch-core/src/formats/pnpm/lines.rs @@ -26,7 +26,8 @@ pub(crate) fn section_bounds(lines: &[String], name: &str) -> Option<(usize, usi /// The importer dependency fields pnpm writes into the env document of a /// two-document lock. -const ENV_DOC_IMPORTER_FIELDS: [&str; 2] = ["configDependencies", "packageManagerDependencies"]; +pub(crate) const ENV_DOC_IMPORTER_FIELDS: [&str; 2] = + ["configDependencies", "packageManagerDependencies"]; /// Split a lock into `(prefix, project document)`, `prefix + project == /// text`. pnpm >= 11 writes `pnpm-lock.yaml` as two YAML documents when the diff --git a/crates/socket-patch-core/src/formats/pnpm/mod.rs b/crates/socket-patch-core/src/formats/pnpm/mod.rs index 1b06aba9e..feb05405b 100644 --- a/crates/socket-patch-core/src/formats/pnpm/mod.rs +++ b/crates/socket-patch-core/src/formats/pnpm/mod.rs @@ -33,12 +33,14 @@ pub(crate) mod workspace; pub use version::{matches_file_id_build_keys, root_only_workspace_breaks_add}; +pub use grammar::PnpmDocument; pub(crate) use grammar::{ entry_bundled, entry_field, is_pnpm_lock_text, Bundled, Entry, Resolution, }; pub(crate) use hosted::plan_hosted; use std::collections::HashSet; +use std::sync::OnceLock; use crate::constants::npm_family::PNPM_LOCK; use crate::formats::text::strip_bom; @@ -58,6 +60,9 @@ pub(crate) struct PnpmPackage<'a> { /// The entry's `resolution:` map; `None` when it has none or the /// grammar refuses it (duplicate keys, nested values, aliases). pub(crate) resolution: Option>, + /// The lock document the entry is in ([`grammar::documents`]): the + /// project lock or pnpm 11+'s env lockfile, each its own install tree. + pub(crate) document: PnpmDocument, } impl<'a> PnpmPackage<'a> { @@ -78,16 +83,22 @@ impl<'a> PnpmPackage<'a> { } } -/// Every `packages:` entry of a pnpm lock text, in lock order — the ONE -/// entry walk the inventory, lockfile discovery and repair share. Every -/// entry is returned (registry, rekeyed vendored, hosted, directory, git); -/// each consumer applies its own key and resolution rules. -pub(crate) fn pnpm_packages(text: &str) -> Vec> { +/// Every `packages:` entry of a pnpm lock text, in lock order, each tagged +/// with its document — the ONE entry walk every reader shares (through +/// [`PnpmLock::packages`]). Every entry is returned (registry, rekeyed +/// vendored, hosted, directory, git); each consumer applies its own key +/// and resolution rules. +fn pnpm_packages(text: &str) -> Vec> { + let documents = grammar::documents(text); grammar::entries(text) .into_iter() .map(|entry| PnpmPackage { key: grammar::unquote(entry.key.trim()), resolution: grammar::resolution(&entry), + document: documents + .iter() + .find(|(span, _)| span.contains(&entry.offset)) + .map_or(PnpmDocument::Project, |(_, kind)| *kind), entry, }) .collect() @@ -368,16 +379,21 @@ pub(crate) fn check_v9_lock_version(text: &str) -> Result<(), String> { // ── the model ── /// One pnpm lock, parsed once (see the module docs). +/// +/// Every document is read. pnpm 11+ writes its env lockfile +/// (`configDependencies`, `packageManagerDependencies`) as a first +/// document; its entries are installed too (into pnpm's config store, not +/// `node_modules`), so they are part of the lock, tagged +/// [`PnpmDocument::Env`]: a reader that keeps install trees apart +/// (lockfile discovery's contests) reads the env document as a tree of its +/// own ([`PnpmPackage::document`]). pub struct PnpmLock<'t> { text: &'t str, - /// The project document ([`grammar::main_document`]): pnpm 11+ writes - /// its env lockfile (`configDependencies`, `packageManagerDependencies`) - /// as a first document whose packages the install never links (#1949). - doc: &'t str, - /// Entries of the project document only, with offsets into `text`. + /// Every document's entries, with offsets into `text`. packages: Vec>, - /// [`vendored_npm_uuids`] of the project document. - vendored: HashSet, + /// [`vendored_npm_uuids`] of the whole lock, computed on first use (the + /// store crawler never asks). + vendored: OnceLock>, } /// One `packages:` entry whose resolution names a tarball — a candidate @@ -392,18 +408,10 @@ pub(crate) struct PnpmTarballRef<'p> { impl<'t> PnpmLock<'t> { /// Parse a lock text (any content: a non-pnpm text has no entries). pub fn parse(text: &'t str) -> Self { - let doc = grammar::main_document(strip_bom(text)); - // `doc` is a suffix of `text`: parse the whole text so offsets stay - // relative to it, and keep the entries that start in `doc`. - let doc_start = text.len() - doc.len(); PnpmLock { text, - doc, - packages: pnpm_packages(text) - .into_iter() - .filter(|p| p.entry.offset >= doc_start) - .collect(), - vendored: vendored_npm_uuids(doc), + packages: pnpm_packages(text), + vendored: OnceLock::new(), } } @@ -416,23 +424,40 @@ impl<'t> PnpmLock<'t> { is_pnpm_lock_text(self.text) } - /// Every `packages:` entry, in lock order. + /// Every `packages:` entry of every document, in lock order, each + /// tagged with its [`PnpmDocument`]. pub(crate) fn packages(&self) -> &[PnpmPackage<'t>] { &self.packages } + /// The entries of one document kind, in lock order. + pub(crate) fn packages_of( + &self, + document: PnpmDocument, + ) -> impl Iterator> { + self.packages.iter().filter(move |p| p.document == document) + } + + /// The document kinds the lock has entries in, project first. + pub(crate) fn documents(&self) -> Vec { + let mut kinds: Vec = self.packages.iter().map(|p| p.document).collect(); + kinds.sort(); + kinds.dedup(); + kinds + } + fn has_packages_section(&self) -> bool { - self.doc + self.text .lines() .any(|l| l.trim_end_matches('\r') == "packages:") } /// The registry inventory: one entry per plain-registry packages key - /// (every key generation, [`pnpm_registry_key`]), flow and block - /// resolutions, CRLF included. A resolution the grammar refuses leaves - /// the entry listed without a verifier; our own vendored tarballs are - /// not registry dependencies and are dropped. `None` when the lock has - /// no `packages:` section. + /// (every key generation, [`pnpm_registry_key`]) of every document, + /// flow and block resolutions, CRLF included. A resolution the grammar + /// refuses leaves the entry listed without a verifier; our own vendored + /// tarballs are not registry dependencies and are dropped. `None` when + /// the lock has no `packages:` section. pub fn entries(&self) -> Option> { if !self.has_packages_section() { return None; @@ -504,7 +529,9 @@ impl<'t> PnpmLock<'t> { /// package.json even when nothing matches it). CRLF locks read like LF /// ones. pub fn vendored_in_use(&self, uuid: &str) -> bool { - self.vendored.contains(uuid) + self.vendored + .get_or_init(|| vendored_npm_uuids(self.text)) + .contains(uuid) } } @@ -524,54 +551,40 @@ pub(crate) struct InstalledPackages { impl InstalledPackages { /// Read a current lockfile in any lock generation (v5.4 `/name/1.0.0`, /// v6 `/name@1.0.0`, v9 `name@1.0.0` keys; a non-registry entry by its - /// `name:` / `version:` fields, or a v9 key's name). `None` when the - /// text is not a pnpm lock, or holds an entry whose package cannot be - /// named — the caller must then keep every store entry. + /// `name:` / `version:` fields, the last of each winning, or a v9 key's + /// name), through [`PnpmLock`]'s entry walk. `None` when the text is not + /// a pnpm lock, or holds an entry whose package cannot be named — the + /// caller must then keep every store entry. + /// + /// Every document counts. pnpm writes the current lockfile as the + /// project lock alone (the env lockfile lives only in `pnpm-lock.yaml`, + /// and config dependencies install into `node_modules/.pnpm-config`, not + /// the virtual store), so a second document is not expected here; if + /// one appears, its entries keep their store entries too, the side that + /// never hides an installed package. pub(crate) fn from_lock_text(text: &str) -> Option { if !grammar::is_pnpm_lock_text(text) { return None; } - // One line walk (the scan reads this on every run, for every - // package of the install): two-space `packages:` keys and their - // four-space `name:` / `version:` fields, nothing else. + let lock = PnpmLock::parse(text); let mut out = Self::default(); - let mut entries = 0usize; - let mut in_packages = false; - let mut current: Option<(&str, Option<&str>, Option<&str>)> = None; - let mut finish = |entry: Option<(&str, Option<&str>, Option<&str>)>, out: &mut Self| { - let Some((key, field_name, field_version)) = entry else { - return Some(()); - }; - entries += 1; - out.insert_entry(key, field_name, field_version) - }; - for raw in grammar::main_document(strip_bom(text)).split('\n') { - let line = raw.strip_suffix('\r').unwrap_or(raw); - if !line.is_empty() && !line.starts_with([' ', '#']) { - finish(current.take(), &mut out)?; - in_packages = line == "packages:"; - continue; - } - if !in_packages { - continue; - } - if let Some((key, _, _)) = lines::parse_key_line(line, 2) { - finish(current.take(), &mut out)?; - current = Some((key, None, None)); - } else if let (Some(entry), Some((field, _, value))) = - (current.as_mut(), lines::parse_key_line(line, 4)) - { - let value = Some(grammar::unquote(value.trim())).filter(|v| !v.is_empty()); - match field { - "name" => entry.1 = value, - "version" => entry.2 = value, - _ => {} + for package in lock.packages() { + let (mut name, mut version) = (None, None); + for line in package.entry.body.split('\n') { + let line = line.strip_suffix('\r').unwrap_or(line); + if let Some((field, _, value)) = lines::parse_key_line(line, 4) { + let value = Some(grammar::unquote(value.trim())).filter(|v| !v.is_empty()); + match field { + "name" => name = value, + "version" => version = value, + _ => {} + } } } + out.insert_entry(package.key, name, version)?; } - finish(current.take(), &mut out)?; // A lock listing no package at all says nothing about the store. - (entries > 0).then_some(out) + (!lock.packages().is_empty()).then_some(out) } /// Record one `packages:` entry; `None` when its package cannot be @@ -768,12 +781,15 @@ mod tests { assert!(!PnpmLock::parse(importers).resolves("left-pad", "1.3.0")); } - /// #1949: a pnpm 11+ lock's env document (config dependencies the - /// install never links) is not part of the inventory: `entries()`, - /// `resolves()`, the vendored uuids and `InstalledPackages` all read - /// the project document alone, and agree with each other. + /// pnpm 11+'s env document is part of the lock (owner decision, + /// lock-model divergence N4; this reverses #1949's skip): pnpm installs + /// its config dependencies, and the hosted planner and the v9 vendored + /// in-use check already read it. `entries()`, `resolves()`, the + /// vendored uuids and `InstalledPackages` all read both documents and + /// agree with each other; every entry is tagged with its document, so + /// discovery keeps the env document a tree of its own. #[test] - fn env_document_is_not_in_the_inventory() { + fn env_document_is_read_as_its_own_tree() { let env = format!( "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n cfg-dep:\n specifier: 2.0.0\n version: 2.0.0\n\npackages:\n\n cfg-dep@2.0.0:\n resolution: {{integrity: sha512-c}}\n\n b@file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz:\n resolution: {{integrity: sha512-x, tarball: file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz}}\n version: 1.0.0\n\n" ); @@ -792,17 +808,44 @@ mod tests { .collect(); assert_eq!( names, - [("left-pad".to_string(), "1.3.0".to_string())], + [ + ("cfg-dep".to_string(), "2.0.0".to_string()), + ("left-pad".to_string(), "1.3.0".to_string()) + ], "{text:?}" ); assert!(lock.resolves("left-pad", "1.3.0")); - assert!(!lock.resolves("cfg-dep", "2.0.0")); - assert!(!lock.vendored_in_use(UUID)); - assert!(lock.packages().iter().all(|p| p.entry.offset >= env.len())); + assert!(lock.resolves("cfg-dep", "2.0.0")); + assert!(lock.vendored_in_use(UUID)); + let tagged: Vec<_> = lock + .packages() + .iter() + .map(|p| (p.key, p.document)) + .collect(); + assert_eq!( + tagged, + [ + ("cfg-dep@2.0.0", PnpmDocument::Env), + ( + format!("b@file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz").as_str(), + PnpmDocument::Env + ), + ("left-pad@1.3.0", PnpmDocument::Project), + ], + "{text:?}" + ); + assert_eq!(lock.documents(), [PnpmDocument::Project, PnpmDocument::Env]); let installed = InstalledPackages::from_lock_text(&text).unwrap(); assert!(installed.contains("left-pad", "1.3.0")); - assert!(!installed.contains("cfg-dep", "2.0.0")); + assert!(installed.contains("cfg-dep", "2.0.0")); + assert!(installed.contains("b", "1.0.0")); } + // One document: the project, whatever its importers say. + let lone = env.trim_start_matches("---\n"); + assert!(PnpmLock::parse(lone) + .packages() + .iter() + .all(|p| p.document == PnpmDocument::Project)); // Offsets still index the whole text. let text = format!("{env}{project}"); let lock = PnpmLock::parse(&text); @@ -813,6 +856,61 @@ mod tests { ); } + /// N4: documents are told apart by content, not position. Before, the + /// readers took the text after the LAST `---` as the project, so a + /// lock with an extra document chose its own "project" and hid the + /// rest; now every document is read, as the tree its importers say. + #[test] + fn documents_are_identified_by_content_not_position() { + let env = "lockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n cfg-dep:\n specifier: 2.0.0\n version: 2.0.0\n\npackages:\n\n cfg-dep@2.0.0:\n resolution: {integrity: sha512-c}\n"; + let project = |dep: &str| { + format!( + "lockfileVersion: '9.0'\n\nimporters:\n\n .:\n dependencies:\n {dep}:\n specifier: 1.0.0\n version: 1.0.0\n\npackages:\n\n {dep}@1.0.0:\n resolution: {{integrity: sha512-x}}\n" + ) + }; + let kinds = |text: &str| -> Vec<(String, PnpmDocument)> { + PnpmLock::parse(text) + .packages() + .iter() + .map(|p| (p.key.to_string(), p.document)) + .collect() + }; + let own = |pairs: &[(&str, PnpmDocument)]| -> Vec<(String, PnpmDocument)> { + pairs.iter().map(|(k, d)| (k.to_string(), *d)).collect() + }; + use PnpmDocument::{Env, Project}; + // The env document after the project still reads as the env tree. + let swapped = format!("---\n{}---\n{env}", project("a")); + assert_eq!( + kinds(&swapped), + own(&[("a@1.0.0", Project), ("cfg-dep@2.0.0", Env)]) + ); + // A first document that is not the env document is a project one, + // and its entries are no longer skipped. + let two = format!("---\n{}---\n{}", project("a"), project("b")); + assert_eq!( + kinds(&two), + own(&[("a@1.0.0", Project), ("b@1.0.0", Project)]) + ); + // A third document is read too, CRLF and a BOM included. + let three = format!("---\n{env}---\n{}---\n{}", project("a"), project("b")); + for text in [ + three.clone(), + three.replace('\n', "\r\n"), + format!("\u{feff}{three}"), + ] { + assert_eq!( + kinds(&text), + own(&[ + ("cfg-dep@2.0.0", Env), + ("a@1.0.0", Project), + ("b@1.0.0", Project) + ]), + "{text:?}" + ); + } + } + #[test] fn vendored_in_use_reads_v9_and_legacy_keys_and_crlf() { let v9 = format!( diff --git a/crates/socket-patch-core/src/formats/pnpm/workspace.rs b/crates/socket-patch-core/src/formats/pnpm/workspace.rs index 5c0d12b7f..63b83c9fd 100644 --- a/crates/socket-patch-core/src/formats/pnpm/workspace.rs +++ b/crates/socket-patch-core/src/formats/pnpm/workspace.rs @@ -202,11 +202,17 @@ pub(crate) fn lock_has_member_importers>(lines: &[S]) -> bool { } /// [`lock_has_member_importers`] of a whole `pnpm-lock.yaml` text: its -/// project document alone ([`super::grammar::main_document`]), so importers +/// project documents alone ([`super::grammar::documents`]), so importers /// of a pnpm 11+ env document never count. pub(crate) fn lock_text_has_member_importers(text: &str) -> bool { - let doc = super::grammar::main_document(strip_bom(text)); - lock_has_member_importers(&doc.split('\n').collect::>()) + use super::grammar::{documents, PnpmDocument}; + documents(text) + .into_iter() + .filter(|(_, kind)| *kind == PnpmDocument::Project) + .any(|(span, _)| { + let doc = strip_bom(&text[span]); + lock_has_member_importers(&doc.split('\n').collect::>()) + }) } /// `text` with the contiguous run of lines `block` removed, when that diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs index 1bb703280..ef0031a0e 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs @@ -1276,15 +1276,16 @@ fn pnpm_registry_is_own(registry: Option<&str>) -> bool { /// env lockfile document ahead of the project lock (only pnpm >= 11 /// writes one, so its `.npmrc` pnpm settings are ignored). pub(crate) fn pnpm_settings_major(text: &str, pm_major: Option) -> Option { - use crate::formats::pnpm::grammar::{is_pnpm_lock_text, main_document}; + use crate::formats::pnpm::grammar::{documents, PnpmDocument}; use crate::formats::pnpm::lock_version_major; let legacy_lock = lock_version_major(text).is_some_and(|major| major < 9) || text .lines() .any(|line| line.starts_with("shrinkwrapVersion:")); - let main = main_document(text); - let env_document = is_pnpm_lock_text(&text[..text.len() - main.len()]); + let env_document = documents(text) + .iter() + .any(|(_, kind)| *kind == PnpmDocument::Env); pm_major .or(legacy_lock.then_some(8)) .or(env_document.then_some(11)) @@ -1504,17 +1505,17 @@ fn pnpm_include_tarball( /// URL can still appear with it off, under a second registry). With no /// bare one, a tarball pnpm could have derived proves it on. `None`: no /// such resolution (an unconventional URL is recorded either way). Only -/// the main document counts: pnpm 11+'s env lockfile document records its +/// the project documents count: pnpm 11+'s env lockfile document records its /// resolutions bare whatever the setting. fn pnpm_lock_tarball_evidence( text: &str, registry: impl Fn(&str) -> Option, is_hosted: impl Fn(&str) -> bool, ) -> Option { - use crate::formats::pnpm::{classify_pnpm_key, grammar::main_document, pnpm_packages, PnpmKey}; + use crate::formats::pnpm::{classify_pnpm_key, PnpmDocument, PnpmKey, PnpmLock}; let mut derived = false; - for package in pnpm_packages(main_document(text)) { + for package in PnpmLock::parse(text).packages_of(PnpmDocument::Project) { let PnpmKey::Registry { name, version } = classify_pnpm_key(package.key) else { continue; }; diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs index bd14058d0..21651261d 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs @@ -2888,7 +2888,8 @@ fn pnpm_resolution_tokens_cover_maps_the_grammar_refuses() { let lock = format!( "lockfileVersion: '9.0'\n\npackages:\n\n x@1.0.0:\n resolution:\n tarball: {url}\n nested:\n a: b\n\n y@1.0.0:\n resolution: {{integrity: sha512-a}}\n resolution: {{tarball: '{url}'}}\n\n z@1.0.0:\n resolution: {{integrity: sha512-z,\n tarball: \"{url}\"}}\n\n ok@1.0.0:\n resolution: {{integrity: sha512-ok}}\n" ); - let packages = crate::formats::pnpm::pnpm_packages(&lock); + let parsed = crate::formats::pnpm::PnpmLock::parse(&lock); + let packages = parsed.packages(); let by_key = |key: &str| packages.iter().find(|p| p.key == key).unwrap(); for key in ["x@1.0.0", "y@1.0.0", "z@1.0.0"] { let package = by_key(key); diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 5c08758c3..427eceecf 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -244,6 +244,36 @@ pub const DIAG_REF_INVALID: &str = "patched_ref_invalid"; /// dependency version, a nuget source with no package mapping). pub const DIAG_REF_UNATTRIBUTABLE: &str = "patched_ref_unattributable"; +/// The label [`pnpm_document_tree`] appends to a pnpm lock's path to key +/// its env lockfile document as an install tree of its own. No file +/// discovery reads ends in it (every lock path ends in `.yaml`). +pub(crate) const PNPM_ENV_TREE_SUFFIX: &str = " (env lockfile)"; + +/// The install-tree key of one document of the pnpm lock at root-relative +/// `file`. pnpm 11+'s env lockfile document (config dependencies, the +/// pinned package manager) installs apart from the project's +/// `node_modules`, so it is its own tree (owner decision, lock-model +/// divergence N4): its records carry ` (env lockfile)` as their +/// file. That keeps its refs and the project document's entries apart in +/// the same-lock and bundled-copy contests ([`Discovery::contest_within_locks`], +/// [`UnwiredCopy`] scopes), keeps its wiring from exempting the project +/// document in [`Discovery::contest_across_locks`], and lets that contest +/// skip it both ways: config dependencies install apart from every +/// `node_modules`, so no other lock decides them and they decide no other +/// lock's ref. Within the env document the same-lock rules apply. The +/// project document keeps `file` itself. [`Discovery::finalize`] maps +/// every key back to `file`, so consumers only ever see real paths; detail +/// texts keep the label. +pub(crate) fn pnpm_document_tree( + file: &str, + document: crate::formats::pnpm::PnpmDocument, +) -> String { + match document { + crate::formats::pnpm::PnpmDocument::Project => file.to_string(), + crate::formats::pnpm::PnpmDocument::Env => format!("{file}{PNPM_ENV_TREE_SUFFIX}"), + } +} + /// One non-fatal discovery finding. The CLI surfaces each as a run warning /// (`code` is the stable routing tag, `file` the root-relative source). #[derive(Debug, Clone, PartialEq, Eq)] @@ -1168,7 +1198,9 @@ impl Discovery { /// 11). This generalizes the npm extractor's shrinkwrap/package-lock /// rule to every pair of locks. PEP 723 script locks (`*.py.lock`) /// neither contest nor are contested: each is scoped to its own script's - /// install. Files of one install tree ([`Discovery::install_trees`]) are + /// install. Neither is a pnpm env lockfile document + /// ([`pnpm_document_tree`]): pnpm installs its config dependencies into + /// its config store whatever else runs. Files of one install tree ([`Discovery::install_trees`]) are /// one lock here: pip resolves a direct reference and a compatible `==` /// pin of the same version in `requirements.txt` and its `-r` includes /// to the reference (#1086). @@ -1193,7 +1225,7 @@ impl Discovery { let mut contested = Vec::new(); let refs = std::mem::take(&mut self.refs); for r in refs { - let other = (!is_script_lock(&r.source_file)) + let other = (!is_script_lock(&r.source_file) && !is_pnpm_env_tree(&r.source_file)) .then(|| { let tree = self.tree_of(&r.source_file); // Only a PyPI ref can match another spelling of its @@ -1212,9 +1244,13 @@ impl Discovery { .find(|e| { let other = self.tree_of(&e.file); // A tree that wires the release itself, in any - // spelling, is no contester. + // spelling, is no contester. A pnpm env + // lockfile installs config dependencies apart + // from every `node_modules`: it contests no + // other tree. e.covers(&r.purl) && other != tree + && !is_pnpm_env_tree(&e.file) && !wiring.get(other).is_some_and(|purls| { purls.contains(&r.purl) || (pypi @@ -1270,6 +1306,41 @@ impl Discovery { } } + /// Map every [`pnpm_document_tree`] key back to its real lock file once + /// the contests that needed the trees apart have run: consumers read, + /// match and display `source_file` / `file` as root-relative paths. + /// Detail texts keep the key's label, which names the document. + fn resolve_document_trees(&mut self) { + fn real(path: &mut PathBuf) { + if let Some(file) = path + .to_str() + .and_then(|p| p.strip_suffix(PNPM_ENV_TREE_SUFFIX)) + { + *path = PathBuf::from(file); + } + } + for r in self + .refs + .iter_mut() + .chain(&mut self.shadowed) + .chain(&mut self.rewirable) + { + real(&mut r.source_file); + } + self.elsewhere.iter_mut().for_each(|e| real(&mut e.file)); + self.unpatched_copies + .iter_mut() + .for_each(|c| real(&mut c.file)); + self.unattested.iter_mut().for_each(|u| real(&mut u.file)); + for c in &mut self.contested { + real(&mut c.file); + real(&mut c.other); + } + self.diagnostics.iter_mut().for_each(|d| real(&mut d.file)); + self.recognized.iter_mut().for_each(|r| real(&mut r.file)); + self.withheld.iter_mut().for_each(|r| real(&mut r.file)); + } + /// The install tree `file` belongs to: its tree's root file, or itself. fn tree_of<'a>(&'a self, file: &'a Path) -> &'a Path { self.install_trees @@ -1425,6 +1496,7 @@ impl Discovery { } fn finalize(&mut self) { + self.resolve_document_trees(); self.elsewhere.sort(); self.elsewhere.dedup(); self.unpatched_copies.sort(); @@ -1605,6 +1677,13 @@ fn is_script_lock(file: &Path) -> bool { crate::utils::python_lock::is_script_lock_name(&file.to_string_lossy()) } +/// Whether `file` is a [`pnpm_document_tree`] key of a pnpm env lockfile +/// document (before [`Discovery::finalize`] maps it back). +fn is_pnpm_env_tree(file: &Path) -> bool { + file.to_str() + .is_some_and(|f| f.ends_with(PNPM_ENV_TREE_SUFFIX)) +} + /// What every extractor receives: the project view plus the hosted-origin /// allowlist, with the guarded-read and identity helpers bolted on. pub(crate) struct DiscoverCtx<'a> { diff --git a/crates/socket-patch-core/src/vex/discover/npm.rs b/crates/socket-patch-core/src/vex/discover/npm.rs index 3193d7f80..567050efa 100644 --- a/crates/socket-patch-core/src/vex/discover/npm.rs +++ b/crates/socket-patch-core/src/vex/discover/npm.rs @@ -53,14 +53,14 @@ use std::path::PathBuf; use serde_json::Value; use super::{ - npm_purl, npm_vendored_tarball_names, parse_json, vendor_ref, CopyTarget, DiscoverCtx, - Discovery, LocateOpts, Located, PatchedRef, UnattestedKind, UnwiredCopy, VendorRef, - DIAG_LOCKFILE_UNPARSEABLE, DIAG_REF_INVALID, DIAG_REF_UNATTRIBUTABLE, + npm_purl, npm_vendored_tarball_names, parse_json, pnpm_document_tree, vendor_ref, CopyTarget, + DiscoverCtx, Discovery, LocateOpts, Located, PatchedRef, UnattestedKind, UnwiredCopy, + VendorRef, DIAG_LOCKFILE_UNPARSEABLE, DIAG_REF_INVALID, DIAG_REF_UNATTRIBUTABLE, }; use crate::constants::npm_family::{NPM_LOCKS, PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY}; use crate::formats::pnpm::{ - classify_pnpm_key, entry_bundled, entry_field, pnpm_registry_key, Bundled, PnpmKey, PnpmLock, - PnpmPackage, + classify_pnpm_key, entry_bundled, entry_field, pnpm_registry_key, Bundled, PnpmDocument, + PnpmKey, PnpmLock, PnpmPackage, }; use crate::formats::text::strip_bom; use crate::utils::digest::is_sri_pin; @@ -642,6 +642,12 @@ fn entry_ref( /// exists, like the rewriter (no precedence between files — rule 10). /// Vendoring refuses Rush (`npm_flavor`), so these only carry hosted refs. /// +/// Every document of a lock is read. pnpm 11+'s env lockfile document +/// (config dependencies, the pinned package manager; identified by its +/// importers, [`PnpmLock::packages`]) is an install tree of its own +/// ([`pnpm_document_tree`]): its refs are emitted, and they never contest +/// or are contested by the project document or another lock. +/// /// ## Entries /// /// Only `packages:` entries are read — the section pnpm resolves and fetches @@ -748,10 +754,29 @@ async fn extract_pnpm_lock( ); return; } + // Each document is its own install tree: pnpm 11+'s env lockfile is + // read under its own tree key ([`pnpm_document_tree`]), so its refs and + // the project document's entries never contest each other. + for document in lock.documents() { + let tree = pnpm_document_tree(file, document); + extract_pnpm_document(ctx, &tree, &lock, document, disabled, out).await; + } +} + +/// The refs and copies of one document of a pnpm lock, recorded under the +/// document's install-tree key `file` (see [`extract_pnpm_lock`]). +async fn extract_pnpm_document( + ctx: &DiscoverCtx<'_>, + file: &str, + lock: &PnpmLock<'_>, + document: PnpmDocument, + disabled: Option<&str>, + out: &mut Discovery, +) { let mut copies: Vec = Vec::new(); let mut bundles: Vec<(&str, Bundled<'_>)> = Vec::new(); let mut withheld: BTreeSet = BTreeSet::new(); - for package in lock.packages() { + for package in lock.packages_of(document) { let withhold = disabled.map(|_| &mut withheld); pnpm_entry_ref(ctx, file, package, &mut copies, withhold, out); bundles.extend(entry_bundled(&package.entry).map(|b| (package.key, b))); @@ -925,7 +950,8 @@ pub(crate) async fn pnpm_lock_file_copies( let ctx = DiscoverCtx::with_origins(root, &[]); let lock = PnpmLock::parse(text); let mut copies: Vec = Vec::new(); - for package in lock.packages() { + // The vendored backend wires the project document alone. + for package in lock.packages_of(PnpmDocument::Project) { // As `pnpm_entry_ref` classifies them: an entry without a tarball // is a directory copy; one with a tarball is a copy unless it is a // Socket wiring (or one discovery rejects as such). @@ -3091,6 +3117,127 @@ mod tests { assert_eq!(out.unattested.len(), 1, "vendored: {:#?}", out.unattested); } + /// pnpm 11+'s env lockfile document is read as an install tree of its + /// own (owner decision, lock-model divergence N4). Before, discovery + /// skipped it (#1949), so a Socket wiring of a config dependency was + /// no ref although the hosted planner writes it. Now it is a ref; the + /// project document's registry entry, `file:` copy and bundled copy of + /// the same release never contest it (and the reverse); its wiring does + /// not exempt the project document from contesting another lock; and + /// it contests no other lock's ref. + #[tokio::test] + async fn pnpm_env_document_is_its_own_install_tree() { + let url = hosted_url("npm", "left-pad", "1.3.0", UUID_A, "left-pad-1.3.0.tgz"); + let wired = + format!(" left-pad@1.3.0:\n resolution: {{integrity: {SRI}, tarball: {url}}}\n\n"); + let registry = " left-pad@1.3.0:\n resolution: {integrity: sha512-UPSTREAM==}\n\n"; + let env = |packages: &str| { + format!( + "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n \ + left-pad:\n specifier: 1.3.0\n version: 1.3.0\n\npackages:\n\n{packages}" + ) + }; + let project = |packages: &str| { + format!( + "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n dependencies:\n \ + left-pad:\n specifier: 1.3.0\n version: 1.3.0\n\npackages:\n\n{packages}" + ) + }; + let hosted = [("pkg:npm/left-pad@1.3.0", UUID_A, WiringMode::Hosted)]; + let file_copy = " left-pad@file:forks/left-pad:\n \ + resolution: {directory: forks/left-pad, type: directory}\n\n"; + let bundle = " host-pkg@1.0.0:\n resolution: {integrity: sha512-HOST==}\n \ + bundledDependencies: true\n\n"; + + // Wired in the env document, the project resolving the same release + // from the registry, from a `file:` copy, or bundling it: a ref. + for (case, project_packages) in [ + ("registry", registry.to_string()), + ("file: copy", format!("{registry}{file_copy}")), + ("bundled copy", format!("{registry}{bundle}")), + ] { + for crlf in [false, true] { + let p = Project::new(); + let text = format!("{}{}", env(&wired), project(&project_packages)); + let text = if crlf { + text.replace('\n', "\r\n") + } else { + text + }; + p.write("pnpm-lock.yaml", text); + p.write( + "forks/left-pad/package.json", + r#"{"name":"left-pad","version":"1.3.0"}"#, + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert_eq!( + out.refs[0].source_file, + std::path::Path::new("pnpm-lock.yaml") + ); + assert!(out.unattested.is_empty(), "{case}: {:#?}", out.unattested); + assert!(out.shadowed.is_empty(), "{case}: {:#?}", out.shadowed); + assert!(out.contested.is_empty(), "{case}: {:#?}", out.contested); + assert_eq!( + out.hosted_claim("pkg:npm/left-pad@1.3.0", UUID_A), + Some(true), + "{case}" + ); + } + } + + // Wired in the project document, the env document resolving the + // same release from the registry: a ref. + let p = Project::new(); + p.write( + "pnpm-lock.yaml", + format!("{}{}", env(registry), project(&wired)), + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert!(out.contested.is_empty(), "{:#?}", out.contested); + + // The env document's registry entry contests no other lock's ref. + let npm_lock = format!( + r#"{{"name":"m","lockfileVersion":3,"packages":{{"":{{"name":"m"}}, + "node_modules/left-pad":{{"version":"1.3.0","resolved":"{url}","integrity":"{SRI}"}}}}}}"# + ); + let p = Project::new(); + p.write("package-lock.json", &npm_lock); + p.write( + "pnpm-lock.yaml", + format!("{}{}", env(registry), project("")), + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert!(out.contested.is_empty(), "{:#?}", out.contested); + + // The env document's wiring does not exempt the project document, + // whose registry entry still contests package-lock.json's ref; the + // env document's own ref stands. + let p = Project::new(); + p.write("package-lock.json", &npm_lock); + p.write( + "pnpm-lock.yaml", + format!("{}{}", env(&wired), project(registry)), + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert_eq!( + out.refs[0].source_file, + std::path::Path::new("pnpm-lock.yaml") + ); + assert_eq!(out.contested.len(), 1, "{:#?}", out.contested); + let c = &out.contested[0]; + assert_eq!( + (c.file.as_path(), c.other.as_path()), + ( + std::path::Path::new("package-lock.json"), + std::path::Path::new("pnpm-lock.yaml") + ) + ); + } + /// The pnpm bundled mark stays in its own lock: a same-version ref in /// another lock (a `package-lock.json` twin) is neither marked nor /// contested by it — the pnpm lock does not record the bundled version, From 182e3b72343c532a4d67e8b4447c164cb101abe3 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 10:52:37 -0600 Subject: [PATCH 08/36] Ask the origin classifier whether a pnpm dep is vendored The hosted pnpm planner names a dep it cannot find "vendored" instead of "not found" when the lock carries the vendor backend's wiring for it. It decided that with a substring test over every line: any line starting `@file:.socket/vendor/`, or an `@:` line containing both `file:` and `.socket/vendor/`. It now reads the lock's packages entries through PnpmLock and asks crate::origin::vendor_ref (pure path grammar, no I/O, no host policy, so allowed in formats/): a v9 `@file:` key, or a legacy bare `file:` key whose `name:` line names the dep, with `` this project's vendored npm artifact. That is the rule discovery reads vendored pnpm keys with (lock-model divergence N5, pnpm part). Behavior changes, pinned by pnpm_vendored_diagnosis_reads_the_rekeyed_entry_through_origin (the backends' real v9 and legacy shapes stay "vendored", per the existing tests): - A `file:` path without a canonical uuid dir, or under another ecosystem's vendor dir, is redirect_pnpm_entry_not_found now. - An `overrides:` line alone, with no rekeyed packages entry behind it, is redirect_pnpm_entry_not_found now: pnpm mirrors overrides from package.json whether or not anything matches (the vendored in-use check already treats it that way). The depscan-sized equivalence fixture's vendored entry had no uuid dir, a shape the backend never writes; it now carries one, and its golden is re-blessed (both digests change because the input text changed). Co-Authored-By: Claude Opus 5.5 --- .../src/formats/pnpm/hosted.rs | 61 +++++++++--------- .../src/patch/redirect/mod.rs | 62 +++++++++++++++++++ .../patch/redirect/pnpm_equivalence_tests.rs | 2 +- .../equivalence/pnpm_depscan_sized.golden | 2 +- 4 files changed, 92 insertions(+), 35 deletions(-) diff --git a/crates/socket-patch-core/src/formats/pnpm/hosted.rs b/crates/socket-patch-core/src/formats/pnpm/hosted.rs index c491bd22c..a99f0585f 100644 --- a/crates/socket-patch-core/src/formats/pnpm/hosted.rs +++ b/crates/socket-patch-core/src/formats/pnpm/hosted.rs @@ -338,44 +338,18 @@ pub(crate) fn plan_hosted( // is named as such: `socket-patch vendor` removes the registry // resolution this grammar looks for (v9 respells the packages key // `@file:.socket/vendor/…`; v5/v6 rekey it to a bare `file:` - // key but keep the `@: file:…` overrides line), so - // the generic not-locked wording would send users on a wild-goose - // `pnpm install` when the real path is a mode switch. Fail-closed - // either way: nothing is rewritten for the dep. + // key with a `name:` line), so the generic not-locked wording would + // send users on a wild-goose `pnpm install` when the real path is a + // mode switch. Fail-closed either way: nothing is rewritten for the + // dep. if !matched_any { - let v9_vendored_key = format!("{fname}@file:"); - let override_key = format!("{fname}@{}", dep.version); // Scanned over the post-splice text, so fold pending splices in. for lock in locks.iter_mut() { lock.materialize(); } - let vendored = locks.iter().any(|lock| { - lock.text.lines().any(|line| { - let t = line.trim_start(); - let t = t.strip_prefix('\'').unwrap_or(t); - // v9 packages/snapshots key (leading `/` in v6 spelling). - // The vendor backend always writes the RELATIVE - // `file:.socket/vendor/…` spelling here, so anchoring on - // it keeps a user's own `file:` dep of the same name - // from being misreported as vendored. - let key = t.strip_prefix('/').unwrap_or(t); - if key - .strip_prefix(&v9_vendored_key) - .is_some_and(|rest| rest.starts_with(".socket/vendor/")) - { - return true; - } - // overrides / root-dep line: `@: file:…` - // (pnpm <=8 absolutizes the value, so only the - // `.socket/vendor/` tail is stable enough to match). - t.strip_prefix(&override_key) - .map(|rest| rest.strip_prefix('\'').unwrap_or(rest)) - .and_then(|rest| rest.strip_prefix(':')) - .is_some_and(|rest| { - rest.contains("file:") && rest.contains(".socket/vendor/") - }) - }) - }); + let vendored = locks + .iter() + .any(|lock| vendors_package(&super::PnpmLock::parse(&lock.text), &fname)); if vendored { result.warnings.push(RewriteWarning { code: "redirect_pnpm_entry_vendored".into(), @@ -404,6 +378,27 @@ pub(crate) fn plan_hosted( } } +/// Whether `lock` has a packages entry the vendor backends rekeyed for +/// package `fname`: v9's `@file:` key or legacy's bare +/// `file:` key with a `name:` line, whose path is this project's +/// vendored npm artifact ([`crate::origin::vendor_ref`]: root-anchored, +/// canonical uuid, traversal-safe leaf) — the one rule lockfile discovery +/// reads vendored pnpm keys with. A user's own `file:` dep of the same name, +/// a `../.socket/vendor/…` path from another checkout, and an `overrides:` +/// line alone (pnpm mirrors it from package.json whether or not anything +/// matches) are not vendoring. +fn vendors_package(lock: &super::PnpmLock<'_>, fname: &str) -> bool { + use super::{classify_pnpm_key, PnpmKey}; + lock.packages().iter().any(|package| { + let (name, path) = match classify_pnpm_key(package.key) { + PnpmKey::V9File { name, path } => (Some(name.trim_start_matches('/')), path), + PnpmKey::LegacyFile { path } => (pnpm::entry_field(&package.entry, "name"), path), + PnpmKey::Registry { .. } | PnpmKey::Other => return false, + }; + name == Some(fname) && crate::origin::vendor_ref(path).is_some_and(|v| v.eco == "npm") + }) +} + /// Test-only reference for the residual gate: every instance of this exact /// name@version in `content` that does not resolve to `artifact_url`. /// Production judges the same predicate inline, per instance, on each diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 706b6eb21..4fbee0155 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -28267,6 +28267,68 @@ packages: ); } + /// Lock-model divergence N5 (pnpm): "is this dep vendored?" reads the + /// rekeyed packages entry with the origin classifier + /// (`origin::vendor_ref`), the rule discovery reads vendored pnpm keys + /// with, instead of a substring test over every line. A path that is + /// not this project's vendored npm artifact (no canonical uuid dir, + /// another ecosystem's dir) and an `overrides:` line with no rekeyed + /// entry behind it are the generic entry-not-found now; the backends' + /// real v9 and legacy shapes stay vendored (see + /// `pnpm_vendored_entry_is_named_vendored_not_entry_not_found`). + #[test] + fn pnpm_vendored_diagnosis_reads_the_rekeyed_entry_through_origin() { + let ovr = npm_override( + "left-pad", + "1.3.0", + "http://patch.test/left-pad-1.3.0.tgz", + "sha512-PATCHED==", + ); + let uuid = "1a2b3c4d-5e6f-4a1b-8c2d-0123456789ab"; + let v9 = |path: &str| { + format!( + "lockfileVersion: '9.0'\n\npackages:\n\n left-pad@file:{path}:\n \ + resolution: {{integrity: sha512-VENDORED==, tarball: file:{path}}}\n \ + version: 1.3.0\n" + ) + }; + let legacy_overrides_only = format!( + "lockfileVersion: '6.0'\n\noverrides:\n left-pad@1.3.0: \ + file:/abs/project/.socket/vendor/npm/{uuid}/left-pad-1.3.0.tgz\n\npackages:\n\n \ + /other@1.0.0:\n resolution: {{integrity: sha512-OTHER==}}\n dev: false\n" + ); + let codes = |lock: &str| { + let files = BTreeMap::from([("pnpm-lock.yaml".to_string(), lock.to_string())]); + let r = rewrite_registry_redirect(&files, std::slice::from_ref(&ovr)); + assert!(r.files.is_empty() && r.edits.is_empty(), "{lock}"); + r.warnings.into_iter().map(|w| w.code).collect::>() + }; + for (case, lock, want) in [ + ( + "the backend's v9 shape", + v9(&format!(".socket/vendor/npm/{uuid}/left-pad-1.3.0.tgz")), + "redirect_pnpm_entry_vendored", + ), + ( + "no uuid dir", + v9(".socket/vendor/npm/left-pad-1.3.0.tgz"), + "redirect_pnpm_entry_not_found", + ), + ( + "another ecosystem's dir", + v9(&format!(".socket/vendor/cargo/{uuid}/left-pad-1.3.0.tgz")), + "redirect_pnpm_entry_not_found", + ), + ( + "an overrides line alone", + legacy_overrides_only, + "redirect_pnpm_entry_not_found", + ), + ] { + assert_eq!(codes(&lock), vec![want], "{case}"); + } + } + /// A classic-lock block whose first line is not a `key:` line /// (hand-mangled) is skipped without derailing the rewrite of the real /// entry — and survives byte-identically. diff --git a/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs b/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs index 2d026a253..7ce3cc6ea 100644 --- a/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs +++ b/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs @@ -232,7 +232,7 @@ fn indexed_pnpm_rewrite_matches_golden_on_depscan_sized_lock_set() { }; let root_extra = vec![ format!( - " vendored-pkg@file:.socket/vendor/npm/vendored-pkg-1.0.0.tgz:\n resolution: {{tarball: file:.socket/vendor/npm/vendored-pkg-1.0.0.tgz}}\n" + " vendored-pkg@file:.socket/vendor/npm/0a1b2c3d-4e5f-4a6b-8c7d-8e9f0a1b2c3d/vendored-pkg-1.0.0.tgz:\n resolution: {{tarball: file:.socket/vendor/npm/0a1b2c3d-4e5f-4a6b-8c7d-8e9f0a1b2c3d/vendored-pkg-1.0.0.tgz}}\n" ), // A second copy of an entry that already points at the hosted // artifact (a re-run): rebuilt == original, no edit. diff --git a/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden b/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden index 8b8c62706..29f1efde0 100644 --- a/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden +++ b/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden @@ -1,3 +1,3 @@ # A depscan-sized seeded pnpm-lock.yaml set + shuffled overrides. # -0 782731128c8cd746 f3e10497dfc8fec6 +0 342d79c926a00044 2cf6f312234e59ca From 38d5a3cf9ff5392704bbd1e36a1a2346022bb74b Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 10:55:42 -0600 Subject: [PATCH 09/36] Read Rush locks always and follow lockfile=false in the inventory Two npm-family inventory rules now follow lockfile discovery (and so what hosted mode pins): - Rush's common and subspace pnpm locks are separate install trees, read whenever rush.json is present, beside whatever lock the flavor router chose. Before, the inventory read them only as a fallback when the root had no lock (lock-model divergence N12). The listing itself is unchanged (#2003 aligns it separately). root_pnpm_lock_wins_over_ rush_fallback pinned the old shadowing; it becomes rush_locks_are_read_beside_the_root_lock. - pnpm's `lockfile` setting off (lockfile_disabled, not under Rush, as in discovery): pnpm neither reads nor writes the root or member locks. Discovery keeps a hosted ref from such a lock but withholds its lock pin; the inventory now keeps the entry (its coordinates and hosted url, which ledger liveness reads) and drops its integrity, so vendor's fetch and repair never verify against a pin the install does not use. Registry entries keep their integrity: a registry release's hash verifies its content whether or not pnpm reads the lock (N14). Pinned by pnpm_lockfile_off_withholds_hosted_pins. "Hosted" here is the default Socket host (origin::hosted_patch_uuid with no configured origins): the inventory has no per-run origin list yet; the install-tree PR moves it onto the single classifier. Co-Authored-By: Claude Opus 5.5 --- .../src/vendor/lock_inventory/npm_family.rs | 73 +++++++++++++-- .../src/vendor/lock_inventory/tests.rs | 92 ++++++++++++++++++- 2 files changed, 151 insertions(+), 14 deletions(-) diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs b/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs index 208934cf1..1dabe294f 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs @@ -8,6 +8,8 @@ use std::path::Path; use crate::constants::npm_family::{BUN_LOCK, PNPM_SHRINKWRAP_LEGACY}; use crate::formats::governing_locks::{npm_governing_family, npm_lock_files, NpmLockFamily}; use crate::formats::yarn::{grammar as yarn_grammar, YarnLockGrammar}; +use crate::origin::hosted_patch_uuid; +use crate::utils::pnpm_workspace::lockfile_disabled; use crate::utils::purl::npm_purl; use crate::vendor::npm_flavor::NpmLockFlavor; @@ -20,7 +22,7 @@ use super::pnpm::{ use super::view::ProjectView; use super::vlt::inventory_vlt_in; use super::yarn::{inventory_yarn_berry_in, inventory_yarn_classic_in}; -use super::{dedup_prefer_integrity, LockfileEntry, UnsupportedNpmLayout}; +use super::{dedup_prefer_integrity, LockIntegrity, LockfileEntry, UnsupportedNpmLayout}; use crate::vendor::npm_flavor::detect_npm_lock_flavor_in; // ── registry view ── @@ -41,8 +43,8 @@ use crate::vendor::npm_flavor::detect_npm_lock_flavor_in; /// failing that, the workspace members' own locks. A pnpm workspace with /// `sharedWorkspaceLockfile: false` reads those beside its root lock /// ([`inventory_pnpm_member_locks_in`], #492). -/// Any remaining probe failure falls back to Rush's common lock when -/// `rush.json` is present. +/// Rush's common and subspace locks are read beside whatever this picks +/// whenever `rush.json` is present ([`inventory_npm_lock_raw_in`]). #[cfg(test)] pub(crate) async fn inventory_npm_lock( project_root: &Path, @@ -61,8 +63,64 @@ pub(super) async fn inventory_npm_lock_in( /// [`inventory_npm_lock`] before its collapse: every /// guarded instance ([`super::inventory_project_every_lock`]). +/// +/// Two rules follow lockfile discovery (`vex::discover::npm`'s +/// `extract_pnpm`), which reads what hosted mode pins: +/// +/// * Rush's common and subspace locks are separate install trees, read +/// whenever `rush.json` is present, beside whatever the router chose +/// (before: only when the root had no lock; lock-model divergence N12). +/// * pnpm's `lockfile` setting off ([`lockfile_disabled`], not under Rush, +/// which runs pnpm with settings of its own): pnpm neither reads nor +/// writes the root or member locks, so a Socket-hosted pin in them is no +/// lockfile basis. Discovery keeps such a ref but withholds its lock pin; +/// here the entry stays listed (what it names is still the dependency +/// set, and its hosted url still shows liveness the wiring) with its +/// integrity dropped, so nothing verifies or fetches against the pin +/// (N14). Registry entries keep theirs: a registry release's integrity +/// is its content verifier whether or not pnpm reads the lock. pub(super) async fn inventory_npm_lock_raw_in( view: &ProjectView<'_>, +) -> Result)>, UnsupportedNpmLayout> { + let mut found = inventory_npm_router_lock_raw_in(view).await?; + let rush = view.exists("rush.json").await; + if let Some((NpmLockFlavor::Pnpm | NpmLockFlavor::PnpmLegacy, entries)) = &mut found { + if !rush && lockfile_disabled(view, false).await.is_some() { + withhold_hosted_pins(entries); + } + } + let rush = guard_npm(inventory_rush_pnpm_locks_in(view).await); + if rush.is_empty() { + return Ok(found); + } + Ok(Some(match found { + Some((flavor, mut entries)) => { + entries.extend(rush); + (flavor, entries) + } + None => (NpmLockFlavor::Pnpm, rush), + })) +} + +/// Drop the integrity of every entry resolving from Socket's patch server +/// (see [`inventory_npm_lock_raw_in`]). +fn withhold_hosted_pins(entries: &mut [LockfileEntry]) { + for entry in entries { + if entry + .resolved + .as_deref() + .and_then(|url| hosted_patch_uuid(url, &[])) + .is_some() + { + entry.integrity = LockIntegrity::None; + } + } +} + +/// The lock the npm flavor router picks, with its fallbacks (see +/// [`inventory_npm_lock`]); Rush's locks are added by the caller. +async fn inventory_npm_router_lock_raw_in( + view: &ProjectView<'_>, ) -> Result)>, UnsupportedNpmLayout> { let (flavor, _warnings) = match detect_npm_lock_flavor_in(view).await { Ok(found) => found, @@ -144,13 +202,8 @@ pub(super) async fn inventory_npm_lock_raw_in( return Ok(Some((NpmLockFlavor::Pnpm, guard_npm(members)))); } } - // Rush monorepos have no root package.json/lock pair; their - // single pnpm source-of-truth lives under common/config/rush/. - // The flavor probe (root-relative) can't see it, so fall back - // explicitly when the root lock is absent but rush.json is - // present. - let rush = inventory_rush_pnpm_locks_in(view).await; - return Ok((!rush.is_empty()).then(|| (NpmLockFlavor::Pnpm, guard_npm(rush)))); + // Rush's locks are read by the caller, whatever the probe says. + return Ok(None); } }; let raw = match flavor { diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs index 21651261d..6ef23cd40 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs @@ -756,10 +756,12 @@ async fn rush_json_without_any_lock_yields_none() { assert!(inventory_npm_lock(tmp.path()).await.unwrap().is_none()); } +/// Lock-model divergence N12: Rush's locks are their own install trees, +/// read whenever `rush.json` is present — beside a root lock too, as +/// hosted mode and lockfile discovery read them. Before, a root lock +/// shadowed them (this test pinned "the root lock wins"). #[tokio::test] -async fn root_pnpm_lock_wins_over_rush_fallback() { - // A plain pnpm project that also happens to carry a stray rush.json - // must route through the normal root-lock path, never the fallback. +async fn rush_locks_are_read_beside_the_root_lock() { let tmp = tempfile::tempdir().unwrap(); write(tmp.path(), "rush.json", r#"{"rushVersion":"5.0.0"}"#).await; write(tmp.path(), "pnpm-lock.yaml", PNPM_LOCK).await; @@ -779,9 +781,91 @@ packages: let (flavor, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); assert_eq!(flavor, NpmLockFlavor::Pnpm); assert!(entries.iter().any(|e| e.name == "left-pad")); + assert!( + entries.iter().any(|e| e.name == "only-in-common"), + "the Rush common lock is read beside the root lock: {entries:?}" + ); + + // Beside another family's lock too. + tokio::fs::remove_file(tmp.path().join("pnpm-lock.yaml")) + .await + .unwrap(); + write(tmp.path(), "package-lock.json", PACKAGE_LOCK).await; + let (flavor, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + assert_eq!(flavor, NpmLockFlavor::PackageLock); + assert!( + entries.iter().any(|e| e.name == "only-in-common"), + "{entries:?}" + ); + + // Without rush.json they are not Rush's. + tokio::fs::remove_file(tmp.path().join("rush.json")) + .await + .unwrap(); + let (_, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); assert!( !entries.iter().any(|e| e.name == "only-in-common"), - "the root lock must win; the rush fallback must not run: {entries:?}" + "{entries:?}" + ); +} + +/// Lock-model divergence N14: with pnpm's `lockfile` setting off, pnpm +/// never reads the lock, so a Socket-hosted pin in it is no lockfile +/// basis. Lockfile discovery keeps the ref and withholds its pin; the +/// inventory keeps the entry (and its hosted url) and drops its +/// integrity, so nothing fetches or verifies against it. Registry entries +/// keep theirs. Before, the inventory ignored the setting. Rush runs pnpm +/// with settings of its own, so its locks are untouched (as in discovery). +#[tokio::test] +async fn pnpm_lockfile_off_withholds_hosted_pins() { + let uuid = "1a2b3c4d-5e6f-4a1b-8c2d-0123456789ab"; + let url = + format!("https://patch.socket.dev/patch/npm/left-pad/1.3.0/tok/{uuid}/left-pad-1.3.0.tgz"); + let lock = format!( + "lockfileVersion: '9.0'\n\npackages:\n\n left-pad@1.3.0:\n \ + resolution: {{integrity: sha512-PATCHED==, tarball: {url}}}\n\n \ + right-pad@2.0.0:\n resolution: {{integrity: sha512-UPSTREAM==}}\n" + ); + let integrity = |entries: &[LockfileEntry], name: &str| entry(entries, name).integrity.clone(); + for (setting, withheld) in [ + (None, false), + (Some((".npmrc", "lockfile=false\n")), true), + (Some(("pnpm-workspace.yaml", "lockfile: false\n")), true), + (Some((".npmrc", "lockfile=true\n")), false), + ] { + let tmp = tempfile::tempdir().unwrap(); + write(tmp.path(), "pnpm-lock.yaml", &lock).await; + if let Some((file, text)) = setting { + write(tmp.path(), file, text).await; + } + let (_, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + let left = entry(&entries, "left-pad"); + assert_eq!(left.resolved.as_deref(), Some(url.as_str()), "{setting:?}"); + assert_eq!( + integrity(&entries, "left-pad"), + if withheld { + LockIntegrity::None + } else { + LockIntegrity::Sri("sha512-PATCHED==".into()) + }, + "{setting:?}" + ); + assert_eq!( + integrity(&entries, "right-pad"), + LockIntegrity::Sri("sha512-UPSTREAM==".into()), + "{setting:?}" + ); + } + + // Rush: the setting is not Rush's, so its lock keeps the pin. + let tmp = tempfile::tempdir().unwrap(); + write(tmp.path(), "rush.json", r#"{"rushVersion":"5.0.0"}"#).await; + write(tmp.path(), ".npmrc", "lockfile=false\n").await; + write_nested(tmp.path(), "common/config/rush/pnpm-lock.yaml", &lock).await; + let (_, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + assert_eq!( + integrity(&entries, "left-pad"), + LockIntegrity::Sri("sha512-PATCHED==".into()) ); } From f63ee08c9a0f6862c4f1e39fefffd338c26459bb Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 10:58:11 -0600 Subject: [PATCH 10/36] Guard pnpm entry reads with an architecture test Outside formats/pnpm, pnpm lock entries are read through PnpmLock (one walk, every document tagged). A new test in formats/pnpm/mod.rs fails on any production use of grammar::entries / pnpm_packages or a hand-rolled `packages:` line walk elsewhere, except the writers that splice by byte range or line position, each allowlisted with a label and a use count: - patch/redirect/upstream/npm.rs (2): the hosted unwind's resolution splices (restore_pnpm_locks); - vendor/pnpm_lock.rs (3): the v9 vendored planner's preflight and line splices; - vendor/pnpm_lock_legacy.rs (6): the 5.4 / 6.0 vendored planner's preflight, splices and revert. Co-Authored-By: Claude Opus 5.5 --- .../src/formats/pnpm/grammar.rs | 8 +- .../socket-patch-core/src/formats/pnpm/mod.rs | 121 ++++++++++++++++++ .../socket-patch-core/src/vex/discover/mod.rs | 4 +- 3 files changed, 128 insertions(+), 5 deletions(-) diff --git a/crates/socket-patch-core/src/formats/pnpm/grammar.rs b/crates/socket-patch-core/src/formats/pnpm/grammar.rs index 10a7a57fc..0e6ae2baf 100644 --- a/crates/socket-patch-core/src/formats/pnpm/grammar.rs +++ b/crates/socket-patch-core/src/formats/pnpm/grammar.rs @@ -1,9 +1,11 @@ //! The `packages:` entry grammar: read pnpm's package blocks without //! reserializing unrelated YAML. The old shrinkwrap and lockfile 5.1/5.2 //! formats use block resolutions; newer locks use flow mappings. Peer -//! suffixes are identities, not part of the version. The hosted planner -//! ([`super::hosted`]), the lock inventory and lockfile discovery all read -//! entries through this one walk. +//! suffixes are identities, not part of the version. Every reader reads +//! entries through [`super::PnpmLock`], which wraps this walk; the writers +//! that splice byte ranges (the hosted planner, [`super::hosted`], and the +//! hosted unwind) call it directly (see `architecture_tests` in +//! `formats/pnpm/mod.rs`). use std::ops::Range; diff --git a/crates/socket-patch-core/src/formats/pnpm/mod.rs b/crates/socket-patch-core/src/formats/pnpm/mod.rs index feb05405b..95f21ef42 100644 --- a/crates/socket-patch-core/src/formats/pnpm/mod.rs +++ b/crates/socket-patch-core/src/formats/pnpm/mod.rs @@ -1065,3 +1065,124 @@ mod tests { assert!(!may_need_store_flag("packages: {}\n")); } } + +/// ARCHITECTURE GUARD: outside `formats/pnpm`, pnpm lock entries are READ +/// through [`PnpmLock`] (one walk, every document tagged), never through +/// the grammar's raw entry walk or a hand-rolled `packages:` line walk. +/// Writers that splice by byte range or line position keep their own +/// walk; each such use is allowlisted below with a label and a count, so +/// a new reader in the same file still fails. +#[cfg(test)] +mod architecture_tests { + use std::path::{Path, PathBuf}; + + /// `(root-relative file, uses, why)`. + const WRITERS: &[(&str, usize, &str)] = &[ + ( + "patch/redirect/upstream/npm.rs", + 2, + "hosted unwind (`restore_pnpm_locks`): splices each pin's \ + resolution byte range back to the registry spelling", + ), + ( + "vendor/pnpm_lock.rs", + 3, + "v9 vendored planner: preflight and splices over line blocks \ + (`lock_has_target_package`, `edit_packages`, `LockIndex`)", + ), + ( + "vendor/pnpm_lock_legacy.rs", + 6, + "5.4 / 6.0 vendored planner: preflight, splices and revert over \ + line blocks", + ), + ]; + + /// Whether one production line reads pnpm entries outside the model. + fn raw_walk(line: &str) -> bool { + let code = line.trim_start(); + if code.starts_with("//") { + return false; + } + [ + "grammar::entries(", + "pnpm::entries(", + "pnpm_grammar::entries(", + "pnpm_packages(", + "== \"packages:\"", + ] + .iter() + .any(|needle| code.contains(needle)) + || (code.contains("\"packages\")") + && (code.contains("bounds(") || code.contains("SectionIndex::build("))) + } + + /// `text` up to its test module. + fn production(text: &str) -> &str { + ["#[cfg(test)]\nmod ", "#[cfg(test)]\npub(crate) mod "] + .iter() + .filter_map(|marker| text.find(marker)) + .min() + .map_or(text, |at| &text[..at]) + } + + fn rs_files(dir: &Path, out: &mut Vec) { + for entry in std::fs::read_dir(dir).expect("read src dir") { + let path = entry.expect("dir entry").path(); + if path.is_dir() { + rs_files(&path, out); + } else if path.extension().is_some_and(|e| e == "rs") { + out.push(path); + } + } + } + + #[test] + fn pnpm_entries_are_read_through_the_model() { + let src = Path::new(env!("CARGO_MANIFEST_DIR")).join("src"); + let mut files = Vec::new(); + rs_files(&src, &mut files); + let mut found: Vec<(String, usize)> = Vec::new(); + for path in files { + let rel = path + .strip_prefix(&src) + .unwrap() + .to_string_lossy() + .replace('\\', "/"); + if rel.starts_with("formats/pnpm/") || rel.ends_with("tests.rs") { + continue; + } + let text = std::fs::read_to_string(&path) + .unwrap() + .replace("\r\n", "\n"); + let uses = production(&text).lines().filter(|l| raw_walk(l)).count(); + if uses > 0 { + found.push((rel, uses)); + } + } + found.sort(); + let mut allowed: Vec<(String, usize)> = WRITERS + .iter() + .map(|(rel, uses, _)| (rel.to_string(), *uses)) + .collect(); + allowed.sort(); + assert_eq!( + found, allowed, + "pnpm lock entries read outside formats/pnpm: read them through \ + PnpmLock::packages (a reader), or allowlist a writer that needs \ + byte or line positions in WRITERS with its reason" + ); + } + + #[test] + fn raw_walk_needles() { + assert!(raw_walk(" for entry in pnpm::entries(&text) {")); + assert!(raw_walk(" in_packages = line == \"packages:\";")); + assert!(raw_walk( + " let (s, e) = section_bounds(lines, \"packages\").unwrap();" + )); + assert!(!raw_walk(" // pnpm::entries(&text)")); + assert!(!raw_walk(" lock.get(\"packages\")")); + assert!(!raw_walk(" for p in PnpmLock::parse(t).packages() {")); + } +} diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 427eceecf..48e9c2a6f 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -30,7 +30,7 @@ //! lockfile supplement, the vendored fetch inventory) read each format //! through ONE shared reader that yields EVERY entry, Socket-owned ones //! included: the npm-family entry models in `lock_inventory` -//! (`npm_lock_nodes`, `pnpm::pnpm_packages`, `yarn::classic_entries` / +//! (`npm_lock_nodes`, `PnpmLock::packages`, `yarn::classic_entries` / //! `berry_entries`, `BunLockb::parse_packages`, `vlt::vlt_lock_model`) and, //! for the other formats, the readers the writers own (`cargo_lock` / //! `cargo_config`, `go_mod_edit` / `go_sum_edit`, `formats::gem`, @@ -4621,7 +4621,7 @@ mod tests { /// ARCHITECTURE GUARD (rule 12): the npm-family extractors read their /// locks through the entry models the lock inventory shares - /// (`classic_entries` / `berry_entries`, `pnpm_packages`, + /// (`classic_entries` / `berry_entries`, `PnpmLock::packages`, /// `bun_text_entries`, `BunLockb::parse_packages`), never through the /// grammar primitives those models wrap, nor a second parser of the /// same lock (a JSON read of `bun.lock`) — so discovery and the From 09fa5a9e50da46d98d37e0720d729de8efd12d02 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 11:02:21 -0600 Subject: [PATCH 11/36] Mark N4, N5 (pnpm), N12 and N14 resolved; rebudget docs/design/lock-model-divergences.md records what PR 2a resolved. Budget: lock-readers 30,976 -> 31,117 (+141), raised by hand: - vex/discover/mod.rs +46: the env document's tree key (pnpm_document_tree), mapping it back to the lock path in finalize, and skipping it in the cross-lock contest; - formats/pnpm/grammar.rs +51: the content-based document split (documents, is_env_document) replacing main_document; - vendor/lock_inventory/npm_family.rs +35: Rush locks beside the routed lock and the lockfile=false pin withholding; - vex/discover/npm.rs +13: one extraction pass per document; - formats/pnpm/workspace.rs +6, lines.rs +1; - formats/pnpm/mod.rs -4 (InstalledPackages on PnpmLock instead of its own line walk), formats/pnpm/hosted.rs -7 (the vendored check). lock-writers 88,314 -> 88,313 (--lower). Co-Authored-By: Claude Opus 5.5 --- .../socket-patch-core/src/patch/redirect/upstream/npm.rs | 4 +--- docs/design/lock-model-divergences.md | 8 ++++---- scripts/loc-budget.json | 4 ++-- 3 files changed, 7 insertions(+), 9 deletions(-) diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs index ef0031a0e..9250b6b6b 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs @@ -1283,9 +1283,7 @@ pub(crate) fn pnpm_settings_major(text: &str, pm_major: Option) -> Option (env lockfile)`, which neither contests nor is contested by the project document or another lock; a document is the env one when its importers carry only `configDependencies` / `packageManagerDependencies`. | | +| N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere. pnpm part **resolved in PR 2a**: the hosted pnpm "vendored?" check reads the rekeyed packages entry with `origin::vendor_ref`. | revert could rewrite another project's wiring | | N6 | `npm-shrinkwrap.json` and `package-lock.json` | Hosted, vendored and discovery read both, with a contest. The inventory reads the shrinkwrap only. | Both, everywhere | | | N7 | npm lockfileVersion | Hosted rewrites v1 and v4. Vendored accepts v2/v3 only. The inventory ignores v1. Discovery reads v1 and v2+. | Hosted, once vendored `file:` wiring under npm 6 and npm 12.1 v4 has fixtures | | | N8 | npm `file:` directory link identity (#1481) | Hosted and discovery give a versionless link its source directory's `name@version`. Vendored matches the entry's own version, wiring the registry copies with no warning. | Hosted | silent vendored success | | N9 | yarn classic blocks reached only through an alias | Hosted skips them. Vendored rewrites them. Discovery counts an alias block as an install of the real package. | Back-port: hosted rewrites alias blocks (vendored supports what hosted lacks) | | | N10 | Locks of several npm-family package managers in one project | Hosted rewrites every family; discovery reads every family with the contest. The inventory and vendored pick one family (vlt > bun > pnpm > yarn > npm). | Union (separate trees) | | | N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | Pending owner decision (see above) | needs owner | -| N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. | target-specific (vendored) | +| N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. **Resolved in PR 2a** (the inventory reads them whenever `rush.json` exists). | target-specific (vendored) | | N13 | Plug'n'Play | Vendored refuses. The hosted berry preflight has no PnP gate. The inventory returns an error, so `scan` sees nothing in a project hosted pinned. | The inventory reads PnP locks. Vendored keeps its refusal. | target-specific (vendored) | -| N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery | target-specific | +| N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery. **Resolved in PR 2a**: under the setting (not under Rush), the inventory keeps a Socket-hosted pnpm entry but drops its integrity, as discovery withholds the lock pin. | target-specific | | N15 | CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml` | Vendored refuses; the rest handle it | Handle it everywhere | | | N16 | Lock versions | pnpm: vendored accepts only 5.4/6.0/9.0; hosted any but early shrinkwrap. vlt: vendored refuses a lock with no version; hosted rewrites and warns. | Hosted | | | N17 | Bun workspaces, yarn classic offline mirror | Refused differently by the two modes | Stays | target-specific | diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json index efdb902f1..c73793400 100644 --- a/scripts/loc-budget.json +++ b/scripts/loc-budget.json @@ -17,7 +17,7 @@ "crates/socket-patch-core/src/utils/pipenv.rs", "crates/socket-patch-core/src/crawlers/deno_crawler.rs" ], - "max": 30976 + "max": 31117 }, "lock-writers": { "paths": [ @@ -30,7 +30,7 @@ "crates/socket-patch-core/src/vendor/npm_origin.rs", "crates/socket-patch-core/src/vendor/test_support" ], - "max": 88314 + "max": 88313 } } } From 613ca3aae6b4b593789831698b185474960a99e7 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Wed, 4 Nov 2026 11:47:02 -0600 Subject: [PATCH 12/36] Address code review of the pnpm reader PR 1. The pnpm architecture guard cut each file at its first `#[cfg(test)] mod`, so a file declaring an out-of-line test module near the top (crawlers/npm_crawler.rs, patch/redirect/mod.rs, vendor/mod.rs) went mostly unchecked. It now drops only the gated items: a `mod x;` line, or an inline item through its closing brace, counted outside comments and string, raw string and char literals (which span lines). production_drops_only_test_gated_items pins it, including late production code of those three files. 2. The env document must be the FIRST document as well as env-shaped, as pnpm and split_project_document read it. An env-shaped later document is a project one, so it cannot skip the cross-lock contest, and pnpm_settings_major no longer reports 11 for it. documents_are_identified_by_position_and_content (was ..._by_content_not_position) and pnpm_include_tarball_ignores_the_env_ lockfile_document pin it. 3. The hosted "vendored?" check compares the version too: another version vendored is redirect_pnpm_entry_not_found. The depscan-sized equivalence fixture's vendored entry gains the `version:` line the backend always writes; golden re-blessed. 4. LockfileEntry gains `pnpm_env`, and dedup_prefer_integrity never lets an env-document instance displace a project one (project first, then a verifier). env_document_instances_never_displace_project_instances. 5. Entries are tagged by their KEY line's offset, so an empty-bodied last key of a document keeps its document. an_entry_with_an_empty_body_keeps_its_document. 6. No pseudo path: discovery records carry an explicit TreePart (File or PnpmEnv) beside the real file (PatchedRef, ResolvedElsewhere, UnpatchedCopy, UnwiredCopy scope); the contests compare it, and the finalize remap is gone. An env ref the same file's project document also wires (same purl, uuid, mode) is dropped in finalize, so the two never both survive, whatever their pins. 7. InstalledPackages walks keys only (PnpmLock::parse_keys, no resolution parse) on the store crawler's hot path. 8. The hosted planner builds each lock's vendored (name, version) set once, from the unspliced text (a splice never touches a key or a name:/version: line), dropped when materialize replaces the text. 9. Rush locks are no flavor's: the routed lock keeps its flavor alone (inventory_npm_lock), and inventory_npm_trees_raw_in unions it with Rush's trees; rush.json is stat'd once through the view (the Rush reader no longer re-checks it), no bool/Vec shadowing. Budget: lock-readers 31,117 -> 31,207 (+90: TreePart and its helpers +30 in vex/discover/mod.rs, PnpmTree +22 in vex/discover/npm.rs, the key-only walk +16 in formats/pnpm/mod.rs, the vendored cache +13 in formats/pnpm/hosted.rs, `pnpm_env: false` in 21 LockfileEntry literals, golden render +7, minus 15 in the inventory's npm_family/pnpm/mod). lock-writers 88,313 -> 88,314 (+1: hosted/engine.rs compares the part). Co-Authored-By: Claude Opus 5.5 --- .../src/commands/scan/discovery.rs | 1 + .../src/formats/cargo/mod.rs | 1 + .../src/formats/composer/mod.rs | 1 + .../socket-patch-core/src/formats/gem/mod.rs | 1 + .../src/formats/pnpm/grammar.rs | 23 +- .../src/formats/pnpm/hosted.rs | 66 ++-- .../socket-patch-core/src/formats/pnpm/mod.rs | 284 ++++++++++++++++-- crates/socket-patch-core/src/hosted/engine.rs | 1 + .../src/patch/redirect/mod.rs | 8 + .../patch/redirect/pnpm_equivalence_tests.rs | 2 +- .../src/patch/redirect/upstream/npm.rs | 7 + .../src/vendor/lock_inventory/golang.rs | 1 + .../src/vendor/lock_inventory/mod.rs | 36 ++- .../src/vendor/lock_inventory/npm_family.rs | 97 +++--- .../src/vendor/lock_inventory/pnpm.rs | 17 +- .../src/vendor/lock_inventory/pypi.rs | 6 + .../src/vendor/lock_inventory/recover.rs | 6 + .../src/vendor/lock_inventory/tests.rs | 74 ++++- .../socket-patch-core/src/vex/discover/mod.rs | 204 +++++++------ .../socket-patch-core/src/vex/discover/npm.rs | 138 ++++++--- .../src/vex/discover/testing/golden.rs | 16 +- .../equivalence/pnpm_depscan_sized.golden | 2 +- docs/design/lock-model-divergences.md | 2 +- scripts/loc-budget.json | 4 +- 24 files changed, 717 insertions(+), 281 deletions(-) diff --git a/crates/socket-patch-cli/src/commands/scan/discovery.rs b/crates/socket-patch-cli/src/commands/scan/discovery.rs index 0562e26ee..fd27bc50d 100644 --- a/crates/socket-patch-cli/src/commands/scan/discovery.rs +++ b/crates/socket-patch-cli/src/commands/scan/discovery.rs @@ -722,6 +722,7 @@ mod tests { resolved: None, integrity: LockIntegrity::None, source_kind: SourceKind::Unspecified, + pnpm_env: false, } } diff --git a/crates/socket-patch-core/src/formats/cargo/mod.rs b/crates/socket-patch-core/src/formats/cargo/mod.rs index 081b7f230..459533241 100644 --- a/crates/socket-patch-core/src/formats/cargo/mod.rs +++ b/crates/socket-patch-core/src/formats/cargo/mod.rs @@ -493,6 +493,7 @@ impl CargoLock { out.push(LockfileEntry { ecosystem: "cargo", source_kind, + pnpm_env: false, purl, name: pkg.name.clone(), version, diff --git a/crates/socket-patch-core/src/formats/composer/mod.rs b/crates/socket-patch-core/src/formats/composer/mod.rs index 6e3f8f3e9..25ebb6e85 100644 --- a/crates/socket-patch-core/src/formats/composer/mod.rs +++ b/crates/socket-patch-core/src/formats/composer/mod.rs @@ -183,6 +183,7 @@ impl<'a> ComposerLock<'a> { out.push(LockfileEntry { ecosystem: "composer", source_kind: SourceKind::Unspecified, + pnpm_env: false, name, version, purl, diff --git a/crates/socket-patch-core/src/formats/gem/mod.rs b/crates/socket-patch-core/src/formats/gem/mod.rs index e041581ba..09e40090c 100644 --- a/crates/socket-patch-core/src/formats/gem/mod.rs +++ b/crates/socket-patch-core/src/formats/gem/mod.rs @@ -290,6 +290,7 @@ impl<'t> GemfileLock<'t> { out.push(LockfileEntry { ecosystem: "gem", source_kind: SourceKind::Unspecified, + pnpm_env: false, purl, resolved, name: name.to_string(), diff --git a/crates/socket-patch-core/src/formats/pnpm/grammar.rs b/crates/socket-patch-core/src/formats/pnpm/grammar.rs index 0e6ae2baf..3ba09bf18 100644 --- a/crates/socket-patch-core/src/formats/pnpm/grammar.rs +++ b/crates/socket-patch-core/src/formats/pnpm/grammar.rs @@ -109,14 +109,16 @@ pub enum PnpmDocument { /// is. A text without a column-0 `---` line is one project document. A /// blank document (the empty span before a leading `---`) is dropped. /// -/// pnpm >= 11 writes the env lockfile as a first document ahead of the -/// project lock. A document is identified by its CONTENT, not by its -/// position: in a lock of two or more documents, one whose `importers:` -/// carry only [`ENV_DOC_IMPORTER_FIELDS`] (at least one) is -/// [`PnpmDocument::Env`]; every other document is -/// [`PnpmDocument::Project`]. So a crafted lock cannot hide entries from a -/// reader by adding documents: each one is read, as the tree its content -/// says it is. CRLF reads like LF. +/// pnpm >= 11 writes the env lockfile as the FIRST document, ahead of the +/// project lock, and reads it from there (as the vendored planners' +/// `lines::split_project_document` does). So the env document is told by +/// position AND content: in a lock of two or more documents, the first one +/// is [`PnpmDocument::Env`] when its `importers:` carry only +/// [`ENV_DOC_IMPORTER_FIELDS`] (at least one). Every other document is +/// [`PnpmDocument::Project`], an env-shaped later one included: no +/// document can opt out of the project tree's contests, and a crafted lock +/// cannot hide entries by adding documents, since each one is read. CRLF +/// reads like LF. pub(crate) fn documents(content: &str) -> Vec<(Range, PnpmDocument)> { let mut spans = Vec::new(); let mut start = 0; @@ -146,8 +148,9 @@ pub(crate) fn documents(content: &str) -> Vec<(Range, PnpmDocument)> { let several = spans.len() > 1; spans .into_iter() - .map(|span| { - let kind = if several && is_env_document(&content[span.clone()]) { + .enumerate() + .map(|(i, span)| { + let kind = if several && i == 0 && is_env_document(&content[span.clone()]) { PnpmDocument::Env } else { PnpmDocument::Project diff --git a/crates/socket-patch-core/src/formats/pnpm/hosted.rs b/crates/socket-patch-core/src/formats/pnpm/hosted.rs index a99f0585f..84f2524e2 100644 --- a/crates/socket-patch-core/src/formats/pnpm/hosted.rs +++ b/crates/socket-patch-core/src/formats/pnpm/hosted.rs @@ -4,7 +4,7 @@ //! rewriter's pnpm leg). use std::borrow::Cow; -use std::collections::BTreeMap; +use std::collections::{BTreeMap, HashSet}; use serde_json::Value; @@ -44,6 +44,9 @@ struct PnpmLockState<'f> { pending: Vec<(std::ops::Range, String)>, spliced: std::collections::HashSet, changed: bool, + /// [`vendored_packages`] of `text`, computed on the first unmatched dep + /// and dropped when [`PnpmLockState::materialize`] replaces `text`. + vendored: Option>, } impl<'f> PnpmLockState<'f> { @@ -57,6 +60,7 @@ impl<'f> PnpmLockState<'f> { pending: Vec::new(), spliced: Default::default(), changed: false, + vendored: None, }; state.reindex(); state @@ -80,6 +84,17 @@ impl<'f> PnpmLockState<'f> { self.sorted = sorted; } + /// Whether the lock vendors `fname@version` ([`vendored_packages`]). + /// Read from `text` without folding `pending` in: a resolution splice + /// never touches a packages key or a `name:` / `version:` line (see + /// the type docs), so the answer is the post-splice lock's. + fn vendors(&mut self, fname: &str, version: &str) -> bool { + let text: &str = &self.text; + self.vendored + .get_or_insert_with(|| vendored_packages(text)) + .contains(&(fname.to_string(), version.to_string())) + } + fn entry(&self, i: usize) -> pnpm::Entry<'_> { let (key, body) = &self.entries[i]; pnpm::Entry { @@ -147,6 +162,7 @@ impl<'f> PnpmLockState<'f> { out.push_str(&self.text[cursor..]); self.text = Cow::Owned(out); self.spliced.clear(); + self.vendored = None; self.reindex(); #[cfg(debug_assertions)] debug_assert_eq!( @@ -343,13 +359,9 @@ pub(crate) fn plan_hosted( // mode switch. Fail-closed either way: nothing is rewritten for the // dep. if !matched_any { - // Scanned over the post-splice text, so fold pending splices in. - for lock in locks.iter_mut() { - lock.materialize(); - } let vendored = locks - .iter() - .any(|lock| vendors_package(&super::PnpmLock::parse(&lock.text), &fname)); + .iter_mut() + .any(|lock| lock.vendors(&fname, &dep.version)); if vendored { result.warnings.push(RewriteWarning { code: "redirect_pnpm_entry_vendored".into(), @@ -378,25 +390,33 @@ pub(crate) fn plan_hosted( } } -/// Whether `lock` has a packages entry the vendor backends rekeyed for -/// package `fname`: v9's `@file:` key or legacy's bare -/// `file:` key with a `name:` line, whose path is this project's -/// vendored npm artifact ([`crate::origin::vendor_ref`]: root-anchored, -/// canonical uuid, traversal-safe leaf) — the one rule lockfile discovery -/// reads vendored pnpm keys with. A user's own `file:` dep of the same name, -/// a `../.socket/vendor/…` path from another checkout, and an `overrides:` +/// The `(name, version)` of every packages entry of `text` the vendor +/// backends rekeyed: v9's `@file:` key or legacy's bare +/// `file:` key with a `name:` line, each with its `version:` +/// line, whose path is this project's vendored npm artifact +/// ([`crate::origin::vendor_ref`]: root-anchored, canonical uuid, +/// traversal-safe leaf) — the one rule lockfile discovery reads vendored +/// pnpm keys with. A user's own `file:` dep of the same name, a +/// `../.socket/vendor/…` path from another checkout, and an `overrides:` /// line alone (pnpm mirrors it from package.json whether or not anything /// matches) are not vendoring. -fn vendors_package(lock: &super::PnpmLock<'_>, fname: &str) -> bool { +fn vendored_packages(text: &str) -> HashSet<(String, String)> { use super::{classify_pnpm_key, PnpmKey}; - lock.packages().iter().any(|package| { - let (name, path) = match classify_pnpm_key(package.key) { - PnpmKey::V9File { name, path } => (Some(name.trim_start_matches('/')), path), - PnpmKey::LegacyFile { path } => (pnpm::entry_field(&package.entry, "name"), path), - PnpmKey::Registry { .. } | PnpmKey::Other => return false, - }; - name == Some(fname) && crate::origin::vendor_ref(path).is_some_and(|v| v.eco == "npm") - }) + let lock = super::PnpmLock::parse_keys(text); + lock.packages() + .iter() + .filter_map(|package| { + let (name, path) = match classify_pnpm_key(package.key) { + PnpmKey::V9File { name, path } => (name.trim_start_matches('/'), path), + PnpmKey::LegacyFile { path } => (pnpm::entry_field(&package.entry, "name")?, path), + PnpmKey::Registry { .. } | PnpmKey::Other => return None, + }; + let version = pnpm::entry_field(&package.entry, "version")?; + crate::origin::vendor_ref(path) + .filter(|v| v.eco == "npm") + .map(|_| (name.to_string(), version.to_string())) + }) + .collect() } /// Test-only reference for the residual gate: every instance of this exact diff --git a/crates/socket-patch-core/src/formats/pnpm/mod.rs b/crates/socket-patch-core/src/formats/pnpm/mod.rs index 95f21ef42..2df9b643f 100644 --- a/crates/socket-patch-core/src/formats/pnpm/mod.rs +++ b/crates/socket-patch-core/src/formats/pnpm/mod.rs @@ -88,29 +88,47 @@ impl<'a> PnpmPackage<'a> { /// [`PnpmLock::packages`]). Every entry is returned (registry, rekeyed /// vendored, hosted, directory, git); each consumer applies its own key /// and resolution rules. -fn pnpm_packages(text: &str) -> Vec> { +fn pnpm_packages(text: &str, resolve: Resolve) -> Vec> { let documents = grammar::documents(text); grammar::entries(text) .into_iter() - .map(|entry| PnpmPackage { - key: grammar::unquote(entry.key.trim()), - resolution: grammar::resolution(&entry), - document: documents - .iter() - .find(|(span, _)| span.contains(&entry.offset)) - .map_or(PnpmDocument::Project, |(_, kind)| *kind), - entry, + .map(|entry| { + // Tag by the KEY line: an entry with an empty body (its key the + // last line of a document) has its body offset on the next + // document's `---` line, or at the end of the text. + let key_at = entry.key.as_ptr() as usize - text.as_ptr() as usize; + PnpmPackage { + key: grammar::unquote(entry.key.trim()), + resolution: match resolve { + Resolve::Yes => grammar::resolution(&entry), + Resolve::No => None, + }, + document: documents + .iter() + .find(|(span, _)| span.contains(&key_at)) + .map_or(PnpmDocument::Project, |(_, kind)| *kind), + entry, + } }) .collect() } +/// Whether [`pnpm_packages`] parses each entry's `resolution:` map. +#[derive(Clone, Copy)] +enum Resolve { + Yes, + /// Keys and bodies only: the store crawler's hot path + /// ([`InstalledPackages::from_lock_text`]) never reads a resolution. + No, +} + /// The `(integrity, tarball)` of one recorded `packages:` block — a /// vendored planner's wiring fragment (its key line and body, as /// recorded), each unquoted; `None` unless the fragment is exactly one /// entry with a flat resolution. pub(crate) fn fragment_resolution(block: &[String]) -> Option<(Option, Option)> { let text = format!("packages:\n{}\n", block.join("\n")); - let packages = pnpm_packages(&text); + let packages = pnpm_packages(&text, Resolve::Yes); let [package] = packages.as_slice() else { return None; }; @@ -410,7 +428,18 @@ impl<'t> PnpmLock<'t> { pub fn parse(text: &'t str) -> Self { PnpmLock { text, - packages: pnpm_packages(text), + packages: pnpm_packages(text, Resolve::Yes), + vendored: OnceLock::new(), + } + } + + /// [`PnpmLock::parse`] without the resolution maps (every + /// [`PnpmPackage::resolution`] is `None`), for a reader of keys and + /// entry fields only. + fn parse_keys(text: &'t str) -> Self { + PnpmLock { + text, + packages: pnpm_packages(text, Resolve::No), vendored: OnceLock::new(), } } @@ -478,12 +507,10 @@ impl<'t> PnpmLock<'t> { if tarball.is_some_and(|t| parse_vendor_path(t).is_some()) { continue; } - out.push(LockfileEntry::npm( - name, - version, - tarball.and_then(http_url), - integrity, - )); + let mut entry = + LockfileEntry::npm(name, version, tarball.and_then(http_url), integrity); + entry.pnpm_env = package.document == PnpmDocument::Env; + out.push(entry); } Some(out) } @@ -566,7 +593,9 @@ impl InstalledPackages { if !grammar::is_pnpm_lock_text(text) { return None; } - let lock = PnpmLock::parse(text); + // Keys and `name:` / `version:` lines only: the scan reads this on + // every run, for every package of the install. + let lock = PnpmLock::parse_keys(text); let mut out = Self::default(); for package in lock.packages() { let (mut name, mut version) = (None, None); @@ -856,12 +885,15 @@ mod tests { ); } - /// N4: documents are told apart by content, not position. Before, the - /// readers took the text after the LAST `---` as the project, so a - /// lock with an extra document chose its own "project" and hid the - /// rest; now every document is read, as the tree its importers say. + /// N4: before, the readers took the text after the LAST `---` as the + /// project, so a lock with an extra document chose its own "project" + /// and hid the rest; now every document is read. The env document is + /// the FIRST one, when its importers say so, as pnpm and the vendored + /// planners' `split_project_document` read it: an env-shaped LATER + /// document is a project one, so it cannot opt out of the project + /// tree's contests (code-review finding 2). #[test] - fn documents_are_identified_by_content_not_position() { + fn documents_are_identified_by_position_and_content() { let env = "lockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n cfg-dep:\n specifier: 2.0.0\n version: 2.0.0\n\npackages:\n\n cfg-dep@2.0.0:\n resolution: {integrity: sha512-c}\n"; let project = |dep: &str| { format!( @@ -879,12 +911,13 @@ mod tests { pairs.iter().map(|(k, d)| (k.to_string(), *d)).collect() }; use PnpmDocument::{Env, Project}; - // The env document after the project still reads as the env tree. + // An env-shaped document after the project is a project one. let swapped = format!("---\n{}---\n{env}", project("a")); assert_eq!( kinds(&swapped), - own(&[("a@1.0.0", Project), ("cfg-dep@2.0.0", Env)]) + own(&[("a@1.0.0", Project), ("cfg-dep@2.0.0", Project)]) ); + assert!(!PnpmLock::parse(&swapped).documents().contains(&Env)); // A first document that is not the env document is a project one, // and its entries are no longer skipped. let two = format!("---\n{}---\n{}", project("a"), project("b")); @@ -911,6 +944,32 @@ mod tests { } } + /// An entry is tagged by its KEY line (code-review finding 5): a key + /// whose body is empty — the last line of the env document, right + /// before `---` — has its body offset on the separator, in no document. + #[test] + fn an_entry_with_an_empty_body_keeps_its_document() { + let env = "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n cfg-dep:\n specifier: 2.0.0\n version: 2.0.0\n\npackages:\n\n cfg-dep@2.0.0:\n"; + let project = "---\nlockfileVersion: '9.0'\n\npackages:\n\n a@1.0.0:\n resolution: {integrity: sha512-x}\n\n b@1.0.0:"; + let text = format!("{env}{project}"); + for text in [text.clone(), text.replace('\n', "\r\n")] { + let tagged: Vec<_> = PnpmLock::parse(&text) + .packages() + .iter() + .map(|p| (p.key, p.document)) + .collect(); + assert_eq!( + tagged, + [ + ("cfg-dep@2.0.0", PnpmDocument::Env), + ("a@1.0.0", PnpmDocument::Project), + ("b@1.0.0", PnpmDocument::Project), + ], + "{text:?}" + ); + } + } + #[test] fn vendored_in_use_reads_v9_and_legacy_keys_and_crlf() { let v9 = format!( @@ -1117,13 +1176,134 @@ mod architecture_tests { && (code.contains("bounds(") || code.contains("SectionIndex::build("))) } - /// `text` up to its test module. - fn production(text: &str) -> &str { - ["#[cfg(test)]\nmod ", "#[cfg(test)]\npub(crate) mod "] - .iter() - .filter_map(|marker| text.find(marker)) - .min() - .map_or(text, |at| &text[..at]) + /// The lines of `text` outside `#[cfg(test)]` / `#[cfg(all(test, …))]` + /// items: an out-of-line `mod name;` declaration drops that line only, + /// an inline item (`mod tests { … }`, a helper `fn`) drops through its + /// closing brace, so production code after either is still scanned. + fn production(text: &str) -> Vec<&str> { + let lines: Vec<&str> = text.split('\n').collect(); + let deltas = brace_deltas(text); + let mut out = Vec::new(); + let mut i = 0; + while i < lines.len() { + let attr = lines[i].trim_start(); + if !(attr.starts_with("#[cfg(test)]") || attr.starts_with("#[cfg(all(test")) { + out.push(lines[i]); + i += 1; + continue; + } + // Skip the gated item's other attributes, then the item. + i += 1; + while i < lines.len() && lines[i].trim_start().starts_with("#[") { + i += 1; + } + let mut depth = 0i64; + let mut opened = false; + while i < lines.len() { + let (open, close) = deltas[i]; + let line = lines[i]; + i += 1; + depth += open - close; + opened |= open > 0; + if (!opened && line.trim_end().ends_with(';')) || (opened && depth <= 0) { + break; + } + } + } + out + } + + /// Per line of `text` (split on `\n`), the `{` and `}` in code: outside + /// comments and string, raw string and char literals, which may span + /// lines. + fn brace_deltas(text: &str) -> Vec<(i64, i64)> { + let chars: Vec = text.chars().collect(); + let mut out = vec![(0i64, 0i64)]; + let mut i = 0; + let ident = |c: char| c.is_alphanumeric() || c == '_'; + // Advance past `n` chars, opening a new line entry at each `\n`. + let step = |out: &mut Vec<(i64, i64)>, i: &mut usize, n: usize| { + for _ in 0..n { + if chars.get(*i) == Some(&'\n') { + out.push((0, 0)); + } + *i += 1; + } + }; + while i < chars.len() { + let c = chars[i]; + let next = chars.get(i + 1).copied(); + if c == '/' && next == Some('/') { + while i < chars.len() && chars[i] != '\n' { + i += 1; + } + } else if c == '/' && next == Some('*') { + let mut depth = 0; + while i < chars.len() { + if chars[i] == '/' && chars.get(i + 1) == Some(&'*') { + depth += 1; + step(&mut out, &mut i, 2); + } else if chars[i] == '*' && chars.get(i + 1) == Some(&'/') { + depth -= 1; + step(&mut out, &mut i, 2); + if depth == 0 { + break; + } + } else { + step(&mut out, &mut i, 1); + } + } + } else if c == 'r' + && matches!(next, Some('"' | '#')) + && (i == 0 || !ident(chars[i - 1]) || chars[i - 1] == 'b') + { + let hashes = chars[i + 1..].iter().take_while(|&&h| h == '#').count(); + if chars.get(i + 1 + hashes) != Some(&'"') { + step(&mut out, &mut i, 1); + continue; + } + step(&mut out, &mut i, 2 + hashes); + while i < chars.len() { + let closes = chars[i] == '"' + && chars[i + 1..] + .iter() + .take(hashes) + .filter(|&&h| h == '#') + .count() + == hashes; + if closes { + step(&mut out, &mut i, 1 + hashes); + break; + } + step(&mut out, &mut i, 1); + } + } else if c == '"' { + step(&mut out, &mut i, 1); + while i < chars.len() && chars[i] != '"' { + let n = if chars[i] == '\\' { 2 } else { 1 }; + step(&mut out, &mut i, n); + } + step(&mut out, &mut i, 1); + } else if c == '\'' && next == Some('\\') { + // An escaped char literal (`'\''`, `'\u{7b}'`). + step(&mut out, &mut i, 2); + while i < chars.len() && chars[i] != '\'' { + step(&mut out, &mut i, 1); + } + step(&mut out, &mut i, 1); + } else if c == '\'' && chars.get(i + 2) == Some(&'\'') { + step(&mut out, &mut i, 3); + } else { + let last = out.last_mut().expect("one entry per line"); + match c { + '{' => last.0 += 1, + '}' => last.1 += 1, + _ => {} + } + step(&mut out, &mut i, 1); + } + } + out } fn rs_files(dir: &Path, out: &mut Vec) { @@ -1155,7 +1335,10 @@ mod architecture_tests { let text = std::fs::read_to_string(&path) .unwrap() .replace("\r\n", "\n"); - let uses = production(&text).lines().filter(|l| raw_walk(l)).count(); + let uses = production(&text) + .into_iter() + .filter(|l| raw_walk(l)) + .count(); if uses > 0 { found.push((rel, uses)); } @@ -1185,4 +1368,39 @@ mod architecture_tests { assert!(!raw_walk(" lock.get(\"packages\")")); assert!(!raw_walk(" for p in PnpmLock::parse(t).packages() {")); } + + /// Code-review finding 1: an out-of-line test module declared near the + /// top of a file (`crawlers/npm_crawler.rs`, `patch/redirect/mod.rs`, + /// `vendor/mod.rs` do) used to end the scan there, so a raw walk deep + /// in the file went unseen. Only the gated items are dropped now. + #[test] + fn production_drops_only_test_gated_items() { + let src = "use a;\n#[cfg(test)]\nmod oracle;\n#[cfg(all(test, unix))]\n#[path = \"x.rs\"]\nmod x_tests;\n\ + #[cfg(test)]\nfn helper() {\n let s = \"}\";\n let c = '{';\n if true { pnpm::entries(t); }\n}\n\ + fn reader<'a>(t: &'a str) {\n for e in pnpm::entries(t) {}\n}\n\ + #[cfg(test)]\nmod tests {\n fn t() { pnpm::entries(\"{\"); }\n \ + const S: &str = \"a {\n{ b\";\n const R: &str = r#\"{\"{\n\"#;\n /* { */\n}\n\ + fn after() { let x = line == \"packages:\"; }\n"; + let kept = production(src); + assert_eq!(kept.iter().filter(|l| raw_walk(l)).count(), 2, "{kept:#?}"); + assert!(kept.iter().all(|l| !l.contains("mod oracle"))); + assert!(kept.iter().any(|l| l.contains("fn after()"))); + assert!(kept.iter().all(|l| !l.contains("fn helper"))); + // The real files that declare test modules early keep their late + // production code. + let src = Path::new(env!("CARGO_MANIFEST_DIR")).join("src"); + for (rel, late) in [ + ("crawlers/npm_crawler.rs", "fn is_safe_npm_component("), + ("patch/redirect/mod.rs", "fn rewrite_golang_target("), + ("vendor/mod.rs", "pub async fn vendored_purl_keys("), + ] { + let text = std::fs::read_to_string(src.join(rel)) + .unwrap() + .replace("\r\n", "\n"); + assert!( + production(&text).iter().any(|l| l.starts_with(late)), + "{rel}: production code before its test module was not scanned" + ); + } + } } diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index 09901663b..1fb9c4c41 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -2194,6 +2194,7 @@ async fn unattributed_pins( for c in discovery.unpatched_copies.iter().filter(|c| { c.purl == r.purl && c.file == r.source_file + && c.part == r.part && matches!( c.file.file_name().and_then(|n| n.to_str()), Some("pnpm-lock.yaml" | "shrinkwrap.yaml" | "yarn.lock") diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 4fbee0155..0201fbb31 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -28324,6 +28324,14 @@ packages: legacy_overrides_only, "redirect_pnpm_entry_not_found", ), + // Code-review finding 3: another version vendored is not this + // one (the dep's version is simply not locked). + ( + "another version vendored", + v9(&format!(".socket/vendor/npm/{uuid}/left-pad-1.2.0.tgz")) + .replace("version: 1.3.0", "version: 1.2.0"), + "redirect_pnpm_entry_not_found", + ), ] { assert_eq!(codes(&lock), vec![want], "{case}"); } diff --git a/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs b/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs index 7ce3cc6ea..8a00efcb7 100644 --- a/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs +++ b/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs @@ -232,7 +232,7 @@ fn indexed_pnpm_rewrite_matches_golden_on_depscan_sized_lock_set() { }; let root_extra = vec![ format!( - " vendored-pkg@file:.socket/vendor/npm/0a1b2c3d-4e5f-4a6b-8c7d-8e9f0a1b2c3d/vendored-pkg-1.0.0.tgz:\n resolution: {{tarball: file:.socket/vendor/npm/0a1b2c3d-4e5f-4a6b-8c7d-8e9f0a1b2c3d/vendored-pkg-1.0.0.tgz}}\n" + " vendored-pkg@file:.socket/vendor/npm/0a1b2c3d-4e5f-4a6b-8c7d-8e9f0a1b2c3d/vendored-pkg-1.0.0.tgz:\n resolution: {{tarball: file:.socket/vendor/npm/0a1b2c3d-4e5f-4a6b-8c7d-8e9f0a1b2c3d/vendored-pkg-1.0.0.tgz}}\n version: 1.0.0\n" ), // A second copy of an entry that already points at the hosted // artifact (a re-run): rebuilt == original, no edit. diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs index 9250b6b6b..7b68418c8 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs @@ -3802,6 +3802,13 @@ mod tests { // A lone leading `---` marker is no env document. let marker_only = format!("---\n{}", lock("")); assert!(include(&marker_only, None, Some(RC_ON), None)); + // Neither is an env-shaped document AFTER the project one: pnpm + // writes the env document first (code-review finding 2). + let env_last = format!("---\n{}{env}", lock("")); + assert_eq!(super::pnpm_settings_major(&pinned_only, None), Some(11)); + assert_eq!(super::pnpm_settings_major(&env_last, None), None); + // Its bare entry is project evidence: the setting is off. + assert!(!include(&env_last, None, Some(RC_ON), None)); // A bare entry in the main document still proves it off. let bare = " b@2.0.0:\n resolution: {integrity: sha512-B==}\n"; assert!(!include( diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/golang.rs b/crates/socket-patch-core/src/vendor/lock_inventory/golang.rs index 25636f85d..f95d3a65f 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/golang.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/golang.rs @@ -43,6 +43,7 @@ pub(super) async fn inventory_go_sum_raw_in(view: &ProjectView<'_>) -> Option = Vec::new(); let mut unsupported: Vec = Vec::new(); - let npm = if every { - npm_family::inventory_npm_lock_raw_in(view).await - } else { - npm_family::inventory_npm_lock_in(view).await - }; - match npm { - Ok(Some((_, entries))) => out.extend(entries), - Ok(None) => {} + match npm_family::inventory_npm_trees_raw_in(view).await { + Ok(entries) if every => out.extend(entries), + Ok(entries) => out.extend(dedup_prefer_integrity(entries)), Err(diag) => unsupported.push(diag), } unsupported.extend(gem::unsupported_gem_layout_in(view).await); @@ -349,20 +350,23 @@ async fn union_views_in( (out, unsupported) } -/// Collapse duplicate (name, version) instances, preferring one that -/// carries a verifier. Each view applies it ONCE, at its public boundary, -/// over its `*_raw` body (collapsing is associative over concatenation — -/// the first verifier-bearing instance wins, else the first — so a union of -/// raw sub-views collapses to what collapsing each sub-view first would). +/// Collapse duplicate (name, version) instances, preferring a project +/// instance over a pnpm env-lockfile one ([`LockfileEntry::pnpm_env`]: a +/// config dependency never stands in for the project's own copy, whose +/// `resolved` url ledger liveness reads), then one that carries a +/// verifier. Each view applies it ONCE, at its public boundary, over its +/// `*_raw` body (collapsing is associative over concatenation — the first +/// most-preferred instance wins — so a union of raw sub-views collapses to +/// what collapsing each sub-view first would). fn dedup_prefer_integrity(raw: Vec) -> Vec { + let rank = |e: &LockfileEntry| (!e.pnpm_env, e.integrity != LockIntegrity::None); let mut seen: HashMap<(String, String), usize> = HashMap::new(); let mut out: Vec = Vec::new(); for entry in raw { let key = (entry.name.clone(), entry.version.clone()); match seen.get(&key) { Some(&i) => { - if out[i].integrity == LockIntegrity::None && entry.integrity != LockIntegrity::None - { + if rank(&entry) > rank(&out[i]) { out[i] = entry; } } diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs b/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs index 1dabe294f..f8bc82c6a 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs @@ -14,6 +14,8 @@ use crate::utils::purl::npm_purl; use crate::vendor::npm_flavor::NpmLockFlavor; use super::bun::{bun_text_lock_drives, inventory_bun_binary_in, inventory_bun_in}; +#[cfg(test)] +use super::dedup_prefer_integrity; use super::npm::inventory_package_lock_in; use super::pnpm::{ inventory_pnpm_lock_in, inventory_pnpm_lock_rel_in, inventory_pnpm_member_locks_in, @@ -22,7 +24,7 @@ use super::pnpm::{ use super::view::ProjectView; use super::vlt::inventory_vlt_in; use super::yarn::{inventory_yarn_berry_in, inventory_yarn_classic_in}; -use super::{dedup_prefer_integrity, LockIntegrity, LockfileEntry, UnsupportedNpmLayout}; +use super::{LockIntegrity, LockfileEntry, UnsupportedNpmLayout}; use crate::vendor::npm_flavor::detect_npm_lock_flavor_in; // ── registry view ── @@ -42,64 +44,73 @@ use crate::vendor::npm_flavor::detect_npm_lock_flavor_in; /// the pnpm <=2-era `shrinkwrap.yaml` (same v5 grammar, older filename) or, /// failing that, the workspace members' own locks. A pnpm workspace with /// `sharedWorkspaceLockfile: false` reads those beside its root lock -/// ([`inventory_pnpm_member_locks_in`], #492). -/// Rush's common and subspace locks are read beside whatever this picks -/// whenever `rush.json` is present ([`inventory_npm_lock_raw_in`]). +/// ([`inventory_pnpm_member_locks_in`], #492). The flavor names that +/// routed lock; Rush's locks, which no flavor routes, are read beside it by +/// [`inventory_npm_trees_raw_in`]. #[cfg(test)] pub(crate) async fn inventory_npm_lock( project_root: &Path, ) -> Result)>, UnsupportedNpmLayout> { - inventory_npm_lock_in(&ProjectView::Disk(project_root)).await + let view = ProjectView::Disk(project_root); + let rush = view.exists("rush.json").await; + inventory_npm_lock_raw_in(&view, rush) + .await + .map(|found| found.map(|(flavor, raw)| (flavor, dedup_prefer_integrity(raw)))) } -/// [`inventory_npm_lock`] over a [`ProjectView`]. -pub(super) async fn inventory_npm_lock_in( - view: &ProjectView<'_>, -) -> Result)>, UnsupportedNpmLayout> { - inventory_npm_lock_raw_in(view) +/// Every npm-family tree the inventory reads, collapsed: what +/// [`super::inventory_project`] lists for npm. +#[cfg(test)] +pub(crate) async fn inventory_npm_trees( + project_root: &Path, +) -> Result, UnsupportedNpmLayout> { + inventory_npm_trees_raw_in(&ProjectView::Disk(project_root)) .await - .map(|found| found.map(|(flavor, raw)| (flavor, dedup_prefer_integrity(raw)))) + .map(dedup_prefer_integrity) } -/// [`inventory_npm_lock`] before its collapse: every -/// guarded instance ([`super::inventory_project_every_lock`]). -/// -/// Two rules follow lockfile discovery (`vex::discover::npm`'s -/// `extract_pnpm`), which reads what hosted mode pins: -/// -/// * Rush's common and subspace locks are separate install trees, read -/// whenever `rush.json` is present, beside whatever the router chose -/// (before: only when the root had no lock; lock-model divergence N12). -/// * pnpm's `lockfile` setting off ([`lockfile_disabled`], not under Rush, -/// which runs pnpm with settings of its own): pnpm neither reads nor -/// writes the root or member locks, so a Socket-hosted pin in them is no -/// lockfile basis. Discovery keeps such a ref but withholds its lock pin; -/// here the entry stays listed (what it names is still the dependency -/// set, and its hosted url still shows liveness the wiring) with its -/// integrity dropped, so nothing verifies or fetches against the pin -/// (N14). Registry entries keep theirs: a registry release's integrity -/// is its content verifier whether or not pnpm reads the lock. -pub(super) async fn inventory_npm_lock_raw_in( +/// Every guarded npm-family instance, uncollapsed: the routed lock +/// ([`inventory_npm_lock`]) and, whenever `rush.json` is present, Rush's +/// common and subspace locks, separate install trees beside it as hosted +/// mode and lockfile discovery read them (before: only when the root had +/// no lock; lock-model divergence N12). +pub(super) async fn inventory_npm_trees_raw_in( view: &ProjectView<'_>, +) -> Result, UnsupportedNpmLayout> { + let rush = view.exists("rush.json").await; + let mut out = inventory_npm_lock_raw_in(view, rush) + .await? + .map(|(_, entries)| entries) + .unwrap_or_default(); + if rush { + out.extend(guard_npm(inventory_rush_pnpm_locks_in(view).await)); + } + Ok(out) +} + +/// The routed lock's guarded instances ([`inventory_npm_lock`] before its +/// collapse). With pnpm's `lockfile` setting off ([`lockfile_disabled`]; +/// not under Rush, `rush` being whether `rush.json` exists, which runs +/// pnpm with settings of its own, as in lockfile discovery's +/// `extract_pnpm`), pnpm neither reads nor writes the root or member +/// locks, so a Socket-hosted pin in them is no lockfile basis. Discovery +/// keeps such a ref but withholds its lock pin; here the entry stays +/// listed (what it names is still the dependency set, and its hosted url +/// still shows ledger liveness the wiring) with its integrity dropped, so +/// nothing verifies or fetches against the pin (N14). Registry entries +/// keep theirs: a registry release's integrity is its content verifier +/// whether or not pnpm reads the lock. +async fn inventory_npm_lock_raw_in( + view: &ProjectView<'_>, + rush: bool, ) -> Result)>, UnsupportedNpmLayout> { let mut found = inventory_npm_router_lock_raw_in(view).await?; - let rush = view.exists("rush.json").await; if let Some((NpmLockFlavor::Pnpm | NpmLockFlavor::PnpmLegacy, entries)) = &mut found { if !rush && lockfile_disabled(view, false).await.is_some() { withhold_hosted_pins(entries); } } - let rush = guard_npm(inventory_rush_pnpm_locks_in(view).await); - if rush.is_empty() { - return Ok(found); - } - Ok(Some(match found { - Some((flavor, mut entries)) => { - entries.extend(rush); - (flavor, entries) - } - None => (NpmLockFlavor::Pnpm, rush), - })) + Ok(found) } /// Drop the integrity of every entry resolving from Socket's patch server @@ -118,7 +129,7 @@ fn withhold_hosted_pins(entries: &mut [LockfileEntry]) { } /// The lock the npm flavor router picks, with its fallbacks (see -/// [`inventory_npm_lock`]); Rush's locks are added by the caller. +/// [`inventory_npm_lock`]). async fn inventory_npm_router_lock_raw_in( view: &ProjectView<'_>, ) -> Result)>, UnsupportedNpmLayout> { diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs b/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs index 9efe7dfd7..ea0c64927 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs @@ -102,11 +102,11 @@ fn pnpm_lock_text_inventory(text: &str) -> Option> { /// `common/config/subspaces//pnpm-lock.yaml`. `rush install` copies /// the source lock into common/temp and runs pnpm there. /// -/// Only called (via [`inventory_npm_lock`]) when there is NO root lock but -/// `rush.json` is present, so it never shadows a plain pnpm project. The +/// Called (via `npm_family::inventory_npm_trees_raw_in`) only when +/// `rush.json` is present, which that caller checks once through the +/// view; the locks are read beside whatever lock the router picks. The /// subspace directory is read sorted for deterministic output. Missing -/// files/dirs are skipped fail-soft; the caller drops the whole result when -/// it comes back empty. +/// files/dirs are skipped fail-soft. pub(super) async fn inventory_rush_pnpm_locks_in(view: &ProjectView<'_>) -> Vec { let project = match view { ProjectView::Disk(_) | ProjectView::Snapshot(_) => { @@ -115,9 +115,6 @@ pub(super) async fn inventory_rush_pnpm_locks_in(view: &ProjectView<'_>) -> Vec< } ProjectView::Memory(project) => *project, }; - if !project.contains("rush.json") { - return Vec::new(); - } let mut out = Vec::new(); if let Some(entries) = inventory_pnpm_lock_rel_in(view, RUSH_COMMON_LOCK_REL).await { out.extend(entries); @@ -135,12 +132,6 @@ pub(super) async fn inventory_rush_pnpm_locks_in(view: &ProjectView<'_>) -> Vec< } async fn inventory_rush_pnpm_locks(project_root: &Path) -> Vec { - if tokio::fs::metadata(project_root.join("rush.json")) - .await - .is_err() - { - return Vec::new(); - } let mut out = Vec::new(); // The single source-of-truth lock. diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs b/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs index b1822066a..0280a0ad9 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs @@ -375,6 +375,7 @@ pub(super) fn python_lock_inventory(text: &str) -> Option> { out.push(LockfileEntry { ecosystem: "pypi", source_kind: SourceKind::Unspecified, + pnpm_env: false, purl, name, version: version.to_string(), @@ -416,6 +417,7 @@ async fn inventory_poetry_lock(view: &ProjectView<'_>) -> Option) -> Option) -> Option) -> Option .map(|(name, version, purl, _)| LockfileEntry { ecosystem: "pypi", source_kind: SourceKind::Unspecified, + pnpm_env: false, purl, name, version, @@ -722,6 +727,7 @@ async fn inventory_requirements_txt(view: &ProjectView<'_>) -> Option, integrity: LockIntegrity| LockfileEntry { ecosystem: "npm", source_kind: SourceKind::Unspecified, + pnpm_env: false, purl: format!("pkg:npm/{name}@{version}"), name: name.to_string(), version: version.to_string(), diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs index 6ef23cd40..68f264cac 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs @@ -741,8 +741,9 @@ packages: ) .await; - let (flavor, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); - assert_eq!(flavor, NpmLockFlavor::Pnpm); + // No lock is routed (Rush's are no flavor's), and both are read. + assert!(inventory_npm_lock(tmp.path()).await.unwrap().is_none()); + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); // Union across the common lock and the subspace lock. assert_eq!(entry(&entries, "left-pad").version, "1.3.0"); assert_eq!(entry(&entries, "only-in-subspace").version, "9.9.9"); @@ -754,6 +755,7 @@ async fn rush_json_without_any_lock_yields_none() { let tmp = tempfile::tempdir().unwrap(); write(tmp.path(), "rush.json", r#"{"rushVersion":"5.0.0"}"#).await; assert!(inventory_npm_lock(tmp.path()).await.unwrap().is_none()); + assert!(inventory_npm_trees(tmp.path()).await.unwrap().is_empty()); } /// Lock-model divergence N12: Rush's locks are their own install trees, @@ -778,8 +780,12 @@ packages: ) .await; - let (flavor, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + // The routed lock keeps its flavor and holds only its own entries + // (code-review finding 9: Rush's are no flavor's). + let (flavor, routed) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); assert_eq!(flavor, NpmLockFlavor::Pnpm); + assert!(!routed.iter().any(|e| e.name == "only-in-common")); + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); assert!(entries.iter().any(|e| e.name == "left-pad")); assert!( entries.iter().any(|e| e.name == "only-in-common"), @@ -791,24 +797,77 @@ packages: .await .unwrap(); write(tmp.path(), "package-lock.json", PACKAGE_LOCK).await; - let (flavor, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + let (flavor, routed) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); assert_eq!(flavor, NpmLockFlavor::PackageLock); + assert!(!routed.iter().any(|e| e.name == "only-in-common")); + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); assert!( entries.iter().any(|e| e.name == "only-in-common"), "{entries:?}" ); + // The project union lists them too. + assert!(inventory_project(tmp.path()) + .await + .iter() + .any(|e| e.name == "only-in-common")); // Without rush.json they are not Rush's. tokio::fs::remove_file(tmp.path().join("rush.json")) .await .unwrap(); - let (_, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); assert!( !entries.iter().any(|e| e.name == "only-in-common"), "{entries:?}" ); } +/// Code-review finding 4: the env document's registry instance of a +/// release comes first in the lock and may carry the only verifier (the +/// project's hosted instance loses its pin under `lockfile=false`), but a +/// config dependency never stands in for the project's own copy: the +/// collapsed view keeps the project's hosted instance, and the +/// every-instance view (ledger liveness) keeps both. +#[tokio::test] +async fn env_document_instances_never_displace_project_instances() { + let uuid = "1a2b3c4d-5e6f-4a1b-8c2d-0123456789ab"; + let url = + format!("https://patch.socket.dev/patch/npm/left-pad/1.3.0/tok/{uuid}/left-pad-1.3.0.tgz"); + let lock = format!( + "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n \ + left-pad:\n specifier: 1.3.0\n version: 1.3.0\n\npackages:\n\n \ + left-pad@1.3.0:\n resolution: {{integrity: sha512-UPSTREAM==}}\n\n---\n\ + lockfileVersion: '9.0'\n\npackages:\n\n left-pad@1.3.0:\n \ + resolution: {{integrity: sha512-PATCHED==, tarball: {url}}}\n" + ); + for lockfile_off in [false, true] { + let tmp = tempfile::tempdir().unwrap(); + write(tmp.path(), "pnpm-lock.yaml", &lock).await; + if lockfile_off { + write(tmp.path(), ".npmrc", "lockfile=false\n").await; + } + let (_, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + let left = entry(&entries, "left-pad"); + assert_eq!( + left.resolved.as_deref(), + Some(url.as_str()), + "{lockfile_off}" + ); + assert!(!left.pnpm_env, "{lockfile_off}"); + let every: Vec<_> = inventory_project_every_lock(tmp.path()) + .await + .into_iter() + .filter(|e| e.name == "left-pad") + .map(|e| (e.pnpm_env, e.resolved)) + .collect(); + assert_eq!( + every, + [(true, None), (false, Some(url.clone()))], + "{lockfile_off}" + ); + } +} + /// Lock-model divergence N14: with pnpm's `lockfile` setting off, pnpm /// never reads the lock, so a Socket-hosted pin in it is no lockfile /// basis. Lockfile discovery keeps the ref and withholds its pin; the @@ -862,7 +921,7 @@ async fn pnpm_lockfile_off_withholds_hosted_pins() { write(tmp.path(), "rush.json", r#"{"rushVersion":"5.0.0"}"#).await; write(tmp.path(), ".npmrc", "lockfile=false\n").await; write_nested(tmp.path(), "common/config/rush/pnpm-lock.yaml", &lock).await; - let (_, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); assert_eq!( integrity(&entries, "left-pad"), LockIntegrity::Sri("sha512-PATCHED==".into()) @@ -1382,6 +1441,7 @@ async fn lookup_matches_cargo_and_golang_purls() { LockfileEntry { ecosystem: "cargo", source_kind: SourceKind::Unspecified, + pnpm_env: false, name: "serde".into(), version: "1.0.200".into(), purl: "pkg:cargo/serde@1.0.200".into(), @@ -1391,6 +1451,7 @@ async fn lookup_matches_cargo_and_golang_purls() { LockfileEntry { ecosystem: "golang", source_kind: SourceKind::Unspecified, + pnpm_env: false, name: "github.com/x/y".into(), version: "v1.0.0".into(), purl: "pkg:golang/github.com/x/y@v1.0.0".into(), @@ -3777,6 +3838,7 @@ async fn lookup_matches_by_purl_identity() { let entry = |ecosystem: &'static str, name: &str, version: &str| LockfileEntry { ecosystem, source_kind: SourceKind::Unspecified, + pnpm_env: false, name: name.into(), version: version.into(), purl: format!("pkg:{ecosystem}/{name}@{version}"), diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 48e9c2a6f..f798ca957 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -182,7 +182,7 @@ //! `vlt-lock.json`), so discovery and the inventory see one entry walk //! per format (an architecture test in this module enforces it). -use std::collections::{BTreeMap, BTreeSet, HashMap}; +use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; use std::path::{Path, PathBuf}; use std::sync::Mutex; @@ -244,33 +244,32 @@ pub const DIAG_REF_INVALID: &str = "patched_ref_invalid"; /// dependency version, a nuget source with no package mapping). pub const DIAG_REF_UNATTRIBUTABLE: &str = "patched_ref_unattributable"; -/// The label [`pnpm_document_tree`] appends to a pnpm lock's path to key -/// its env lockfile document as an install tree of its own. No file -/// discovery reads ends in it (every lock path ends in `.yaml`). -pub(crate) const PNPM_ENV_TREE_SUFFIX: &str = " (env lockfile)"; - -/// The install-tree key of one document of the pnpm lock at root-relative -/// `file`. pnpm 11+'s env lockfile document (config dependencies, the -/// pinned package manager) installs apart from the project's -/// `node_modules`, so it is its own tree (owner decision, lock-model -/// divergence N4): its records carry ` (env lockfile)` as their -/// file. That keeps its refs and the project document's entries apart in -/// the same-lock and bundled-copy contests ([`Discovery::contest_within_locks`], -/// [`UnwiredCopy`] scopes), keeps its wiring from exempting the project -/// document in [`Discovery::contest_across_locks`], and lets that contest -/// skip it both ways: config dependencies install apart from every -/// `node_modules`, so no other lock decides them and they decide no other -/// lock's ref. Within the env document the same-lock rules apply. The -/// project document keeps `file` itself. [`Discovery::finalize`] maps -/// every key back to `file`, so consumers only ever see real paths; detail -/// texts keep the label. -pub(crate) fn pnpm_document_tree( - file: &str, - document: crate::formats::pnpm::PnpmDocument, -) -> String { - match document { - crate::formats::pnpm::PnpmDocument::Project => file.to_string(), - crate::formats::pnpm::PnpmDocument::Env => format!("{file}{PNPM_ENV_TREE_SUFFIX}"), +/// Which install tree of its file a record belongs to: the install-tree +/// key is the file (its [`Discovery::install_trees`] root) plus this part. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum TreePart { + /// The file's own tree (every format but the one below). + #[default] + File, + /// pnpm 11+'s env lockfile document (config dependencies, the pinned + /// package manager), which installs apart from the project's + /// `node_modules`: its own tree (owner decision, lock-model divergence + /// N4). Its refs and the project document's entries of the same lock + /// never contest each other ([`Discovery::contest_within_locks`], + /// [`UnwiredCopy`] scopes), its wiring never exempts the project + /// document in [`Discovery::contest_across_locks`], and that contest + /// skips it both ways: no other lock decides a config dependency, and a + /// config dependency decides no other lock's ref. Within the document + /// the same-lock rules apply. + PnpmEnv, +} + +impl From for TreePart { + fn from(document: crate::formats::pnpm::PnpmDocument) -> Self { + match document { + crate::formats::pnpm::PnpmDocument::Project => TreePart::File, + crate::formats::pnpm::PnpmDocument::Env => TreePart::PnpmEnv, + } } } @@ -328,6 +327,8 @@ pub struct PatchedRef { /// package from, whose dependency sub-maps decide which dependencies /// install beside it (#1538). pub yarn_classic_block: Option, + /// The install tree of `source_file` the wiring is in ([`TreePart`]). + pub part: TreePart, } /// A hosted yarn classic pin's lock block, as lock-only VEX reads it @@ -386,9 +387,16 @@ impl PatchedRef { integrity_required, url: url.map(str::to_string), yarn_classic_block: None, + part: TreePart::File, } } + /// This ref, in install tree `part` of its file. + pub(crate) fn in_part(mut self, part: TreePart) -> Self { + self.part = part; + self + } + /// A vendored ref for the artifact `vref` names. pub fn vendored( purl: String, @@ -406,6 +414,7 @@ impl PatchedRef { integrity_required: false, url: None, yarn_classic_block: None, + part: TreePart::File, } } @@ -507,6 +516,8 @@ pub struct ResolvedElsewhere { /// which pip compares as a string /// ([`Discovery::resolved_elsewhere_as_spelled`]). pub as_spelled: bool, + /// The install tree of `file` the entry is in ([`TreePart`]). + pub part: TreePart, } impl ResolvedElsewhere { @@ -578,6 +589,8 @@ pub struct UnpatchedCopy { /// What makes the build patch that copy, when it is not the usual /// "depend on the registry release instead" (a bundled copy, #758). pub remedy: Option, + /// The install tree of `file` the entry is in ([`TreePart`]). + pub part: TreePart, } /// A lock entry that installs SOME package (at `version`, when known) under @@ -684,8 +697,9 @@ pub enum CopyTarget { /// extractor has run ([`Discovery::unattest_unwired_copies`]). #[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)] pub struct UnwiredCopy { - /// Only refs of this root-relative file; `None` = refs of any file. - pub(crate) scope: Option, + /// Only refs of this root-relative file, in this tree of it; `None` = + /// refs of any file. + pub(crate) scope: Option<(PathBuf, TreePart)>, pub(crate) target: CopyTarget, /// Root-relative file that shows the copy. pub(crate) file: PathBuf, @@ -696,7 +710,11 @@ pub struct UnwiredCopy { impl UnwiredCopy { fn covers(&self, r: &PatchedRef) -> bool { - if self.scope.as_ref().is_some_and(|f| *f != r.source_file) { + if self + .scope + .as_ref() + .is_some_and(|(f, part)| *f != r.source_file || *part != r.part) + { return false; } match &self.target { @@ -1004,11 +1022,22 @@ impl Discovery { /// lock's size (#993), and every reader before then only asks whether /// some entry matches. pub(crate) fn resolved_elsewhere(&mut self, file: &str, purl: Option) { + self.resolved_elsewhere_in(file, TreePart::File, purl); + } + + /// [`Discovery::resolved_elsewhere`] in install tree `part` of `file`. + pub(crate) fn resolved_elsewhere_in( + &mut self, + file: &str, + part: TreePart, + purl: Option, + ) { let Some(purl) = purl else { return }; self.elsewhere.push(ResolvedElsewhere { purl: canonical_base_purl(&purl), file: PathBuf::from(file), as_spelled: false, + part, }); } @@ -1021,6 +1050,7 @@ impl Discovery { purl: canonical_base_purl(&purl), file: PathBuf::from(file), as_spelled: true, + part: TreePart::File, }); } @@ -1040,6 +1070,7 @@ impl Discovery { purl, file: PathBuf::from(file), as_spelled: false, + part: TreePart::File, }); } @@ -1057,11 +1088,23 @@ impl Discovery { purl: Option, key: &str, how: &str, + ) { + self.unpatched_copy_in(file, TreePart::File, purl, key, how); + } + + /// [`Discovery::unpatched_copy`] in install tree `part` of `file`. + pub(crate) fn unpatched_copy_in( + &mut self, + file: &str, + part: TreePart, + purl: Option, + key: &str, + how: &str, ) { let Some(purl) = purl else { return; }; - self.resolved_elsewhere(file, Some(purl.clone())); + self.resolved_elsewhere_in(file, part, Some(purl.clone())); // Deduplicated once, in `finalize`: a per-push scan is quadratic // over a lock with thousands of registry blocks. self.unpatched_copies.push(UnpatchedCopy { @@ -1070,6 +1113,7 @@ impl Discovery { key: key.to_string(), how: how.to_string(), remedy: None, + part, }); } @@ -1121,7 +1165,9 @@ impl Discovery { let purls: BTreeSet<&str> = self .refs .iter() - .filter(|r| r.source_file == u.file) + // A file-level copy: no extractor records one in a pnpm + // env lockfile, whose refs are a tree of their own. + .filter(|r| r.source_file == u.file && r.part == TreePart::File) .filter(|r| match &u.version { Some(v) => r.purl.rsplit_once('@').is_some_and(|(_, rv)| rv == v), None => true, @@ -1140,6 +1186,7 @@ impl Discovery { key: u.key.clone(), how: u.how.clone(), remedy: None, + part: TreePart::File, }); } } @@ -1162,7 +1209,7 @@ impl Discovery { let copy = self .unpatched_copies .iter() - .find(|c| c.purl == r.purl && c.file == r.source_file) + .find(|c| c.purl == r.purl && c.file == r.source_file && c.part == r.part) .cloned(); match copy { Some(c) => { @@ -1199,15 +1246,16 @@ impl Discovery { /// rule to every pair of locks. PEP 723 script locks (`*.py.lock`) /// neither contest nor are contested: each is scoped to its own script's /// install. Neither is a pnpm env lockfile document - /// ([`pnpm_document_tree`]): pnpm installs its config dependencies into - /// its config store whatever else runs. Files of one install tree ([`Discovery::install_trees`]) are - /// one lock here: pip resolves a direct reference and a compatible `==` - /// pin of the same version in `requirements.txt` and its `-r` includes - /// to the reference (#1086). + /// ([`TreePart::PnpmEnv`]): pnpm installs its config dependencies into + /// its config store whatever else runs, and its wiring exempts no + /// other tree. Files of one install tree ([`Discovery::install_trees`]) + /// are one lock here: pip resolves a direct reference and a compatible + /// `==` pin of the same version in `requirements.txt` and its `-r` + /// includes to the reference (#1086). fn contest_across_locks(&mut self) { // The purls each install tree wires. let mut wiring: BTreeMap> = BTreeMap::new(); - for r in &self.refs { + for r in self.refs.iter().filter(|r| r.part == TreePart::File) { wiring .entry(self.tree_of(&r.source_file).to_path_buf()) .or_default() @@ -1225,7 +1273,7 @@ impl Discovery { let mut contested = Vec::new(); let refs = std::mem::take(&mut self.refs); for r in refs { - let other = (!is_script_lock(&r.source_file) && !is_pnpm_env_tree(&r.source_file)) + let other = (!is_script_lock(&r.source_file) && r.part == TreePart::File) .then(|| { let tree = self.tree_of(&r.source_file); // Only a PyPI ref can match another spelling of its @@ -1250,7 +1298,7 @@ impl Discovery { // other tree. e.covers(&r.purl) && other != tree - && !is_pnpm_env_tree(&e.file) + && e.part == TreePart::File && !wiring.get(other).is_some_and(|purls| { purls.contains(&r.purl) || (pypi @@ -1306,39 +1354,33 @@ impl Discovery { } } - /// Map every [`pnpm_document_tree`] key back to its real lock file once - /// the contests that needed the trees apart have run: consumers read, - /// match and display `source_file` / `file` as root-relative paths. - /// Detail texts keep the key's label, which names the document. - fn resolve_document_trees(&mut self) { - fn real(path: &mut PathBuf) { - if let Some(file) = path - .to_str() - .and_then(|p| p.strip_suffix(PNPM_ENV_TREE_SUFFIX)) - { - *path = PathBuf::from(file); - } - } - for r in self - .refs - .iter_mut() - .chain(&mut self.shadowed) - .chain(&mut self.rewirable) - { - real(&mut r.source_file); - } - self.elsewhere.iter_mut().for_each(|e| real(&mut e.file)); - self.unpatched_copies - .iter_mut() - .for_each(|c| real(&mut c.file)); - self.unattested.iter_mut().for_each(|u| real(&mut u.file)); - for c in &mut self.contested { - real(&mut c.file); - real(&mut c.other); - } - self.diagnostics.iter_mut().for_each(|d| real(&mut d.file)); - self.recognized.iter_mut().for_each(|r| real(&mut r.file)); - self.withheld.iter_mut().for_each(|r| real(&mut r.file)); + /// Drop a pnpm env-lockfile ref ([`TreePart::PnpmEnv`]) that the same + /// file's project document also wires (the same `(purl, uuid, mode)`): + /// one wiring of the package in that file, whose project instance is + /// the one an installed `node_modules` copy is verified against, even + /// when the two pins differ. + fn drop_env_twins(refs: &mut Vec) { + let project: HashSet<(PathBuf, String, String, WiringMode)> = refs + .iter() + .filter(|r| r.part == TreePart::File) + .map(|r| { + ( + r.source_file.clone(), + r.purl.clone(), + r.uuid.clone(), + r.mode, + ) + }) + .collect(); + refs.retain(|r| { + r.part == TreePart::File + || !project.contains(&( + r.source_file.clone(), + r.purl.clone(), + r.uuid.clone(), + r.mode, + )) + }); } /// The install tree `file` belongs to: its tree's root file, or itself. @@ -1496,7 +1538,6 @@ impl Discovery { } fn finalize(&mut self) { - self.resolve_document_trees(); self.elsewhere.sort(); self.elsewhere.dedup(); self.unpatched_copies.sort(); @@ -1506,12 +1547,14 @@ impl Discovery { self.contested.sort(); self.contested.dedup(); for refs in [&mut self.refs, &mut self.shadowed, &mut self.rewirable] { + Self::drop_env_twins(refs); refs.sort_by(|a, b| { - (&a.source_file, &a.purl, &a.uuid, a.mode).cmp(&( + (&a.source_file, &a.purl, &a.uuid, a.mode, a.part).cmp(&( &b.source_file, &b.purl, &b.uuid, b.mode, + b.part, )) }); refs.dedup(); @@ -1677,13 +1720,6 @@ fn is_script_lock(file: &Path) -> bool { crate::utils::python_lock::is_script_lock_name(&file.to_string_lossy()) } -/// Whether `file` is a [`pnpm_document_tree`] key of a pnpm env lockfile -/// document (before [`Discovery::finalize`] maps it back). -fn is_pnpm_env_tree(file: &Path) -> bool { - file.to_str() - .is_some_and(|f| f.ends_with(PNPM_ENV_TREE_SUFFIX)) -} - /// What every extractor receives: the project view plus the hosted-origin /// allowlist, with the guarded-read and identity helpers bolted on. pub(crate) struct DiscoverCtx<'a> { diff --git a/crates/socket-patch-core/src/vex/discover/npm.rs b/crates/socket-patch-core/src/vex/discover/npm.rs index 567050efa..96d63c64e 100644 --- a/crates/socket-patch-core/src/vex/discover/npm.rs +++ b/crates/socket-patch-core/src/vex/discover/npm.rs @@ -53,9 +53,9 @@ use std::path::PathBuf; use serde_json::Value; use super::{ - npm_purl, npm_vendored_tarball_names, parse_json, pnpm_document_tree, vendor_ref, CopyTarget, - DiscoverCtx, Discovery, LocateOpts, Located, PatchedRef, UnattestedKind, UnwiredCopy, - VendorRef, DIAG_LOCKFILE_UNPARSEABLE, DIAG_REF_INVALID, DIAG_REF_UNATTRIBUTABLE, + npm_purl, npm_vendored_tarball_names, parse_json, vendor_ref, CopyTarget, DiscoverCtx, + Discovery, LocateOpts, Located, PatchedRef, TreePart, UnattestedKind, UnwiredCopy, VendorRef, + DIAG_LOCKFILE_UNPARSEABLE, DIAG_REF_INVALID, DIAG_REF_UNATTRIBUTABLE, }; use crate::constants::npm_family::{NPM_LOCKS, PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY}; use crate::formats::pnpm::{ @@ -645,7 +645,7 @@ fn entry_ref( /// Every document of a lock is read. pnpm 11+'s env lockfile document /// (config dependencies, the pinned package manager; identified by its /// importers, [`PnpmLock::packages`]) is an install tree of its own -/// ([`pnpm_document_tree`]): its refs are emitted, and they never contest +/// ([`TreePart::PnpmEnv`]): its refs are emitted, and they never contest /// or are contested by the project document or another lock. /// /// ## Entries @@ -755,19 +755,39 @@ async fn extract_pnpm_lock( return; } // Each document is its own install tree: pnpm 11+'s env lockfile is - // read under its own tree key ([`pnpm_document_tree`]), so its refs and - // the project document's entries never contest each other. + // read as [`TreePart::PnpmEnv`], so its refs and the project document's + // entries never contest each other. for document in lock.documents() { - let tree = pnpm_document_tree(file, document); + let tree = PnpmTree::of(file, document); extract_pnpm_document(ctx, &tree, &lock, document, disabled, out).await; } } -/// The refs and copies of one document of a pnpm lock, recorded under the -/// document's install-tree key `file` (see [`extract_pnpm_lock`]). +/// One install tree of a pnpm lock: the lock's root-relative `file`, the +/// [`TreePart`] its records carry, and the `label` its diagnostics name it +/// by (` (env lockfile)` for the env document). +struct PnpmTree<'a> { + file: &'a str, + part: TreePart, + label: String, +} + +impl<'a> PnpmTree<'a> { + fn of(file: &'a str, document: PnpmDocument) -> Self { + let part = TreePart::from(document); + let label = match part { + TreePart::File => file.to_string(), + TreePart::PnpmEnv => format!("{file} (env lockfile)"), + }; + PnpmTree { file, part, label } + } +} + +/// The refs and copies of one document of a pnpm lock, recorded in its +/// install tree (see [`extract_pnpm_lock`]). async fn extract_pnpm_document( ctx: &DiscoverCtx<'_>, - file: &str, + tree: &PnpmTree<'_>, lock: &PnpmLock<'_>, document: PnpmDocument, disabled: Option<&str>, @@ -778,17 +798,18 @@ async fn extract_pnpm_document( let mut withheld: BTreeSet = BTreeSet::new(); for package in lock.packages_of(document) { let withhold = disabled.map(|_| &mut withheld); - pnpm_entry_ref(ctx, file, package, &mut copies, withhold, out); + pnpm_entry_ref(ctx, tree, package, &mut copies, withhold, out); bundles.extend(entry_bundled(&package.entry).map(|b| (package.key, b))); } - record_pnpm_file_copies(ctx, file, copies, out).await; - record_pnpm_bundled_copies(file, bundles, out); + record_pnpm_file_copies(ctx, tree, copies, out).await; + record_pnpm_bundled_copies(tree, bundles, out); + let label = &tree.label; if let (Some(source), false) = (disabled, withheld.is_empty()) { out.diag( DIAG_REF_UNATTRIBUTABLE, - file, + tree.file, format!( - "{file}: {}, so its hosted pins of {} are not attested from the lock alone, \ + "{label}: {}, so its hosted pins of {} are not attested from the lock alone, \ only from an installed tree whose every copy verifies; \ `socket-patch scan --mode vendored` wires them without a lock", crate::utils::pnpm_workspace::describe_lockfile_disabled(source), @@ -814,7 +835,12 @@ async fn extract_pnpm_document( /// stay refs: the wiring is intact and no rewire could clear the bundled /// copy, so the ledgers' liveness gates (`vendor --check`, `scan`) keep /// seeing them live, and the copy is no cross-lock evidence either. -fn record_pnpm_bundled_copies(file: &str, bundles: Vec<(&str, Bundled<'_>)>, out: &mut Discovery) { +fn record_pnpm_bundled_copies( + tree: &PnpmTree<'_>, + bundles: Vec<(&str, Bundled<'_>)>, + out: &mut Discovery, +) { + let (file, label) = (tree.file, &tree.label); for (parent, bundled) in bundles { let (targets, what) = match bundled { Bundled::All => ( @@ -831,11 +857,11 @@ fn record_pnpm_bundled_copies(file: &str, bundles: Vec<(&str, Bundled<'_>)>, out }; for target in targets { out.unwired_copy(UnwiredCopy { - scope: Some(PathBuf::from(file)), + scope: Some((PathBuf::from(file), tree.part)), target, file: PathBuf::from(file), detail: format!( - "{file} entry `{parent}` {what}, which pnpm unpacks from that package's \ + "{label} entry `{parent}` {what}, which pnpm unpacks from that package's \ own tarball without locking its version, so no Socket wiring reaches it" ), kind: UnattestedKind::BundledCopy, @@ -865,11 +891,11 @@ struct PnpmFileCopy { /// read is left alone. async fn record_pnpm_file_copies( ctx: &DiscoverCtx<'_>, - file: &str, + tree: &PnpmTree<'_>, copies: Vec, out: &mut Discovery, ) { - let lock_dir = std::path::Path::new(file) + let lock_dir = std::path::Path::new(tree.file) .parent() .map(|d| d.to_string_lossy().replace('\\', "/")) .unwrap_or_default(); @@ -919,8 +945,9 @@ async fn record_pnpm_file_copies( } else { "tarball" }; - out.unpatched_copy( - file, + out.unpatched_copy_in( + tree.file, + tree.part, npm_purl(&name, &version), ©.key, &format!( @@ -977,7 +1004,8 @@ pub(crate) async fn pnpm_lock_file_copies( })); } let mut scratch = Discovery::default(); - record_pnpm_file_copies(&ctx, file, copies, &mut scratch).await; + let tree = PnpmTree::of(file, PnpmDocument::Project); + record_pnpm_file_copies(&ctx, &tree, copies, &mut scratch).await; let purl = crate::utils::purl_key::canonical_base_purl(&purl); scratch .unpatched_copies @@ -1009,12 +1037,13 @@ fn pnpm_file_copy(package: &PnpmPackage<'_>, directory: bool) -> Option, - file: &str, + tree: &PnpmTree<'_>, package: &PnpmPackage<'_>, copies: &mut Vec, withheld: Option<&mut BTreeSet>, out: &mut Discovery, ) { + let (file, label) = (tree.file, &tree.label); let key = package.key; let Some(resolution) = &package.resolution else { if resolution_is_socket_shaped(ctx, package) { @@ -1022,7 +1051,7 @@ fn pnpm_entry_ref( DIAG_REF_INVALID, file, format!( - "{file}: packages entry {key:?} references a Socket patch through a \ + "{label}: packages entry {key:?} references a Socket patch through a \ resolution that is not a flat pnpm resolution map; it is ignored" ), ); @@ -1031,7 +1060,7 @@ fn pnpm_entry_ref( }; let Some(tarball) = resolution.tarball() else { // A plain registry entry (integrity only) or a directory/git dep. - out.resolved_elsewhere(file, pnpm_registry_key_purl(key)); + out.resolved_elsewhere_in(file, tree.part, pnpm_registry_key_purl(key)); copies.extend(pnpm_file_copy(package, true)); return; }; @@ -1041,14 +1070,14 @@ fn pnpm_entry_ref( .map(|sri| LockIntegrity::Sri(sri.to_string())); let located = ctx.locate(tarball, LocateOpts::LITERAL_CHECKED); if let Some(vref) = located.vendored { - pnpm_vendored_ref(file, package, tarball, &vref, integrity, out); + pnpm_vendored_ref(tree, package, tarball, &vref, integrity, out); } else if located.decorated_leaf { // pnpm fetches the literal file (`#` / `::` included). out.diag( DIAG_REF_INVALID, file, format!( - "{file}: packages entry {key:?} (tarball {tarball:?}) is not a literal \ + "{label}: packages entry {key:?} (tarball {tarball:?}) is not a literal \ .socket/vendor/npm// path; it is ignored" ), ); @@ -1058,7 +1087,7 @@ fn pnpm_entry_ref( DIAG_REF_INVALID, file, format!( - "{file}: packages entry {key:?} resolves from the Socket patch server but \ + "{label}: packages entry {key:?} resolves from the Socket patch server but \ its key is not a registry name@version" ), ); @@ -1072,25 +1101,20 @@ fn pnpm_entry_ref( } None => integrity, }; - out.push(PatchedRef::hosted( - purl, - uuid, - file, - Some(tarball), - integrity, - true, - )); + out.push( + PatchedRef::hosted(purl, uuid, file, Some(tarball), integrity, true).in_part(tree.part), + ); } else { // A registry-keyed entry fetching some other tarball, or a user's // `file:` tarball. - out.resolved_elsewhere(file, pnpm_registry_key_purl(key)); + out.resolved_elsewhere_in(file, tree.part, pnpm_registry_key_purl(key)); copies.extend(pnpm_file_copy(package, false)); } } /// A vendored packages entry (see [`extract_pnpm`] for the key shapes). fn pnpm_vendored_ref( - file: &str, + tree: &PnpmTree<'_>, package: &PnpmPackage<'_>, tarball: &str, vref: &VendorRef, @@ -1101,8 +1125,11 @@ fn pnpm_vendored_ref( let invalid = |out: &mut Discovery, why: &str| { out.diag( DIAG_REF_INVALID, - file, - format!("{file}: packages entry {key:?} (tarball {tarball:?}) {why}; it is ignored"), + tree.file, + format!( + "{}: packages entry {key:?} (tarball {tarball:?}) {why}; it is ignored", + tree.label + ), ); }; // (name, the key's own `file:` path if the key was rekeyed). @@ -1113,7 +1140,7 @@ fn pnpm_vendored_ref( PnpmKey::V9File { name, path } => (Some(name), Some(path)), // Registry-form key with a hand-pointed `file:` tarball. PnpmKey::Registry { name, version } => { - return finish_vendored(file, name, version, vref, integrity, invalid, out) + return finish_vendored(tree, name, version, vref, integrity, invalid, out) } PnpmKey::Other => return invalid(out, "has no registry name@version key"), }; @@ -1128,11 +1155,11 @@ fn pnpm_vendored_ref( let Some(version) = entry_field(&package.entry, "version") else { return invalid(out, "has no (single) `version:` line"); }; - finish_vendored(file, name, version, vref, integrity, invalid, out) + finish_vendored(tree, name, version, vref, integrity, invalid, out) } fn finish_vendored( - file: &str, + tree: &PnpmTree<'_>, name: &str, version: &str, vref: &VendorRef, @@ -1149,7 +1176,7 @@ fn finish_vendored( &format!("names {purl} but its tarball is not that package's vendored npm tarball"), ); } - out.push(PatchedRef::vendored(purl, vref, file, integrity)); + out.push(PatchedRef::vendored(purl, vref, tree.file, integrity).in_part(tree.part)); } /// [`pnpm_registry_key`] (the lock inventory's key grammar) as a validated @@ -3171,10 +3198,13 @@ mod tests { ); let out = p.discover().await; assert_refs(&out, &hosted); + // The real lock path, with the tree carried beside it + // (code-review finding 6: no pseudo path). assert_eq!( out.refs[0].source_file, std::path::Path::new("pnpm-lock.yaml") ); + assert_eq!(out.refs[0].part, TreePart::PnpmEnv, "{case}"); assert!(out.unattested.is_empty(), "{case}: {:#?}", out.unattested); assert!(out.shadowed.is_empty(), "{case}: {:#?}", out.shadowed); assert!(out.contested.is_empty(), "{case}: {:#?}", out.contested); @@ -3195,8 +3225,26 @@ mod tests { ); let out = p.discover().await; assert_refs(&out, &hosted); + assert_eq!(out.refs[0].part, TreePart::File); assert!(out.contested.is_empty(), "{:#?}", out.contested); + // Wired in both documents, the pins differing: ONE ref, the project + // document's (code-review finding 6: no twin refs of one file). + let env_wired = wired.replace(SRI, "sha512-ENVPIN=="); + let p = Project::new(); + p.write( + "pnpm-lock.yaml", + format!("{}{}", env(&env_wired), project(&wired)), + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert_eq!(out.refs.len(), 1, "{:#?}", out.refs); + assert_eq!(out.refs[0].part, TreePart::File); + assert_eq!( + out.refs[0].locked_integrity, + Some(LockIntegrity::Sri(SRI.into())) + ); + // The env document's registry entry contests no other lock's ref. let npm_lock = format!( r#"{{"name":"m","lockfileVersion":3,"packages":{{"":{{"name":"m"}}, diff --git a/crates/socket-patch-core/src/vex/discover/testing/golden.rs b/crates/socket-patch-core/src/vex/discover/testing/golden.rs index 54509be0e..589d75633 100644 --- a/crates/socket-patch-core/src/vex/discover/testing/golden.rs +++ b/crates/socket-patch-core/src/vex/discover/testing/golden.rs @@ -37,7 +37,7 @@ use std::path::{Path, PathBuf}; use serde_json::{json, Map, Value}; use crate::vex::discover::{ - ContestedRef, Diag, Discovery, PatchedRef, Recognized, ResolvedElsewhere, Unattested, + ContestedRef, Diag, Discovery, PatchedRef, Recognized, ResolvedElsewhere, TreePart, Unattested, UnlockedPin, UnpatchedCopy, WiringMode, }; @@ -150,8 +150,9 @@ fn render(out: &Discovery, root: &Path) -> Value { url, // The block's lines; the lock itself is the golden's input. yarn_classic_block: _, + part, } = r; - json!({ + let mut rendered = json!({ "purl": purl, "uuid": uuid, "mode": mode(*m), @@ -161,7 +162,13 @@ fn render(out: &Discovery, root: &Path) -> Value { "integrity_required": integrity_required, "url": url, "lockfile_basis_ok": r.lockfile_basis_ok(), - }) + }); + // Only off the default, so the goldens of single-tree files stay + // byte-identical. + if *part != TreePart::File { + rendered["part"] = json!(format!("{part:?}")); + } + rendered }; let refs: Vec = refs.iter().map(render_ref).collect(); let diagnostics: Vec = diagnostics @@ -255,6 +262,8 @@ fn render(out: &Discovery, root: &Path) -> Value { key, how, remedy, + // A pnpm env-lockfile copy contests only its own tree. + part: _, } = c; let mut copy = json!({ "purl": purl, @@ -301,6 +310,7 @@ fn render(out: &Discovery, root: &Path) -> Value { key, how, remedy, + part: _, } = c; let mut copy = json!({ "purl": purl, diff --git a/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden b/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden index 29f1efde0..5b9361677 100644 --- a/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden +++ b/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden @@ -1,3 +1,3 @@ # A depscan-sized seeded pnpm-lock.yaml set + shuffled overrides. # -0 342d79c926a00044 2cf6f312234e59ca +0 b120df27c737dd90 1ab3b36d03d3b3f8 diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md index 9f486b460..a25f67760 100644 --- a/docs/design/lock-model-divergences.md +++ b/docs/design/lock-model-divergences.md @@ -82,7 +82,7 @@ follow from the install-tree decision. | N1 | pnpm `gitBranchLockfile` with a branch lock present: is the root lock the install tree? | Hosted withholds the root locks and refuses (`hosted/engine.rs` ~930). Vendored refuses. Discovery and the inventory never check the setting. | Hosted | false attestation | | N2 | npm user patches (#711) and `bun patch` (#367) | Hosted skips that one dependency and records it as refused. Vendored refuses the whole npm v4 lock, or the one bun dependency. Discovery never reads `patchedDependencies` / `"patched"` and attests a pre-#711 hosted pin. | Hosted's per-dependency skip, in discovery too | false attestation | | N3 | A workspace member's stray npm/bun/vlt lock | Discovery treats it as absent. Hosted and vendored refuse. The inventory reads it as the project's lock. | The inventory ignores it | | -| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by content, not position. **Resolved in PR 2a**: every reader reads both documents through `PnpmLock`, each entry tagged by document; discovery keys the env document as the tree ` (env lockfile)`, which neither contests nor is contested by the project document or another lock; a document is the env one when its importers carry only `configDependencies` / `packageManagerDependencies`. | | +| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by content, not position. **Resolved in PR 2a**: every reader reads both documents through `PnpmLock`, each entry tagged by document; discovery records carry the env document's install tree (`TreePart::PnpmEnv`) beside the real lock path, and that tree neither contests nor is contested by the project document or another lock; the env document is the FIRST document when its importers carry only `configDependencies` / `packageManagerDependencies` (pnpm's own position rule), and every other document is read as the project. | | | N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere. pnpm part **resolved in PR 2a**: the hosted pnpm "vendored?" check reads the rekeyed packages entry with `origin::vendor_ref`. | revert could rewrite another project's wiring | | N6 | `npm-shrinkwrap.json` and `package-lock.json` | Hosted, vendored and discovery read both, with a contest. The inventory reads the shrinkwrap only. | Both, everywhere | | | N7 | npm lockfileVersion | Hosted rewrites v1 and v4. Vendored accepts v2/v3 only. The inventory ignores v1. Discovery reads v1 and v2+. | Hosted, once vendored `file:` wiring under npm 6 and npm 12.1 v4 has fixtures | | diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json index c73793400..b31d39de7 100644 --- a/scripts/loc-budget.json +++ b/scripts/loc-budget.json @@ -17,7 +17,7 @@ "crates/socket-patch-core/src/utils/pipenv.rs", "crates/socket-patch-core/src/crawlers/deno_crawler.rs" ], - "max": 31117 + "max": 31207 }, "lock-writers": { "paths": [ @@ -30,7 +30,7 @@ "crates/socket-patch-core/src/vendor/npm_origin.rs", "crates/socket-patch-core/src/vendor/test_support" ], - "max": 88313 + "max": 88314 } } } From ad40d1dbe71f9e840215334d2e129734975ac647 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Thu, 5 Nov 2026 13:23:55 -0600 Subject: [PATCH 13/36] Plan pnpm rewires with one core for both modes Add formats/pnpm/plan.rs, the planning core hosted and vendored mode share. They now differ only in where a rewired entry points: * Target::check_lock: one lock-version table. Hosted takes every lock but pnpm 1's early shrinkwrap. Vendored takes 5.4 / 6.0 / 9.0 and routes each dialect from it. This replaces unsupported_early_shrinkwrap, check_v9_lock_version and the legacy check_lock_version. * match_key: one lookup of how a key names name@version, used by the hosted lookup, the hosted "vendored?" hint, and both vendored dialects' target, split-entry and rekey checks. It finds the registry entry with its peer suffix, or a vendored rekey read with origin::vendor_ref plus the artifact leaf. * Target::repoints: the all-or-nothing gate's per-instance predicate. Hosted splices any parsable peer context; vendored rekeys the plain entry only. Within the vendored planner, the v9 and legacy dialects now share the dep-pair revert (DepPair), the block revert (revert_block, re-sorting legacy's bare file: rekey), target_blocks, and dep_entries. dep_entries also serves the importer edit, the 6.0 root-dep edit, the ref gate and the lock index. The ref gate's index and scan paths run one set of predicates. Behavior changes, each pinned by a test: * N15: the vendored planner and reverter edit a CRLF pnpm-lock.yaml or pnpm-workspace.yaml like its LF twin and write it back CRLF. Only a file mixing CRLF and LF is still refused (vendor_lockfile_crlf_unsupported). * N5 (vendored planner): a sibling checkout's ../.socket/vendor rekey is no longer taken for this project's entry. * The hosted "vendored" hint now judges a rekey by its artifact leaf, as the vendored planner does, instead of the entry's version: line. N16 is kept as a target-specific refusal: no lock version beyond 5.4 / 6.0 / 9.0 has a captured fixture that proves the vendored rekey. Co-Authored-By: Claude Opus 5.5 --- .../tests/in_process_vendor_pnpm_takeover.rs | 59 +- .../src/formats/pnpm/grammar.rs | 32 - .../src/formats/pnpm/hosted.rs | 86 +- .../socket-patch-core/src/formats/pnpm/mod.rs | 86 +- .../src/formats/pnpm/plan.rs | 201 +++ .../src/patch/redirect/mod.rs | 22 + crates/socket-patch-core/src/vendor/mod.rs | 15 +- .../src/vendor/npm_common.rs | 2 +- .../socket-patch-core/src/vendor/pnpm_lock.rs | 1145 +++++++++-------- .../src/vendor/pnpm_lock_legacy.rs | 366 ++---- 10 files changed, 1007 insertions(+), 1007 deletions(-) create mode 100644 crates/socket-patch-core/src/formats/pnpm/plan.rs diff --git a/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs b/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs index 7d4bcc49d..60ff577bd 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs @@ -1,7 +1,7 @@ //! Hermetic hosted → vendored takeover tests through the built binary for //! pnpm projects whose shape the pnpm vendored backend refuses although //! hosted mode accepts it (#853): a `catalog:` dependency -//! (`vendor_lock_entry_unsupported`), a CRLF `pnpm-lock.yaml` +//! (`vendor_lock_entry_unsupported`), a `pnpm-lock.yaml` mixing CRLF and LF //! (`vendor_lockfile_crlf_unsupported`) and a conflicting user override //! (a range) in `pnpm-workspace.yaml` (`vendor_override_conflict`). //! @@ -54,6 +54,8 @@ enum Shape { Catalog, /// A plain dependency whose lock is converted to CRLF after hosting. Crlf, + /// A plain dependency whose lock mixes CRLF and LF after hosting. + MixedEol, /// A plain dependency plus a conflicting user /// `overrides: { left-pad: ^1.3.0 }` in pnpm-workspace.yaml. Override, @@ -90,7 +92,7 @@ fn write_pnpm_project(root: &Path, shape: Shape) { Shape::Catalog => format!("packages:\n - .\ncatalog:\n {NAME}: {VERSION}\n"), Shape::Override => format!("overrides:\n {NAME}: ^{VERSION}\n"), Shape::ExactPin => format!("overrides:\n {NAME}: {VERSION}\n"), - Shape::Crlf | Shape::Plain | Shape::TwoDocument => String::new(), + Shape::Crlf | Shape::MixedEol | Shape::Plain | Shape::TwoDocument => String::new(), }; if !workspace.is_empty() { std::fs::write(root.join("pnpm-workspace.yaml"), workspace).unwrap(); @@ -117,7 +119,7 @@ fn write_pnpm_project(root: &Path, shape: Shape) { )), Shape::Override => lock.push_str(&format!("overrides:\n {NAME}: ^{VERSION}\n\n")), Shape::ExactPin => lock.push_str(&format!("overrides:\n {NAME}: {VERSION}\n\n")), - Shape::Crlf | Shape::Plain | Shape::TwoDocument => {} + Shape::Crlf | Shape::MixedEol | Shape::Plain | Shape::TwoDocument => {} } let specifier = match shape { Shape::Catalog => "'catalog:'".to_string(), @@ -305,7 +307,8 @@ fn snapshot(root: &Path) -> Vec<(&'static str, Option>)> { /// The hosted project: pristine lock, then a real `scan --mode hosted` /// (and, for [`Shape::Crlf`], the lock converted to CRLF as a -/// `core.autocrlf` checkout would leave it). Returns the hosted wiring. +/// `core.autocrlf` checkout would leave it; for [`Shape::MixedEol`], its +/// first line only). Returns the hosted wiring. fn host_project(root: &Path, api: &str, shape: Shape) -> Vec<(&'static str, Option>)> { write_pnpm_project(root, shape); let (code, env) = run_mode(root, api, "scan", "hosted", &[]); @@ -316,8 +319,10 @@ fn host_project(root: &Path, api: &str, shape: Shape) -> Vec<(&'static str, Opti lock.contains(HOSTED_URL), "hosted mode must pin the {shape:?} lock entry:\n{lock}\n{env:#}" ); - if matches!(shape, Shape::Crlf) { - std::fs::write(&lock_path, lock.replace('\n', "\r\n")).unwrap(); + match shape { + Shape::Crlf => std::fs::write(&lock_path, lock.replace('\n', "\r\n")).unwrap(), + Shape::MixedEol => std::fs::write(&lock_path, lock.replacen('\n', "\r\n", 1)).unwrap(), + _ => {} } snapshot(root) } @@ -402,17 +407,21 @@ async fn get_vendored_over_hosted_pnpm_catalog_dep_keeps_the_hosted_pin() { refused_takeover_keeps_hosted_pin(Shape::Catalog, "get", "vendor_lock_entry_unsupported").await; } -/// #853: a CRLF `pnpm-lock.yaml` (a `core.autocrlf` checkout). +/// #853: a `pnpm-lock.yaml` mixing CRLF and LF. (A CRLF lock, a +/// `core.autocrlf` checkout, takes over since lock-model 2b: see +/// [`scan_vendored_over_hosted_pnpm_crlf_lock_takes_over_and_stays_crlf`].) #[tokio::test(flavor = "multi_thread")] -async fn scan_vendored_over_hosted_pnpm_crlf_lock_keeps_the_hosted_pin() { - refused_takeover_keeps_hosted_pin(Shape::Crlf, "scan", "vendor_lockfile_crlf_unsupported") +async fn scan_vendored_over_hosted_pnpm_mixed_eol_lock_keeps_the_hosted_pin() { + refused_takeover_keeps_hosted_pin(Shape::MixedEol, "scan", "vendor_lockfile_crlf_unsupported") .await; } -/// #853: `get --mode vendored` over a CRLF lock (a project-level refusal). +/// #853: `get --mode vendored` over a mixed-ending lock (a project-level +/// refusal). #[tokio::test(flavor = "multi_thread")] -async fn get_vendored_over_hosted_pnpm_crlf_lock_keeps_the_hosted_pin() { - refused_takeover_keeps_hosted_pin(Shape::Crlf, "get", "vendor_lockfile_crlf_unsupported").await; +async fn get_vendored_over_hosted_pnpm_mixed_eol_lock_keeps_the_hosted_pin() { + refused_takeover_keeps_hosted_pin(Shape::MixedEol, "get", "vendor_lockfile_crlf_unsupported") + .await; } /// #853: a conflicting user override (a range) in `pnpm-workspace.yaml`. @@ -522,6 +531,32 @@ async fn scan_vendored_over_hosted_pnpm_plain_dep_still_takes_over() { ); } +/// Lock-model 2b (divergence N15): a CRLF `pnpm-lock.yaml` (a +/// `core.autocrlf` checkout) takes over like its LF twin, and the vendored +/// lock keeps its CRLF endings. It used to be refused +/// (`vendor_lockfile_crlf_unsupported`) and keep the hosted pin. +#[tokio::test(flavor = "multi_thread")] +async fn scan_vendored_over_hosted_pnpm_crlf_lock_takes_over_and_stays_crlf() { + let server = MockServer::start().await; + mock_api(&server).await; + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + host_project(root, &server.uri(), Shape::Crlf); + + let (exit, env) = run_mode(root, &server.uri(), "scan", "vendored", &[]); + assert_eq!(exit, 0, "the CRLF takeover must succeed: {env:#}"); + let lock = std::fs::read_to_string(root.join("pnpm-lock.yaml")).unwrap(); + assert!(!lock.contains(HOSTED_URL), "{lock}"); + assert!( + lock.contains(&format!(".socket/vendor/npm/{UUID}/")), + "the lock must point at the vendored artifact:\n{lock}" + ); + assert!( + !lock.replace("\r\n", "").contains('\n'), + "the lock keeps its CRLF endings: {lock:?}" + ); +} + /// #854: a user exact pin equal to the vendored version in /// `pnpm-workspace.yaml` (the pnpm 10.5+/11/12 override map) is taken over /// like the same pin in package.json: the workspace value becomes the diff --git a/crates/socket-patch-core/src/formats/pnpm/grammar.rs b/crates/socket-patch-core/src/formats/pnpm/grammar.rs index 3ba09bf18..9f7ce6e4e 100644 --- a/crates/socket-patch-core/src/formats/pnpm/grammar.rs +++ b/crates/socket-patch-core/src/formats/pnpm/grammar.rs @@ -13,21 +13,6 @@ use crate::formats::text::strip_bom; use super::lines::ENV_DOC_IMPORTER_FIELDS; -/// Early pnpm 1 writes shrinkwrapVersion 3 without a minor version and -/// unconditionally drops registry tarball URLs on install. Its frozen flag -/// cannot preserve this redirect (verified with pnpm 1.0.0). -pub(crate) fn unsupported_early_shrinkwrap(content: &str) -> bool { - let content = strip_bom(content); - let version = content - .lines() - .find_map(|line| line.strip_prefix("shrinkwrapVersion:")); - let minor = content - .lines() - .find_map(|line| line.strip_prefix("shrinkwrapMinorVersion:")); - version.is_some_and(|v| v.trim().trim_matches(['\'', '"']) == "3") - && !minor.is_some_and(|v| v.trim().parse::().is_ok_and(|v| v > 0)) -} - pub(crate) struct Entry<'a> { pub key: &'a str, pub body: &'a str, @@ -270,23 +255,6 @@ pub(crate) fn suffix<'a>(key: &'a str, name: &str, version: &str) -> Option<&'a Some(suffix) } -pub(crate) fn supported_suffix(suffix: &str) -> bool { - if suffix.is_empty() || suffix.starts_with('_') { - return true; - } - let mut depth = 0usize; - for c in suffix.chars() { - match c { - '(' => depth += 1, - ')' if depth > 0 => depth -= 1, - ')' => return false, - _ if depth == 0 => return false, - _ => {} - } - } - depth == 0 -} - pub(crate) struct Resolution<'a> { pub range: Range, pub fields: Vec<(&'a str, &'a str)>, diff --git a/crates/socket-patch-core/src/formats/pnpm/hosted.rs b/crates/socket-patch-core/src/formats/pnpm/hosted.rs index 84f2524e2..14f8e05bb 100644 --- a/crates/socket-patch-core/src/formats/pnpm/hosted.rs +++ b/crates/socket-patch-core/src/formats/pnpm/hosted.rs @@ -11,6 +11,7 @@ use serde_json::Value; use crate::patch::redirect::{full_name, DepOverride, FileEdit, RewriteResult, RewriteWarning}; use super::grammar as pnpm; +use super::plan::{match_key, KeyMatch, Target}; /// Whether `entry` resolves to exactly `artifact_url` — the per-instance /// residual-gate predicate. @@ -42,11 +43,8 @@ struct PnpmLockState<'f> { /// Entry indices sorted by normalized (unquoted, `/`-stripped) key. sorted: Vec, pending: Vec<(std::ops::Range, String)>, - spliced: std::collections::HashSet, + spliced: HashSet, changed: bool, - /// [`vendored_packages`] of `text`, computed on the first unmatched dep - /// and dropped when [`PnpmLockState::materialize`] replaces `text`. - vendored: Option>, } impl<'f> PnpmLockState<'f> { @@ -54,13 +52,12 @@ impl<'f> PnpmLockState<'f> { let mut state = PnpmLockState { path, text: Cow::Borrowed(text), - early_shrinkwrap: pnpm::unsupported_early_shrinkwrap(text), + early_shrinkwrap: Target::Hosted.check_lock(text).is_err(), entries: Vec::new(), sorted: Vec::new(), pending: Vec::new(), spliced: Default::default(), changed: false, - vendored: None, }; state.reindex(); state @@ -84,17 +81,6 @@ impl<'f> PnpmLockState<'f> { self.sorted = sorted; } - /// Whether the lock vendors `fname@version` ([`vendored_packages`]). - /// Read from `text` without folding `pending` in: a resolution splice - /// never touches a packages key or a `name:` / `version:` line (see - /// the type docs), so the answer is the post-splice lock's. - fn vendors(&mut self, fname: &str, version: &str) -> bool { - let text: &str = &self.text; - self.vendored - .get_or_insert_with(|| vendored_packages(text)) - .contains(&(fname.to_string(), version.to_string())) - } - fn entry(&self, i: usize) -> pnpm::Entry<'_> { let (key, body) = &self.entries[i]; pnpm::Entry { @@ -111,20 +97,32 @@ impl<'f> PnpmLockState<'f> { } /// Entries whose key names `fname@version` (any suffix), in file order — - /// the same set a full [`pnpm::suffix`] scan of the logical lock yields. + /// the same set a full [`match_key`] scan of the logical lock yields. fn hits(&mut self, fname: &str, version: &str) -> Vec { - let hits = self.lookup(fname, version); + let hits = self.lookup(fname, version, false); if hits.iter().any(|i| self.spliced.contains(i)) { self.materialize(); - return self.lookup(fname, version); + return self.lookup(fname, version, false); } hits } - fn lookup(&self, fname: &str, version: &str) -> Vec { + /// Whether a vendored rewire rekeyed `fname@version` + /// ([`KeyMatch::Rekeyed`]). Read from `text` without folding `pending` + /// in: a resolution splice never touches a key (see the type docs). + fn vendors(&self, fname: &str, version: &str) -> bool { + !self.lookup(fname, version, true).is_empty() + } + + /// The entries [`match_key`] matches as a registry instance or, with + /// `rekeyed`, as a vendored rekey, found by their key prefixes. + fn lookup(&self, fname: &str, version: &str, rekeyed: bool) -> Vec { + let prefixes = match rekeyed { + false => [format!("{fname}@{version}"), format!("{fname}/{version}")], + true => [format!("{fname}@file:"), "file:".to_string()], + }; let mut out = Vec::new(); - for sep in ['@', '/'] { - let prefix = format!("{fname}{sep}{version}"); + for prefix in &prefixes { let start = self .sorted .partition_point(|&i| self.norm_key(i) < prefix.as_str()); @@ -137,7 +135,11 @@ impl<'f> PnpmLockState<'f> { } out.sort_unstable(); out.dedup(); - out.retain(|&i| pnpm::suffix(self.entry(i).key, fname, version).is_some()); + out.retain(|&i| match match_key(self.entry(i).key, fname, version) { + Some(KeyMatch::Registry { .. }) => !rekeyed, + Some(KeyMatch::Rekeyed) => rekeyed, + None => false, + }); out } @@ -162,7 +164,6 @@ impl<'f> PnpmLockState<'f> { out.push_str(&self.text[cursor..]); self.text = Cow::Owned(out); self.spliced.clear(); - self.vendored = None; self.reindex(); #[cfg(debug_assertions)] debug_assert_eq!( @@ -268,7 +269,7 @@ pub(crate) fn plan_hosted( let entry = lock.entry(i); let suffix = pnpm::suffix(entry.key, &fname, &dep.version) .expect("hits only holds entries naming this dep"); - let resolution = if pnpm::supported_suffix(suffix) { + let resolution = if Target::Hosted.repoints(suffix) { pnpm::resolution(&entry) } else { None @@ -359,9 +360,7 @@ pub(crate) fn plan_hosted( // mode switch. Fail-closed either way: nothing is rewritten for the // dep. if !matched_any { - let vendored = locks - .iter_mut() - .any(|lock| lock.vendors(&fname, &dep.version)); + let vendored = locks.iter().any(|lock| lock.vendors(&fname, &dep.version)); if vendored { result.warnings.push(RewriteWarning { code: "redirect_pnpm_entry_vendored".into(), @@ -390,35 +389,6 @@ pub(crate) fn plan_hosted( } } -/// The `(name, version)` of every packages entry of `text` the vendor -/// backends rekeyed: v9's `@file:` key or legacy's bare -/// `file:` key with a `name:` line, each with its `version:` -/// line, whose path is this project's vendored npm artifact -/// ([`crate::origin::vendor_ref`]: root-anchored, canonical uuid, -/// traversal-safe leaf) — the one rule lockfile discovery reads vendored -/// pnpm keys with. A user's own `file:` dep of the same name, a -/// `../.socket/vendor/…` path from another checkout, and an `overrides:` -/// line alone (pnpm mirrors it from package.json whether or not anything -/// matches) are not vendoring. -fn vendored_packages(text: &str) -> HashSet<(String, String)> { - use super::{classify_pnpm_key, PnpmKey}; - let lock = super::PnpmLock::parse_keys(text); - lock.packages() - .iter() - .filter_map(|package| { - let (name, path) = match classify_pnpm_key(package.key) { - PnpmKey::V9File { name, path } => (name.trim_start_matches('/'), path), - PnpmKey::LegacyFile { path } => (pnpm::entry_field(&package.entry, "name")?, path), - PnpmKey::Registry { .. } | PnpmKey::Other => return None, - }; - let version = pnpm::entry_field(&package.entry, "version")?; - crate::origin::vendor_ref(path) - .filter(|v| v.eco == "npm") - .map(|_| (name.to_string(), version.to_string())) - }) - .collect() -} - /// Test-only reference for the residual gate: every instance of this exact /// name@version in `content` that does not resolve to `artifact_url`. /// Production judges the same predicate inline, per instance, on each diff --git a/crates/socket-patch-core/src/formats/pnpm/mod.rs b/crates/socket-patch-core/src/formats/pnpm/mod.rs index 2df9b643f..b94801dfb 100644 --- a/crates/socket-patch-core/src/formats/pnpm/mod.rs +++ b/crates/socket-patch-core/src/formats/pnpm/mod.rs @@ -28,6 +28,7 @@ pub(crate) mod grammar; pub(crate) mod hosted; pub(crate) mod lines; pub(crate) mod modules_yaml; +pub(crate) mod plan; mod version; pub(crate) mod workspace; @@ -286,28 +287,28 @@ fn head_lock_version(text: &str) -> Option { /// looping them back to "re-lock with pnpm >= 9" would hand them the lock /// they have. pub fn sniff_lock_grammar(text: &str) -> Result { - match head_lock_version(text).as_deref() { + let version = head_lock_version(text); + match version.as_deref() { Some("9.0") => Ok(PnpmLockGrammar::V9), Some("5.4") => Ok(PnpmLockGrammar::V54), Some("6.0") => Ok(PnpmLockGrammar::V60), - Some(v) => { - let major = v.split('.').next().and_then(|m| m.parse::().ok()); + v => { + let major = v.and_then(|v| v.split('.').next()?.parse::().ok()); + let v = v.map_or("no lockfileVersion in its head".into(), |v| { + format!("lockfileVersion {v}") + }); Err(match major { - Some(m) if m < 9 => format!( - "{PNPM_LOCK} has lockfileVersion {v}; supported versions are 5.4 \ - (pnpm 7), 6.0 (pnpm 8), and 9.0 (pnpm >= 9) — re-lock with pnpm >= 9" - ), - _ => format!( - "{PNPM_LOCK} has lockfileVersion {v}; this socket-patch build supports \ + Some(m) if m >= 9 => format!( + "{PNPM_LOCK} has {v}; this socket-patch build supports \ lockfileVersions 5.4, 6.0, and 9.0 — re-lock with a pnpm release that \ emits one of them, or update socket-patch" ), + _ => format!( + "{PNPM_LOCK} has {v}; supported versions are 5.4 (pnpm 7), 6.0 \ + (pnpm 8), and 9.0 (pnpm >= 9) — re-lock with pnpm >= 9" + ), }) } - None => Err(format!( - "{PNPM_LOCK} has no lockfileVersion in its head; supported versions are 5.4, \ - 6.0, and 9.0 — re-lock with pnpm >= 9" - )), } } @@ -357,43 +358,6 @@ pub fn is_shrinkwrap_lock(text: &str) -> bool { .any(|line| line.starts_with("shrinkwrapVersion:")) } -/// The lockfileVersion the v9 vendored planner splices. -const V9_LOCK_VERSION: &str = "9.0"; - -/// `lockfileVersion: '9.0'` head check (accept pnpm's single quotes plus -/// double-quoted/bare spellings) — the v9 vendored planner's own guard. -/// The flavor router sniffs with [`sniff_lock_grammar`] instead, whose -/// allowlist also routes the legacy 5.4/6.0 grammars to their planner; -/// this check only fires if a non-9.0 lock reaches the v9 planner directly. -pub(crate) fn check_v9_lock_version(text: &str) -> Result<(), String> { - match head_lock_version(text) { - Some(v) if v == V9_LOCK_VERSION => Ok(()), - Some(v) => { - // The remedy must point the right way: 5.x (pnpm 7) / 6.x - // (pnpm 8) locks predate the v9 grammar and upgrading pnpm - // re-locks them, but a HIGHER version means the user's pnpm - // already outgrew this build — telling them "re-lock with - // pnpm >= 9" would loop them back to the lock they have. - let major = v.split('.').next().and_then(|m| m.parse::().ok()); - Err(match major { - Some(m) if m < 9 => format!( - "{PNPM_LOCK} has lockfileVersion {v}; only {V9_LOCK_VERSION} is \ - supported — re-lock with pnpm >= 9" - ), - _ => format!( - "{PNPM_LOCK} has lockfileVersion {v}; this socket-patch build supports \ - lockfileVersion {V9_LOCK_VERSION} — re-lock with a pnpm release \ - that emits it, or update socket-patch" - ), - }) - } - None => Err(format!( - "{PNPM_LOCK} has no lockfileVersion in its head; only \ - {V9_LOCK_VERSION} is supported — re-lock with pnpm >= 9" - )), - } -} - // ── the model ── /// One pnpm lock, parsed once (see the module docs). @@ -1046,11 +1010,6 @@ mod tests { may_need_store_flag(plain), "{plain}" ); - assert_eq!( - check_v9_lock_version(&bom), - check_v9_lock_version(plain), - "{plain}" - ); let keys = |text: &str| { PnpmLock::parse(text).entries().map(|e| { e.into_iter() @@ -1067,9 +1026,9 @@ mod tests { assert_eq!(lock_version_major(&format!("\u{feff}{v9}")), Some(9)); assert!(may_need_store_flag(&format!("\u{feff}{v52}"))); assert!(may_need_store_flag(&format!("\u{feff}{shrinkwrap}"))); - assert!(grammar::unsupported_early_shrinkwrap( - "\u{feff}shrinkwrapVersion: 3\n" - )); + assert!(plan::Target::Hosted + .check_lock("\u{feff}shrinkwrapVersion: 3\n") + .is_err()); // Exactly one BOM is encoding; a second one is content, as for pnpm. assert!(!is_pnpm_lock_text( "\u{feff}\u{feff}lockfileVersion: '9.0'\n" @@ -1146,14 +1105,15 @@ mod architecture_tests { ( "vendor/pnpm_lock.rs", 3, - "v9 vendored planner: preflight and splices over line blocks \ - (`lock_has_target_package`, `edit_packages`, `LockIndex`)", + "vendored planner (both dialects): the shared lookup and rekey \ + over line blocks (`lock_has_target_package`, `edit_packages`, \ + `LockIndex`), keys read with `plan::match_key`", ), ( "vendor/pnpm_lock_legacy.rs", - 6, - "5.4 / 6.0 vendored planner: preflight, splices and revert over \ - line blocks", + 4, + "5.4 / 6.0 dialect: its ref gate, sorted rekey, dep-ref edit and \ + revert over line blocks", ), ]; diff --git a/crates/socket-patch-core/src/formats/pnpm/plan.rs b/crates/socket-patch-core/src/formats/pnpm/plan.rs new file mode 100644 index 000000000..a2fa56c1d --- /dev/null +++ b/crates/socket-patch-core/src/formats/pnpm/plan.rs @@ -0,0 +1,201 @@ +//! The pnpm planning core both modes share. Hosted ([`super::hosted`]) and +//! vendored (`vendor::pnpm_lock`, `vendor::pnpm_lock_legacy`) mode differ +//! only in where a rewired entry points ([`Target`]); this module owns what +//! they agree on: +//! +//! * which lock versions each target's edit has fixtures for +//! ([`Target::check_lock`]); +//! * how a lock key names a patched `name@version` ([`match_key`]): the +//! registry entry with its peer suffix, or the entry a vendored rewire +//! already rekeyed; +//! * the all-or-nothing gate: an instance the target's edit cannot repoint +//! ([`Target::repoints`]) refuses the release in every lock, so no lock +//! is left half-rewired. +//! +//! The edits themselves stay with the targets: hosted splices each +//! instance's `resolution:`; vendored rekeys the entry to its `file:` +//! artifact and mirrors the override pnpm cross-checks. + +use super::grammar; +use super::{sniff_lock_grammar, PnpmLockGrammar}; +use crate::constants::npm_family::PNPM_LOCK; +use crate::formats::text::strip_bom; +use crate::vendor::npm_common::tgz_rel_leaf; + +/// Where a rewired entry points. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Target { + /// A Socket-hosted tarball URL (`scan --mode hosted`): the instance's + /// `resolution:` is spliced in place. + Hosted, + /// A committed `.socket/vendor/npm//` tarball (`vendor`): + /// the entry is rekeyed to `file:`. + Vendored, +} + +impl Target { + /// Whether this target's edit can rewrite a lock with `text`'s head, + /// and in which grammar (`None`: the hosted splice reads every + /// grammar the same way). Hosted takes every lock but pnpm 1's early + /// shrinkwrap (shrinkwrapVersion 3 without a minor version), whose + /// installers drop registry tarball URLs whatever the frozen flag says + /// (verified with pnpm 1.0.0). Vendored takes the grammars its rekey was + /// captured against (5.4, 6.0, 9.0): another version's `file:` override + /// shape has no pnpm-blessed fixture, so it is refused, + /// target-specifically (divergences N16). + pub(crate) fn check_lock(self, text: &str) -> Result, String> { + if self == Target::Vendored { + return sniff_lock_grammar(text).map(Some); + } + let field = |name: &str| strip_bom(text).lines().find_map(|l| l.strip_prefix(name)); + let early = field("shrinkwrapVersion:") + .is_some_and(|v| v.trim().trim_matches(['\'', '"']) == "3") + && !field("shrinkwrapMinorVersion:") + .is_some_and(|v| v.trim().parse::().is_ok_and(|v| v > 0)); + match early { + true => Err(format!( + "{PNPM_LOCK} uses early pnpm 1 shrinkwrapVersion 3 without a supported \ + minor version" + )), + false => Ok(None), + } + } + + /// Whether this target's edit repoints a registry instance whose key + /// carries `suffix` after `name@version`. Hosted splices every peer + /// context it can parse (pnpm 5's `_peer@1` or balanced `(peer@1)` + /// groups); vendored rekeys the plain entry only, since a peer-suffixed + /// rekey has no fixture and would dangle. + pub(crate) fn repoints(self, suffix: &str) -> bool { + if self == Target::Vendored || suffix.is_empty() || suffix.starts_with('_') { + return suffix.is_empty() || self == Target::Hosted; + } + let mut depth = 0usize; + for c in suffix.chars() { + match c { + '(' => depth += 1, + ')' if depth > 0 => depth -= 1, + _ if depth == 0 => return false, + _ => {} + } + } + depth == 0 + } +} +/// How one `packages:` / `snapshots:` key names a patched release. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum KeyMatch<'k> { + /// The registry entry, with whatever follows `name@version` (empty, or + /// a peer suffix such as `(react@18.2.0)` / `_react@18.2.0`). + Registry { suffix: &'k str }, + /// The entry a vendored rewire rekeyed: v9's `name@file:` or + /// legacy's bare `file:`, where the artifact is this + /// project's vendored tarball of this release + /// ([`crate::origin::vendor_ref`]: root-anchored, so a sibling + /// checkout's `../.socket/vendor/…` is not ours; any uuid). + Rekeyed, +} + +/// How `key` names `name@version`, if it does (see [`KeyMatch`]). Keys in +/// every grammar's spelling (`name@1.0.0`, `/name@1.0.0`, `/name/1.0.0`, +/// quoted) read alike. +pub(crate) fn match_key<'k>(key: &'k str, name: &str, version: &str) -> Option> { + if let Some(suffix) = grammar::suffix(key, name, version) { + return Some(KeyMatch::Registry { suffix }); + } + let key = grammar::unquote(key); + let path = key + .strip_prefix(name) + .and_then(|rest| rest.strip_prefix("@file:")) + .or_else(|| key.strip_prefix("file:"))?; + // The leaf binding matters: a project can vendor one package at several + // versions, and an edit must never take a sibling version's entry. + crate::origin::vendor_ref(path) + .filter(|v| v.eco == "npm" && v.leaf == tgz_rel_leaf(name, version)) + .map(|_| KeyMatch::Rekeyed) +} + +#[cfg(test)] +mod tests { + use super::*; + + const ART: &str = ".socket/vendor/npm/0b8d2c3e-4f5a-4b6c-8d7e-9f0a1b2c3d4e/left-pad-1.3.0.tgz"; + + #[test] + fn keys_of_every_grammar_name_the_release() { + for key in [ + "left-pad@1.3.0", + "/left-pad@1.3.0", + "/left-pad/1.3.0", + "'left-pad@1.3.0'", + ] { + assert_eq!( + match_key(key, "left-pad", "1.3.0"), + Some(KeyMatch::Registry { suffix: "" }), + "{key}" + ); + } + assert_eq!( + match_key("left-pad@1.3.0(react@18.2.0)", "left-pad", "1.3.0"), + Some(KeyMatch::Registry { + suffix: "(react@18.2.0)" + }) + ); + assert_eq!(match_key("left-pad@1.3.01", "left-pad", "1.3.0"), None); + assert_eq!( + match_key(&format!("left-pad@file:{ART}"), "left-pad", "1.3.0"), + Some(KeyMatch::Rekeyed) + ); + assert_eq!( + match_key(&format!("file:{ART}"), "left-pad", "1.3.0"), + Some(KeyMatch::Rekeyed) + ); + // A sibling version's artifact, another package's key, and a + // sibling checkout's path are not this release's rekey. + assert_eq!(match_key(&format!("file:{ART}"), "left-pad", "1.3.1"), None); + assert_eq!( + match_key(&format!("is-odd@file:{ART}"), "left-pad", "1.3.0"), + None + ); + assert_eq!( + match_key(&format!("left-pad@file:../{ART}"), "left-pad", "1.3.0"), + None + ); + } + + #[test] + fn each_target_repoints_what_its_edit_has_fixtures_for() { + assert!(Target::Hosted.repoints("")); + assert!(Target::Hosted.repoints("(react@18.2.0)")); + assert!(Target::Hosted.repoints("_react@18.2.0")); + assert!(!Target::Hosted.repoints("(react@18.2.0")); + assert!(Target::Vendored.repoints("")); + assert!(!Target::Vendored.repoints("(react@18.2.0)")); + assert!(!Target::Vendored.repoints("_react@18.2.0")); + } + + /// Divergence N16: hosted reads every lock but early shrinkwrap, while + /// vendored keeps its refusal for every version its rekey was not + /// captured against (target-specific). + #[test] + fn lock_versions_each_target_rewrites() { + for (head, vendored) in [ + ("lockfileVersion: '9.0'\n", Some(PnpmLockGrammar::V9)), + ("lockfileVersion: '6.0'\n", Some(PnpmLockGrammar::V60)), + ("lockfileVersion: 5.4\n", Some(PnpmLockGrammar::V54)), + ("lockfileVersion: 5.3\n", None), + ("lockfileVersion: 5.1\n", None), + ("lockfileVersion: '6.1'\n", None), + ("lockfileVersion: '10.0'\n", None), + ("shrinkwrapVersion: 3\nshrinkwrapMinorVersion: 9\n", None), + ] { + assert_eq!(Target::Hosted.check_lock(head), Ok(None), "{head}"); + assert_eq!( + Target::Vendored.check_lock(head).ok().flatten(), + vendored, + "{head}" + ); + } + assert!(Target::Hosted.check_lock("shrinkwrapVersion: 3\n").is_err()); + } +} diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 0201fbb31..165450ab4 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -28332,6 +28332,28 @@ packages: .replace("version: 1.3.0", "version: 1.2.0"), "redirect_pnpm_entry_not_found", ), + // Lock-model 2b: the hint reads the key with the planning core's + // `match_key`, the rule the vendored planner rekeys by, so the + // artifact leaf names the release and the `version:` line is not + // consulted. Before, a rekey without a `version:` line was "not + // found" and one whose leaf names another release but whose + // `version:` line matched was "vendored". + ( + "a rekey without a version line", + v9(&format!(".socket/vendor/npm/{uuid}/left-pad-1.3.0.tgz")) + .replace(" version: 1.3.0\n", ""), + "redirect_pnpm_entry_vendored", + ), + ( + "another release's artifact under this version line", + v9(&format!(".socket/vendor/npm/{uuid}/left-pad-1.2.0.tgz")), + "redirect_pnpm_entry_not_found", + ), + ( + "a sibling checkout's artifact", + v9(&format!("../.socket/vendor/npm/{uuid}/left-pad-1.3.0.tgz")), + "redirect_pnpm_entry_not_found", + ), ] { assert_eq!(codes(&lock), vec![want], "{case}"); } diff --git a/crates/socket-patch-core/src/vendor/mod.rs b/crates/socket-patch-core/src/vendor/mod.rs index dd732ca7f..df2e1fd7b 100644 --- a/crates/socket-patch-core/src/vendor/mod.rs +++ b/crates/socket-patch-core/src/vendor/mod.rs @@ -989,8 +989,8 @@ pub async fn lock_text_refusals( /// deletes; a gate that depended on it would make this under-predict, and /// the wet run still refuses. Yarn's restores rewrite key and checksum /// text, and legacy pnpm 7/8 locks (5.4 / 6.0) are not lock-text gated at -/// all, so neither is predicted here: a CRLF legacy lock, an override -/// conflict or an unsupported legacy entry previews `would_vendor` while +/// all, so neither is predicted here: a mixed line-ending legacy lock, an +/// override conflict or an unsupported legacy entry previews `would_vendor` while /// the wet takeover refuses it (keeping the hosted pin, #963) — a known /// preview gap the contract names. pub async fn pnpm_takeover_lock_text_refusals( @@ -2236,10 +2236,15 @@ mod pnpm_takeover_lock_text_refusal_tests { tmp } + /// Divergences N15: a CRLF hosted lock (an autocrlf checkout) takes over + /// like its LF twin; only a lock mixing CRLF and LF is refused. #[tokio::test] - async fn crlf_hosted_pnpm_lock_is_refused() { + async fn only_a_mixed_line_ending_hosted_pnpm_lock_is_refused() { let tmp = project("pnpm-lock.yaml", &HOSTED_V9.replace('\n', "\r\n")); let refused = pnpm_takeover_lock_text_refusals(tmp.path(), &[(PURL, UUID)]).await; + assert!(refused.is_empty(), "{refused:?}"); + let tmp = project("pnpm-lock.yaml", &HOSTED_V9.replacen('\n', "\r\n", 1)); + let refused = pnpm_takeover_lock_text_refusals(tmp.path(), &[(PURL, UUID)]).await; assert_eq!( refused.get(PURL).map(|(code, _)| *code), Some("vendor_lockfile_crlf_unsupported"), @@ -2266,8 +2271,8 @@ mod pnpm_takeover_lock_text_refusal_tests { } /// A known preview gap (CLI_CONTRACT, #853): a legacy pnpm 7/8 lock - /// (5.4 / 6.0) is not lock-text gated, so even a CRLF one the wet - /// takeover refuses (`vendor_lockfile_crlf_unsupported`) previews + /// (5.4 / 6.0) is not lock-text gated, so even one the wet takeover + /// refuses (a mixed line-ending lock, `vendor_lockfile_crlf_unsupported`) previews /// `would_vendor`. The wet run's refusal keeps the hosted pin (#963). #[tokio::test] async fn legacy_pnpm_project_is_out_of_scope() { diff --git a/crates/socket-patch-core/src/vendor/npm_common.rs b/crates/socket-patch-core/src/vendor/npm_common.rs index 48de7a329..0e477052c 100644 --- a/crates/socket-patch-core/src/vendor/npm_common.rs +++ b/crates/socket-patch-core/src/vendor/npm_common.rs @@ -659,7 +659,7 @@ pub(crate) fn is_safe_npm_name(name: &str) -> bool { /// The artifact path under the uuid dir: `[@scope/]-.tgz`, /// with the scope kept as a real subdirectory. -pub(super) fn tgz_rel_leaf(name: &str, version: &str) -> String { +pub(crate) fn tgz_rel_leaf(name: &str, version: &str) -> String { format!("{}.tgz", pkg_rel_leaf(name, version)) } diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock.rs b/crates/socket-patch-core/src/vendor/pnpm_lock.rs index 7547e780d..d80b04d29 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock.rs @@ -69,6 +69,7 @@ use super::npm_common::{ use super::parse_memo::ParseMemo; use super::path::parse_vendor_path; use super::pnpm_build_approvals as build_approvals; +use super::pnpm_lock_legacy::swap_block_sorted; use super::revert::{self, KeepPolicy}; use super::source::PackageSource; use super::state::{PnpmMeta, VendorEntry, WiringAction, WiringRecord}; @@ -78,10 +79,12 @@ use crate::formats::pnpm::lines::{ indent_of, next_block, parse_key_line, section_bounds, split_lines, split_project_document, unquote_value, yaml_key, yaml_key_like, YamlBlock, }; +use crate::formats::pnpm::plan::{match_key, KeyMatch, Target}; use crate::formats::pnpm::workspace; -use crate::formats::pnpm::{check_v9_lock_version as check_lock_version, vendored_npm_uuids}; +use crate::formats::pnpm::{vendored_npm_uuids, PnpmLockGrammar}; const PACKAGE_JSON: &str = "package.json"; +use crate::utils::line_endings::{to_lf, LineEndings}; use crate::utils::pnpm_workspace::{governing_workspace_file, PNPM_WORKSPACE}; /// Refusal code for a workspace member whose pnpm settings live in an @@ -173,6 +176,23 @@ impl PnpmDialect { fn allows_revert_file(self, file: &str) -> bool { REVERT_ALLOWLIST.contains(&file) && (self == Self::V9 || file != PNPM_WORKSPACE) } + + /// The grammar of a lock this dialect rewrites: one the vendored target + /// takes ([`Target::check_lock`]) and this dialect splices. + pub(super) fn check_lock(self, text: &str) -> Result { + let grammar = Target::Vendored + .check_lock(text)? + .unwrap_or(PnpmLockGrammar::V9); + match (self, grammar) { + (Self::V9, PnpmLockGrammar::V9) + | (Self::Legacy, PnpmLockGrammar::V54 | PnpmLockGrammar::V60) => Ok(grammar), + _ => Err(format!( + "{PNPM_LOCK} is a {} lock, which the {} planner does not write", + grammar.describe(), + self.flavor() + )), + } + } } pub(super) async fn vendor_pnpm_dialect( @@ -301,7 +321,9 @@ impl NpmLockBackend for PnpmBackend { mut pkg, mut lock, lock_prefix, + lock_eol, ws_text, + ws_eol, }, effective_key, } = plan; @@ -395,15 +417,18 @@ impl NpmLockBackend for PnpmBackend { .render(&pkg) .map_err(|e| format!("cannot serialize {PACKAGE_JSON}: {e}"))?; // An env document ahead of the project lock is carried over verbatim. + // Each file is written back in its own line endings (`lf_text`). let lock_out = format!("{lock_prefix}{}", lock.lines().join("\n")); + let ws_out = ws_edit.new_text.as_deref().map(|t| ws_eol.restore(t)); + let ws_was = ws_text.as_deref().map(|t| ws_eol.restore(t)); commit_surfaces( project_root, pkg_changed.then_some(new_pkg_bytes.as_slice()), &pkg_bytes, - ws_edit.new_text.as_deref().map(str::as_bytes), - ws_text.as_deref().map(str::as_bytes), + ws_out.as_deref().map(str::as_bytes), + ws_was.as_deref().map(str::as_bytes), ws_edit.created_file, - lock_changed.then_some(lock_out.as_bytes()), + lock_changed.then_some(lock_eol.restore(&lock_out).as_bytes()), ) .await?; if lock_changed { @@ -459,7 +484,11 @@ struct PnpmProject { lock: ProjectLock, /// Everything ahead of the project document, written back verbatim. lock_prefix: String, + /// The line endings the lock and the workspace file are written back + /// with; their texts here are LF ([`lf_text`]). + lock_eol: LineEndings, ws_text: Option, + ws_eol: LineEndings, } enum ProjectLock { @@ -564,13 +593,7 @@ async fn read_project( ))); } }; - let grammar = match dialect { - PnpmDialect::V9 => { - check_lock_version(&lock_text).map(|()| crate::formats::pnpm::PnpmLockGrammar::V9) - } - PnpmDialect::Legacy => super::pnpm_lock_legacy::check_lock_version(&lock_text), - }; - let grammar = match grammar { + let grammar = match dialect.check_lock(&lock_text) { Ok(grammar) => grammar, Err(detail) => { return Err(Box::new(refused( @@ -579,23 +602,7 @@ async fn read_project( ))); } }; - // CRLF line endings (Windows autocrlf checkouts) break every structural - // probe below — `split_lines` keeps the trailing `\r`, so section headers - // like `packages:` never match — which would otherwise surface as a - // misleading "no packages entry … run `pnpm install`" refusal. pnpm - // itself tolerates CRLF, so name the real cause and fail closed before - // any probe runs. - if lock_text.contains('\r') { - return Err(Box::new(refused( - "vendor_lockfile_crlf_unsupported", - format!( - "{PNPM_LOCK} has CRLF line endings, which this rewriter cannot edit \ - byte-faithfully — normalize the file to LF (re-run `pnpm install`, \ - or add `pnpm-lock.yaml text eol=lf` to .gitattributes and re-checkout) \ - and retry" - ), - ))); - } + let (lock_text, lock_eol) = lf_text(PNPM_LOCK, lock_text)?; if dialect == PnpmDialect::Legacy { return Ok(PnpmProject { pkg_bytes, @@ -606,7 +613,9 @@ async fn read_project( grammar, )?), lock_prefix: String::new(), + lock_eol, ws_text: None, + ws_eol: LineEndings::None, }); } let (lock_prefix, project_text) = match split_v9_lock(&lock_text) { @@ -636,21 +645,20 @@ async fn read_project( // encoding, permissions, a FIFO) must refuse — treating it as absent // would route into the create path, which OVERWRITES the user's // workspace definition with the root-only scaffold. - let ws_text: Option = - match read_regular_to_string(&project_root.join(PNPM_WORKSPACE)).await { - Ok(text) => Some(text), - Err(e) if e.kind() == std::io::ErrorKind::NotFound => None, - Err(e) => { - return Err(Box::new(refused( - "vendor_lockfile_missing", - format!( - "cannot read {PNPM_WORKSPACE}: {e} — vendoring mirrors the override \ + let (ws_text, ws_eol) = match read_regular_to_string(&project_root.join(PNPM_WORKSPACE)).await { + Ok(text) => lf_text(PNPM_WORKSPACE, text).map(|(t, eol)| (Some(t), eol))?, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => (None, LineEndings::None), + Err(e) => { + return Err(Box::new(refused( + "vendor_lockfile_missing", + format!( + "cannot read {PNPM_WORKSPACE}: {e} — vendoring mirrors the override \ into it (pnpm >= 11 reads overrides from pnpm-workspace.yaml); fix \ the file and retry" - ), - ))); - } - }; + ), + ))); + } + }; // A workspace member with its own lock (`sharedWorkspaceLockfile: // false`) and no workspace file of its own: pnpm reads `overrides:` // only from the workspace root's file, so the create path below would @@ -677,28 +685,36 @@ async fn read_project( ))); } } - // Same CRLF posture as the lock above: the workspace splices match exact - // LF lines, so a CRLF file dodges the conflict checks and the edit - // appends a DUPLICATE `overrides:` section — a duplicated mapping key - // pnpm refuses to parse. Fail closed naming the real cause. - if ws_text.as_deref().is_some_and(|t| t.contains('\r')) { - return Err(Box::new(refused( - "vendor_lockfile_crlf_unsupported", - format!( - "{PNPM_WORKSPACE} has CRLF line endings, which this rewriter cannot edit \ - byte-faithfully — normalize the file to LF and retry" - ), - ))); - } Ok(PnpmProject { pkg_bytes, pkg, lock: ProjectLock::V9(lines), lock_prefix, + lock_eol, ws_text, + ws_eol, }) } +/// A pnpm file's text with LF line endings, and the endings it is written +/// back with. A CRLF file (a Windows autocrlf checkout) is read and edited +/// like its LF twin and written back CRLF, as hosted mode's byte splices +/// keep it (divergences N15). A file that mixes endings is refused: no one +/// ending writes it back byte-faithfully. +fn lf_text(file: &str, text: String) -> Result<(String, LineEndings), Box> { + match LineEndings::of(&text) { + LineEndings::Mixed => Err(Box::new(refused( + "vendor_lockfile_crlf_unsupported", + format!( + "{file} mixes CRLF and LF line endings, which this rewriter cannot write \ + back byte-faithfully — normalize the file to one line ending and retry" + ), + ))), + LineEndings::Crlf => Ok((text.replace("\r\n", "\n"), LineEndings::Crlf)), + eol => Ok((text, eol)), + } +} + /// `(prefix, project document)` of a v9 lock ([`split_project_document`]), /// the project document's own head version-checked. `Err` names why the /// lock's documents are not pnpm's layout. @@ -706,7 +722,7 @@ fn split_v9_lock(text: &str) -> Result<(&str, &str), String> { let (prefix, project) = split_project_document(text).map_err(|why| format!("{PNPM_LOCK} {why}"))?; if !prefix.is_empty() { - check_lock_version(project).map_err(|e| { + PnpmDialect::V9.check_lock(project).map_err(|e| { format!("the project document of {PNPM_LOCK} fails its version check: {e}") })?; } @@ -770,7 +786,7 @@ fn preflight_package( { return Err(Box::new(refused("vendor_override_conflict", detail))); } - if !lock_has_target_package_in(lines, name, version) { + if !lock_has_target_package(lines, name, version) { return Err(Box::new(refused( "vendor_lock_entry_not_found", format!( @@ -834,7 +850,7 @@ pub async fn pnpm_entry_in_use(entry: &VendorEntry, project_root: &Path) -> Opti let text = read_regular_to_string(&project_root.join(PNPM_LOCK)) .await .ok()?; - if check_lock_version(&text).is_err() { + if PnpmDialect::V9.check_lock(&text).is_err() { return None; } // Every `packages:`/`snapshots:` block key resolving into @@ -1008,6 +1024,7 @@ pub(super) async fn revert_pnpm_dialect( // file degrades to a warning and the artifact removal still proceeds). let mut lock_lines: Option> = None; let mut lock_prefix = String::new(); + let mut lock_eol = LineEndings::None; if touches_lock { let read = match lock_text { Some(read) => read, @@ -1015,18 +1032,21 @@ pub(super) async fn revert_pnpm_dialect( }; match read { // The wiring lives in the project document; an env document - // ahead of it is written back verbatim. - Ok(text) => match dialect { - PnpmDialect::V9 => { - let (prefix, project) = match split_v9_lock(&text) { + // ahead of it is written back verbatim, in the lock's own line + // endings (N15). + Ok(text) => { + lock_eol = LineEndings::of(&text); + let text = to_lf(&text); + let (prefix, project) = match dialect { + PnpmDialect::V9 => match split_v9_lock(&text) { Ok(split) => split, Err(why) => return RevertOutcome::failed(why), - }; - lock_prefix = prefix.to_string(); - lock_lines = Some(split_lines(project)); - } - PnpmDialect::Legacy => lock_lines = Some(split_lines(&text)), - }, + }, + PnpmDialect::Legacy => ("", &*text), + }; + lock_prefix = prefix.to_string(); + lock_lines = Some(split_lines(project)); + } Err(e) if e.kind() == std::io::ErrorKind::NotFound => { outcome.warnings.push(VendorWarning::new( "vendor_lockfile_missing", @@ -1126,7 +1146,9 @@ pub(super) async fn revert_pnpm_dialect( if let Some(lines) = &lock_lines { if let Err(e) = atomic_write_bytes_preserving_mode( &project_root.join(PNPM_LOCK), - format!("{lock_prefix}{}", lines.join("\n")).as_bytes(), + lock_eol + .restore(&format!("{lock_prefix}{}", lines.join("\n"))) + .as_bytes(), ) .await { @@ -1231,18 +1253,10 @@ async fn revert_workspace( } Err(e) => return Err(format!("cannot read {PNPM_WORKSPACE}: {e}")), }; - // CRLF blindness fails CLOSED: the splices below match exact LF lines, - // so a CRLF-converted file would read an Added override as "already - // converged" and let the artifact it still points at be removed. Drift, - // not convergence — the keep gate holds the artifact until the user - // normalizes the file. - if text.contains('\r') { - warnings.push(drifted(format!( - "{PNPM_WORKSPACE} has CRLF line endings the revert splices cannot edit; \ - left alone — normalize the file to LF and re-run `vendor --revert`" - ))); - return Ok(()); - } + // A CRLF file (an autocrlf checkout of what vendoring wrote) is read + // like its LF twin and written back CRLF (N15). + let eol = LineEndings::of(&text); + let text = to_lf(&text).into_owned(); // Fast path: a file we created that is still byte-identical to the // scaffold we wrote → delete it (byte-restore to "no file"). @@ -1278,7 +1292,7 @@ async fn revert_workspace( } dirty |= build_approvals::revert_ws(&mut lines, approvals); if dirty { - atomic_write_bytes_preserving_mode(&path, lines.join("\n").as_bytes()) + atomic_write_bytes_preserving_mode(&path, eol.restore(&lines.join("\n")).as_bytes()) .await .map_err(|e| format!("cannot write {PNPM_WORKSPACE}: {e}"))?; } @@ -1405,12 +1419,9 @@ impl EditCtx<'_> { vendor_value_is_for(value, self.name, self.version) } - /// Is `key` our rekeyed `name@file:` packages/snapshots key for - /// THIS name@version (any uuid)? - fn is_ours_key(&self, key: &str) -> bool { - key.strip_prefix(self.name) - .and_then(|rest| rest.strip_prefix("@file:")) - .is_some_and(|rest| self.is_ours(rest)) + /// [`target_key`] of a packages/snapshots key for this release. + pub(super) fn target_key(&self, key: &str) -> Option { + target_key(key, self.name, self.version) } /// The per-importer `specifier:` spelling: re-relativized for nested @@ -1678,56 +1689,26 @@ pub(super) fn check_lock_override( Ok(()) } -/// Pre-flight: does the lock have a packages entry vendoring can target — -/// the registry `name@version` key, or our own rekeyed `name@file:` key -/// (the in-sync / stale-uuid re-run)? The ours-probe binds to THIS -/// version's leaf — a sibling version's vendored entry is not a target -/// (see `vendor_value_is_for`). -fn lock_has_target_package(lines: &[String], name: &str, version: &str) -> bool { - let Some((start, end)) = section_bounds(lines, "packages") else { - return false; - }; - let reg_key = format!("{name}@{version}"); - let ours_prefix = format!("{name}@file:"); - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - if block.key == reg_key { - return true; - } - if let Some(rest) = block.key.strip_prefix(&ours_prefix) { - if vendor_value_is_for(rest, name, version) { - return true; - } - } - i = block.end; +/// Whether a packages/snapshots `key` is one the vendored edit targets for +/// `name@version` ([`match_key`]): `Some(false)` for the registry entry the +/// rekey repoints ([`Target::repoints`]: no peer suffix), `Some(true)` for a +/// rekey of this release already made (the in-sync / stale-uuid re-run), +/// `None` for every other key. The rekey probe binds to THIS version's leaf +/// and to this project's `.socket/vendor/`. +pub(super) fn target_key(key: &str, name: &str, version: &str) -> Option { + match match_key(key, name, version)? { + KeyMatch::Registry { suffix } => Target::Vendored.repoints(suffix).then_some(false), + KeyMatch::Rekeyed => Some(true), } - false } -/// [`lock_has_target_package`] over the backend's lock lines: answered from -/// the section index while the lines are the memoized split (the -/// candidate blocks are exactly those whose key is the registry key or -/// starts with `name@file:`, in lock order), else by the scan. -fn lock_has_target_package_in(lines: &LockLines, name: &str, version: &str) -> bool { - let Some(index) = lines.index() else { - return lock_has_target_package(lines, name, version); - }; - if index.packages.bounds.is_none() { - return false; - } - let reg_key = format!("{name}@{version}"); - let ours_prefix = format!("{name}@file:"); - index - .packages - .candidates(®_key, name) - .into_iter() - .any(|block| { - block.key == reg_key - || block - .key - .strip_prefix(&ours_prefix) - .is_some_and(|rest| vendor_value_is_for(rest, name, version)) - }) +/// Pre-flight: does the lock have a packages entry vendoring can target +/// ([`target_key`])? From the section index while the lines are the +/// memoized split, else by the scan ([`matching_blocks`]). +pub(super) fn lock_has_target_package(lines: &L, name: &str, version: &str) -> bool { + lines.bounds("packages").is_some_and(|(start, end)| { + !matching_blocks(lines, "packages", start, end, name, version).is_empty() + }) } /// Pre-flight fail-closed guard against reference forms the surgery does @@ -1821,60 +1802,37 @@ fn check_rewritable_refs_with( i = catalog.end; } } - if let Some(index) = index { - for candidate in index.snapshot_ref_candidates(®_key, name, version) { - match candidate { - SnapshotCandidate::Key(block) => { - let key = &index.snapshots.blocks[block].key; - if key.starts_with(&key_peer_prefix) { - return refuse("a peer-suffixed snapshot key", key); - } - } - SnapshotCandidate::Body(line) => { - let Some((dep, _repr, rest)) = parse_key_line(&lines[line], 6) else { - continue; - }; - let rest = unquote_value(rest); - if rest == reg_key || rest.starts_with(&key_peer_prefix) { - return refuse("an aliased snapshot reference", rest); - } - if dep == name && rest.starts_with(&val_peer_prefix) { - return refuse("a peer-suffixed snapshot reference", rest); - } - } + // The positions the checks below read: from the index, the earliest + // position each refusal can hold at; by the scan, every one. + let scanned; + let (snapshot_refs, importer_refs): (Box + '_>, _) = + match index { + Some(index) => ( + Box::new( + index + .snapshot_ref_candidates(®_key, name, version) + .into_iter(), + ), + index.importer_ref_candidates(®_key, name, version), + ), + None => { + scanned = importer_deps(lines); + ( + Box::new(snapshot_positions(lines)), + scanned.iter().collect(), + ) } - } - for entry in index.importer_ref_candidates(®_key, name, version) { - let dep = entry.dep.as_str(); - if let Some(v) = entry.ver.as_deref().map(unquote_value) { - if v == reg_key || v.starts_with(&key_peer_prefix) { - return refuse("an aliased importer version", v); - } - if dep == name && v.starts_with(&val_peer_prefix) { - return refuse("a peer-suffixed importer version", v); - } - if dep == name && v == version { - if let Some(s) = entry.spec.as_deref() { - if unquote_value(s).starts_with("catalog:") { - return refuse_catalog( - &format!("importer `{}` specifier", entry.importer), - s, - ); - } - } + }; + for candidate in snapshot_refs { + match candidate { + SnapshotCandidate::Key(header) => { + let key = parse_key_line(&lines[header], 2).map_or("", |(key, ..)| key); + if key.starts_with(&key_peer_prefix) { + return refuse("a peer-suffixed snapshot key", key); } } - } - return Ok(()); - } - if let Some((start, end)) = section_bounds(lines, "snapshots") { - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - if block.key.starts_with(&key_peer_prefix) { - return refuse("a peer-suffixed snapshot key", &block.key); - } - for line in &lines[block.header + 1..block.end] { - let Some((dep, _repr, rest)) = parse_key_line(line, 6) else { + SnapshotCandidate::Body(line) => { + let Some((dep, _repr, rest)) = parse_key_line(&lines[line], 6) else { continue; }; let rest = unquote_value(rest); @@ -1885,52 +1843,73 @@ fn check_rewritable_refs_with( return refuse("a peer-suffixed snapshot reference", rest); } } - i = block.end; } } - if let Some((start, end)) = section_bounds(lines, "importers") { - let mut i = start + 1; - while let Some(importer) = next_block(lines, i, end) { - let mut k = importer.header + 1; - while k < importer.end { - let Some((dep, _repr, rest)) = parse_key_line(&lines[k], 6) else { - k += 1; - continue; - }; - if !rest.is_empty() { - k += 1; - continue; - } - let (spec, ver, f) = dep_field_lines(lines, k + 1, importer.end, 8); - if let Some((_, v)) = &ver { - let v = unquote_value(v); - if v == reg_key || v.starts_with(&key_peer_prefix) { - return refuse("an aliased importer version", v); - } - if dep == name && v.starts_with(&val_peer_prefix) { - return refuse("a peer-suffixed importer version", v); - } - if dep == name && v == version { - if let Some((_, s)) = spec { - // pnpm spells the default catalog `'catalog:'` - // (quoted) and named ones `catalog:` (bare). - if unquote_value(&s).starts_with("catalog:") { - return refuse_catalog( - &format!("importer `{}` specifier", importer.key), - &s, - ); - } - } - } - } - k = f; + for entry in importer_refs { + let dep = entry.dep.as_str(); + let Some(v) = entry.ver.as_deref().map(unquote_value) else { + continue; + }; + if v == reg_key || v.starts_with(&key_peer_prefix) { + return refuse("an aliased importer version", v); + } + if dep == name && v.starts_with(&val_peer_prefix) { + return refuse("a peer-suffixed importer version", v); + } + // pnpm spells the default catalog `'catalog:'` (quoted) and named + // ones `catalog:` (bare). + if let Some(s) = entry + .spec + .as_deref() + .filter(|_| dep == name && v == version) + { + if unquote_value(s).starts_with("catalog:") { + return refuse_catalog(&format!("importer `{}` specifier", entry.importer), s); } - i = importer.end; } } Ok(()) } +/// Every snapshot block header and body line, in lock order: the scan's +/// [`SnapshotCandidate`]s. +fn snapshot_positions(lines: &[String]) -> impl Iterator + '_ { + let blocks = section_bounds(lines, "snapshots").into_iter(); + blocks + .flat_map(move |(start, end)| { + std::iter::successors(next_block(lines, start + 1, end), move |b| { + next_block(lines, b.end, end) + }) + }) + .flat_map(|b| { + std::iter::once(SnapshotCandidate::Key(b.header)) + .chain((b.header + 1..b.end).map(SnapshotCandidate::Body)) + }) +} + +/// Every importer's [`DepEntry`]s, in lock order. +fn importer_deps(lines: &[String]) -> Vec { + let Some((start, end)) = section_bounds(lines, "importers") else { + return Vec::new(); + }; + let mut out = Vec::new(); + let mut i = start + 1; + while let Some(importer) = next_block(lines, i, end) { + out.extend( + dep_entries(lines, importer.header + 1, importer.end, 6) + .into_iter() + .map(|e| ImporterDep { + importer: importer.key.clone(), + dep: e.dep, + spec: e.spec.map(|(_, s)| s), + ver: e.ver.map(|(_, v)| v), + }), + ); + i = importer.end; + } + out +} + // ───────────────────────── package.json override ───────────────────────── /// Add/refresh `pnpm.overrides[@] = file:` on the @@ -2390,30 +2369,50 @@ pub(super) fn overrides_record( } } -/// Locate a dep entry's `specifier:`/`version:` field lines at `indent` -/// (4 for legacy root deps, 8 for modern importers). Returns their values plus -/// the index of the first non-field line. -#[allow(clippy::type_complexity)] -pub(super) fn dep_field_lines( +/// One dependency entry with nested `specifier:` / `version:` fields (each +/// with its line): a v9 importer dep (key at 6, fields at 8) or a legacy +/// 6.0 root dep (2 / 4). +pub(super) struct DepEntry { + pub(super) dep: String, + pub(super) spec: Option<(usize, String)>, + pub(super) ver: Option<(usize, String)>, +} + +/// Every [`DepEntry`] keyed at `indent` in `[start, end)`, in order. A key +/// with an inline value is not one; unknown fields are skipped. +pub(super) fn dep_entries( lines: &[String], - mut f: usize, + start: usize, end: usize, indent: usize, -) -> (Option<(usize, String)>, Option<(usize, String)>, usize) { - let mut spec = None; - let mut ver = None; - while f < end { - let Some((field, _repr, fval)) = parse_key_line(&lines[f], indent) else { - break; +) -> Vec { + let mut out = Vec::new(); + let mut k = start; + while k < end { + let Some((dep, _repr, "")) = parse_key_line(&lines[k], indent) else { + k += 1; + continue; }; - match field { - "specifier" => spec = Some((f, fval.to_string())), - "version" => ver = Some((f, fval.to_string())), - _ => {} + let mut entry = DepEntry { + dep: dep.to_string(), + spec: None, + ver: None, + }; + k += 1; + while let Some((field, _repr, value)) = lines[..end] + .get(k) + .and_then(|l| parse_key_line(l, indent + 2)) + { + match field { + "specifier" => entry.spec = Some((k, value.to_string())), + "version" => entry.ver = Some((k, value.to_string())), + _ => {} + } + k += 1; } - f += 1; + out.push(entry); } - (spec, ver, f) + out } /// Edit 2: every importer's dep entry for the exact `name@version` — @@ -2430,92 +2429,40 @@ fn edit_importers( let mut changed = false; let mut i = start + 1; while let Some(importer) = next_block(lines.read(), i, end) { - let importer_key = importer.key.clone(); + let importer_key = importer.key; // Dep entries sit at 6-space indent under the 4-space dep-type // headers; their fields at 8. - let mut k = importer.header + 1; - while k < importer.end { - let Some((dep, _repr, rest)) = parse_key_line(&lines.read()[k], 6) else { - k += 1; + for entry in dep_entries(lines.read(), importer.header + 1, importer.end, 6) { + let (Some((si, old_spec)), Some((vi, old_ver))) = (entry.spec, entry.ver) else { continue; }; - if dep != ctx.name || !rest.is_empty() { - k += 1; + let target = old_ver == ctx.version || (old_ver != ctx.spec && ctx.is_ours(&old_ver)); + if entry.dep != ctx.name || !target { continue; } - let (spec_idx, ver_idx, f) = dep_field_lines(lines.read(), k + 1, importer.end, 8); - if let (Some((si, old_spec)), Some((vi, old_ver))) = (spec_idx, ver_idx) { - let target = - old_ver == ctx.version || (old_ver != ctx.spec && ctx.is_ours(&old_ver)); - if target { - let was_ours = ctx.is_ours(&old_ver); - let importer_spec = ctx.spec_for_importer(&importer_key); - // Built before the splices below: `dep` borrows `lines[k]`. - let record = WiringRecord { - file: PNPM_LOCK.to_string(), - kind: KIND_LOCK_IMPORTER_DEP.to_string(), - action: WiringAction::Rewritten, - key: Some(format!("{importer_key}|{dep}")), - original: if was_ours { - None - } else { - Some(serde_json::json!({ - "specifier": old_spec, - "version": old_ver, - })) - }, - new: Some(serde_json::json!({ - "specifier": importer_spec, - "version": ctx.spec, - })), - }; - let lines = lines.write(); - lines[si] = format!(" specifier: {importer_spec}"); - lines[vi] = format!(" version: {}", ctx.spec); - wiring.push(record); - changed = true; - } - } - k = f; + let importer_spec = ctx.spec_for_importer(&importer_key); + wiring.push(WiringRecord { + file: PNPM_LOCK.to_string(), + kind: KIND_LOCK_IMPORTER_DEP.to_string(), + action: WiringAction::Rewritten, + key: Some(format!("{importer_key}|{}", entry.dep)), + original: (!ctx.is_ours(&old_ver)) + .then(|| serde_json::json!({ "specifier": old_spec, "version": old_ver })), + new: Some(serde_json::json!({ + "specifier": importer_spec, + "version": ctx.spec, + })), + }); + let lines = lines.write(); + lines[si] = format!(" specifier: {importer_spec}"); + lines[vi] = format!(" version: {}", ctx.spec); + changed = true; } i = importer.end; } Ok(changed) } -/// Fail closed on a half-drifted lock: when a `[start, end)` section -/// carries BOTH the registry-keyed entry and a socket file:-keyed entry -/// for this package, a rekey would splice a DUPLICATE mapping key (pnpm -/// refuses to parse those) and surgery cannot decide which block carries -/// the truth. -/// -/// `matches` is the section's [`matching_blocks`]: every other block is -/// neither, so it cannot change the answer. -fn check_no_split_entry( - matches: &[YamlBlock], - ctx: &EditCtx<'_>, - section: &str, -) -> Result<(), String> { - let reg_key = ctx.reg_key(); - let mut has_registry = false; - let mut has_ours = false; - for block in matches { - if block.key == reg_key { - has_registry = true; - } else if ctx.is_ours_key(&block.key) { - has_ours = true; - } - } - if has_registry && has_ours { - return Err(format!( - "{section} section carries BOTH `{reg_key}` and a `{}@file:…` entry (a \ - half-edited lock); run `pnpm install` to re-resolve it, then re-vendor", - ctx.name - )); - } - Ok(()) -} - /// Edit 3: rekey the `packages:` entry and rewrite its body — /// `resolution: {integrity: , tarball: }`, a `version:` line /// inserted after it, `deprecated:` dropped, everything else verbatim. @@ -2524,16 +2471,15 @@ fn edit_packages( ctx: &EditCtx<'_>, wiring: &mut Vec, ) -> Result { - let (start, end) = lines.bounds("packages").ok_or("no packages: section")?; + let bounds = lines.bounds("packages").ok_or("no packages: section")?; let reg_key = ctx.reg_key(); let new_key = ctx.new_key(); - let matches = matching_blocks(lines, "packages", start, end, ctx); - check_no_split_entry(&matches, ctx, "packages")?; + let matches = target_blocks(lines, "packages", bounds, ctx.name, ctx.version)?; // The first registry-or-ours block is the one rewritten (every path // below returns). if let Some(block) = matches.into_iter().next() { - let is_ours_key = ctx.is_ours_key(&block.key); + let is_ours_key = ctx.target_key(&block.key) == Some(true); let original_lines: Vec = lines.read()[block.header..block.end].to_vec(); let expected_resolution = format!( " resolution: {{integrity: {}, tarball: {}}}", @@ -2605,17 +2551,16 @@ fn edit_snapshot_rekey( ctx: &EditCtx<'_>, wiring: &mut Vec, ) -> Result { - let Some((start, end)) = lines.bounds("snapshots") else { + let Some(bounds) = lines.bounds("snapshots") else { return Ok(false); // a lock without snapshots has nothing to rekey }; let reg_key = ctx.reg_key(); let new_key = ctx.new_key(); - let matches = matching_blocks(lines, "snapshots", start, end, ctx); - check_no_split_entry(&matches, ctx, "snapshots")?; + let matches = target_blocks(lines, "snapshots", bounds, ctx.name, ctx.version)?; // The first registry-or-ours block decides (every path below returns). if let Some(block) = matches.into_iter().next() { - let is_ours_key = ctx.is_ours_key(&block.key); + let is_ours_key = ctx.target_key(&block.key) == Some(true); if block.key == new_key { return Ok(false); // in sync } @@ -2725,24 +2670,23 @@ fn edit_snapshot_refs( Ok(changed) } -/// The blocks of section `name` (`[start, end)`, its current bounds) whose -/// key is the registry key `name@version` or our `name@file:` key for this -/// version, in lock order — the only blocks the packages/snapshots edits -/// act on. From the index while the lines are the memoized split (its -/// candidates are a superset, filtered by the same predicate), else by the -/// scan. +/// The blocks of `section` (`[start, end)`, its current bounds) whose key is +/// a [`target_key`] of `name@version`, in lock order — the only blocks the +/// packages/snapshots edits act on. From the index while the lines are the +/// memoized split (its candidates are a superset, filtered by the same +/// predicate), else by the scan. fn matching_blocks( lines: &L, - name: &str, + section: &str, start: usize, end: usize, - ctx: &EditCtx<'_>, + name: &str, + version: &str, ) -> Vec { - let reg_key = ctx.reg_key(); - let keep = |key: &str| key == reg_key || ctx.is_ours_key(key); - if let Some(section) = lines.index().and_then(|index| index.section(name)) { + let keep = |key: &str| target_key(key, name, version).is_some(); + if let Some(section) = lines.index().and_then(|index| index.section(section)) { return section - .candidates(®_key, ctx.name) + .candidates(name, version) .into_iter() .filter(|block| keep(&block.key)) .map(IndexedBlock::to_block) @@ -2760,6 +2704,32 @@ fn matching_blocks( found } +/// [`matching_blocks`], failing closed on a half-drifted lock: when the +/// section carries BOTH the registry entry and a rekey of it, a rekey would +/// splice a DUPLICATE mapping key (pnpm refuses to parse those) and surgery +/// cannot decide which block carries the truth. +pub(super) fn target_blocks( + lines: &L, + section: &str, + (start, end): (usize, usize), + name: &str, + version: &str, +) -> Result, String> { + let matches = matching_blocks(lines, section, start, end, name, version); + let rekeyed = |want| move |b: &&YamlBlock| target_key(&b.key, name, version) == Some(want); + match ( + matches.iter().find(rekeyed(false)), + matches.iter().any(|b| rekeyed(true)(&b)), + ) { + (Some(registry), true) => Err(format!( + "{section} section carries BOTH `{}` and a vendored `file:` rekey of it (a \ + half-edited lock); run `pnpm install` to re-resolve it, then re-vendor", + registry.key + )), + _ => Ok(matches), + } +} + // ─────────────────────── memoized split + section index ──────────────────── /// The run's `pnpm-lock.yaml` split. A vendored run reads the lock once per @@ -2839,7 +2809,7 @@ impl std::ops::Deref for LockLines { /// Read / write access to lock lines for the edits, which run both on the /// backend's [`LockLines`] and on a plain `Vec` (unit tests). `index` is /// `Some` only while the lines are exactly the ones it was built from. -trait EditLines { +pub(super) trait EditLines { fn read(&self) -> &Vec; fn write(&mut self) -> &mut Vec; fn index(&self) -> Option<&LockIndex> { @@ -2922,8 +2892,9 @@ impl IndexedBlock { /// The blocks of one top-level section, in lock order, as the scans walk /// them ([`section_bounds`] + [`next_block`]), with two lookups: blocks by -/// exact key, and blocks by the key text before each `@file:` in it (so -/// `name` finds every key starting `name@file:`). +/// key (one leading `/` dropped, as [`match_key`] reads it), and blocks by +/// the key text before each `@file:` in it (so `name` finds every key +/// starting `name@file:`, and `""` every bare `file:` key). #[derive(Default)] struct SectionIndex { bounds: Option<(usize, usize)>, @@ -2945,11 +2916,12 @@ impl SectionIndex { while let Some(block) = next_block(lines, i, end) { i = block.end; let at = index.blocks.len(); - index.by_key.entry(block.key.clone()).or_default().push(at); - for (p, _) in block.key.match_indices("@file:") { + let key = block.key.strip_prefix('/').unwrap_or(&block.key); + index.by_key.entry(key.to_string()).or_default().push(at); + for (p, _) in format!("@{}", block.key).match_indices("@file:") { index .by_file_prefix - .entry(block.key[..p].to_string()) + .entry(block.key[..p.saturating_sub(1)].to_string()) .or_default() .push(at); } @@ -2964,14 +2936,18 @@ impl SectionIndex { index } - /// Every block whose key is `reg_key` or starts with `name@file:`, in - /// lock order. - fn candidates(&self, reg_key: &str, name: &str) -> Vec<&IndexedBlock> { - let mut at: Vec = self - .by_key - .get(reg_key) - .into_iter() - .chain(self.by_file_prefix.get(name)) + /// Every block whose key may name `name@version` without a suffix + /// ([`match_key`]): its registry key in either separator, or a + /// `name@file:` / bare `file:` key, in lock order. + fn candidates(&self, name: &str, version: &str) -> Vec<&IndexedBlock> { + let mut at: Vec = [format!("{name}@{version}"), format!("{name}/{version}")] + .iter() + .filter_map(|key| self.by_key.get(key)) + .chain( + [name, ""] + .iter() + .filter_map(|n| self.by_file_prefix.get(*n)), + ) .flatten() .copied() .collect(); @@ -2992,7 +2968,7 @@ struct ImporterDep { } /// A position the snapshot scan of [`check_rewritable_refs`] may refuse -/// at: a block header (by block) or a body line. +/// at: a block header or a body line (by line). enum SnapshotCandidate { Key(usize), Body(usize), @@ -3004,7 +2980,7 @@ enum SnapshotCandidate { /// its package up instead of scanning; `*_paren` maps key on the text /// before each `(` (so `x` finds everything starting `x(`). #[derive(Default)] -struct LockIndex { +pub(super) struct LockIndex { /// First line index of every column-0 line, and all column-0 lines in /// order: [`section_bounds`] for any section. headers: HashMap, @@ -3086,53 +3062,33 @@ impl LockIndex { } } - if let Some((start, end)) = section_bounds(lines, "importers") { - let mut i = start + 1; - while let Some(importer) = next_block(lines, i, end) { - let mut k = importer.header + 1; - while k < importer.end { - let Some((dep, _repr, rest)) = parse_key_line(&lines[k], 6) else { - k += 1; - continue; - }; - if !rest.is_empty() { - k += 1; - continue; - } - let (spec, ver, f) = dep_field_lines(lines, k + 1, importer.end, 8); - if let Some((_, v)) = &ver { - let at = index.importer_deps.len(); - let v = unquote_value(v); - index.first_importer_ver.entry(v.to_string()).or_insert(at); - for prefix in paren_prefixes(v) { - index - .first_importer_ver_paren - .entry(prefix.to_string()) - .or_insert(at); - index - .first_importer_dep_ver_paren - .entry((dep.to_string(), prefix.to_string())) - .or_insert(at); - } - if spec - .as_ref() - .is_some_and(|(_, s)| unquote_value(s).starts_with("catalog:")) - { - index - .first_importer_catalog - .entry((dep.to_string(), v.to_string())) - .or_insert(at); - } - index.importer_deps.push(ImporterDep { - importer: importer.key.clone(), - dep: dep.to_string(), - spec: spec.map(|(_, s)| s), - ver: ver.map(|(_, v)| v), - }); - } - k = f; - } - i = importer.end; + index.importer_deps = importer_deps(lines); + index.importer_deps.retain(|entry| entry.ver.is_some()); + for (at, entry) in index.importer_deps.iter().enumerate() { + let (dep, v) = ( + &entry.dep, + unquote_value(entry.ver.as_deref().unwrap_or("")), + ); + index.first_importer_ver.entry(v.to_string()).or_insert(at); + for prefix in paren_prefixes(v) { + index + .first_importer_ver_paren + .entry(prefix.to_string()) + .or_insert(at); + index + .first_importer_dep_ver_paren + .entry((dep.clone(), prefix.to_string())) + .or_insert(at); + } + if entry + .spec + .as_deref() + .is_some_and(|s| unquote_value(s).starts_with("catalog:")) + { + index + .first_importer_catalog + .entry((dep.clone(), v.to_string())) + .or_insert(at); } } index @@ -3166,10 +3122,8 @@ impl LockIndex { ) -> Vec { let mut found: Vec<(usize, SnapshotCandidate)> = Vec::new(); if let Some(&block) = self.first_snapshot_key_paren.get(reg_key) { - found.push(( - self.snapshots.blocks[block].header, - SnapshotCandidate::Key(block), - )); + let header = self.snapshots.blocks[block].header; + found.push((header, SnapshotCandidate::Key(header))); } let body = [ self.first_snapshot_rest.get(reg_key), @@ -3413,84 +3367,106 @@ fn revert_importer_dep( )); return; }; + let pair = DepPair { + rec, + key, + dep, + entry_uuid, + label: "importer dep", + }; let mut i = start + 1; while let Some(importer) = next_block(lines, i, end) { - if importer.key != importer_key { - i = importer.end; - continue; + if importer.key == importer_key { + return pair.restore(lines, importer.header + 1, importer.end, 6, dirty, warnings); } - let mut k = importer.header + 1; - while k < importer.end { - let Some((d, _repr, rest)) = parse_key_line(&lines[k], 6) else { - k += 1; - continue; - }; - if d != dep || !rest.is_empty() { - k += 1; - continue; - } - let (spec_idx, ver_idx, _) = dep_field_lines(lines, k + 1, importer.end, 8); - let (Some((si, live_spec)), Some((vi, live_ver))) = (spec_idx, ver_idx) else { - break; - }; - // ALREADY CONVERGED: both fields already equal the recorded - // pre-vendor original — an earlier partial revert (or the - // user, by hand) already restored this record. Not drift: - // stay silent so the drift-skip keep gate can converge. - if let Some(original) = rec.original.as_ref() { - if original.get("specifier").and_then(Value::as_str) == Some(live_spec.as_str()) - && original.get("version").and_then(Value::as_str) == Some(live_ver.as_str()) - { - return; - } - } - let new_ver = rec - .new - .as_ref() - .and_then(|n| n.get("version")) - .and_then(Value::as_str); - let ours = Some(live_ver.as_str()) == new_ver - || parse_vendor_path(&live_ver) - .is_some_and(|p| p.eco == "npm" && p.uuid == entry_uuid); - if !ours { - warnings.push(drifted(format!( - "importer dep `{key}` was re-resolved since vendoring ({live_ver}); left alone" - ))); - return; - } - let Some(original) = rec.original.as_ref() else { - warnings.push(drifted(format!( - "importer dep `{key}` has no recorded pre-vendor original; left as-is \ - (re-run `pnpm install` to re-resolve it)" - ))); - return; - }; - let (Some(orig_spec), Some(orig_ver)) = ( - original.get("specifier").and_then(Value::as_str), - original.get("version").and_then(Value::as_str), - ) else { - warnings.push(drifted(format!( - "importer dep `{key}` original is malformed" - ))); - return; - }; - lines[si] = format!(" specifier: {orig_spec}"); - lines[vi] = format!(" version: {orig_ver}"); - *dirty = true; + i = importer.end; + } + pair.removed(warnings); +} + +/// One recorded `specifier:` / `version:` pair to restore: a v9 importer +/// dep or a legacy 6.0 root dep. +pub(super) struct DepPair<'a> { + pub(super) rec: &'a WiringRecord, + pub(super) key: &'a str, + pub(super) dep: &'a str, + pub(super) entry_uuid: &'a str, + /// Warning prefix (`importer dep` / `root dep`). + pub(super) label: &'a str, +} + +impl DepPair<'_> { + fn removed(&self, warnings: &mut Vec) { + warnings.push(removed(format!( + "{} `{}` no longer exists; nothing to restore", + self.label, self.key + ))); + } + + /// Restore the pair of the `dep` entry at `indent` in `[start, end)` + /// ([`dep_entries`]); a missing entry is "removed". + pub(super) fn restore( + &self, + lines: &mut [String], + start: usize, + end: usize, + indent: usize, + dirty: &mut bool, + warnings: &mut Vec, + ) { + let (rec, key, label) = (self.rec, self.key, self.label); + let entry = dep_entries(lines, start, end, indent) + .into_iter() + .find(|e| e.dep == self.dep); + let Some((Some((si, live_spec)), Some((vi, live_ver)))) = entry.map(|e| (e.spec, e.ver)) + else { + return self.removed(warnings); + }; + // ALREADY CONVERGED: both fields already equal the recorded + // pre-vendor original — an earlier partial revert (or the user, by + // hand) already restored this record. Not drift: stay silent so the + // drift-skip keep gate can converge. + fn field<'v>(v: Option<&'v Value>, f: &str) -> Option<&'v str> { + v.and_then(|v| v.get(f)).and_then(Value::as_str) + } + let original = rec.original.as_ref(); + if original.is_some() + && field(original, "specifier") == Some(&live_spec) + && field(original, "version") == Some(&live_ver) + { return; } - break; + let ours = field(rec.new.as_ref(), "version") == Some(&live_ver) + || parse_vendor_path(&live_ver) + .is_some_and(|p| p.eco == "npm" && p.uuid == self.entry_uuid); + if !ours { + return warnings.push(drifted(format!( + "{label} `{key}` was re-resolved since vendoring ({live_ver}); left alone" + ))); + } + if original.is_none() { + return warnings.push(drifted(format!( + "{label} `{key}` has no recorded pre-vendor original; left as-is \ + (re-run `pnpm install` to re-resolve it)" + ))); + } + let (Some(orig_spec), Some(orig_ver)) = + (field(original, "specifier"), field(original, "version")) + else { + return warnings.push(drifted(format!("{label} `{key}` original is malformed"))); + }; + let pad = " ".repeat(indent + 2); + lines[si] = format!("{pad}specifier: {orig_spec}"); + lines[vi] = format!("{pad}version: {orig_ver}"); + *dirty = true; } - warnings.push(removed(format!( - "importer dep `{key}` no longer exists; nothing to restore" - ))); } /// Restore a rekeyed packages/snapshots block: locate the block by the NEW /// key (from `rec.new`'s first line), verify ownership, splice the original /// lines back — merged with any dependency ref ANOTHER entry rewrote inside /// the block since (#830, [`merge_live_dep_refs`]). -fn revert_block( +pub(super) fn revert_block( lines: &mut Vec, rec: &WiringRecord, key: &str, @@ -3525,11 +3501,12 @@ fn revert_block( continue; } // Ours iff the live block is exactly what we wrote, or its key still - // points into OUR uuid dir (a re-serialized but unmoved entry). + // points into OUR uuid dir (a re-serialized but unmoved entry): v9's + // `name@file:` or legacy's bare `file:`. let live: Vec = lines[block.header..block.end].to_vec(); - let key_is_ours = new_key.rsplit_once("@file:").is_some_and(|(_, p)| { - parse_vendor_path(p).is_some_and(|v| v.eco == "npm" && v.uuid == entry_uuid) - }); + let path = new_key.rsplit_once("@file:").map_or(new_key, |(_, p)| p); + let key_is_ours = + parse_vendor_path(path).is_some_and(|v| v.eco == "npm" && v.uuid == entry_uuid); if live != new_lines && !key_is_ours { warnings.push(drifted(format!( "{section} entry `{new_key}` was changed since vendoring; left alone" @@ -3544,7 +3521,25 @@ fn revert_block( return; }; let restored = merge_live_dep_refs(&original, &new_lines, &live); - lines.splice(block.header..block.end, restored); + // Legacy's bare `file:` rekey moved the block across the `/` vs + // `file:` sort boundary, so it goes back to its byte-sorted place + // (byte-identity with the pre-vendor lock); v9 rekeys in place. + if !new_key.starts_with("file:") { + lines.splice(block.header..block.end, restored); + } else if let Some(orig_key) = original.first().and_then(|l| parse_key_line(l, 2)) { + let orig_key = orig_key.0.to_string(); + if swap_block_sorted(lines, section, new_key, &orig_key, &restored).is_err() { + warnings.push(drifted(format!( + "{section} entry `{new_key}` vanished mid-restore; left alone" + ))); + return; + } + } else { + warnings.push(drifted(format!( + "{section} entry `{key}` original is malformed" + ))); + return; + } *dirty = true; return; } @@ -5702,28 +5697,44 @@ snapshots: assert!(matches!(fx.vendor(false).await, VendorOutcome::Done { .. })); } - /// A CRLF pnpm-lock.yaml (Windows autocrlf checkout) passes the version - /// sniff (`str::lines` strips `\r`) but breaks every structural probe — - /// the refusal must name the line endings as the cause, not the - /// misleading "no packages entry … run `pnpm install`". + /// Divergences N15: a CRLF pnpm-lock.yaml (a Windows autocrlf checkout) + /// is wired like its LF twin and keeps its CRLF endings, as hosted + /// mode's byte splices keep them, and the revert restores the CRLF + /// bytes. Vendoring used to refuse it (`vendor_lockfile_crlf_unsupported`). #[tokio::test] - async fn crlf_lock_refuses_naming_line_endings() { + async fn crlf_lock_is_wired_like_its_lf_twin_and_stays_crlf() { let crlf_lock = P1_BEFORE_LOCK.replace('\n', "\r\n"); let fx = fixture_with(P1_BEFORE_PKG, &crlf_lock).await; + let (result, entry, _) = expect_done(fx.vendor(false).await); + assert!(result.success, "{:?}", result.error); + let wired = P1_AFTER_LOCK.replace(SPIKE_INTEGRITY, &fx.actual_integrity().await); + assert_eq!(fx.read(PNPM_LOCK).await, wired.replace('\n', "\r\n")); + assert_eq!(fx.read(PACKAGE_JSON).await, P1_AFTER_PKG); + + let outcome = revert_pnpm(&entry.unwrap(), fx.root(), false).await; + assert!(outcome.success, "{:?}", outcome.error); + assert!(outcome.warnings.is_empty(), "{:?}", outcome.warnings); + assert_eq!(fx.read(PNPM_LOCK).await, crlf_lock, "CRLF bytes restored"); + assert_eq!(fx.read(PACKAGE_JSON).await, P1_BEFORE_PKG); + assert!(!fx.root().join(".socket/vendor/npm").join(UUID).exists()); + } + + /// N15's one remaining refusal: a lock mixing CRLF and LF has no single + /// ending to write back, so it is refused naming the line endings (not + /// the misleading "no packages entry … run `pnpm install`"). + #[tokio::test] + async fn mixed_line_ending_lock_refuses_naming_line_endings() { + let mixed = P1_BEFORE_LOCK.replacen('\n', "\r\n", 1); + let fx = fixture_with(P1_BEFORE_PKG, &mixed).await; let detail = expect_refused(fx.vendor(false).await, "vendor_lockfile_crlf_unsupported"); - assert!(detail.contains("CRLF"), "{detail}"); - assert!( - !detail.contains("make sure the package is installed"), - "no misleading install remedy: {detail}" - ); - assert_eq!(fx.read(PNPM_LOCK).await, crlf_lock, "lock untouched"); + assert!(detail.contains("mixes CRLF and LF"), "{detail}"); + assert_eq!(fx.read(PNPM_LOCK).await, mixed, "lock untouched"); assert_eq!(fx.read(PACKAGE_JSON).await, P1_BEFORE_PKG, "pkg untouched"); assert!( !fx.root().join(".socket/vendor").exists(), "refusal stages no artifact" ); } - #[tokio::test] async fn dry_run_writes_nothing() { let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; @@ -6361,7 +6372,9 @@ snapshots: fn lock_version_remedy_is_version_aware() { use crate::formats::pnpm::{sniff_lock_grammar, PnpmLockGrammar}; - assert!(check_lock_version("lockfileVersion: '9.0'\n").is_ok()); + assert!(PnpmDialect::V9 + .check_lock("lockfileVersion: '9.0'\n") + .is_ok()); assert_eq!( sniff_lock_grammar("lockfileVersion: '9.0'\n"), Ok(PnpmLockGrammar::V9) @@ -6381,7 +6394,7 @@ snapshots: // …while the v9 BACKEND's own guard still holds them out (they can // only reach `vendor_pnpm` through a router bug). for head in ["lockfileVersion: 5.4\n", "lockfileVersion: '6.0'\n"] { - assert!(check_lock_version(head).is_err(), "{head}"); + assert!(PnpmDialect::V9.check_lock(head).is_err(), "{head}"); } // Too old for the allowlist (pnpm 6's 5.3): upgrade pnpm. @@ -6395,14 +6408,14 @@ snapshots: assert!(err.contains("update socket-patch"), "{err}"); assert!(!err.contains("re-lock with pnpm >= 9"), "{err}"); // The v9 backend's guard agrees on the direction. - let err = check_lock_version(head).unwrap_err(); + let err = PnpmDialect::V9.check_lock(head).unwrap_err(); assert!(err.contains("update socket-patch"), "{err}"); } // No version in the head: unrecognizably old — upgrade advice stands. let err = sniff_lock_grammar("importers:\n").unwrap_err(); assert!(err.contains("re-lock with pnpm >= 9"), "{err}"); - assert!(check_lock_version("importers:\n").is_err()); + assert!(PnpmDialect::V9.check_lock("importers:\n").is_err()); } #[test] @@ -7256,69 +7269,66 @@ snapshots: assert_eq!(fx.read(PNPM_LOCK).await, P1_BEFORE_LOCK, "lock untouched"); } - /// A CRLF pnpm-workspace.yaml (Windows autocrlf checkout) dodges every - /// structural probe — the conflict checks miss the existing `overrides:` - /// section and the edit appends a DUPLICATE one, which pnpm rejects as a - /// duplicated mapping key (bricking every pnpm command). Same fail-closed - /// posture as the lock's CRLF refusal. + /// N15 for pnpm-workspace.yaml: a CRLF file (Windows autocrlf checkout) + /// is read like its LF twin, so the existing `overrides:` section is found + /// and extended (never a DUPLICATE section, which pnpm rejects), and the + /// file keeps its CRLF endings. Vendoring used to refuse the file. #[tokio::test] - async fn crlf_workspace_file_refuses_instead_of_duplicating_overrides() { + async fn crlf_workspace_file_gains_the_override_in_its_own_section() { let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; let crlf_ws = "packages:\r\n - 'packages/*'\r\noverrides:\r\n other-pkg: 2.0.0\r\n"; write_ws(&fx, crlf_ws).await; - let detail = expect_refused(fx.vendor(false).await, "vendor_lockfile_crlf_unsupported"); - assert!(detail.contains(PNPM_WORKSPACE), "{detail}"); - assert!(detail.contains("CRLF"), "{detail}"); + let (result, entry, _) = expect_done(fx.vendor(false).await); + assert!(result.success, "{:?}", result.error); + let ws = fx.read(PNPM_WORKSPACE).await; + assert_eq!(ws.matches("overrides:").count(), 1, "{ws:?}"); + assert!(ws.contains(" other-pkg: 2.0.0\r\n"), "{ws:?}"); + assert!( + ws.contains(&format!("left-pad@1.3.0: file:{}\r\n", fx.rel_tgz())), + "{ws:?}" + ); + assert!(!ws.replace("\r\n", "").contains('\n'), "stays CRLF: {ws:?}"); + + let outcome = revert_pnpm(&entry.unwrap(), fx.root(), false).await; + assert!(outcome.success, "{:?}", outcome.error); assert_eq!( fx.read(PNPM_WORKSPACE).await, crlf_ws, - "workspace untouched" + "CRLF bytes restored" ); - assert_eq!(fx.read(PNPM_LOCK).await, P1_BEFORE_LOCK, "lock untouched"); - assert_eq!(fx.read(PACKAGE_JSON).await, P1_BEFORE_PKG, "pkg untouched"); } - /// The revert twin of the CRLF blindness: a workspace file converted to - /// CRLF since vendoring makes the Added override read as "already - /// converged" (the section probe misses it), so revert would remove the - /// artifact while the live override still points at it — pnpm >= 11 then - /// resolves through a deleted tarball. Must be a drift-keep instead. + /// The revert twin: a project vendored with LF files and then checked + /// out with autocrlf (every text file CRLF) reverts cleanly. The created + /// workspace scaffold reads as the scaffold and is deleted, the lock is + /// restored in CRLF, and the artifact goes. Before N15 the CRLF workspace + /// file was a drift-keep that held the artifact until the user + /// normalized it. #[tokio::test] - async fn crlf_workspace_revert_is_a_drift_keep_not_silent_convergence() { + async fn crlf_checkout_of_a_vendored_project_reverts_cleanly() { let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; let (_, entry, _) = expect_done(fx.vendor(false).await); let entry = entry.unwrap(); - let crlf_ws = fx.read(PNPM_WORKSPACE).await.replace('\n', "\r\n"); - tokio::fs::write(fx.root().join(PNPM_WORKSPACE), &crlf_ws) - .await - .unwrap(); + for file in [PNPM_WORKSPACE, PNPM_LOCK] { + let crlf = fx.read(file).await.replace('\n', "\r\n"); + tokio::fs::write(fx.root().join(file), crlf).await.unwrap(); + } let outcome = revert_pnpm(&entry, fx.root(), false).await; assert!(outcome.success, "{:?}", outcome.error); - assert!( - outcome - .warnings - .iter() - .any(|w| w.code == "vendor_lock_entry_drifted" && w.detail.contains(PNPM_WORKSPACE)), - "CRLF workspace must surface as drift: {:?}", - outcome.warnings - ); - assert!( - outcome.kept_artifact, - "the still-referenced artifact must be kept" - ); - assert!( - fx.root() - .join(format!(".socket/vendor/npm/{UUID}")) - .exists(), - "artifact dir survives the drift-keep" - ); + assert!(outcome.warnings.is_empty(), "{:?}", outcome.warnings); + assert!(!outcome.kept_artifact); + assert!(!fx.root().join(PNPM_WORKSPACE).exists(), "scaffold deleted"); assert_eq!( - fx.read(PNPM_WORKSPACE).await, - crlf_ws, - "CRLF file left alone" + fx.read(PNPM_LOCK).await, + P1_BEFORE_LOCK.replace('\n', "\r\n"), + "lock restored, still CRLF" ); + assert!(!fx + .root() + .join(format!(".socket/vendor/npm/{UUID}")) + .exists()); } /// A CRLF lock (a Windows autocrlf checkout) must never read as @@ -9774,19 +9784,60 @@ snapshots: assert_eq!(check_rewritable_refs(&lines, "left-pad", "1.3.0"), Ok(())); } + #[test] + fn a_sibling_checkouts_rekey_is_not_a_target() { + // Lock-model 2b (divergence N5, vendored planner): the planner reads + // its own rekeyed keys with the planning core's `match_key` + // (`origin::vendor_ref`, root-anchored), as hosted mode and discovery + // do. Before, `parse_vendor_path` found the anchor anywhere, so a + // sibling checkout's `../.socket/vendor/…` rekey was "ours" and got + // rekeyed into this project's artifact. + let art = format!(".socket/vendor/npm/{UUID}/left-pad-1.3.0.tgz"); + let target = |key: &str| target_key(key, "left-pad", "1.3.0"); + assert_eq!(target(&format!("left-pad@file:{art}")), Some(true)); + assert_eq!(target(&format!("file:{art}")), Some(true), "legacy rekey"); + assert_eq!(target("left-pad@1.3.0"), Some(false)); + assert_eq!( + target("/left-pad/1.3.0"), + Some(false), + "legacy registry key" + ); + assert_eq!(target("left-pad@1.3.0(react@18.2.0)"), None, "peer context"); + let sibling = format!("left-pad@file:../{art}"); + assert_eq!(target(&sibling), None); + let lock = split_lines(&format!( + "lockfileVersion: '9.0'\n\npackages:\n\n {sibling}:\n resolution: \ + {{integrity: sha512-x, tarball: file:../{art}}}\n version: 1.3.0\n" + )); + assert!(!lock_has_target_package(&lock, "left-pad", "1.3.0")); + } + /// A dep entry carrying a field that is neither `specifier` nor /// `version` (hand-edited or future-pnpm) is skipped, not misparsed: - /// both known fields are still found and the cursor lands past the - /// whole field run. + /// both known fields are still found and the entry ends at the first + /// non-field line. #[test] - fn dep_field_lines_ignores_unknown_fields() { + fn dep_entries_ignore_unknown_fields() { let lines = split_lines( - " specifier: ^1.0.0\n engines: whatever\n version: 1.0.0\n other:", + " dep:\n specifier: ^1.0.0\n engines: whatever\n version: 1.0.0\n other:\n inline: 1.0.0", + ); + let entries = dep_entries(&lines, 0, lines.len(), 6); + let got: Vec<_> = entries + .iter() + .map(|e| (e.dep.as_str(), e.spec.clone(), e.ver.clone())) + .collect(); + assert_eq!( + got, + vec![ + ( + "dep", + Some((1, "^1.0.0".to_string())), + Some((3, "1.0.0".to_string())) + ), + ("other", None, None), + ], + "fields end at the first non-field line; an inline value is no entry" ); - let (spec, ver, f) = dep_field_lines(&lines, 0, lines.len(), 8); - assert_eq!(spec, Some((0, "^1.0.0".to_string()))); - assert_eq!(ver, Some((2, "1.0.0".to_string()))); - assert_eq!(f, 3, "cursor stops at the first non-field line"); } /// The importer/snapshot edits are no-ops on a lock without their @@ -10122,7 +10173,7 @@ snapshots: let peer_suffixed = P1_BEFORE_LOCK.replace(" left-pad@1.3.0: {}", " left-pad@1.3.0(react@18.2.0): {}"); let absent = P1_BEFORE_LOCK.replace("left-pad@1.3.0", "left-pad@1.3.1"); - let crlf = P1_BEFORE_LOCK.replace('\n', "\r\n"); + let mixed = P1_BEFORE_LOCK.replacen('\n', "\r\n", 1); let legacy = P1_BEFORE_LOCK.replace("lockfileVersion: '9.0'", "lockfileVersion: '6.0'"); let cases: [(&str, &str, &str, &str); 6] = [ ( @@ -10150,9 +10201,9 @@ snapshots: "vendor_lock_entry_not_found", ), ( - "CRLF lock", + "mixed line-ending lock", P1_BEFORE_PKG, - &crlf, + &mixed, "vendor_lockfile_crlf_unsupported", ), ( @@ -10428,7 +10479,7 @@ snapshots: "seed {seed} refs {name}@{version}\n{text}" ); assert_eq!( - lock_has_target_package_in(&shared(&lines), name, version), + lock_has_target_package(&shared(&lines), name, version), lock_has_target_package(&lines, name, version), "seed {seed} target {name}@{version}\n{text}" ); @@ -10519,7 +10570,9 @@ snapshots: P1_BEFORE_LOCK, ))))), lock_prefix: String::new(), + lock_eol: LineEndings::Lf, ws_text: None, + ws_eol: LineEndings::None, }; let doc_of = |p: &PnpmProject| match &p.lock { ProjectLock::V9(LockLines::Shared(doc)) => Arc::clone(doc), @@ -10593,8 +10646,8 @@ snapshots: let ProjectLock::V9(lines) = &second.lock else { unreachable!() }; - assert!(!lock_has_target_package_in(lines, "left-pad", "1.3.0")); - assert!(lock_has_target_package_in(lines, "right-pad", "1.3.0")); + assert!(!lock_has_target_package(lines, "left-pad", "1.3.0")); + assert!(lock_has_target_package(lines, "right-pad", "1.3.0")); } /// pnpm-workspace.yaml spellings and document shapes the overrides surgery diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs b/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs index 549524546..09fc48634 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs @@ -64,9 +64,9 @@ use crate::utils::fs::read_regular_to_string; use super::common::refused; use super::path::parse_vendor_path; use super::pnpm_lock::{ - dep_field_lines, drifted, lines_value, merge_live_dep_refs, ref_record_blocks, removed, - revert_overrides_line, same_modulo_dep_refs, value_lines, vendor_value_is_for, DepRef, EditCtx, - PnpmDialect, KIND_LOCK_OVERRIDES, KIND_LOCK_PACKAGE, + dep_entries, drifted, lines_value, lock_has_target_package, ref_record_blocks, removed, + revert_block, revert_overrides_line, target_blocks, target_key, vendor_value_is_for, DepPair, + DepRef, EditCtx, PnpmDialect, KIND_LOCK_OVERRIDES, KIND_LOCK_PACKAGE, }; use super::source::PackageSource; use super::state::{VendorEntry, WiringAction, WiringRecord}; @@ -76,7 +76,7 @@ use crate::formats::pnpm::lines::{ next_block, parse_key_line, section_bounds, split_lines, unquote_value, yaml_key, yaml_key_like, yaml_value, }; -use crate::formats::pnpm::{sniff_lock_grammar, PnpmLock, PnpmLockGrammar}; +use crate::formats::pnpm::{PnpmLock, PnpmLockGrammar}; /// The [`VendorEntry::flavor`] string legacy wirings are stamped with. /// Distinct from the v9 backend's `"pnpm"` so an older binary (which has no @@ -230,15 +230,6 @@ pub(super) struct LegacyLock { abs_root: String, } -pub(super) fn check_lock_version(text: &str) -> Result { - match sniff_lock_grammar(text) { - Ok(PnpmLockGrammar::V9) => Err(format!( - "{PNPM_LOCK} is a lockfileVersion 9.0 lock; not a legacy grammar" - )), - result => result, - } -} - pub(super) fn read_lock( lock_text: &str, project_root: &Path, @@ -310,7 +301,7 @@ pub(super) fn preflight_package( if let Err(detail) = check_rewritable_refs(&lock.lines, &ctx) { return Err(Box::new(refused("vendor_lock_entry_unsupported", detail))); } - if !lock_has_target_package(&lock.lines, &ctx) { + if !lock_has_target_package(&lock.lines, name, version) { return Err(Box::new(refused( "vendor_lock_entry_not_found", format!( @@ -341,10 +332,7 @@ pub async fn pnpm_legacy_entry_in_use(entry: &VendorEntry, project_root: &Path) let text = read_regular_to_string(&project_root.join(PNPM_LOCK)) .await .ok()?; - match sniff_lock_grammar(&text) { - Ok(PnpmLockGrammar::V54 | PnpmLockGrammar::V60) => {} - _ => return None, - } + PnpmDialect::Legacy.check_lock(&text).ok()?; Some(PnpmLock::parse(&text).vendored_in_use(&entry.uuid)) } @@ -399,24 +387,6 @@ impl LegacyLock { // ─────────────────────────── pre-flight checks ─────────────────────────── -/// Does the lock have a packages entry vendoring can target — the grammar's -/// registry key, or our rekeyed `file:` key (the in-sync / stale-uuid -/// re-run)? -fn lock_has_target_package(lines: &[String], ctx: &Ctx<'_>) -> bool { - let Some((start, end)) = section_bounds(lines, "packages") else { - return false; - }; - let reg_key = ctx.reg_key(); - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - if block.key == reg_key || ctx.is_ours(&block.key) { - return true; - } - i = block.end; - } - false -} - /// Fail-closed guard against legacy reference forms the surgery does not /// rewrite: PEER-SUFFIXED dep paths (`/name/version_peer…` in v5.4, /// `/name@version(peer…)` in v6.0) and ALIASED references (a dep whose @@ -436,29 +406,20 @@ fn check_rewritable_refs(lines: &[String], ctx: &Ctx<'_>) -> Result<(), String> ctx.name, ctx.version )) }; - // Root dep maps (v5.4 bare values; v6.0 version: fields). + // Root dep maps: v5.4 bare values, v6.0 nested `version:` fields. for section in ROOT_DEP_SECTIONS { let Some((start, end)) = section_bounds(lines, section) else { continue; }; - let mut k = start + 1; - while k < end { - let Some((dep, _repr, rest)) = parse_key_line(&lines[k], 2) else { - k += 1; - continue; - }; - let value = if rest.is_empty() { - // v6.0 nested shape: read the version: field. - let (_, ver, f) = dep_field_lines(lines, k + 1, end, 4); - k = f; - match ver { - Some((_, v)) => v, - None => continue, - } - } else { - k += 1; - rest.to_string() - }; + let bare = lines[start + 1..end] + .iter() + .filter_map(|line| parse_key_line(line, 2)) + .filter(|(_, _, value)| !value.is_empty()) + .map(|(dep, _, value)| (dep.to_string(), value.to_string())); + let nested = dep_entries(lines, start + 1, end, 2) + .into_iter() + .filter_map(|entry| Some((entry.dep, entry.ver?.1))); + for (dep, value) in bare.chain(nested) { if value == reg_key || value.starts_with(&key_peer_prefix) { return refuse("an aliased root dependency", &value); } @@ -617,52 +578,32 @@ fn edit_root_deps_v60( let Some((start, end)) = section_bounds(lines, section) else { continue; }; - let mut k = start + 1; - while k < end { - let Some((dep, _repr, rest)) = parse_key_line(&lines[k], 2) else { - k += 1; + for entry in dep_entries(lines, start + 1, end, 2) { + let (Some((si, old_spec)), Some((vi, old_ver))) = (entry.spec, entry.ver) else { continue; }; - if dep != ctx.name || !rest.is_empty() { - k += 1; + if entry.dep != ctx.name || !(old_ver == ctx.version || ctx.is_ours(&old_ver)) { continue; } - let (spec_f, ver_f, f) = dep_field_lines(lines, k + 1, end, 4); - if let (Some((si, old_spec)), Some((vi, old_ver))) = (spec_f, ver_f) { - let target = old_ver == ctx.version || ctx.is_ours(&old_ver); - if target { - hit = true; - if old_ver == ctx.spec && old_spec == abs_yaml { - k = f; - continue; // in sync - } - let was_ours = ctx.is_ours(&old_ver); - // Built before the splices below: `dep` borrows `lines[k]`. - let record = WiringRecord { - file: PNPM_LOCK.to_string(), - kind: KIND_LOCK_ROOT_DEP_PAIR.to_string(), - action: WiringAction::Rewritten, - key: Some(format!("{section}|{dep}")), - original: if was_ours { - None - } else { - Some(serde_json::json!({ - "specifier": old_spec, - "version": old_ver, - })) - }, - new: Some(serde_json::json!({ - "specifier": ctx.abs_spec, - "version": ctx.spec, - })), - }; - lines[si] = format!(" specifier: {abs_yaml}"); - lines[vi] = format!(" version: {}", ctx.spec); - wiring.push(record); - changed = true; - } + hit = true; + if old_ver == ctx.spec && old_spec == abs_yaml { + continue; // in sync } - k = f; + wiring.push(WiringRecord { + file: PNPM_LOCK.to_string(), + kind: KIND_LOCK_ROOT_DEP_PAIR.to_string(), + action: WiringAction::Rewritten, + key: Some(format!("{section}|{}", entry.dep)), + original: (!ctx.is_ours(&old_ver)) + .then(|| serde_json::json!({ "specifier": old_spec, "version": old_ver })), + new: Some(serde_json::json!({ + "specifier": ctx.abs_spec, + "version": ctx.spec, + })), + }); + lines[si] = format!(" specifier: {abs_yaml}"); + lines[vi] = format!(" version: {}", ctx.spec); + changed = true; } } Ok((changed, hit)) @@ -677,37 +618,14 @@ fn edit_packages( ctx: &Ctx<'_>, wiring: &mut Vec, ) -> Result { - let (start, end) = section_bounds(lines, "packages").ok_or("no packages: section")?; + let bounds = section_bounds(lines, "packages").ok_or("no packages: section")?; let reg_key = ctx.reg_key(); let new_key = ctx.new_key(); - - // Fail closed on a half-drifted lock carrying BOTH spellings. - let mut has_registry = false; - let mut has_ours = false; - let mut j = start + 1; - while let Some(block) = next_block(lines, j, end) { - if block.key == reg_key { - has_registry = true; - } else if ctx.is_ours(&block.key) { - has_ours = true; - } - j = block.end; - } - if has_registry && has_ours { - return Err(format!( - "packages section carries BOTH `{reg_key}` and a `file:…` entry (a \ - half-edited lock); run `pnpm install` to re-resolve it, then re-vendor" - )); - } - - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - let is_registry = block.key == reg_key; - let is_ours_key = ctx.is_ours(&block.key); - if !is_registry && !is_ours_key { - i = block.end; - continue; - } + let matches = target_blocks(&*lines, "packages", bounds, ctx.name, ctx.version)?; + // The first registry-or-ours block is the one rewritten (every path + // below returns). + if let Some(block) = matches.into_iter().next() { + let is_ours_key = target_key(&block.key, ctx.name, ctx.version) == Some(true); let original_lines: Vec = lines[block.header..block.end].to_vec(); let expected_resolution = format!( " resolution: {{integrity: {}, tarball: {}}}", @@ -844,7 +762,7 @@ fn edit_pkg_dep_refs( /// sort before `file:`-keys). pnpm sorts package keys with the `sort-keys` /// default compare (JS code-unit order), which plain Rust `str` ordering /// matches for these ASCII keys. -fn swap_block_sorted( +pub(super) fn swap_block_sorted( lines: &mut Vec, section: &str, old_key: &str, @@ -940,7 +858,7 @@ pub(super) fn revert_lock_record( KIND_LOCK_ROOT_DEP_PAIR => { revert_root_dep_pair(lines, rec, key, entry_uuid, dirty, warnings) } - KIND_LOCK_PACKAGE => revert_package_block(lines, rec, key, entry_uuid, dirty, warnings), + KIND_LOCK_PACKAGE => revert_block(lines, rec, key, "packages", entry_uuid, dirty, warnings), KIND_LOCK_PKG_DEP_REF => revert_pkg_dep_ref(lines, rec, key, entry_uuid, dirty, warnings), other => warnings.push(drifted(format!( "unknown wiring kind `{other}` for `{key}`; left alone" @@ -1030,156 +948,14 @@ fn revert_root_dep_pair( ))); return; }; - let mut k = start + 1; - while k < end { - let Some((d, _repr, rest)) = parse_key_line(&lines[k], 2) else { - k += 1; - continue; - }; - if d != dep || !rest.is_empty() { - k += 1; - continue; - } - let (spec_f, ver_f, _) = dep_field_lines(lines, k + 1, end, 4); - let (Some((si, live_spec)), Some((vi, live_ver))) = (spec_f, ver_f) else { - break; - }; - // ALREADY CONVERGED: both fields already equal the recorded - // pre-vendor original — an earlier partial revert (or the user, by - // hand) already restored this record. Not drift: stay silent so - // the drift-skip keep gate can converge. - if let Some(original) = rec.original.as_ref() { - if original.get("specifier").and_then(Value::as_str) == Some(live_spec.as_str()) - && original.get("version").and_then(Value::as_str) == Some(live_ver.as_str()) - { - return; - } - } - let new_ver = rec - .new - .as_ref() - .and_then(|n| n.get("version")) - .and_then(Value::as_str); - let ours = Some(live_ver.as_str()) == new_ver - || parse_vendor_path(&live_ver).is_some_and(|p| p.eco == "npm" && p.uuid == entry_uuid); - if !ours { - warnings.push(drifted(format!( - "root dep `{key}` was re-resolved since vendoring ({live_ver}); left alone" - ))); - return; - } - let Some(original) = rec.original.as_ref() else { - warnings.push(drifted(format!( - "root dep `{key}` has no recorded pre-vendor original; left as-is \ - (re-run `pnpm install` to re-resolve it)" - ))); - return; - }; - let (Some(orig_spec), Some(orig_ver)) = ( - original.get("specifier").and_then(Value::as_str), - original.get("version").and_then(Value::as_str), - ) else { - warnings.push(drifted(format!("root dep `{key}` original is malformed"))); - return; - }; - lines[si] = format!(" specifier: {orig_spec}"); - lines[vi] = format!(" version: {orig_ver}"); - *dirty = true; - return; - } - warnings.push(removed(format!( - "root dep `{key}` no longer exists; nothing to restore" - ))); -} - -/// Restore the rekeyed packages block: locate it by the NEW `file:` key, -/// verify ownership, then reinsert the ORIGINAL block at its byte-sorted -/// position (the rekey moved it across the `/` vs `file:` sort boundary, so -/// an in-place splice would restore it out of order and break byte-identity -/// with the pre-vendor lock) — merged with any dependency ref ANOTHER entry -/// rewrote inside the block since (#830, [`merge_live_dep_refs`]). -fn revert_package_block( - lines: &mut Vec, - rec: &WiringRecord, - key: &str, - entry_uuid: &str, - dirty: &mut bool, - warnings: &mut Vec, -) { - let Some(new_lines) = rec.new.as_ref().and_then(value_lines) else { - warnings.push(drifted(format!( - "record for `{key}` has no `new` fragment; left alone" - ))); - return; - }; - let Some((new_key, _repr, _rest)) = new_lines.first().and_then(|l| parse_key_line(l, 2)) else { - warnings.push(drifted(format!( - "record for `{key}` has a malformed fragment" - ))); - return; - }; - let Some((start, end)) = section_bounds(lines, "packages") else { - warnings.push(drifted(format!( - "packages section is gone; `{key}` not restored" - ))); - return; + let pair = DepPair { + rec, + key, + dep, + entry_uuid, + label: "root dep", }; - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - if block.key != new_key { - i = block.end; - continue; - } - let live: Vec = lines[block.header..block.end].to_vec(); - let key_is_ours = - parse_vendor_path(new_key).is_some_and(|p| p.eco == "npm" && p.uuid == entry_uuid); - if live != new_lines && !key_is_ours { - warnings.push(drifted(format!( - "packages entry `{new_key}` was changed since vendoring; left alone" - ))); - return; - } - let Some(original) = rec.original.as_ref().and_then(value_lines) else { - warnings.push(drifted(format!( - "packages entry `{key}` has no recorded pre-vendor original; left as-is \ - (re-run `pnpm install` to re-resolve it)" - ))); - return; - }; - let Some((orig_key, _r, _v)) = original.first().and_then(|l| parse_key_line(l, 2)) else { - warnings.push(drifted(format!( - "packages entry `{key}` original is malformed" - ))); - return; - }; - let restored = merge_live_dep_refs(&original, &new_lines, &live); - if swap_block_sorted(lines, "packages", new_key, orig_key, &restored).is_err() { - warnings.push(drifted(format!( - "packages entry `{new_key}` vanished mid-restore; left alone" - ))); - return; - } - *dirty = true; - return; - } - // ALREADY CONVERGED: an earlier partial revert restored this record — - // the restore rekeys the block back to its pre-vendor key, so the - // recorded `file:` key no longer matches while the original block is - // live (up to the dependency refs the merge kept). Not drift: stay - // silent so the drift-skip keep gate can converge instead of keeping - // the artifacts forever. - if let Some(orig) = rec.original.as_ref().and_then(value_lines) { - let mut j = start + 1; - while let Some(block) = next_block(lines, j, end) { - if same_modulo_dep_refs(&lines[block.header..block.end], &orig) { - return; - } - j = block.end; - } - } - warnings.push(removed(format!( - "packages entry `{new_key}` no longer exists; nothing to restore" - ))); + pair.restore(lines, start + 1, end, 2, dirty, warnings); } /// Restore one dependent packages block's dep ref to this entry. Keyed by @@ -2622,14 +2398,21 @@ packages: assert_eq!(fx.read(PNPM_LOCK).await, lock, "refusal writes nothing"); } - /// CRLF and non-allowlisted lock versions refuse with the same codes - /// and byte-untouched files as the v9 backend. + /// Divergences N15 on a legacy lock: a CRLF 5.4 lock is wired like its + /// LF twin (the captured pnpm 7 shape) and stays CRLF; vendoring used to + /// refuse it. Non-allowlisted lock versions still refuse, with + /// byte-untouched files, as the v9 backend does (N16, target-specific). #[tokio::test] - async fn crlf_and_foreign_versions_refuse() { + async fn crlf_is_wired_and_foreign_versions_refuse() { let crlf = T7_BEFORE_LOCK.replace('\n', "\r\n"); let fx = fixture_with(T_BEFORE_PKG, &crlf).await; - expect_refused(fx.vendor(false).await, "vendor_lockfile_crlf_unsupported"); - assert_eq!(fx.read(PNPM_LOCK).await, crlf, "refusal writes nothing"); + let (result, entry, _) = expect_done(fx.vendor(false).await); + assert!(result.success, "{:?}", result.error); + let wired = fx.expected_lock(T7_AFTER_LOCK).await; + assert_eq!(fx.read(PNPM_LOCK).await, wired.replace('\n', "\r\n")); + let outcome = revert_pnpm_legacy(&entry.unwrap(), fx.root(), false).await; + assert!(outcome.success, "{:?}", outcome.error); + assert_eq!(fx.read(PNPM_LOCK).await, crlf, "CRLF bytes restored"); // pnpm 6's 5.3 is NOT allowlisted. let old = T7_BEFORE_LOCK.replace("lockfileVersion: 5.4", "lockfileVersion: 5.3"); @@ -2858,8 +2641,8 @@ packages: ); } - /// Wiring records left alone during a WIRED revert (here: a - /// CRLF-converted lock whose LF-exact probes all miss) must KEEP the + /// Wiring records left alone during a WIRED revert (here: a root dep + /// re-resolved by hand since vendoring) must KEEP the /// artifact dir — the lock still resolves through the tarball, and /// deleting it bricks every subsequent install (the lossiness guard, /// present in the v9/npm/bun backends). @@ -2868,8 +2651,12 @@ packages: let fx = fixture_with(T_BEFORE_PKG, T7_BEFORE_LOCK).await; let (_, entry, _) = expect_done(fx.vendor(false).await); let entry = entry.unwrap(); - let crlf = fx.read(PNPM_LOCK).await.replace('\n', "\r\n"); - tokio::fs::write(fx.root().join(PNPM_LOCK), &crlf) + let drifted = fx.read(PNPM_LOCK).await.replacen( + &format!(" left-pad: file:{}\n", fx.rel_tgz()), + " left-pad: 1.3.1\n", + 1, + ); + tokio::fs::write(fx.root().join(PNPM_LOCK), &drifted) .await .unwrap(); @@ -2890,12 +2677,11 @@ packages: ); assert!( fx.root().join(fx.rel_tgz()).exists(), - "the CRLF lock still resolves through the tarball; deleting it bricks installs" + "the lock still resolves through the tarball; deleting it bricks installs" ); - assert_eq!( - fx.read(PNPM_LOCK).await, - crlf, - "the drifted lock is left alone" + assert!( + fx.read(PNPM_LOCK).await.contains(" left-pad: 1.3.1\n"), + "the drifted line is left alone" ); } From 4a84f2da4ef3cdf6e3b18a5bfe36b1cbb57bf4e7 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Thu, 5 Nov 2026 13:23:55 -0600 Subject: [PATCH 14/36] Mark N5, N15 and N16 for the pnpm planner merge; rebudget N5's vendored-planner part and N15 are resolved in PR 2b. N16 stays target-specific for pnpm vendored mode. scripts/loc-budget.py --lower: lock-readers 31207 -> 31201, lock-writers 88314 -> 88093. Co-Authored-By: Claude Opus 5.5 --- docs/design/lock-model-divergences.md | 6 +++--- scripts/loc-budget.json | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md index a25f67760..c77e7e7f3 100644 --- a/docs/design/lock-model-divergences.md +++ b/docs/design/lock-model-divergences.md @@ -83,7 +83,7 @@ follow from the install-tree decision. | N2 | npm user patches (#711) and `bun patch` (#367) | Hosted skips that one dependency and records it as refused. Vendored refuses the whole npm v4 lock, or the one bun dependency. Discovery never reads `patchedDependencies` / `"patched"` and attests a pre-#711 hosted pin. | Hosted's per-dependency skip, in discovery too | false attestation | | N3 | A workspace member's stray npm/bun/vlt lock | Discovery treats it as absent. Hosted and vendored refuse. The inventory reads it as the project's lock. | The inventory ignores it | | | N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by content, not position. **Resolved in PR 2a**: every reader reads both documents through `PnpmLock`, each entry tagged by document; discovery records carry the env document's install tree (`TreePart::PnpmEnv`) beside the real lock path, and that tree neither contests nor is contested by the project document or another lock; the env document is the FIRST document when its importers carry only `configDependencies` / `packageManagerDependencies` (pnpm's own position rule), and every other document is read as the project. | | -| N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere. pnpm part **resolved in PR 2a**: the hosted pnpm "vendored?" check reads the rekeyed packages entry with `origin::vendor_ref`. | revert could rewrite another project's wiring | +| N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere. pnpm part **resolved in PR 2a**: the hosted pnpm "vendored?" check reads the rekeyed packages entry with `origin::vendor_ref`. **PR 2b**: both modes read pnpm keys with one rule, `formats::pnpm::plan::match_key` (`origin::vendor_ref` plus the artifact leaf), so the vendored planner no longer takes a sibling checkout's rekey for its own. The vendored reverter still finds its wiring with `parse_vendor_path` (values such as pnpm 7/8's absolute specifier need it). | revert could rewrite another project's wiring | | N6 | `npm-shrinkwrap.json` and `package-lock.json` | Hosted, vendored and discovery read both, with a contest. The inventory reads the shrinkwrap only. | Both, everywhere | | | N7 | npm lockfileVersion | Hosted rewrites v1 and v4. Vendored accepts v2/v3 only. The inventory ignores v1. Discovery reads v1 and v2+. | Hosted, once vendored `file:` wiring under npm 6 and npm 12.1 v4 has fixtures | | | N8 | npm `file:` directory link identity (#1481) | Hosted and discovery give a versionless link its source directory's `name@version`. Vendored matches the entry's own version, wiring the registry copies with no warning. | Hosted | silent vendored success | @@ -93,8 +93,8 @@ follow from the install-tree decision. | N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. **Resolved in PR 2a** (the inventory reads them whenever `rush.json` exists). | target-specific (vendored) | | N13 | Plug'n'Play | Vendored refuses. The hosted berry preflight has no PnP gate. The inventory returns an error, so `scan` sees nothing in a project hosted pinned. | The inventory reads PnP locks. Vendored keeps its refusal. | target-specific (vendored) | | N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery. **Resolved in PR 2a**: under the setting (not under Rush), the inventory keeps a Socket-hosted pnpm entry but drops its integrity, as discovery withholds the lock pin. | target-specific | -| N15 | CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml` | Vendored refuses; the rest handle it | Handle it everywhere | | -| N16 | Lock versions | pnpm: vendored accepts only 5.4/6.0/9.0; hosted any but early shrinkwrap. vlt: vendored refuses a lock with no version; hosted rewrites and warns. | Hosted | | +| N15 | CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml` | Vendored refuses; the rest handle it | Handle it everywhere. **Resolved in PR 2b**: the vendored planner and reverter edit a CRLF lock or workspace file like its LF twin and write it back CRLF. Only a file mixing CRLF and LF is still refused (`vendor_lockfile_crlf_unsupported`), since no single ending writes it back byte-faithfully. | | +| N16 | Lock versions | pnpm: vendored accepts only 5.4/6.0/9.0; hosted any but early shrinkwrap. vlt: vendored refuses a lock with no version; hosted rewrites and warns. | Hosted. **pnpm, PR 2b**: one table, `plan::Target::check_lock`. Vendored keeps its 5.4/6.0/9.0 range: no other version has a captured pnpm `file:` override shape that proves its rekey, so the other versions stay refused, target-specifically. vlt is unchanged. | target-specific (pnpm vendored) | | N17 | Bun workspaces, yarn classic offline mirror | Refused differently by the two modes | Stays | target-specific | | N18 | An unparseable npm lock | Hosted warns and skips. Vendored refuses the primary lock. Discovery diagnoses. The inventory returns nothing silently. | Discovery's diagnostic, surfaced by every consumer | | | N19 | yarn.lock with a stray top-level line | Discovery rejects the file; hosted and the inventory scan its blocks | Discovery's rule | | diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json index b31d39de7..8ffa1aab4 100644 --- a/scripts/loc-budget.json +++ b/scripts/loc-budget.json @@ -17,7 +17,7 @@ "crates/socket-patch-core/src/utils/pipenv.rs", "crates/socket-patch-core/src/crawlers/deno_crawler.rs" ], - "max": 31207 + "max": 31201 }, "lock-writers": { "paths": [ @@ -30,7 +30,7 @@ "crates/socket-patch-core/src/vendor/npm_origin.rs", "crates/socket-patch-core/src/vendor/test_support" ], - "max": 88314 + "max": 88093 } } } From 56392e18261c4fe8c5c5e047b6cfa273ed65be73 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Thu, 5 Nov 2026 14:37:39 -0600 Subject: [PATCH 15/36] Shadow a shrinkwrap.yaml beside a pnpm-lock.yaml (N11) Owner decision on divergence N11: pnpm never reads a pnpm <= 2 shrinkwrap.yaml it has a pnpm-lock.yaml for, so that file is shadowed. The planning core's planned_locks leaves it out, so hosted mode never rewrites it and it no longer takes part in the all-or-nothing gate. Before, an entry the splice cannot repoint, or pnpm 1's early-shrinkwrap head, refused the dependency in every lock. Discovery keeps sweeping it for stale Socket identities, and vendored mode never reads it. A shrinkwrap.yaml that is the only pnpm lock (nested ones are shadowed only by their own sibling) is rewritten as before. No redirect golden changes. Co-Authored-By: Claude Opus 5.5 --- crates/socket-patch-cli/CLI_CONTRACT.md | 2 +- .../src/formats/pnpm/hosted.rs | 37 ++++------ .../src/formats/pnpm/plan.rs | 54 ++++++++++++-- .../src/patch/redirect/mod.rs | 70 +++++++++++++++++++ docs/design/lock-model-divergences.md | 4 +- docs/ecosystems.md | 3 +- 6 files changed, 135 insertions(+), 35 deletions(-) diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index a3ed909d6..3f491176e 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -272,7 +272,7 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc **pnpm per-branch locks (#556)**: with `gitBranchLockfile` on (a top-level `gitBranchLockfile: true` in the root `pnpm-workspace.yaml`; when the YAML leaves the key unset, `git-branch-lockfile=true` in the root `.npmrc`, as pnpm 10 and older read it; on disk also `pnpm_config_git_branch_lockfile` (pnpm 11 and newer) or `npm_config_git_branch_lockfile` (pnpm 10 and older; either when the major is unknown, read as for the `lockfile` setting above) set to `true`; a workspace member with no `pnpm-workspace.yaml` of its own reads the governing ancestor's file and the `.npmrc` beside it), pnpm installs a git branch from its own `pnpm-lock..yaml` whenever that lock exists, in every directory it installs a lock from: the root, and under `sharedWorkspaceLockfile: false` each member's (#492). When the setting is on AND at least one such `pnpm-lock..yaml` exists (at the root, or in a member that installs from its own lock), hosted mode leaves every root pnpm lock (`pnpm-lock.yaml`, `shrinkwrap.yaml`) and `pnpm-workspace.yaml` untouched (every member lock too), confirms no npm pin from them and warns `redirect_pnpm_git_branch_lockfile` (in place of `redirect_pnpm_no_lockfile` when the branch lock is the only one); the detail names the setting and the branch lock(s), and the remedy is to turn `gitBranchLockfile` off and fold the branch locks into `pnpm-lock.yaml` with `pnpm install --merge-git-branch-lockfiles`, then re-run. A vendored→hosted takeover of a pnpm-vendored purl is refused with the same code (the vendored wiring and ledger stay byte-for-byte). Vendored mode refuses the same projects with `vendor_pnpm_git_branch_lockfile`, before any write and in the pre-flight. The setting with no branch lock (main, or a branch whose dependencies never changed) installs from `pnpm-lock.yaml`, which both modes wire as usual; a stray branch lock with the setting off is ignored. The in-memory engine fetches root `pnpm-lock.*.yaml` names presence-only and refuses the same way. -`redirect_pnpm_no_lockfile` names pnpm when installer markers exist without a lock; `redirect_pnpm_entry_vendored` identifies a vendored entry instead of reporting it missing. Supported `shrinkwrap.yaml` files are writable lockfiles, not read-only markers. +`redirect_pnpm_no_lockfile` names pnpm when installer markers exist without a lock; `redirect_pnpm_entry_vendored` identifies a vendored entry instead of reporting it missing. Supported `shrinkwrap.yaml` files are writable lockfiles, not read-only markers, when no `pnpm-lock.yaml` sits beside them. pnpm never reads a `shrinkwrap.yaml` it has a `pnpm-lock.yaml` for, so that one is shadowed: hosted mode never rewrites it and leaves it out of the all-or-nothing gate, so an entry it cannot repoint (or pnpm 1's early shrinkwrap head) refuses nothing, and `vex` only sweeps it for stale Socket identities. Vendored mode never writes it either. **vlt hosted-mode contract**: `scan` / `get --mode hosted` rewrite, in `vlt-lock.json`, every default-registry node of a granted `name@version` (the `''` / `npm` segment or a URL segment equal to the lock's scalar `registry`, both DepID grammars, every peer and modifier variant): slot [2] becomes the granted sha512 and slot [3] the hosted URL (appended to a 3-tuple); the DepID, flags and trailing slots, the line ending and every other byte stay. `options` is never edited and `vlt.json` is only read. A lock with another `lockfileVersion` (decided on the raw JSON token), a BOM, a non-object body or a `nodes` section outside vlt's one-node-per-line layout refuses the whole lock (`redirect_vlt_lock_unsupported`). **Confirmation**: vlt drives when its install state (`node_modules/.vlt-lock.json` or `node_modules/.vlt/`) is present or no other npm-family lock is; then only `vlt-lock.json` confirms a uuid. Otherwise every lock is rewritten, `redirect_vlt_sibling_lockfiles` warns, and the other locks' rules confirm, including a dep `vlt-lock.json` merely does not wire (`redirect_vlt_entry_not_found`, `redirect_vlt_entry_vendored`, or `redirect_vlt_custom_registry_skipped` alone). Whichever lock drives, a dep the vlt rewriter refuses (`redirect_vlt_missing_sha512`, `redirect_vlt_unsupported_lock_key`) is never confirmed by any lock, although a sibling lock may already carry its rewritten URL. **Artifact preflight**: before any takeover or write (dry runs included), each granted artifact with a default-registry instance is fetched once as vlt fetches it and must verify, else the dep is withheld (`redirect_vlt_artifact_unverifiable`, see the tag table). **Heal**: stale installed copies of Socket-owned nodes are removed so the next `vlt install` extracts the patched bytes, and `rollback` / `remove` do the same for the registry bytes (`--no-vlt-install-cleanup` keeps them; optional dependencies' copies are always kept); `redirect_vlt_reinstall_required` says what happened and what to run. The same-run `--vex` never attests a vlt package whose installed copy is stale or unchecked, whose lock a vlt release may ignore (`redirect_vlt_lockfile_version_missing`, `redirect_vlt_old_lockfile_ignored`, `redirect_vlt_scalar_registry_ignored`), or which also resolves from a non-default registry (`redirect_vlt_custom_registry_skipped`). `vlt.json` or vlt install state without `vlt-lock.json` warns `redirect_vlt_no_lockfile` instead of `redirect_npm_no_lockfile`. `rollback` / `remove` restore each hosted node's slots [2] and [3] from the version document of the registry the node resolves against (slot [3] is its `dist.tarball`, as vlt writes it; `upstream_registry_fallback` when that registry can't be read), following the lock's own slot-[3] convention (see "Hosted unwind coverage"). Tested releases: `docs/testing/vlt-compatibility.md`. diff --git a/crates/socket-patch-core/src/formats/pnpm/hosted.rs b/crates/socket-patch-core/src/formats/pnpm/hosted.rs index 14f8e05bb..e16f3979d 100644 --- a/crates/socket-patch-core/src/formats/pnpm/hosted.rs +++ b/crates/socket-patch-core/src/formats/pnpm/hosted.rs @@ -11,7 +11,7 @@ use serde_json::Value; use crate::patch::redirect::{full_name, DepOverride, FileEdit, RewriteResult, RewriteWarning}; use super::grammar as pnpm; -use super::plan::{match_key, KeyMatch, Target}; +use super::plan::{match_key, planned_locks, KeyMatch, Target}; /// Whether `entry` resolves to exactly `artifact_url` — the per-instance /// residual-gate predicate. @@ -107,13 +107,6 @@ impl<'f> PnpmLockState<'f> { hits } - /// Whether a vendored rewire rekeyed `fname@version` - /// ([`KeyMatch::Rekeyed`]). Read from `text` without folding `pending` - /// in: a resolution splice never touches a key (see the type docs). - fn vendors(&self, fname: &str, version: &str) -> bool { - !self.lookup(fname, version, true).is_empty() - } - /// The entries [`match_key`] matches as a registry instance or, with /// `rekeyed`, as a vendored rekey, found by their key prefixes. fn lookup(&self, fname: &str, version: &str, rekeyed: bool) -> Vec { @@ -135,10 +128,9 @@ impl<'f> PnpmLockState<'f> { } out.sort_unstable(); out.dedup(); - out.retain(|&i| match match_key(self.entry(i).key, fname, version) { - Some(KeyMatch::Registry { .. }) => !rekeyed, - Some(KeyMatch::Rekeyed) => rekeyed, - None => false, + out.retain(|&i| { + match_key(self.entry(i).key, fname, version) + .is_some_and(|m| (m == KeyMatch::Rekeyed) == rekeyed) }); out } @@ -192,17 +184,10 @@ pub(crate) fn plan_hosted( let npm: Vec<&DepOverride> = overrides.iter().filter(|o| o.ecosystem == "npm").collect(); // A pnpm lock lives at the project root or at any nested path (e.g. Rush // repos keep them under `common/config/rush/`); every such files-map key - // is rewritten under the same grammar. Deterministic order: BTreeMap - // iterates keys sorted, so goldens are stable across every lock in the set. - let lock_keys: Vec<&String> = files - .keys() - .filter(|k| { - matches!( - k.rsplit('/').next(), - Some("pnpm-lock.yaml" | "shrinkwrap.yaml") - ) - }) - .collect(); + // is rewritten under the same grammar, except a `shrinkwrap.yaml` a + // `pnpm-lock.yaml` shadows ([`planned_locks`]). Deterministic order: + // BTreeMap iterates keys sorted, so goldens are stable across every lock. + let lock_keys = planned_locks(&files.keys().collect::>()); if npm.is_empty() || lock_keys.is_empty() { return; } @@ -360,7 +345,11 @@ pub(crate) fn plan_hosted( // mode switch. Fail-closed either way: nothing is rewritten for the // dep. if !matched_any { - let vendored = locks.iter().any(|lock| lock.vendors(&fname, &dep.version)); + // A rekey ([`KeyMatch::Rekeyed`]) is read from `text` without + // folding `pending` in: a resolution splice never touches a key. + let vendored = locks + .iter() + .any(|lock| !lock.lookup(&fname, &dep.version, true).is_empty()); if vendored { result.warnings.push(RewriteWarning { code: "redirect_pnpm_entry_vendored".into(), diff --git a/crates/socket-patch-core/src/formats/pnpm/plan.rs b/crates/socket-patch-core/src/formats/pnpm/plan.rs index a2fa56c1d..54361ecb2 100644 --- a/crates/socket-patch-core/src/formats/pnpm/plan.rs +++ b/crates/socket-patch-core/src/formats/pnpm/plan.rs @@ -18,7 +18,6 @@ use super::grammar; use super::{sniff_lock_grammar, PnpmLockGrammar}; -use crate::constants::npm_family::PNPM_LOCK; use crate::formats::text::strip_bom; use crate::vendor::npm_common::tgz_rel_leaf; @@ -52,13 +51,10 @@ impl Target { .is_some_and(|v| v.trim().trim_matches(['\'', '"']) == "3") && !field("shrinkwrapMinorVersion:") .is_some_and(|v| v.trim().parse::().is_ok_and(|v| v > 0)); - match early { - true => Err(format!( - "{PNPM_LOCK} uses early pnpm 1 shrinkwrapVersion 3 without a supported \ - minor version" - )), - false => Ok(None), + if early { + return Err("early pnpm 1 shrinkwrapVersion 3 without a minor version".into()); } + Ok(None) } /// Whether this target's edit repoints a registry instance whose key @@ -82,6 +78,29 @@ impl Target { depth == 0 } } +/// The pnpm locks among `keys` (root-relative paths) a planner rewrites, in +/// their order: every `pnpm-lock.yaml`, and a pnpm <= 2 `shrinkwrap.yaml` +/// only when no `pnpm-lock.yaml` sits beside it. pnpm never reads a +/// `shrinkwrap.yaml` it has a `pnpm-lock.yaml` for, so that one is +/// SHADOWED (owner decision on divergence N11): never rewritten, never part +/// of the all-or-nothing gate (debris in it cannot refuse a dependency), +/// and only swept for stale Socket identities by lockfile discovery. +pub(crate) fn planned_locks<'k>(keys: &[&'k String]) -> Vec<&'k String> { + // `dir` keeps its trailing `/` (empty at the root). + let shadowed = |key: &str| { + key.strip_suffix("shrinkwrap.yaml").is_some_and(|dir| { + keys.iter() + .any(|k| k.strip_suffix("pnpm-lock.yaml") == Some(dir)) + }) + }; + let planned = |key: &&String| match key.rsplit('/').next() { + Some("pnpm-lock.yaml") => true, + Some("shrinkwrap.yaml") => !shadowed(key), + _ => false, + }; + keys.iter().copied().filter(planned).collect() +} + /// How one `packages:` / `snapshots:` key names a patched release. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub(crate) enum KeyMatch<'k> { @@ -163,6 +182,27 @@ mod tests { ); } + /// N11 (owner decision): a `shrinkwrap.yaml` with a `pnpm-lock.yaml` + /// beside it is shadowed; one without is planned. + #[test] + fn a_pnpm_lock_shadows_its_sibling_shrinkwrap() { + let keys: Vec = [ + "a/pnpm-lock.yaml", + "a/shrinkwrap.yaml", + "b/shrinkwrap.yaml", + "package.json", + "pnpm-lock.yaml", + "shrinkwrap.yaml", + ] + .map(String::from) + .into(); + let refs: Vec<&String> = keys.iter().collect(); + assert_eq!( + planned_locks(&refs), + ["a/pnpm-lock.yaml", "b/shrinkwrap.yaml", "pnpm-lock.yaml"] + ); + } + #[test] fn each_target_repoints_what_its_edit_has_fixtures_for() { assert!(Target::Hosted.repoints("")); diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 165450ab4..fbac8d8db 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -28359,6 +28359,76 @@ packages: } } + /// Divergence N11 (owner decision, lock-model 2b): a pnpm <= 2 + /// `shrinkwrap.yaml` beside a `pnpm-lock.yaml` is shadowed. pnpm reads + /// only the `pnpm-lock.yaml`, so the hosted planner never rewrites the + /// shrinkwrap and leaves it out of the all-or-nothing gate: debris in it + /// (a key the splice cannot repoint, or pnpm 1's early shrinkwrap head) + /// no longer refuses the dependency in every lock. A `shrinkwrap.yaml` + /// that is the only pnpm lock is still rewritten. + #[test] + fn pnpm_shrinkwrap_beside_a_pnpm_lock_is_shadowed() { + let url = "http://patch.test/left-pad-1.3.0.tgz"; + let ovr = npm_override("left-pad", "1.3.0", url, "sha512-PATCHED=="); + let lock = "lockfileVersion: '9.0'\n\npackages:\n\n left-pad@1.3.0:\n \ + resolution: {integrity: sha512-OLD==}\n"; + let shrinkwrap = |head: &str, key: &str| { + format!( + "{head}\n\npackages:\n\n {key}:\n resolution: {{integrity: sha512-OLD==}}\n" + ) + }; + let supported = "shrinkwrapVersion: 3\nshrinkwrapMinorVersion: 9"; + let run = |files: &[(&str, String)]| { + let files: BTreeMap = files + .iter() + .map(|(k, v)| (k.to_string(), v.clone())) + .collect(); + rewrite_registry_redirect(&files, std::slice::from_ref(&ovr)) + }; + for (case, shrink) in [ + ("a plain entry", shrinkwrap(supported, "/left-pad/1.3.0")), + ( + "an unparseable key", + shrinkwrap(supported, "/left-pad/1.3.0)("), + ), + ( + "an early shrinkwrap", + shrinkwrap("shrinkwrapVersion: 3", "/left-pad/1.3.0"), + ), + ] { + let r = run(&[ + ("pnpm-lock.yaml", lock.to_string()), + ("shrinkwrap.yaml", shrink), + ]); + assert_eq!( + r.files.keys().collect::>(), + ["pnpm-lock.yaml"], + "{case}: only the governing lock is written: {:?}", + r.warnings + ); + assert!(r.files["pnpm-lock.yaml"].contains(url), "{case}"); + assert!(r.warnings.is_empty(), "{case}: {:?}", r.warnings); + assert!(r.refused_pnpm_uuids.is_empty(), "{case}"); + assert!(r.edits.iter().all(|e| e.path == "pnpm-lock.yaml"), "{case}"); + } + // A nested `shrinkwrap.yaml` is shadowed only by its own sibling. + let r = run(&[ + ("pnpm-lock.yaml", lock.to_string()), + ( + "sub/shrinkwrap.yaml", + shrinkwrap(supported, "/left-pad/1.3.0"), + ), + ]); + assert!( + r.files.contains_key("sub/shrinkwrap.yaml"), + "{:?}", + r.warnings + ); + // The only pnpm lock: rewritten as before. + let r = run(&[("shrinkwrap.yaml", shrinkwrap(supported, "/left-pad/1.3.0"))]); + assert!(r.files["shrinkwrap.yaml"].contains(url), "{:?}", r.warnings); + } + /// A classic-lock block whose first line is not a `key:` line /// (hand-mangled) is skipped without derailing the rewrite of the real /// entry — and survives byte-identically. diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md index c77e7e7f3..00af98b9b 100644 --- a/docs/design/lock-model-divergences.md +++ b/docs/design/lock-model-divergences.md @@ -31,7 +31,7 @@ follow from the install-tree decision. | J12 | scala-cli | Guidance only (`patch/redirect/scala_guidance.rs`) | Back-port: hosted wires scala-cli through the merged planner (a same-GAV Socket repository instead of a suffixed version). | | P3 | Is a shadowed `pdm.lock` rewritten? | Only when it governs (`pdm_drives`, ~564) | Rewrite it like every other tree, per the install-tree decision. | | P4 | Are `-r` includes of `requirements.txt` rewritten? | Root file only (`patch/redirect/requirements.rs` ~208) | Follow includes, per the install-tree decision. | -| N11 | Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` rewritten? | Yes; its all-or-nothing gate covers both files | **Pending.** Proposed: treat it as shadowed (swept for stale identities, never written). pnpm never reads it, and an unparseable debris key in it can refuse the dependency in every lock. | +| N11 | Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` rewritten? | Yes; its all-or-nothing gate covers both files | **Resolved (owner: shadowed) in PR 2b.** Swept for stale identities, never written, and out of the all-or-nothing gate (`formats::pnpm::plan::planned_locks`). pnpm never reads it, and an unparseable debris key in it used to refuse the dependency in every lock. A `shrinkwrap.yaml` that is the only pnpm lock is rewritten as before. | ## JVM @@ -89,7 +89,7 @@ follow from the install-tree decision. | N8 | npm `file:` directory link identity (#1481) | Hosted and discovery give a versionless link its source directory's `name@version`. Vendored matches the entry's own version, wiring the registry copies with no warning. | Hosted | silent vendored success | | N9 | yarn classic blocks reached only through an alias | Hosted skips them. Vendored rewrites them. Discovery counts an alias block as an install of the real package. | Back-port: hosted rewrites alias blocks (vendored supports what hosted lacks) | | | N10 | Locks of several npm-family package managers in one project | Hosted rewrites every family; discovery reads every family with the contest. The inventory and vendored pick one family (vlt > bun > pnpm > yarn > npm). | Union (separate trees) | | -| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | Pending owner decision (see above) | needs owner | +| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | **Resolved (owner: shadowed) in PR 2b**: hosted never rewrites it and leaves it out of the gate; discovery keeps sweeping it; vendored never reads it. A lone `shrinkwrap.yaml` keeps today's behavior. | | | N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. **Resolved in PR 2a** (the inventory reads them whenever `rush.json` exists). | target-specific (vendored) | | N13 | Plug'n'Play | Vendored refuses. The hosted berry preflight has no PnP gate. The inventory returns an error, so `scan` sees nothing in a project hosted pinned. | The inventory reads PnP locks. Vendored keeps its refusal. | target-specific (vendored) | | N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery. **Resolved in PR 2a**: under the setting (not under Rush), the inventory keeps a Socket-hosted pnpm entry but drops its integrity, as discovery withholds the lock pin. | target-specific | diff --git a/docs/ecosystems.md b/docs/ecosystems.md index b03dc6665..9e7d42b11 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -171,7 +171,8 @@ Each Legacy format has an upgrade path and an undo path. Both work in v5: from its legacy mirror, alias nodes included) and rewires both locks in npm 12's dual-lock state. Majors 6–12 are measured in [npm compatibility](testing/npm-compatibility.md). -- **pnpm** — hosted rewriting supports legacy `shrinkwrap.yaml` (pnpm 1/2), +- **pnpm** — hosted rewriting supports legacy `shrinkwrap.yaml` (pnpm 1/2; never + rewritten beside a `pnpm-lock.yaml`, which pnpm reads instead), lockfileVersion 5.x (pnpm 3–7), 6.0 (pnpm 8), and 9.0 (pnpm 9–12). Early pnpm 1 locks with shrinkwrapVersion 3 and no positive minor version are refused because those installers discard hosted URLs; the tested From cc3ad0c95a1de1b3191f7701a63c5ca26395bc3e Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Thu, 5 Nov 2026 23:16:36 -0600 Subject: [PATCH 16/36] Address the code review of the pnpm planner merge - Read every vendored pnpm text file through one reader, read_pnpm_text. It normalizes CRLF and rejects mixed endings. The revert now reads a CRLF two-document lock: it used to fail on `---\r`. A lock or workspace file that mixes CRLF and LF now fails the revert before any write, as vendoring refuses it; the revert used to rewrite it all-LF. - N11: a project that pins pnpm 1 / 2 (install record or package.json packageManager) reads only shrinkwrap.yaml, so there the stray pnpm-lock.yaml is the shadowed one. Hosted (planned_locks) and discovery share governing_pnpm_lock. The pin reader moves to utils::pnpm_workspace::pinned_pnpm_major, which pnpm_major_from reuses. - sniff_lock_grammar: an unparseable lockfileVersion major keeps the update-socket-patch remedy. - Perf: SectionIndex::build no longer formats a string per block. dep_entries is lazy and borrowed, so the non-index ref gate stops at the first refusal. matching_blocks classifies each block once, and target_block returns it with its rekey flag. - Target::check_lock is split into hosted_refuses_lock and the vendored dialect's sniff_lock_grammar check, with no dead default. Target::repoints is an explicit match per target. Budget, against the base: lock-readers 31206 (base 31207), lock-writers 88115 (base 88314). Co-Authored-By: Claude Opus 5.5 --- crates/socket-patch-cli/CLI_CONTRACT.md | 2 +- .../src/formats/pnpm/hosted.rs | 19 +- .../socket-patch-core/src/formats/pnpm/mod.rs | 15 +- .../src/formats/pnpm/plan.rs | 225 ++++--- .../src/patch/redirect/mod.rs | 20 + .../src/utils/pnpm_workspace.rs | 44 +- .../socket-patch-core/src/vendor/pnpm_lock.rs | 620 ++++++++++-------- .../src/vendor/pnpm_lock_legacy.rs | 30 +- .../socket-patch-core/src/vex/discover/npm.rs | 48 +- docs/design/lock-model-divergences.md | 4 +- docs/ecosystems.md | 3 +- scripts/loc-budget.json | 4 +- 12 files changed, 618 insertions(+), 416 deletions(-) diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 3f491176e..4b7c6ec51 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -272,7 +272,7 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc **pnpm per-branch locks (#556)**: with `gitBranchLockfile` on (a top-level `gitBranchLockfile: true` in the root `pnpm-workspace.yaml`; when the YAML leaves the key unset, `git-branch-lockfile=true` in the root `.npmrc`, as pnpm 10 and older read it; on disk also `pnpm_config_git_branch_lockfile` (pnpm 11 and newer) or `npm_config_git_branch_lockfile` (pnpm 10 and older; either when the major is unknown, read as for the `lockfile` setting above) set to `true`; a workspace member with no `pnpm-workspace.yaml` of its own reads the governing ancestor's file and the `.npmrc` beside it), pnpm installs a git branch from its own `pnpm-lock..yaml` whenever that lock exists, in every directory it installs a lock from: the root, and under `sharedWorkspaceLockfile: false` each member's (#492). When the setting is on AND at least one such `pnpm-lock..yaml` exists (at the root, or in a member that installs from its own lock), hosted mode leaves every root pnpm lock (`pnpm-lock.yaml`, `shrinkwrap.yaml`) and `pnpm-workspace.yaml` untouched (every member lock too), confirms no npm pin from them and warns `redirect_pnpm_git_branch_lockfile` (in place of `redirect_pnpm_no_lockfile` when the branch lock is the only one); the detail names the setting and the branch lock(s), and the remedy is to turn `gitBranchLockfile` off and fold the branch locks into `pnpm-lock.yaml` with `pnpm install --merge-git-branch-lockfiles`, then re-run. A vendored→hosted takeover of a pnpm-vendored purl is refused with the same code (the vendored wiring and ledger stay byte-for-byte). Vendored mode refuses the same projects with `vendor_pnpm_git_branch_lockfile`, before any write and in the pre-flight. The setting with no branch lock (main, or a branch whose dependencies never changed) installs from `pnpm-lock.yaml`, which both modes wire as usual; a stray branch lock with the setting off is ignored. The in-memory engine fetches root `pnpm-lock.*.yaml` names presence-only and refuses the same way. -`redirect_pnpm_no_lockfile` names pnpm when installer markers exist without a lock; `redirect_pnpm_entry_vendored` identifies a vendored entry instead of reporting it missing. Supported `shrinkwrap.yaml` files are writable lockfiles, not read-only markers, when no `pnpm-lock.yaml` sits beside them. pnpm never reads a `shrinkwrap.yaml` it has a `pnpm-lock.yaml` for, so that one is shadowed: hosted mode never rewrites it and leaves it out of the all-or-nothing gate, so an entry it cannot repoint (or pnpm 1's early shrinkwrap head) refuses nothing, and `vex` only sweeps it for stale Socket identities. Vendored mode never writes it either. +`redirect_pnpm_no_lockfile` names pnpm when installer markers exist without a lock; `redirect_pnpm_entry_vendored` identifies a vendored entry instead of reporting it missing. Supported `shrinkwrap.yaml` files are writable lockfiles, not read-only markers, when no `pnpm-lock.yaml` sits beside them. pnpm >= 3 never reads a `shrinkwrap.yaml` it has a `pnpm-lock.yaml` for, so that one is shadowed (unless the project pins pnpm 1 or 2 in `node_modules/.modules.yaml` or package.json's `packageManager`: those read only `shrinkwrap.yaml`, so there the stray `pnpm-lock.yaml` is the shadowed one): hosted mode never rewrites it and leaves it out of the all-or-nothing gate, so an entry it cannot repoint (or pnpm 1's early shrinkwrap head) refuses nothing, and `vex` only sweeps it for stale Socket identities. Vendored mode never writes it either. **vlt hosted-mode contract**: `scan` / `get --mode hosted` rewrite, in `vlt-lock.json`, every default-registry node of a granted `name@version` (the `''` / `npm` segment or a URL segment equal to the lock's scalar `registry`, both DepID grammars, every peer and modifier variant): slot [2] becomes the granted sha512 and slot [3] the hosted URL (appended to a 3-tuple); the DepID, flags and trailing slots, the line ending and every other byte stay. `options` is never edited and `vlt.json` is only read. A lock with another `lockfileVersion` (decided on the raw JSON token), a BOM, a non-object body or a `nodes` section outside vlt's one-node-per-line layout refuses the whole lock (`redirect_vlt_lock_unsupported`). **Confirmation**: vlt drives when its install state (`node_modules/.vlt-lock.json` or `node_modules/.vlt/`) is present or no other npm-family lock is; then only `vlt-lock.json` confirms a uuid. Otherwise every lock is rewritten, `redirect_vlt_sibling_lockfiles` warns, and the other locks' rules confirm, including a dep `vlt-lock.json` merely does not wire (`redirect_vlt_entry_not_found`, `redirect_vlt_entry_vendored`, or `redirect_vlt_custom_registry_skipped` alone). Whichever lock drives, a dep the vlt rewriter refuses (`redirect_vlt_missing_sha512`, `redirect_vlt_unsupported_lock_key`) is never confirmed by any lock, although a sibling lock may already carry its rewritten URL. **Artifact preflight**: before any takeover or write (dry runs included), each granted artifact with a default-registry instance is fetched once as vlt fetches it and must verify, else the dep is withheld (`redirect_vlt_artifact_unverifiable`, see the tag table). **Heal**: stale installed copies of Socket-owned nodes are removed so the next `vlt install` extracts the patched bytes, and `rollback` / `remove` do the same for the registry bytes (`--no-vlt-install-cleanup` keeps them; optional dependencies' copies are always kept); `redirect_vlt_reinstall_required` says what happened and what to run. The same-run `--vex` never attests a vlt package whose installed copy is stale or unchecked, whose lock a vlt release may ignore (`redirect_vlt_lockfile_version_missing`, `redirect_vlt_old_lockfile_ignored`, `redirect_vlt_scalar_registry_ignored`), or which also resolves from a non-default registry (`redirect_vlt_custom_registry_skipped`). `vlt.json` or vlt install state without `vlt-lock.json` warns `redirect_vlt_no_lockfile` instead of `redirect_npm_no_lockfile`. `rollback` / `remove` restore each hosted node's slots [2] and [3] from the version document of the registry the node resolves against (slot [3] is its `dist.tarball`, as vlt writes it; `upstream_registry_fallback` when that registry can't be read), following the lock's own slot-[3] convention (see "Hosted unwind coverage"). Tested releases: `docs/testing/vlt-compatibility.md`. diff --git a/crates/socket-patch-core/src/formats/pnpm/hosted.rs b/crates/socket-patch-core/src/formats/pnpm/hosted.rs index e16f3979d..54908d9dc 100644 --- a/crates/socket-patch-core/src/formats/pnpm/hosted.rs +++ b/crates/socket-patch-core/src/formats/pnpm/hosted.rs @@ -11,7 +11,7 @@ use serde_json::Value; use crate::patch::redirect::{full_name, DepOverride, FileEdit, RewriteResult, RewriteWarning}; use super::grammar as pnpm; -use super::plan::{match_key, planned_locks, KeyMatch, Target}; +use super::plan::{hosted_refuses_lock, match_key, planned_locks, KeyMatch, Target}; /// Whether `entry` resolves to exactly `artifact_url` — the per-instance /// residual-gate predicate. @@ -52,7 +52,7 @@ impl<'f> PnpmLockState<'f> { let mut state = PnpmLockState { path, text: Cow::Borrowed(text), - early_shrinkwrap: Target::Hosted.check_lock(text).is_err(), + early_shrinkwrap: hosted_refuses_lock(text), entries: Vec::new(), sorted: Vec::new(), pending: Vec::new(), @@ -119,12 +119,8 @@ impl<'f> PnpmLockState<'f> { let start = self .sorted .partition_point(|&i| self.norm_key(i) < prefix.as_str()); - out.extend( - self.sorted[start..] - .iter() - .take_while(|&&i| self.norm_key(i).starts_with(prefix.as_str())) - .copied(), - ); + let rest = self.sorted[start..].iter(); + out.extend(rest.take_while(|&&i| self.norm_key(i).starts_with(prefix.as_str()))); } out.sort_unstable(); out.dedup(); @@ -184,10 +180,11 @@ pub(crate) fn plan_hosted( let npm: Vec<&DepOverride> = overrides.iter().filter(|o| o.ecosystem == "npm").collect(); // A pnpm lock lives at the project root or at any nested path (e.g. Rush // repos keep them under `common/config/rush/`); every such files-map key - // is rewritten under the same grammar, except a `shrinkwrap.yaml` a - // `pnpm-lock.yaml` shadows ([`planned_locks`]). Deterministic order: + // is rewritten under the same grammar, except the one of a + // `pnpm-lock.yaml` / `shrinkwrap.yaml` pair pnpm never reads + // ([`planned_locks`]). Deterministic order: // BTreeMap iterates keys sorted, so goldens are stable across every lock. - let lock_keys = planned_locks(&files.keys().collect::>()); + let lock_keys = planned_locks(files); if npm.is_empty() || lock_keys.is_empty() { return; } diff --git a/crates/socket-patch-core/src/formats/pnpm/mod.rs b/crates/socket-patch-core/src/formats/pnpm/mod.rs index b94801dfb..45aca1c30 100644 --- a/crates/socket-patch-core/src/formats/pnpm/mod.rs +++ b/crates/socket-patch-core/src/formats/pnpm/mod.rs @@ -293,17 +293,20 @@ pub fn sniff_lock_grammar(text: &str) -> Result { Some("5.4") => Ok(PnpmLockGrammar::V54), Some("6.0") => Ok(PnpmLockGrammar::V60), v => { - let major = v.and_then(|v| v.split('.').next()?.parse::().ok()); + // An old (or missing) version is fixed by upgrading pnpm; a newer or + // unreadable one means pnpm outgrew this build. + let major = v.map(|v| v.split('.').next().and_then(|m| m.parse::().ok())); + let old = matches!(major, None | Some(Some(..9))); let v = v.map_or("no lockfileVersion in its head".into(), |v| { format!("lockfileVersion {v}") }); - Err(match major { - Some(m) if m >= 9 => format!( + Err(match old { + false => format!( "{PNPM_LOCK} has {v}; this socket-patch build supports \ lockfileVersions 5.4, 6.0, and 9.0 — re-lock with a pnpm release that \ emits one of them, or update socket-patch" ), - _ => format!( + true => format!( "{PNPM_LOCK} has {v}; supported versions are 5.4 (pnpm 7), 6.0 \ (pnpm 8), and 9.0 (pnpm >= 9) — re-lock with pnpm >= 9" ), @@ -1026,9 +1029,7 @@ mod tests { assert_eq!(lock_version_major(&format!("\u{feff}{v9}")), Some(9)); assert!(may_need_store_flag(&format!("\u{feff}{v52}"))); assert!(may_need_store_flag(&format!("\u{feff}{shrinkwrap}"))); - assert!(plan::Target::Hosted - .check_lock("\u{feff}shrinkwrapVersion: 3\n") - .is_err()); + assert!(plan::hosted_refuses_lock("\u{feff}shrinkwrapVersion: 3\n")); // Exactly one BOM is encoding; a second one is content, as for pnpm. assert!(!is_pnpm_lock_text( "\u{feff}\u{feff}lockfileVersion: '9.0'\n" diff --git a/crates/socket-patch-core/src/formats/pnpm/plan.rs b/crates/socket-patch-core/src/formats/pnpm/plan.rs index 54361ecb2..34d5fb051 100644 --- a/crates/socket-patch-core/src/formats/pnpm/plan.rs +++ b/crates/socket-patch-core/src/formats/pnpm/plan.rs @@ -4,7 +4,9 @@ //! they agree on: //! //! * which lock versions each target's edit has fixtures for -//! ([`Target::check_lock`]); +//! ([`hosted_refuses_lock`]; vendored takes the grammars its rekey was +//! captured against, [`super::sniff_lock_grammar`], and refuses the rest, +//! target-specifically: divergences N16); //! * how a lock key names a patched `name@version` ([`match_key`]): the //! registry entry with its peer suffix, or the entry a vendored rewire //! already rekeyed; @@ -17,8 +19,12 @@ //! artifact and mirrors the override pnpm cross-checks. use super::grammar; -use super::{sniff_lock_grammar, PnpmLockGrammar}; + +use std::collections::BTreeMap; + +use crate::constants::npm_family::{PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY}; use crate::formats::text::strip_bom; +use crate::utils::pnpm_workspace::pinned_pnpm_major; use crate::vendor::npm_common::tgz_rel_leaf; /// Where a rewired entry points. @@ -32,73 +38,77 @@ pub(crate) enum Target { Vendored, } -impl Target { - /// Whether this target's edit can rewrite a lock with `text`'s head, - /// and in which grammar (`None`: the hosted splice reads every - /// grammar the same way). Hosted takes every lock but pnpm 1's early - /// shrinkwrap (shrinkwrapVersion 3 without a minor version), whose - /// installers drop registry tarball URLs whatever the frozen flag says - /// (verified with pnpm 1.0.0). Vendored takes the grammars its rekey was - /// captured against (5.4, 6.0, 9.0): another version's `file:` override - /// shape has no pnpm-blessed fixture, so it is refused, - /// target-specifically (divergences N16). - pub(crate) fn check_lock(self, text: &str) -> Result, String> { - if self == Target::Vendored { - return sniff_lock_grammar(text).map(Some); - } - let field = |name: &str| strip_bom(text).lines().find_map(|l| l.strip_prefix(name)); - let early = field("shrinkwrapVersion:") - .is_some_and(|v| v.trim().trim_matches(['\'', '"']) == "3") - && !field("shrinkwrapMinorVersion:") - .is_some_and(|v| v.trim().parse::().is_ok_and(|v| v > 0)); - if early { - return Err("early pnpm 1 shrinkwrapVersion 3 without a minor version".into()); - } - Ok(None) - } +/// Whether the hosted splice refuses a lock: pnpm 1's early shrinkwrap +/// (shrinkwrapVersion 3 without a minor version), whose installers drop +/// registry tarball URLs whatever the frozen flag says (verified with pnpm +/// 1.0.0). Hosted takes every other lock version (divergences N16). +pub(crate) fn hosted_refuses_lock(text: &str) -> bool { + let field = |name: &str| strip_bom(text).lines().find_map(|l| l.strip_prefix(name)); + field("shrinkwrapVersion:").is_some_and(|v| v.trim().trim_matches(['\'', '"']) == "3") + && !field("shrinkwrapMinorVersion:") + .is_some_and(|v| v.trim().parse::().is_ok_and(|v| v > 0)) +} +impl Target { /// Whether this target's edit repoints a registry instance whose key - /// carries `suffix` after `name@version`. Hosted splices every peer - /// context it can parse (pnpm 5's `_peer@1` or balanced `(peer@1)` - /// groups); vendored rekeys the plain entry only, since a peer-suffixed - /// rekey has no fixture and would dangle. + /// carries `suffix` after `name@version`. pub(crate) fn repoints(self, suffix: &str) -> bool { - if self == Target::Vendored || suffix.is_empty() || suffix.starts_with('_') { - return suffix.is_empty() || self == Target::Hosted; - } - let mut depth = 0usize; - for c in suffix.chars() { - match c { - '(' => depth += 1, - ')' if depth > 0 => depth -= 1, - _ if depth == 0 => return false, - _ => {} + match self { + // The rekey moves the plain entry only: a peer-suffixed rekey + // has no fixture and would dangle. + Target::Vendored => suffix.is_empty(), + // The splice repoints every peer context it can parse: none, + // pnpm 5's `_peer@1`, or balanced `(peer@1)` groups. + Target::Hosted if suffix.is_empty() || suffix.starts_with('_') => true, + Target::Hosted => { + let mut depth = 0usize; + for c in suffix.chars() { + match c { + '(' => depth += 1, + ')' if depth > 0 => depth -= 1, + _ if depth == 0 => return false, + _ => {} + } + } + depth == 0 } } - depth == 0 } } -/// The pnpm locks among `keys` (root-relative paths) a planner rewrites, in -/// their order: every `pnpm-lock.yaml`, and a pnpm <= 2 `shrinkwrap.yaml` -/// only when no `pnpm-lock.yaml` sits beside it. pnpm never reads a -/// `shrinkwrap.yaml` it has a `pnpm-lock.yaml` for, so that one is -/// SHADOWED (owner decision on divergence N11): never rewritten, never part -/// of the all-or-nothing gate (debris in it cannot refuse a dependency), -/// and only swept for stale Socket identities by lockfile discovery. -pub(crate) fn planned_locks<'k>(keys: &[&'k String]) -> Vec<&'k String> { - // `dir` keeps its trailing `/` (empty at the root). - let shadowed = |key: &str| { - key.strip_suffix("shrinkwrap.yaml").is_some_and(|dir| { - keys.iter() - .any(|k| k.strip_suffix("pnpm-lock.yaml") == Some(dir)) - }) - }; - let planned = |key: &&String| match key.rsplit('/').next() { - Some("pnpm-lock.yaml") => true, - Some("shrinkwrap.yaml") => !shadowed(key), - _ => false, + +/// The lock pnpm reads in a directory holding `pnpm-lock.yaml` (`lock`) +/// and/or a pnpm <= 2 `shrinkwrap.yaml` (`shrinkwrap`), given the pnpm major +/// the project pins ([`pinned_pnpm_major`]); the other one is SHADOWED +/// (owner decision on divergence N11): never rewritten, never part of the +/// all-or-nothing gate (debris in it cannot refuse a dependency), only swept +/// for stale Socket identities by lockfile discovery. pnpm 1 / 2 read only +/// `shrinkwrap.yaml`; pnpm >= 3 read only `pnpm-lock.yaml`. With no pin, +/// `pnpm-lock.yaml` governs (pnpm <= 2 never writes one). +pub(crate) fn governing_pnpm_lock( + lock: bool, + shrinkwrap: bool, + pinned_major: Option, +) -> Option<&'static str> { + let pnpm_1_or_2 = pinned_major.is_some_and(|major| major <= 2); + let governs = shrinkwrap && (!lock || pnpm_1_or_2); + (governs.then_some(PNPM_SHRINKWRAP_LEGACY)).or(lock.then_some(PNPM_LOCK)) +} + +/// The pnpm locks of a hosted run's `files` (root-relative paths) a planner +/// rewrites, in key order: per directory, the [`governing_pnpm_lock`], read +/// with the evidence `files` holds for that directory. +pub(crate) fn planned_locks(files: &BTreeMap) -> Vec<&String> { + let planned = |key: &&String| { + let leaf = key.rsplit('/').next().unwrap_or(key); + // `dir` keeps its trailing `/` (empty at the root). + let in_dir = |rel: &str| format!("{}{rel}", &key[..key.len() - leaf.len()]); + let pinned = pinned_pnpm_major(|rel| files.get(&in_dir(rel)).cloned()); + let has = |rel: &str| files.contains_key(&in_dir(rel)); + matches!(leaf, PNPM_LOCK | PNPM_SHRINKWRAP_LEGACY) + && governing_pnpm_lock(has(PNPM_LOCK), has(PNPM_SHRINKWRAP_LEGACY), pinned) + == Some(leaf) }; - keys.iter().copied().filter(planned).collect() + files.keys().filter(planned).collect() } /// How one `packages:` / `snapshots:` key names a patched release. @@ -136,6 +146,7 @@ pub(crate) fn match_key<'k>(key: &'k str, name: &str, version: &str) -> Option = [ - "a/pnpm-lock.yaml", - "a/shrinkwrap.yaml", - "b/shrinkwrap.yaml", - "package.json", - "pnpm-lock.yaml", - "shrinkwrap.yaml", - ] - .map(String::from) - .into(); - let refs: Vec<&String> = keys.iter().collect(); + fn the_pinned_pnpm_major_picks_the_governing_lock() { + // No pin: `pnpm-lock.yaml` governs a pair, and each lone lock itself. + assert_eq!(governing_pnpm_lock(true, true, None), Some(PNPM_LOCK)); assert_eq!( - planned_locks(&refs), - ["a/pnpm-lock.yaml", "b/shrinkwrap.yaml", "pnpm-lock.yaml"] + governing_pnpm_lock(false, true, None), + Some(PNPM_SHRINKWRAP_LEGACY) + ); + assert_eq!(governing_pnpm_lock(false, false, Some(2)), None); + // pnpm 1 / 2 read only `shrinkwrap.yaml`; pnpm >= 3 only the lock. + assert_eq!( + governing_pnpm_lock(true, true, Some(2)), + Some(PNPM_SHRINKWRAP_LEGACY) + ); + assert_eq!(governing_pnpm_lock(true, true, Some(3)), Some(PNPM_LOCK)); + assert_eq!(governing_pnpm_lock(true, false, Some(1)), Some(PNPM_LOCK)); + + let files = |pin: Option<&str>| { + let mut files: BTreeMap = [ + "a/pnpm-lock.yaml", + "a/shrinkwrap.yaml", + "b/shrinkwrap.yaml", + "pnpm-lock.yaml", + "shrinkwrap.yaml", + ] + .into_iter() + .map(|k| (k.to_string(), String::new())) + .collect(); + if let Some(pin) = pin { + let pkg = format!(r#"{{"packageManager": "pnpm@{pin}"}}"#); + files.insert("package.json".into(), pkg); + } + files + }; + let planned = |files: &BTreeMap| { + planned_locks(files) + .into_iter() + .cloned() + .collect::>() + }; + // No evidence, or pnpm >= 3: each `pnpm-lock.yaml` shadows its + // sibling shrinkwrap. + for pin in [None, Some("9.15.0")] { + assert_eq!( + planned(&files(pin)), + ["a/pnpm-lock.yaml", "b/shrinkwrap.yaml", "pnpm-lock.yaml"], + "{pin:?}" + ); + } + // pnpm 2 pinned at the root: there `shrinkwrap.yaml` governs and the + // stray `pnpm-lock.yaml` is shadowed; `a/` has no evidence of its own. + assert_eq!( + planned(&files(Some("2.25.7"))), + ["a/pnpm-lock.yaml", "b/shrinkwrap.yaml", "shrinkwrap.yaml"] + ); + // The install record outranks package.json. + let mut record = files(Some("9.15.0")); + record.insert( + "node_modules/.modules.yaml".into(), + "packageManager: pnpm@2.25.7\n".into(), + ); + assert_eq!( + planned(&record).last().map(String::as_str), + Some("shrinkwrap.yaml") ); } @@ -229,13 +290,9 @@ mod tests { ("lockfileVersion: '10.0'\n", None), ("shrinkwrapVersion: 3\nshrinkwrapMinorVersion: 9\n", None), ] { - assert_eq!(Target::Hosted.check_lock(head), Ok(None), "{head}"); - assert_eq!( - Target::Vendored.check_lock(head).ok().flatten(), - vendored, - "{head}" - ); + assert!(!hosted_refuses_lock(head), "{head}"); + assert_eq!(sniff_lock_grammar(head).ok(), vendored, "{head}"); } - assert!(Target::Hosted.check_lock("shrinkwrapVersion: 3\n").is_err()); + assert!(hosted_refuses_lock("shrinkwrapVersion: 3\n")); } } diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index fbac8d8db..d96c18c77 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -28424,6 +28424,26 @@ packages: "{:?}", r.warnings ); + // A project pinning pnpm 2 installs from `shrinkwrap.yaml` (review + // finding): it is rewritten and the stray `pnpm-lock.yaml` is the + // shadowed one, so debris in it refuses nothing. + let pnpm2 = r#"{"packageManager": "pnpm@2.25.7"}"#.to_string(); + let r = run(&[ + ("package.json", pnpm2), + ( + "pnpm-lock.yaml", + lock.replace("left-pad@1.3.0:", "left-pad@1.3.0)(:"), + ), + ("shrinkwrap.yaml", shrinkwrap(supported, "/left-pad/1.3.0")), + ]); + assert_eq!( + r.files.keys().collect::>(), + ["shrinkwrap.yaml"], + "{:?}", + r.warnings + ); + assert!(r.files["shrinkwrap.yaml"].contains(url)); + assert!(r.refused_pnpm_uuids.is_empty(), "{:?}", r.warnings); // The only pnpm lock: rewritten as before. let r = run(&[("shrinkwrap.yaml", shrinkwrap(supported, "/left-pad/1.3.0"))]); assert!(r.files["shrinkwrap.yaml"].contains(url), "{:?}", r.warnings); diff --git a/crates/socket-patch-core/src/utils/pnpm_workspace.rs b/crates/socket-patch-core/src/utils/pnpm_workspace.rs index 8671daab2..483ed46f6 100644 --- a/crates/socket-patch-core/src/utils/pnpm_workspace.rs +++ b/crates/socket-patch-core/src/utils/pnpm_workspace.rs @@ -890,36 +890,54 @@ fn lockfile_off(setting: impl Fn(&str, &str) -> Option) -> Option) -> Option { + let texts = read_texts(view, &[MODULES_YAML, "package.json", PNPM_LOCK]).await; + pnpm_major_from(|rel| texts.get(rel).cloned()) +} + +/// The pnpm major the project at `view` pins ([`pinned_pnpm_major`]). +pub(crate) async fn pinned_pnpm_major_in(view: &ProjectView<'_>) -> Option { + let texts = read_texts(view, &[MODULES_YAML, "package.json"]).await; + pinned_pnpm_major(|rel| texts.get(rel).cloned()) +} + +/// The readable files of `rels` under `view`, by path. +async fn read_texts<'r>(view: &ProjectView<'_>, rels: &[&'r str]) -> BTreeMap<&'r str, String> { let mut texts = BTreeMap::new(); - for rel in [MODULES_YAML, "package.json", PNPM_LOCK] { + for &rel in rels { if let Ok(text) = view.read_text(rel).await { texts.insert(rel, text); } } - pnpm_major_from(|rel| texts.get(rel).cloned()) + texts } /// The installed pnpm's record of the project. pub(crate) const MODULES_YAML: &str = "node_modules/.modules.yaml"; -/// The pnpm major whose settings reading applies to a project whose files -/// `read` returns by project-relative path: the one that installed it -/// ([`MODULES_YAML`]), else the one package.json pins, else what the root -/// lock's grammar implies (`pnpm_settings_major`). -pub(crate) fn pnpm_major_from(read: impl Fn(&str) -> Option) -> Option { - use crate::patch::redirect::upstream::npm::{ - modules_yaml_pnpm_major, package_json_pnpm_major, pnpm_settings_major, - }; - let pinned = read(MODULES_YAML) +/// The pnpm major a project pins, from its files as `read` returns them by +/// project-relative path: the one that installed it ([`MODULES_YAML`]), +/// else the one package.json's `packageManager` names. +pub(crate) fn pinned_pnpm_major(read: impl Fn(&str) -> Option) -> Option { + use crate::patch::redirect::upstream::npm::{modules_yaml_pnpm_major, package_json_pnpm_major}; + let modules = read(MODULES_YAML); + modules .as_deref() .and_then(modules_yaml_pnpm_major) .or_else(|| { read("package.json") .as_deref() .and_then(package_json_pnpm_major) - }); + }) +} + +/// The pnpm major whose settings reading applies to a project whose files +/// `read` returns by project-relative path: the one it pins +/// ([`pinned_pnpm_major`]), else what the root lock's grammar implies +/// (`pnpm_settings_major`). +pub(crate) fn pnpm_major_from(read: impl Fn(&str) -> Option) -> Option { + let pinned = pinned_pnpm_major(&read); let lock = read(PNPM_LOCK).unwrap_or_default(); - pnpm_settings_major(&lock, pinned) + crate::patch::redirect::upstream::npm::pnpm_settings_major(&lock, pinned) } /// What pnpm does with its `lockfile` setting off, for the refusals and diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock.rs b/crates/socket-patch-core/src/vendor/pnpm_lock.rs index d80b04d29..9d032904a 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock.rs @@ -80,11 +80,12 @@ use crate::formats::pnpm::lines::{ unquote_value, yaml_key, yaml_key_like, YamlBlock, }; use crate::formats::pnpm::plan::{match_key, KeyMatch, Target}; +use crate::formats::pnpm::sniff_lock_grammar; use crate::formats::pnpm::workspace; use crate::formats::pnpm::{vendored_npm_uuids, PnpmLockGrammar}; const PACKAGE_JSON: &str = "package.json"; -use crate::utils::line_endings::{to_lf, LineEndings}; +use crate::utils::line_endings::LineEndings; use crate::utils::pnpm_workspace::{governing_workspace_file, PNPM_WORKSPACE}; /// Refusal code for a workspace member whose pnpm settings live in an @@ -178,11 +179,10 @@ impl PnpmDialect { } /// The grammar of a lock this dialect rewrites: one the vendored target - /// takes ([`Target::check_lock`]) and this dialect splices. + /// rewrites ([`sniff_lock_grammar`]: 5.4, 6.0 and 9.0, the grammars its + /// rekey was captured against; N16) and this dialect splices. pub(super) fn check_lock(self, text: &str) -> Result { - let grammar = Target::Vendored - .check_lock(text)? - .unwrap_or(PnpmLockGrammar::V9); + let grammar = sniff_lock_grammar(text)?; match (self, grammar) { (Self::V9, PnpmLockGrammar::V9) | (Self::Legacy, PnpmLockGrammar::V54 | PnpmLockGrammar::V60) => Ok(grammar), @@ -417,7 +417,7 @@ impl NpmLockBackend for PnpmBackend { .render(&pkg) .map_err(|e| format!("cannot serialize {PACKAGE_JSON}: {e}"))?; // An env document ahead of the project lock is carried over verbatim. - // Each file is written back in its own line endings (`lf_text`). + // Each file is written back in its own line endings (`read_pnpm_text`). let lock_out = format!("{lock_prefix}{}", lock.lines().join("\n")); let ws_out = ws_edit.new_text.as_deref().map(|t| ws_eol.restore(t)); let ws_was = ws_text.as_deref().map(|t| ws_eol.restore(t)); @@ -485,7 +485,7 @@ struct PnpmProject { /// Everything ahead of the project document, written back verbatim. lock_prefix: String, /// The line endings the lock and the workspace file are written back - /// with; their texts here are LF ([`lf_text`]). + /// with; their texts here are LF ([`read_pnpm_text`]). lock_eol: LineEndings, ws_text: Option, ws_eol: LineEndings, @@ -584,15 +584,9 @@ async fn read_project( { return Err(Box::new(refused(refusal.0, refusal.1))); } - let lock_text = match read_regular_to_string(&project_root.join(PNPM_LOCK)).await { - Ok(text) => text, - Err(e) => { - return Err(Box::new(refused( - "vendor_lockfile_missing", - format!("cannot read {PNPM_LOCK}: {e} — run `pnpm install` first"), - ))); - } - }; + let (lock_text, lock_eol) = read_pnpm_text(project_root, PNPM_LOCK) + .await + .map_err(|e| read_refusal(PNPM_LOCK, e, "run `pnpm install` first"))?; let grammar = match dialect.check_lock(&lock_text) { Ok(grammar) => grammar, Err(detail) => { @@ -602,7 +596,6 @@ async fn read_project( ))); } }; - let (lock_text, lock_eol) = lf_text(PNPM_LOCK, lock_text)?; if dialect == PnpmDialect::Legacy { return Ok(PnpmProject { pkg_bytes, @@ -645,18 +638,16 @@ async fn read_project( // encoding, permissions, a FIFO) must refuse — treating it as absent // would route into the create path, which OVERWRITES the user's // workspace definition with the root-only scaffold. - let (ws_text, ws_eol) = match read_regular_to_string(&project_root.join(PNPM_WORKSPACE)).await { - Ok(text) => lf_text(PNPM_WORKSPACE, text).map(|(t, eol)| (Some(t), eol))?, + let (ws_text, ws_eol) = match read_pnpm_text(project_root, PNPM_WORKSPACE).await { + Ok((text, eol)) => (Some(text), eol), Err(e) if e.kind() == std::io::ErrorKind::NotFound => (None, LineEndings::None), Err(e) => { - return Err(Box::new(refused( - "vendor_lockfile_missing", - format!( - "cannot read {PNPM_WORKSPACE}: {e} — vendoring mirrors the override \ - into it (pnpm >= 11 reads overrides from pnpm-workspace.yaml); fix \ - the file and retry" - ), - ))); + return Err(read_refusal( + PNPM_WORKSPACE, + e, + "vendoring mirrors the override into it (pnpm >= 11 reads overrides from \ + pnpm-workspace.yaml); fix the file and retry", + )) } }; // A workspace member with its own lock (`sharedWorkspaceLockfile: @@ -696,25 +687,57 @@ async fn read_project( }) } -/// A pnpm file's text with LF line endings, and the endings it is written -/// back with. A CRLF file (a Windows autocrlf checkout) is read and edited -/// like its LF twin and written back CRLF, as hosted mode's byte splices -/// keep it (divergences N15). A file that mixes endings is refused: no one -/// ending writes it back byte-faithfully. -fn lf_text(file: &str, text: String) -> Result<(String, LineEndings), Box> { +/// A pnpm file mixing CRLF and LF (see [`read_pnpm_text`]). +#[derive(Debug)] +struct MixedEndings(&'static str); + +impl std::fmt::Display for MixedEndings { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + f, + "{} mixes CRLF and LF line endings, which this rewriter cannot write back \ + byte-faithfully — normalize the file to one line ending and retry", + self.0 + ) + } +} + +impl std::error::Error for MixedEndings {} + +/// Read `file` under `root` as every vendored pnpm reader does (vendoring, +/// its pre-flight and the revert): its text with LF line endings, and the +/// endings it is written back with. A CRLF file (a Windows autocrlf +/// checkout) is edited like its LF twin and written back CRLF, as hosted +/// mode's byte splices keep it (divergences N15). A file that mixes endings +/// is an error ([`is_mixed`]): no one ending writes it back byte-faithfully, +/// so vendoring refuses it and the revert fails before any write. +async fn read_pnpm_text(root: &Path, file: &'static str) -> std::io::Result<(String, LineEndings)> { + let text = read_regular_to_string(&root.join(file)).await?; match LineEndings::of(&text) { - LineEndings::Mixed => Err(Box::new(refused( - "vendor_lockfile_crlf_unsupported", - format!( - "{file} mixes CRLF and LF line endings, which this rewriter cannot write \ - back byte-faithfully — normalize the file to one line ending and retry" - ), - ))), + LineEndings::Mixed => Err(std::io::Error::other(MixedEndings(file))), LineEndings::Crlf => Ok((text.replace("\r\n", "\n"), LineEndings::Crlf)), eol => Ok((text, eol)), } } +/// Whether a [`read_pnpm_text`] error is a mixed-ending file. +fn is_mixed(e: &std::io::Error) -> bool { + e.get_ref().is_some_and(|inner| inner.is::()) +} + +/// The vendoring refusal for a [`read_pnpm_text`] error other than a +/// missing file: `vendor_lockfile_crlf_unsupported` for mixed endings, else +/// `vendor_lockfile_missing` with `remedy`. +fn read_refusal(file: &str, e: std::io::Error, remedy: &str) -> Box { + Box::new(match is_mixed(&e) { + true => refused("vendor_lockfile_crlf_unsupported", e.to_string()), + false => refused( + "vendor_lockfile_missing", + format!("cannot read {file}: {e} — {remedy}"), + ), + }) +} + /// `(prefix, project document)` of a v9 lock ([`split_project_document`]), /// the project document's own head version-checked. `Err` names why the /// lock's documents are not pnpm's layout. @@ -968,30 +991,58 @@ pub(super) async fn revert_pnpm_dialect( return blocked; } } - // A v9 lock whose documents are not pnpm's layout cannot be reverted - // without guessing which document holds the wiring: fail before any - // surface is touched (the dry run too, so the preview never advertises - // a revert the wet run refuses) instead of reverting package.json and - // the workspace file around a lock left wired to the artifact. - let mut lock_text: Option> = None; - if dialect == PnpmDialect::V9 - && entry + // Read the lock and the workspace file the records touch up front, as + // vendoring reads them ([`read_pnpm_text`]), and fail before any surface + // is touched (the dry run too, so the preview never advertises a revert + // the wet run refuses) when one mixes CRLF and LF, or when a v9 lock's + // documents are not pnpm's layout: neither can be reverted without + // guessing, and reverting package.json and the workspace file around a + // lock left wired to the artifact would desync them. + let touches = |kind: fn(&str) -> bool| { + entry .wiring .iter() - .any(|r| r.file != PACKAGE_JSON && dialect.allows_revert_file(&r.file)) + .any(|r| dialect.allows_revert_file(&r.file) && kind(&r.file)) + }; + let read = |file| read_pnpm_text(project_root, file); + let lock_read = match touches(|f| f != PACKAGE_JSON) { + true => Some(read(PNPM_LOCK).await), + false => None, + }; + let ws_read = match touches(|f| f == PNPM_WORKSPACE) { + true => Some(read(PNPM_WORKSPACE).await), + false => None, + }; + for e in [&lock_read, &ws_read] + .into_iter() + .flatten() + .filter_map(|r| r.as_ref().err()) { - let read = read_regular_to_string(&project_root.join(PNPM_LOCK)).await; - if let Ok(text) = &read { - if let Err(why) = split_v9_lock(text) { - return RevertOutcome::failed(format!( - "{why}, so the vendored wiring of {} cannot be located; nothing was \ - reverted — restore the vendored {PNPM_LOCK} (or re-lock it with pnpm) \ - and re-run the revert", - entry.base_purl - )); - } + if is_mixed(e) { + return RevertOutcome::failed(format!("{e}; nothing was reverted")); } - lock_text = Some(read); + } + let mut lock_prefix = String::new(); + let mut lock_lines: Option> = None; + if let Some(Ok((text, _))) = &lock_read { + // The wiring lives in the project document; an env document ahead + // of it is written back verbatim. + let (prefix, project) = match dialect { + PnpmDialect::V9 => match split_v9_lock(text) { + Ok(split) => split, + Err(why) => { + return RevertOutcome::failed(format!( + "{why}, so the vendored wiring of {} cannot be located; nothing was \ + reverted — restore the vendored {PNPM_LOCK} (or re-lock it with \ + pnpm) and re-run the revert", + entry.base_purl + )) + } + }, + PnpmDialect::Legacy => ("", text.as_str()), + }; + lock_prefix = prefix.to_string(); + lock_lines = Some(split_lines(project)); } if dry_run { return RevertOutcome::ok(); @@ -1001,7 +1052,6 @@ pub(super) async fn revert_pnpm_dialect( // Partition by file through the allowlist (fail-closed skip+warning on // anything else — see REVERT_ALLOWLIST's security note). let mut touches_pkg = false; - let mut touches_lock = false; for rec in &entry.wiring { if !dialect.allows_revert_file(&rec.file) { outcome.warnings.push(VendorWarning::new( @@ -1013,49 +1063,23 @@ pub(super) async fn revert_pnpm_dialect( )); continue; } - if rec.file == PACKAGE_JSON { - touches_pkg = true; - } else { - touches_lock = true; - } + touches_pkg |= rec.file == PACKAGE_JSON; } - // Load both surfaces up front (fail-closed on unparseable; a missing - // file degrades to a warning and the artifact removal still proceeds). - let mut lock_lines: Option> = None; - let mut lock_prefix = String::new(); - let mut lock_eol = LineEndings::None; - if touches_lock { - let read = match lock_text { - Some(read) => read, - None => read_regular_to_string(&project_root.join(PNPM_LOCK)).await, - }; - match read { - // The wiring lives in the project document; an env document - // ahead of it is written back verbatim, in the lock's own line - // endings (N15). - Ok(text) => { - lock_eol = LineEndings::of(&text); - let text = to_lf(&text); - let (prefix, project) = match dialect { - PnpmDialect::V9 => match split_v9_lock(&text) { - Ok(split) => split, - Err(why) => return RevertOutcome::failed(why), - }, - PnpmDialect::Legacy => ("", &*text), - }; - lock_prefix = prefix.to_string(); - lock_lines = Some(split_lines(project)); - } - Err(e) if e.kind() == std::io::ErrorKind::NotFound => { - outcome.warnings.push(VendorWarning::new( - "vendor_lockfile_missing", - format!("{PNPM_LOCK} is missing; lock fragments cannot be restored"), - )); - } - Err(e) => return RevertOutcome::failed(format!("cannot read {PNPM_LOCK}: {e}")), + // A missing lock degrades to a warning and the artifact removal still + // proceeds; an unreadable one fails. + let lock_eol = match lock_read { + Some(Ok((_, eol))) => eol, + Some(Err(e)) if e.kind() == std::io::ErrorKind::NotFound => { + outcome.warnings.push(VendorWarning::new( + "vendor_lockfile_missing", + format!("{PNPM_LOCK} is missing; lock fragments cannot be restored"), + )); + LineEndings::None } - } + Some(Err(e)) => return RevertOutcome::failed(format!("cannot read {PNPM_LOCK}: {e}")), + None => LineEndings::None, + }; let mut pkg_state: Option<(Value, JsonLayout)> = None; if touches_pkg { match read_regular_to_bytes(&project_root.join(PACKAGE_JSON)).await { @@ -1186,9 +1210,10 @@ pub(super) async fn revert_pnpm_dialect( && build_approvals::is_ws_kind(&r.kind) }) .collect(); - if ws_rec.is_some() || !ws_approvals.is_empty() { + if let Some(read) = ws_read.filter(|_| ws_rec.is_some() || !ws_approvals.is_empty()) { if let Err(e) = revert_workspace( project_root, + read, ws_rec, &ws_approvals, entry, @@ -1217,9 +1242,11 @@ pub(super) async fn revert_pnpm_dialect( /// out of a file we edited (restoring the taken-over value, and dropping an /// `overrides:` section we created once it empties). Drift ⇒ warning, left /// alone. Build approvals vendoring added (`approvals`) are taken out too. -/// `Err` is a genuine write failure. +/// `read` is the file as [`read_pnpm_text`] read it (LF, with the endings +/// it is written back in). `Err` is a genuine read or write failure. async fn revert_workspace( project_root: &Path, + read: std::io::Result<(String, LineEndings)>, rec: Option<&WiringRecord>, approvals: &[&WiringRecord], entry: &VendorEntry, @@ -1235,8 +1262,8 @@ async fn revert_workspace( }; let entry_uuid = entry.uuid.as_str(); let path = project_root.join(PNPM_WORKSPACE); - let text = match read_regular_to_string(&path).await { - Ok(t) => t, + let (text, eol) = match read { + Ok(read) => read, Err(e) if e.kind() == std::io::ErrorKind::NotFound => { // ALREADY CONVERGED: for an Added override (no recorded // original) the reverted state is "no override" — a missing @@ -1253,10 +1280,6 @@ async fn revert_workspace( } Err(e) => return Err(format!("cannot read {PNPM_WORKSPACE}: {e}")), }; - // A CRLF file (an autocrlf checkout of what vendoring wrote) is read - // like its LF twin and written back CRLF (N15). - let eol = LineEndings::of(&text); - let text = to_lf(&text).into_owned(); // Fast path: a file we created that is still byte-identical to the // scaffold we wrote → delete it (byte-restore to "no file"). @@ -1419,11 +1442,6 @@ impl EditCtx<'_> { vendor_value_is_for(value, self.name, self.version) } - /// [`target_key`] of a packages/snapshots key for this release. - pub(super) fn target_key(&self, key: &str) -> Option { - target_key(key, self.name, self.version) - } - /// The per-importer `specifier:` spelling: re-relativized for nested /// importers, root-relative for `.` (spike P7). fn spec_for_importer(&self, importer: &str) -> String { @@ -1803,26 +1821,28 @@ fn check_rewritable_refs_with( } } // The positions the checks below read: from the index, the earliest - // position each refusal can hold at; by the scan, every one. - let scanned; - let (snapshot_refs, importer_refs): (Box + '_>, _) = - match index { - Some(index) => ( - Box::new( - index - .snapshot_ref_candidates(®_key, name, version) - .into_iter(), - ), - index.importer_ref_candidates(®_key, name, version), + // position each refusal can hold at; by the scan, every one, read + // lazily so the scan stops at the first refusal. + type Refs<'a, T> = Box + 'a>; + let (snapshot_refs, importer_refs): (Refs, Refs) = match index { + Some(index) => ( + Box::new( + index + .snapshot_ref_candidates(®_key, name, version) + .into_iter(), ), - None => { - scanned = importer_deps(lines); - ( - Box::new(snapshot_positions(lines)), - scanned.iter().collect(), - ) - } - }; + Box::new( + index + .importer_ref_candidates(®_key, name, version) + .into_iter() + .map(ImporterDep::as_ref), + ), + ), + None => ( + Box::new(snapshot_positions(lines)), + Box::new(importer_refs(lines)), + ), + }; for candidate in snapshot_refs { match candidate { SnapshotCandidate::Key(header) => { @@ -1846,23 +1866,18 @@ fn check_rewritable_refs_with( } } for entry in importer_refs { - let dep = entry.dep.as_str(); - let Some(v) = entry.ver.as_deref().map(unquote_value) else { + let Some(v) = entry.ver.map(unquote_value) else { continue; }; if v == reg_key || v.starts_with(&key_peer_prefix) { return refuse("an aliased importer version", v); } - if dep == name && v.starts_with(&val_peer_prefix) { + if entry.dep == name && v.starts_with(&val_peer_prefix) { return refuse("a peer-suffixed importer version", v); } // pnpm spells the default catalog `'catalog:'` (quoted) and named // ones `catalog:` (bare). - if let Some(s) = entry - .spec - .as_deref() - .filter(|_| dep == name && v == version) - { + if let Some(s) = entry.spec.filter(|_| entry.dep == name && v == version) { if unquote_value(s).starts_with("catalog:") { return refuse_catalog(&format!("importer `{}` specifier", entry.importer), s); } @@ -1887,27 +1902,33 @@ fn snapshot_positions(lines: &[String]) -> impl Iterator Vec { - let Some((start, end)) = section_bounds(lines, "importers") else { - return Vec::new(); - }; - let mut out = Vec::new(); - let mut i = start + 1; - while let Some(importer) = next_block(lines, i, end) { - out.extend( - dep_entries(lines, importer.header + 1, importer.end, 6) - .into_iter() - .map(|e| ImporterDep { - importer: importer.key.clone(), - dep: e.dep, - spec: e.spec.map(|(_, s)| s), - ver: e.ver.map(|(_, v)| v), - }), - ); - i = importer.end; - } - out +/// One importer dep entry as the ref gate reads it ([`ImporterDep`], +/// borrowed). +struct ImporterRef<'a> { + importer: &'a str, + dep: &'a str, + spec: Option<&'a str>, + ver: Option<&'a str>, +} + +/// Every importer's [`DepEntry`]s, in lock order, read lazily. +fn importer_refs(lines: &[String]) -> impl Iterator> { + let blocks = section_bounds(lines, "importers").into_iter(); + blocks + .flat_map(move |(start, end)| { + std::iter::successors(next_block(lines, start + 1, end), move |b| { + next_block(lines, b.end, end) + }) + }) + .flat_map(move |importer| { + let key = parse_key_line(&lines[importer.header], 2).map_or("", |(key, ..)| key); + dep_entries(lines, importer.header + 1, importer.end, 6).map(move |e| ImporterRef { + importer: key, + dep: e.dep, + spec: e.spec.map(|(_, s)| s), + ver: e.ver.map(|(_, v)| v), + }) + }) } // ───────────────────────── package.json override ───────────────────────── @@ -2371,48 +2392,58 @@ pub(super) fn overrides_record( /// One dependency entry with nested `specifier:` / `version:` fields (each /// with its line): a v9 importer dep (key at 6, fields at 8) or a legacy -/// 6.0 root dep (2 / 4). -pub(super) struct DepEntry { - pub(super) dep: String, - pub(super) spec: Option<(usize, String)>, - pub(super) ver: Option<(usize, String)>, +/// 6.0 root dep (2 / 4). Borrowed from the lines. +pub(super) struct DepEntry<'l> { + pub(super) dep: &'l str, + pub(super) spec: Option<(usize, &'l str)>, + pub(super) ver: Option<(usize, &'l str)>, } -/// Every [`DepEntry`] keyed at `indent` in `[start, end)`, in order. A key -/// with an inline value is not one; unknown fields are skipped. +impl DepEntry<'_> { + /// `(specifier line, specifier, version line, version)`, owned so the + /// caller can rewrite the lines; `None` unless both fields are present. + pub(super) fn pair(&self) -> Option<(usize, String, usize, String)> { + let ((si, spec), (vi, ver)) = (self.spec?, self.ver?); + Some((si, spec.to_string(), vi, ver.to_string())) + } +} + +/// Every [`DepEntry`] keyed at `indent` in `[start, end)`, in order, read +/// lazily. A key with an inline value is not one; unknown fields are +/// skipped. pub(super) fn dep_entries( lines: &[String], start: usize, end: usize, indent: usize, -) -> Vec { - let mut out = Vec::new(); +) -> impl Iterator> { let mut k = start; - while k < end { - let Some((dep, _repr, "")) = parse_key_line(&lines[k], indent) else { + std::iter::from_fn(move || { + while k < end { k += 1; - continue; - }; - let mut entry = DepEntry { - dep: dep.to_string(), - spec: None, - ver: None, - }; - k += 1; - while let Some((field, _repr, value)) = lines[..end] - .get(k) - .and_then(|l| parse_key_line(l, indent + 2)) - { - match field { - "specifier" => entry.spec = Some((k, value.to_string())), - "version" => entry.ver = Some((k, value.to_string())), - _ => {} + let Some((dep, _repr, "")) = parse_key_line(&lines[k - 1], indent) else { + continue; + }; + let mut entry = DepEntry { + dep, + spec: None, + ver: None, + }; + while let Some((field, _repr, value)) = lines[..end] + .get(k) + .and_then(|l| parse_key_line(l, indent + 2)) + { + match field { + "specifier" => entry.spec = Some((k, value)), + "version" => entry.ver = Some((k, value)), + _ => {} + } + k += 1; } - k += 1; + return Some(entry); } - out.push(entry); - } - out + None + }) } /// Edit 2: every importer's dep entry for the exact `name@version` — @@ -2432,12 +2463,13 @@ fn edit_importers( let importer_key = importer.key; // Dep entries sit at 6-space indent under the 4-space dep-type // headers; their fields at 8. - for entry in dep_entries(lines.read(), importer.header + 1, importer.end, 6) { - let (Some((si, old_spec)), Some((vi, old_ver))) = (entry.spec, entry.ver) else { - continue; - }; + let pairs: Vec<_> = dep_entries(lines.read(), importer.header + 1, importer.end, 6) + .filter(|entry| entry.dep == ctx.name) + .filter_map(|entry| entry.pair()) + .collect(); + for (si, old_spec, vi, old_ver) in pairs { let target = old_ver == ctx.version || (old_ver != ctx.spec && ctx.is_ours(&old_ver)); - if entry.dep != ctx.name || !target { + if !target { continue; } let importer_spec = ctx.spec_for_importer(&importer_key); @@ -2445,7 +2477,7 @@ fn edit_importers( file: PNPM_LOCK.to_string(), kind: KIND_LOCK_IMPORTER_DEP.to_string(), action: WiringAction::Rewritten, - key: Some(format!("{importer_key}|{}", entry.dep)), + key: Some(format!("{importer_key}|{}", ctx.name)), original: (!ctx.is_ours(&old_ver)) .then(|| serde_json::json!({ "specifier": old_spec, "version": old_ver })), new: Some(serde_json::json!({ @@ -2474,12 +2506,11 @@ fn edit_packages( let bounds = lines.bounds("packages").ok_or("no packages: section")?; let reg_key = ctx.reg_key(); let new_key = ctx.new_key(); - let matches = target_blocks(lines, "packages", bounds, ctx.name, ctx.version)?; - // The first registry-or-ours block is the one rewritten (every path // below returns). - if let Some(block) = matches.into_iter().next() { - let is_ours_key = ctx.target_key(&block.key) == Some(true); + if let Some((block, is_ours_key)) = + target_block(lines, "packages", bounds, ctx.name, ctx.version)? + { let original_lines: Vec = lines.read()[block.header..block.end].to_vec(); let expected_resolution = format!( " resolution: {{integrity: {}, tarball: {}}}", @@ -2556,11 +2587,10 @@ fn edit_snapshot_rekey( }; let reg_key = ctx.reg_key(); let new_key = ctx.new_key(); - let matches = target_blocks(lines, "snapshots", bounds, ctx.name, ctx.version)?; - // The first registry-or-ours block decides (every path below returns). - if let Some(block) = matches.into_iter().next() { - let is_ours_key = ctx.target_key(&block.key) == Some(true); + if let Some((block, is_ours_key)) = + target_block(lines, "snapshots", bounds, ctx.name, ctx.version)? + { if block.key == new_key { return Ok(false); // in sync } @@ -2671,10 +2701,11 @@ fn edit_snapshot_refs( } /// The blocks of `section` (`[start, end)`, its current bounds) whose key is -/// a [`target_key`] of `name@version`, in lock order — the only blocks the -/// packages/snapshots edits act on. From the index while the lines are the -/// memoized split (its candidates are a superset, filtered by the same -/// predicate), else by the scan. +/// a [`target_key`] of `name@version`, each with that classification (`true`: +/// a rekey), in lock order — the only blocks the packages/snapshots edits +/// act on. From the index while the lines are the memoized split (its +/// candidates are a superset, filtered by the same predicate), else by the +/// scan. fn matching_blocks( lines: &L, section: &str, @@ -2682,52 +2713,49 @@ fn matching_blocks( end: usize, name: &str, version: &str, -) -> Vec { - let keep = |key: &str| target_key(key, name, version).is_some(); +) -> Vec<(YamlBlock, bool)> { + let classify = |block: YamlBlock| Some((target_key(&block.key, name, version)?, block)); if let Some(section) = lines.index().and_then(|index| index.section(section)) { return section .candidates(name, version) .into_iter() - .filter(|block| keep(&block.key)) - .map(IndexedBlock::to_block) + .filter_map(|block| classify(block.to_block())) + .map(|(rekeyed, block)| (block, rekeyed)) .collect(); } let lines = lines.read(); - let mut found = Vec::new(); - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - i = block.end; - if keep(&block.key) { - found.push(block); - } - } - found + let blocks = std::iter::successors(next_block(lines, start + 1, end), |b| { + next_block(lines, b.end, end) + }); + blocks + .filter_map(classify) + .map(|(rekeyed, block)| (block, rekeyed)) + .collect() } -/// [`matching_blocks`], failing closed on a half-drifted lock: when the -/// section carries BOTH the registry entry and a rekey of it, a rekey would -/// splice a DUPLICATE mapping key (pnpm refuses to parse those) and surgery -/// cannot decide which block carries the truth. -pub(super) fn target_blocks( +/// The block the packages/snapshots edit rewrites: the first +/// [`matching_blocks`] entry, with whether it is a rekey already made. +/// Fails closed on a half-drifted lock: when the section carries BOTH the +/// registry entry and a rekey of it, a rekey would splice a DUPLICATE +/// mapping key (pnpm refuses to parse those) and surgery cannot decide +/// which block carries the truth. +pub(super) fn target_block( lines: &L, section: &str, (start, end): (usize, usize), name: &str, version: &str, -) -> Result, String> { +) -> Result, String> { let matches = matching_blocks(lines, section, start, end, name, version); - let rekeyed = |want| move |b: &&YamlBlock| target_key(&b.key, name, version) == Some(want); - match ( - matches.iter().find(rekeyed(false)), - matches.iter().any(|b| rekeyed(true)(&b)), - ) { - (Some(registry), true) => Err(format!( + let registry = matches.iter().find(|(_, rekeyed)| !rekeyed); + if let Some((registry, _)) = registry.filter(|_| matches.iter().any(|(_, rekeyed)| *rekeyed)) { + return Err(format!( "{section} section carries BOTH `{}` and a vendored `file:` rekey of it (a \ half-edited lock); run `pnpm install` to re-resolve it, then re-vendor", registry.key - )), - _ => Ok(matches), + )); } + Ok(matches.into_iter().next()) } // ─────────────────────── memoized split + section index ──────────────────── @@ -2918,12 +2946,12 @@ impl SectionIndex { let at = index.blocks.len(); let key = block.key.strip_prefix('/').unwrap_or(&block.key); index.by_key.entry(key.to_string()).or_default().push(at); - for (p, _) in format!("@{}", block.key).match_indices("@file:") { - index - .by_file_prefix - .entry(block.key[..p.saturating_sub(1)].to_string()) - .or_default() - .push(at); + // A bare `file:` key files under `""`, a `name@file:` one under `name`. + let bare = block.key.starts_with("file:").then_some(0); + let named = block.key.match_indices("@file:").map(|(p, _)| p); + for p in bare.into_iter().chain(named) { + let prefix = block.key[..p].to_string(); + index.by_file_prefix.entry(prefix).or_default().push(at); } index.blocks.push(IndexedBlock { header: block.header, @@ -2967,6 +2995,17 @@ struct ImporterDep { ver: Option, } +impl ImporterDep { + fn as_ref(&self) -> ImporterRef<'_> { + ImporterRef { + importer: &self.importer, + dep: &self.dep, + spec: self.spec.as_deref(), + ver: self.ver.as_deref(), + } + } +} + /// A position the snapshot scan of [`check_rewritable_refs`] may refuse /// at: a block header or a body line (by line). enum SnapshotCandidate { @@ -3062,8 +3101,15 @@ impl LockIndex { } } - index.importer_deps = importer_deps(lines); - index.importer_deps.retain(|entry| entry.ver.is_some()); + index.importer_deps = importer_refs(lines) + .filter(|entry| entry.ver.is_some()) + .map(|e| ImporterDep { + importer: e.importer.to_string(), + dep: e.dep.to_string(), + spec: e.spec.map(str::to_string), + ver: e.ver.map(str::to_string), + }) + .collect(); for (at, entry) in index.importer_deps.iter().enumerate() { let (dep, v) = ( &entry.dep, @@ -3415,9 +3461,7 @@ impl DepPair<'_> { warnings: &mut Vec, ) { let (rec, key, label) = (self.rec, self.key, self.label); - let entry = dep_entries(lines, start, end, indent) - .into_iter() - .find(|e| e.dep == self.dep); + let entry = dep_entries(lines, start, end, indent).find(|e| e.dep == self.dep); let Some((Some((si, live_spec)), Some((vi, live_ver)))) = entry.map(|e| (e.spec, e.ver)) else { return self.removed(warnings); @@ -3431,13 +3475,13 @@ impl DepPair<'_> { } let original = rec.original.as_ref(); if original.is_some() - && field(original, "specifier") == Some(&live_spec) - && field(original, "version") == Some(&live_ver) + && field(original, "specifier") == Some(live_spec) + && field(original, "version") == Some(live_ver) { return; } - let ours = field(rec.new.as_ref(), "version") == Some(&live_ver) - || parse_vendor_path(&live_ver) + let ours = field(rec.new.as_ref(), "version") == Some(live_ver) + || parse_vendor_path(live_ver) .is_some_and(|p| p.eco == "npm" && p.uuid == self.entry_uuid); if !ours { return warnings.push(drifted(format!( @@ -5999,6 +6043,57 @@ snapshots: format!("---\n{env}\n---\n{project}") } + /// N15 on pnpm 11's two-document lock: vendored, then checked out with + /// autocrlf (`---\r` separators), the lock reverts to its CRLF bytes. + /// Review finding: the revert's document check read the raw CRLF text, + /// refused `---\r` as a marker line, and failed every such revert. + #[tokio::test] + async fn crlf_two_document_lock_reverts() { + let before = two_doc(ENV_DOC_CONFIG_DEP, P1_BEFORE_LOCK); + let fx = fixture_with(P1_BEFORE_PKG, &before).await; + let (_, entry, _) = expect_done(fx.vendor(false).await); + let crlf = fx.read(PNPM_LOCK).await.replace('\n', "\r\n"); + tokio::fs::write(fx.root().join(PNPM_LOCK), &crlf) + .await + .unwrap(); + let outcome = revert_pnpm(&entry.unwrap(), fx.root(), false).await; + assert!(outcome.success, "{:?}", outcome.error); + assert_eq!(fx.read(PNPM_LOCK).await, before.replace('\n', "\r\n")); + } + + /// Review finding: a revert met a lock or workspace file that MIXES CRLF + /// and LF and wrote it back all-LF. It now fails before any write, dry + /// run included, as vendoring refuses such a file: every surface and the + /// artifact stay as they are. + #[tokio::test] + async fn mixed_line_ending_revert_fails_before_any_write() { + for file in [PNPM_LOCK, PNPM_WORKSPACE] { + let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; + let (_, entry, _) = expect_done(fx.vendor(false).await); + let entry = entry.unwrap(); + let mixed = fx.read(file).await.replacen('\n', "\r\n", 1); + tokio::fs::write(fx.root().join(file), &mixed) + .await + .unwrap(); + let pkg = fx.read(PACKAGE_JSON).await; + for dry_run in [true, false] { + let outcome = revert_pnpm(&entry, fx.root(), dry_run).await; + assert!(!outcome.success, "{file}"); + let error = outcome.error.unwrap_or_default(); + assert!( + error.contains(&format!("{file} mixes CRLF and LF")), + "{error}" + ); + } + assert_eq!(fx.read(file).await, mixed, "{file} left alone"); + assert_eq!(fx.read(PACKAGE_JSON).await, pkg, "{file}: nothing reverted"); + assert!( + fx.root().join(fx.rel_tgz()).exists(), + "{file}: artifact kept" + ); + } + } + /// #466: vendoring a two-document lock edits the PROJECT document only — /// byte-for-byte the single-document oracle — and leaves the env /// document untouched, even when a config dependency shares the @@ -6403,7 +6498,12 @@ snapshots: // Future versions: the fix is an allowlisted-emitting pnpm or a // newer build, never "pnpm >= 9" (the user's pnpm already is). - for head in ["lockfileVersion: '9.1'\n", "lockfileVersion: '10.0'\n"] { + // An unparseable major (review finding) is not "old" either. + for head in [ + "lockfileVersion: '9.1'\n", + "lockfileVersion: '10.0'\n", + "lockfileVersion: 'v10'\n", + ] { let err = sniff_lock_grammar(head).unwrap_err(); assert!(err.contains("update socket-patch"), "{err}"); assert!(!err.contains("re-lock with pnpm >= 9"), "{err}"); @@ -9821,19 +9921,13 @@ snapshots: let lines = split_lines( " dep:\n specifier: ^1.0.0\n engines: whatever\n version: 1.0.0\n other:\n inline: 1.0.0", ); - let entries = dep_entries(&lines, 0, lines.len(), 6); - let got: Vec<_> = entries - .iter() - .map(|e| (e.dep.as_str(), e.spec.clone(), e.ver.clone())) + let got: Vec<_> = dep_entries(&lines, 0, lines.len(), 6) + .map(|e| (e.dep, e.spec, e.ver)) .collect(); assert_eq!( got, vec![ - ( - "dep", - Some((1, "^1.0.0".to_string())), - Some((3, "1.0.0".to_string())) - ), + ("dep", Some((1, "^1.0.0")), Some((3, "1.0.0"))), ("other", None, None), ], "fields end at the first non-field line; an inline value is no entry" diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs b/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs index 09fc48634..1c1dafb3e 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs @@ -65,8 +65,8 @@ use super::common::refused; use super::path::parse_vendor_path; use super::pnpm_lock::{ dep_entries, drifted, lines_value, lock_has_target_package, ref_record_blocks, removed, - revert_block, revert_overrides_line, target_blocks, target_key, vendor_value_is_for, DepPair, - DepRef, EditCtx, PnpmDialect, KIND_LOCK_OVERRIDES, KIND_LOCK_PACKAGE, + revert_block, revert_overrides_line, target_block, vendor_value_is_for, DepPair, DepRef, + EditCtx, PnpmDialect, KIND_LOCK_OVERRIDES, KIND_LOCK_PACKAGE, }; use super::source::PackageSource; use super::state::{VendorEntry, WiringAction, WiringRecord}; @@ -415,16 +415,15 @@ fn check_rewritable_refs(lines: &[String], ctx: &Ctx<'_>) -> Result<(), String> .iter() .filter_map(|line| parse_key_line(line, 2)) .filter(|(_, _, value)| !value.is_empty()) - .map(|(dep, _, value)| (dep.to_string(), value.to_string())); + .map(|(dep, _, value)| (dep, value)); let nested = dep_entries(lines, start + 1, end, 2) - .into_iter() .filter_map(|entry| Some((entry.dep, entry.ver?.1))); for (dep, value) in bare.chain(nested) { if value == reg_key || value.starts_with(&key_peer_prefix) { - return refuse("an aliased root dependency", &value); + return refuse("an aliased root dependency", value); } if dep == ctx.name && value.starts_with(&val_peer_prefix) { - return refuse("a peer-suffixed root dependency", &value); + return refuse("a peer-suffixed root dependency", value); } } } @@ -578,11 +577,12 @@ fn edit_root_deps_v60( let Some((start, end)) = section_bounds(lines, section) else { continue; }; - for entry in dep_entries(lines, start + 1, end, 2) { - let (Some((si, old_spec)), Some((vi, old_ver))) = (entry.spec, entry.ver) else { - continue; - }; - if entry.dep != ctx.name || !(old_ver == ctx.version || ctx.is_ours(&old_ver)) { + let pairs: Vec<_> = dep_entries(lines, start + 1, end, 2) + .filter(|entry| entry.dep == ctx.name) + .filter_map(|entry| entry.pair()) + .collect(); + for (si, old_spec, vi, old_ver) in pairs { + if !(old_ver == ctx.version || ctx.is_ours(&old_ver)) { continue; } hit = true; @@ -593,7 +593,7 @@ fn edit_root_deps_v60( file: PNPM_LOCK.to_string(), kind: KIND_LOCK_ROOT_DEP_PAIR.to_string(), action: WiringAction::Rewritten, - key: Some(format!("{section}|{}", entry.dep)), + key: Some(format!("{section}|{}", ctx.name)), original: (!ctx.is_ours(&old_ver)) .then(|| serde_json::json!({ "specifier": old_spec, "version": old_ver })), new: Some(serde_json::json!({ @@ -621,11 +621,11 @@ fn edit_packages( let bounds = section_bounds(lines, "packages").ok_or("no packages: section")?; let reg_key = ctx.reg_key(); let new_key = ctx.new_key(); - let matches = target_blocks(&*lines, "packages", bounds, ctx.name, ctx.version)?; // The first registry-or-ours block is the one rewritten (every path // below returns). - if let Some(block) = matches.into_iter().next() { - let is_ours_key = target_key(&block.key, ctx.name, ctx.version) == Some(true); + if let Some((block, is_ours_key)) = + target_block(&*lines, "packages", bounds, ctx.name, ctx.version)? + { let original_lines: Vec = lines[block.header..block.end].to_vec(); let expected_resolution = format!( " resolution: {{integrity: {}, tarball: {}}}", diff --git a/crates/socket-patch-core/src/vex/discover/npm.rs b/crates/socket-patch-core/src/vex/discover/npm.rs index 96d63c64e..b759d13a9 100644 --- a/crates/socket-patch-core/src/vex/discover/npm.rs +++ b/crates/socket-patch-core/src/vex/discover/npm.rs @@ -58,6 +58,7 @@ use super::{ DIAG_LOCKFILE_UNPARSEABLE, DIAG_REF_INVALID, DIAG_REF_UNATTRIBUTABLE, }; use crate::constants::npm_family::{NPM_LOCKS, PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY}; +use crate::formats::pnpm::plan::governing_pnpm_lock; use crate::formats::pnpm::{ classify_pnpm_key, entry_bundled, entry_field, pnpm_registry_key, Bundled, PnpmDocument, PnpmKey, PnpmLock, PnpmPackage, @@ -627,14 +628,13 @@ fn entry_ref( /// ## Files /// /// * `pnpm-lock.yaml` at the root. -/// * `shrinkwrap.yaml` (pnpm 1 / 2) ONLY when there is no `pnpm-lock.yaml`: -/// pnpm >= 3 never reads it (it migrates it to `pnpm-lock.yaml` once), and -/// pnpm <= 2 never writes `pnpm-lock.yaml`, so a pair means one of them is -/// migration debris — the same call `lock_inventory` makes (it reads -/// shrinkwrap.yaml only on `vendor_lockfile_missing`). The hosted rewriter -/// edits both when both exist, so the live one is covered either way; a -/// stale shrinkwrap left carrying a Socket url must not attest from the -/// lockfile basis after `pnpm-lock.yaml` was reverted. +/// * `shrinkwrap.yaml` (pnpm 1 / 2) ONLY when there is no `pnpm-lock.yaml`, +/// or the project pins pnpm 1 / 2: pnpm >= 3 never reads it (it migrates +/// it to `pnpm-lock.yaml` once), and pnpm <= 2 reads nothing else, so a +/// pair means one of them is migration debris. The choice is the hosted +/// planner's (`formats::pnpm::plan::governing_pnpm_lock`, N11): the +/// other file is never rewritten, and a stale Socket url left in it must +/// not attest from the lockfile basis. /// * Rush (`rush.json` present): `common/config/rush/pnpm-lock.yaml` and every /// `common/config/subspaces//pnpm-lock.yaml` — the only non-root /// locks in scope, exactly the set `scan --mode hosted` collects and @@ -701,15 +701,20 @@ async fn extract_pnpm(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { crate::utils::pnpm_workspace::lockfile_disabled(&ctx.view, false).await }; let disabled = disabled.as_deref(); - let root_lock = ctx.exists(PNPM_LOCK).await; - if root_lock { - extract_pnpm_lock(ctx, PNPM_LOCK, disabled, out).await; - // pnpm never reads the debris shrinkwrap: the patches it still names - // are recognized as NOT wired (module docs, rule 11), so a ledger - // record its stale Socket url alone keeps "mentioned" is dead. - ctx.recognize_ignored(PNPM_SHRINKWRAP_LEGACY).await; - } else { - extract_pnpm_lock(ctx, PNPM_SHRINKWRAP_LEGACY, disabled, out).await; + // The lock pnpm reads, as the hosted planner picks it + // (`formats::pnpm::plan::governing_pnpm_lock`, N11). pnpm never reads + // the other one: the patches it still names are recognized as NOT + // wired (module docs, rule 11), so a ledger record its stale Socket + // url alone keeps "mentioned" is dead. + let pinned = crate::utils::pnpm_workspace::pinned_pnpm_major_in(&ctx.view).await; + let (lock, shrinkwrap) = (PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY); + let both = (ctx.exists(lock).await, ctx.exists(shrinkwrap).await); + if let Some(governing) = governing_pnpm_lock(both.0, both.1, pinned) { + extract_pnpm_lock(ctx, governing, disabled, out).await; + if both == (true, true) { + let shadowed = if governing == lock { shrinkwrap } else { lock }; + ctx.recognize_ignored(shadowed).await; + } } if rush { // The common lock, then each subspace's, sorted for deterministic @@ -3574,6 +3579,15 @@ mod tests { out.recognized_files(UUID_A, WiringMode::Hosted), vec![std::path::Path::new("shrinkwrap.yaml")] ); + // N11 review finding: a project that pins pnpm 1 / 2 installs from + // `shrinkwrap.yaml`, so there it is the lock that attests and the + // stray `pnpm-lock.yaml` is the shadowed one, as the hosted planner + // picks them (`plan::governing_pnpm_lock`). + with_lock.write("package.json", r#"{"packageManager": "pnpm@2.25.7"}"#); + assert_refs( + &run(&with_lock).await, + &[("pkg:npm/left-pad@1.3.0", UUID_A, WiringMode::Hosted)], + ); } /// `--patch-server-url` deployments count; the same URL without the diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md index 00af98b9b..83bed89fa 100644 --- a/docs/design/lock-model-divergences.md +++ b/docs/design/lock-model-divergences.md @@ -31,7 +31,7 @@ follow from the install-tree decision. | J12 | scala-cli | Guidance only (`patch/redirect/scala_guidance.rs`) | Back-port: hosted wires scala-cli through the merged planner (a same-GAV Socket repository instead of a suffixed version). | | P3 | Is a shadowed `pdm.lock` rewritten? | Only when it governs (`pdm_drives`, ~564) | Rewrite it like every other tree, per the install-tree decision. | | P4 | Are `-r` includes of `requirements.txt` rewritten? | Root file only (`patch/redirect/requirements.rs` ~208) | Follow includes, per the install-tree decision. | -| N11 | Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` rewritten? | Yes; its all-or-nothing gate covers both files | **Resolved (owner: shadowed) in PR 2b.** Swept for stale identities, never written, and out of the all-or-nothing gate (`formats::pnpm::plan::planned_locks`). pnpm never reads it, and an unparseable debris key in it used to refuse the dependency in every lock. A `shrinkwrap.yaml` that is the only pnpm lock is rewritten as before. | +| N11 | Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` rewritten? | Yes; its all-or-nothing gate covers both files | **Resolved (owner: shadowed) in PR 2b.** Swept for stale identities, never written, and out of the all-or-nothing gate (`formats::pnpm::plan::governing_pnpm_lock`, which hosted and discovery share). When the project pins pnpm 1 / 2 (install record or package.json `packageManager`), which reads only `shrinkwrap.yaml`, the stray `pnpm-lock.yaml` is the shadowed one. pnpm never reads it, and an unparseable debris key in it used to refuse the dependency in every lock. A `shrinkwrap.yaml` that is the only pnpm lock is rewritten as before. | ## JVM @@ -89,7 +89,7 @@ follow from the install-tree decision. | N8 | npm `file:` directory link identity (#1481) | Hosted and discovery give a versionless link its source directory's `name@version`. Vendored matches the entry's own version, wiring the registry copies with no warning. | Hosted | silent vendored success | | N9 | yarn classic blocks reached only through an alias | Hosted skips them. Vendored rewrites them. Discovery counts an alias block as an install of the real package. | Back-port: hosted rewrites alias blocks (vendored supports what hosted lacks) | | | N10 | Locks of several npm-family package managers in one project | Hosted rewrites every family; discovery reads every family with the contest. The inventory and vendored pick one family (vlt > bun > pnpm > yarn > npm). | Union (separate trees) | | -| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | **Resolved (owner: shadowed) in PR 2b**: hosted never rewrites it and leaves it out of the gate; discovery keeps sweeping it; vendored never reads it. A lone `shrinkwrap.yaml` keeps today's behavior. | | +| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | **Resolved (owner: shadowed) in PR 2b**: hosted never rewrites it and leaves it out of the gate; discovery keeps sweeping it; vendored never reads it. A pnpm 1 / 2 pin flips the pair: `shrinkwrap.yaml` governs, `pnpm-lock.yaml` is shadowed. A lone `shrinkwrap.yaml` keeps today's behavior. | | | N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. **Resolved in PR 2a** (the inventory reads them whenever `rush.json` exists). | target-specific (vendored) | | N13 | Plug'n'Play | Vendored refuses. The hosted berry preflight has no PnP gate. The inventory returns an error, so `scan` sees nothing in a project hosted pinned. | The inventory reads PnP locks. Vendored keeps its refusal. | target-specific (vendored) | | N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery. **Resolved in PR 2a**: under the setting (not under Rush), the inventory keeps a Socket-hosted pnpm entry but drops its integrity, as discovery withholds the lock pin. | target-specific | diff --git a/docs/ecosystems.md b/docs/ecosystems.md index 9e7d42b11..9a3d1f688 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -172,7 +172,8 @@ Each Legacy format has an upgrade path and an undo path. Both work in v5: 12's dual-lock state. Majors 6–12 are measured in [npm compatibility](testing/npm-compatibility.md). - **pnpm** — hosted rewriting supports legacy `shrinkwrap.yaml` (pnpm 1/2; never - rewritten beside a `pnpm-lock.yaml`, which pnpm reads instead), + rewritten beside a `pnpm-lock.yaml`, which pnpm >= 3 reads instead; a + project pinning pnpm 1/2 keeps `shrinkwrap.yaml` and leaves the stray lock), lockfileVersion 5.x (pnpm 3–7), 6.0 (pnpm 8), and 9.0 (pnpm 9–12). Early pnpm 1 locks with shrinkwrapVersion 3 and no positive minor version are refused because those installers discard hosted URLs; the tested diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json index 8ffa1aab4..f5ebc49a3 100644 --- a/scripts/loc-budget.json +++ b/scripts/loc-budget.json @@ -17,7 +17,7 @@ "crates/socket-patch-core/src/utils/pipenv.rs", "crates/socket-patch-core/src/crawlers/deno_crawler.rs" ], - "max": 31201 + "max": 31206 }, "lock-writers": { "paths": [ @@ -30,7 +30,7 @@ "crates/socket-patch-core/src/vendor/npm_origin.rs", "crates/socket-patch-core/src/vendor/test_support" ], - "max": 88093 + "max": 88115 } } } From e7df82058ea24126708f363adc7b6b7dc33f57c4 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Thu, 5 Nov 2026 23:34:40 -0600 Subject: [PATCH 17/36] Refuse vendoring a pnpm 1 / 2 project beside its shrinkwrap.yaml Vendored mode now reads the same plan::governing_pnpm_lock decision as hosted mode and discovery. When the project pins pnpm 1 or 2 (install record, else package.json packageManager), pnpm installs from shrinkwrap.yaml and ignores pnpm-lock.yaml. Vendored mode does not write the shrinkwrap grammar, so it now refuses with vendor_lockfile_version_unsupported before any write, instead of wiring the stray pnpm-lock.yaml. With no pin, nothing changes. Documented in CLI_CONTRACT.md and the N11 divergence row. The mixed-ending read error is now an io::ErrorKind::Unsupported error instead of a custom type, which keeps lock-writers within budget (88113). Co-Authored-By: Claude Opus 5.5 --- crates/socket-patch-cli/CLI_CONTRACT.md | 2 +- .../socket-patch-core/src/vendor/pnpm_lock.rs | 98 ++++++++++++------- docs/design/lock-model-divergences.md | 2 +- scripts/loc-budget.json | 2 +- 4 files changed, 66 insertions(+), 38 deletions(-) diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 4b7c6ec51..f133269e6 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -272,7 +272,7 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc **pnpm per-branch locks (#556)**: with `gitBranchLockfile` on (a top-level `gitBranchLockfile: true` in the root `pnpm-workspace.yaml`; when the YAML leaves the key unset, `git-branch-lockfile=true` in the root `.npmrc`, as pnpm 10 and older read it; on disk also `pnpm_config_git_branch_lockfile` (pnpm 11 and newer) or `npm_config_git_branch_lockfile` (pnpm 10 and older; either when the major is unknown, read as for the `lockfile` setting above) set to `true`; a workspace member with no `pnpm-workspace.yaml` of its own reads the governing ancestor's file and the `.npmrc` beside it), pnpm installs a git branch from its own `pnpm-lock..yaml` whenever that lock exists, in every directory it installs a lock from: the root, and under `sharedWorkspaceLockfile: false` each member's (#492). When the setting is on AND at least one such `pnpm-lock..yaml` exists (at the root, or in a member that installs from its own lock), hosted mode leaves every root pnpm lock (`pnpm-lock.yaml`, `shrinkwrap.yaml`) and `pnpm-workspace.yaml` untouched (every member lock too), confirms no npm pin from them and warns `redirect_pnpm_git_branch_lockfile` (in place of `redirect_pnpm_no_lockfile` when the branch lock is the only one); the detail names the setting and the branch lock(s), and the remedy is to turn `gitBranchLockfile` off and fold the branch locks into `pnpm-lock.yaml` with `pnpm install --merge-git-branch-lockfiles`, then re-run. A vendored→hosted takeover of a pnpm-vendored purl is refused with the same code (the vendored wiring and ledger stay byte-for-byte). Vendored mode refuses the same projects with `vendor_pnpm_git_branch_lockfile`, before any write and in the pre-flight. The setting with no branch lock (main, or a branch whose dependencies never changed) installs from `pnpm-lock.yaml`, which both modes wire as usual; a stray branch lock with the setting off is ignored. The in-memory engine fetches root `pnpm-lock.*.yaml` names presence-only and refuses the same way. -`redirect_pnpm_no_lockfile` names pnpm when installer markers exist without a lock; `redirect_pnpm_entry_vendored` identifies a vendored entry instead of reporting it missing. Supported `shrinkwrap.yaml` files are writable lockfiles, not read-only markers, when no `pnpm-lock.yaml` sits beside them. pnpm >= 3 never reads a `shrinkwrap.yaml` it has a `pnpm-lock.yaml` for, so that one is shadowed (unless the project pins pnpm 1 or 2 in `node_modules/.modules.yaml` or package.json's `packageManager`: those read only `shrinkwrap.yaml`, so there the stray `pnpm-lock.yaml` is the shadowed one): hosted mode never rewrites it and leaves it out of the all-or-nothing gate, so an entry it cannot repoint (or pnpm 1's early shrinkwrap head) refuses nothing, and `vex` only sweeps it for stale Socket identities. Vendored mode never writes it either. +`redirect_pnpm_no_lockfile` names pnpm when installer markers exist without a lock; `redirect_pnpm_entry_vendored` identifies a vendored entry instead of reporting it missing. Supported `shrinkwrap.yaml` files are writable lockfiles, not read-only markers, when no `pnpm-lock.yaml` sits beside them. pnpm >= 3 never reads a `shrinkwrap.yaml` it has a `pnpm-lock.yaml` for, so that one is shadowed (unless the project pins pnpm 1 or 2 in `node_modules/.modules.yaml` or package.json's `packageManager`: those read only `shrinkwrap.yaml`, so there the stray `pnpm-lock.yaml` is the shadowed one): hosted mode never rewrites it and leaves it out of the all-or-nothing gate, so an entry it cannot repoint (or pnpm 1's early shrinkwrap head) refuses nothing, and `vex` only sweeps it for stale Socket identities. Vendored mode never writes it either, and refuses a project that pins pnpm 1 or 2 beside a `shrinkwrap.yaml` (`vendor_lockfile_version_unsupported`: it does not write the shrinkwrap grammar, and the stray `pnpm-lock.yaml` would patch nothing pnpm installs). **vlt hosted-mode contract**: `scan` / `get --mode hosted` rewrite, in `vlt-lock.json`, every default-registry node of a granted `name@version` (the `''` / `npm` segment or a URL segment equal to the lock's scalar `registry`, both DepID grammars, every peer and modifier variant): slot [2] becomes the granted sha512 and slot [3] the hosted URL (appended to a 3-tuple); the DepID, flags and trailing slots, the line ending and every other byte stay. `options` is never edited and `vlt.json` is only read. A lock with another `lockfileVersion` (decided on the raw JSON token), a BOM, a non-object body or a `nodes` section outside vlt's one-node-per-line layout refuses the whole lock (`redirect_vlt_lock_unsupported`). **Confirmation**: vlt drives when its install state (`node_modules/.vlt-lock.json` or `node_modules/.vlt/`) is present or no other npm-family lock is; then only `vlt-lock.json` confirms a uuid. Otherwise every lock is rewritten, `redirect_vlt_sibling_lockfiles` warns, and the other locks' rules confirm, including a dep `vlt-lock.json` merely does not wire (`redirect_vlt_entry_not_found`, `redirect_vlt_entry_vendored`, or `redirect_vlt_custom_registry_skipped` alone). Whichever lock drives, a dep the vlt rewriter refuses (`redirect_vlt_missing_sha512`, `redirect_vlt_unsupported_lock_key`) is never confirmed by any lock, although a sibling lock may already carry its rewritten URL. **Artifact preflight**: before any takeover or write (dry runs included), each granted artifact with a default-registry instance is fetched once as vlt fetches it and must verify, else the dep is withheld (`redirect_vlt_artifact_unverifiable`, see the tag table). **Heal**: stale installed copies of Socket-owned nodes are removed so the next `vlt install` extracts the patched bytes, and `rollback` / `remove` do the same for the registry bytes (`--no-vlt-install-cleanup` keeps them; optional dependencies' copies are always kept); `redirect_vlt_reinstall_required` says what happened and what to run. The same-run `--vex` never attests a vlt package whose installed copy is stale or unchecked, whose lock a vlt release may ignore (`redirect_vlt_lockfile_version_missing`, `redirect_vlt_old_lockfile_ignored`, `redirect_vlt_scalar_registry_ignored`), or which also resolves from a non-default registry (`redirect_vlt_custom_registry_skipped`). `vlt.json` or vlt install state without `vlt-lock.json` warns `redirect_vlt_no_lockfile` instead of `redirect_npm_no_lockfile`. `rollback` / `remove` restore each hosted node's slots [2] and [3] from the version document of the registry the node resolves against (slot [3] is its `dist.tarball`, as vlt writes it; `upstream_registry_fallback` when that registry can't be read), following the lock's own slot-[3] convention (see "Hosted unwind coverage"). Tested releases: `docs/testing/vlt-compatibility.md`. diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock.rs b/crates/socket-patch-core/src/vendor/pnpm_lock.rs index 9d032904a..4d616350d 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock.rs @@ -74,12 +74,12 @@ use super::revert::{self, KeepPolicy}; use super::source::PackageSource; use super::state::{PnpmMeta, VendorEntry, WiringAction, WiringRecord}; use super::{RevertOpts, RevertOutcome, VendorOutcome, VendorWarning}; -use crate::constants::npm_family::PNPM_LOCK; +use crate::constants::npm_family::{PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY}; use crate::formats::pnpm::lines::{ indent_of, next_block, parse_key_line, section_bounds, split_lines, split_project_document, unquote_value, yaml_key, yaml_key_like, YamlBlock, }; -use crate::formats::pnpm::plan::{match_key, KeyMatch, Target}; +use crate::formats::pnpm::plan::{governing_pnpm_lock, match_key, KeyMatch, Target}; use crate::formats::pnpm::sniff_lock_grammar; use crate::formats::pnpm::workspace; use crate::formats::pnpm::{vendored_npm_uuids, PnpmLockGrammar}; @@ -577,13 +577,24 @@ async fn read_project( ))); } }; - if let Some(refusal) = git_branch_lock_refusal( - &crate::vendor::lock_inventory::ProjectView::Disk(project_root), - ) - .await - { + let view = crate::vendor::lock_inventory::ProjectView::Disk(project_root); + if let Some(refusal) = git_branch_lock_refusal(&view).await { return Err(Box::new(refused(refusal.0, refusal.1))); } + // pnpm 1 / 2 read only `shrinkwrap.yaml`, whose grammar the rekey has no + // fixture for: there `pnpm-lock.yaml` is stray, and wiring it would + // patch nothing pnpm installs (N11, the choice hosted and discovery make). + let shrinkwrap = view.exists_no_follow(PNPM_SHRINKWRAP_LEGACY).await; + let pinned = crate::utils::pnpm_workspace::pinned_pnpm_major_in(&view).await; + if governing_pnpm_lock(true, shrinkwrap, pinned) == Some(PNPM_SHRINKWRAP_LEGACY) { + let why = "the project pins pnpm 1 / 2, which installs from shrinkwrap.yaml and ignores \ + pnpm-lock.yaml; vendored mode does not write that grammar — upgrade pnpm \ + and re-lock, or use `--mode hosted`"; + return Err(Box::new(refused( + "vendor_lockfile_version_unsupported", + why, + ))); + } let (lock_text, lock_eol) = read_pnpm_text(project_root, PNPM_LOCK) .await .map_err(|e| read_refusal(PNPM_LOCK, e, "run `pnpm install` first"))?; @@ -687,49 +698,34 @@ async fn read_project( }) } -/// A pnpm file mixing CRLF and LF (see [`read_pnpm_text`]). -#[derive(Debug)] -struct MixedEndings(&'static str); - -impl std::fmt::Display for MixedEndings { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - write!( - f, - "{} mixes CRLF and LF line endings, which this rewriter cannot write back \ - byte-faithfully — normalize the file to one line ending and retry", - self.0 - ) - } -} - -impl std::error::Error for MixedEndings {} - /// Read `file` under `root` as every vendored pnpm reader does (vendoring, /// its pre-flight and the revert): its text with LF line endings, and the /// endings it is written back with. A CRLF file (a Windows autocrlf /// checkout) is edited like its LF twin and written back CRLF, as hosted /// mode's byte splices keep it (divergences N15). A file that mixes endings -/// is an error ([`is_mixed`]): no one ending writes it back byte-faithfully, -/// so vendoring refuses it and the revert fails before any write. -async fn read_pnpm_text(root: &Path, file: &'static str) -> std::io::Result<(String, LineEndings)> { +/// is an `Unsupported` error, which no read of a regular file returns: no +/// one ending writes it back byte-faithfully, so vendoring refuses it and +/// the revert fails before any write. +async fn read_pnpm_text(root: &Path, file: &str) -> std::io::Result<(String, LineEndings)> { let text = read_regular_to_string(&root.join(file)).await?; match LineEndings::of(&text) { - LineEndings::Mixed => Err(std::io::Error::other(MixedEndings(file))), + LineEndings::Mixed => Err(std::io::Error::new( + std::io::ErrorKind::Unsupported, + format!( + "{file} mixes CRLF and LF line endings, which this rewriter cannot write back \ + byte-faithfully — normalize the file to one line ending and retry" + ), + )), LineEndings::Crlf => Ok((text.replace("\r\n", "\n"), LineEndings::Crlf)), eol => Ok((text, eol)), } } -/// Whether a [`read_pnpm_text`] error is a mixed-ending file. -fn is_mixed(e: &std::io::Error) -> bool { - e.get_ref().is_some_and(|inner| inner.is::()) -} - /// The vendoring refusal for a [`read_pnpm_text`] error other than a /// missing file: `vendor_lockfile_crlf_unsupported` for mixed endings, else /// `vendor_lockfile_missing` with `remedy`. fn read_refusal(file: &str, e: std::io::Error, remedy: &str) -> Box { - Box::new(match is_mixed(&e) { + Box::new(match e.kind() == std::io::ErrorKind::Unsupported { true => refused("vendor_lockfile_crlf_unsupported", e.to_string()), false => refused( "vendor_lockfile_missing", @@ -1018,7 +1014,7 @@ pub(super) async fn revert_pnpm_dialect( .flatten() .filter_map(|r| r.as_ref().err()) { - if is_mixed(e) { + if e.kind() == std::io::ErrorKind::Unsupported { return RevertOutcome::failed(format!("{e}; nothing was reverted")); } } @@ -6043,6 +6039,38 @@ snapshots: format!("---\n{env}\n---\n{project}") } + /// N11: a project that pins pnpm 1 / 2 installs from `shrinkwrap.yaml`, + /// so its `pnpm-lock.yaml` is stray. Vendoring refuses it, with nothing + /// written, instead of wiring a lock pnpm never reads, which is the + /// choice hosted mode and discovery make (`plan::governing_pnpm_lock`). + /// With no pin, the `pnpm-lock.yaml` governs and is vendored as before. + #[tokio::test] + async fn a_pnpm_2_pin_beside_a_shrinkwrap_refuses() { + let shrinkwrap = "shrinkwrapVersion: 3\nshrinkwrapMinorVersion: 9\n"; + let pinned = P1_BEFORE_PKG.replacen('{', "{\n \"packageManager\": \"pnpm@2.25.7\",", 1); + let fx = fixture_with(&pinned, P1_BEFORE_LOCK).await; + tokio::fs::write(fx.root().join("shrinkwrap.yaml"), shrinkwrap) + .await + .unwrap(); + let detail = expect_refused( + fx.vendor(false).await, + "vendor_lockfile_version_unsupported", + ); + assert!( + detail.contains("pnpm 1 / 2") && detail.contains("shrinkwrap.yaml"), + "{detail}" + ); + assert_eq!(fx.read(PNPM_LOCK).await, P1_BEFORE_LOCK, "lock untouched"); + assert_eq!(fx.read(PACKAGE_JSON).await, pinned, "pkg untouched"); + assert!(!fx.root().join(".socket/vendor").exists()); + + let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; + tokio::fs::write(fx.root().join("shrinkwrap.yaml"), shrinkwrap) + .await + .unwrap(); + assert!(matches!(fx.vendor(false).await, VendorOutcome::Done { .. })); + } + /// N15 on pnpm 11's two-document lock: vendored, then checked out with /// autocrlf (`---\r` separators), the lock reverts to its CRLF bytes. /// Review finding: the revert's document check read the raw CRLF text, diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md index 83bed89fa..c9e5d9ca4 100644 --- a/docs/design/lock-model-divergences.md +++ b/docs/design/lock-model-divergences.md @@ -89,7 +89,7 @@ follow from the install-tree decision. | N8 | npm `file:` directory link identity (#1481) | Hosted and discovery give a versionless link its source directory's `name@version`. Vendored matches the entry's own version, wiring the registry copies with no warning. | Hosted | silent vendored success | | N9 | yarn classic blocks reached only through an alias | Hosted skips them. Vendored rewrites them. Discovery counts an alias block as an install of the real package. | Back-port: hosted rewrites alias blocks (vendored supports what hosted lacks) | | | N10 | Locks of several npm-family package managers in one project | Hosted rewrites every family; discovery reads every family with the contest. The inventory and vendored pick one family (vlt > bun > pnpm > yarn > npm). | Union (separate trees) | | -| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | **Resolved (owner: shadowed) in PR 2b**: hosted never rewrites it and leaves it out of the gate; discovery keeps sweeping it; vendored never reads it. A pnpm 1 / 2 pin flips the pair: `shrinkwrap.yaml` governs, `pnpm-lock.yaml` is shadowed. A lone `shrinkwrap.yaml` keeps today's behavior. | | +| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | **Resolved (owner: shadowed) in PR 2b**: hosted never rewrites it and leaves it out of the gate; discovery keeps sweeping it; vendored never reads it. A pnpm 1 / 2 pin flips the pair: `shrinkwrap.yaml` governs, `pnpm-lock.yaml` is shadowed, and vendored mode refuses the project (`vendor_lockfile_version_unsupported`). A lone `shrinkwrap.yaml` keeps today's behavior. | | | N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. **Resolved in PR 2a** (the inventory reads them whenever `rush.json` exists). | target-specific (vendored) | | N13 | Plug'n'Play | Vendored refuses. The hosted berry preflight has no PnP gate. The inventory returns an error, so `scan` sees nothing in a project hosted pinned. | The inventory reads PnP locks. Vendored keeps its refusal. | target-specific (vendored) | | N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery. **Resolved in PR 2a**: under the setting (not under Rush), the inventory keeps a Socket-hosted pnpm entry but drops its integrity, as discovery withholds the lock pin. | target-specific | diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json index f5ebc49a3..ecd7d7a83 100644 --- a/scripts/loc-budget.json +++ b/scripts/loc-budget.json @@ -30,7 +30,7 @@ "crates/socket-patch-core/src/vendor/npm_origin.rs", "crates/socket-patch-core/src/vendor/test_support" ], - "max": 88115 + "max": 88113 } } } From 0d683937b3f725518af2bf17f263ac9cff06cf2f Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Thu, 5 Nov 2026 23:41:25 -0600 Subject: [PATCH 18/36] Fix the loc-budget counter's cfg(test) and module bugs Review of #2001 found the counter miscounting in several shapes, each now pinned by a test in scripts/tests/test_loc_budget.py: - a cfg(test) struct field, enum variant, parameter or match arm drove the scan below the item's depth and swallowed the code after it; the item now ends at a top-level `,` or before the bracket closing its container - `#[cfg(test)] use a::{..};` left its trailing `;` line counted - cfg predicates are parsed: `all(unix, test)` and nested `all` count, `not(test)` doesn't, and a file opening with `#![cfg(test)]` is test-only - `#[path]` is found among any attributes around the item and resolves against the declaring file's directory, as rustc does - `mod x;` inside a test-only inline module (vex::discover's `testing { mod golden; }`) marks that file test-only - `git ls-files -z` keeps paths with spaces whole, and only the group files plus their module declarers are read - `--files` with an unknown group is a usage error, not a KeyError golden.rs (435 lines) and two `};` lines leave the counts, so the budget drops to 30462 / 88362. Co-Authored-By: Claude Opus 5.5 --- scripts/loc-budget.json | 4 +- scripts/loc-budget.py | 321 ++++++++++++++++++++++++++----- scripts/tests/test_loc_budget.py | 170 +++++++++++++++- 3 files changed, 442 insertions(+), 53 deletions(-) diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json index 28d387299..62f8fe636 100644 --- a/scripts/loc-budget.json +++ b/scripts/loc-budget.json @@ -16,7 +16,7 @@ "crates/socket-patch-core/src/utils/pipenv.rs", "crates/socket-patch-core/src/crawlers/deno_crawler.rs" ], - "max": 30898 + "max": 30462 }, "lock-writers": { "paths": [ @@ -29,7 +29,7 @@ "crates/socket-patch-core/src/vendor/npm_origin.rs", "crates/socket-patch-core/src/vendor/test_support" ], - "max": 88363 + "max": 88362 } } } diff --git a/scripts/loc-budget.py b/scripts/loc-budget.py index 7ef6795a4..b4e024218 100755 --- a/scripts/loc-budget.py +++ b/scripts/loc-budget.py @@ -4,9 +4,13 @@ A line counts when it holds code outside a comment. These never count: * blank and comment-only lines (doc comments included); -* items gated by ``#[cfg(test)]`` or ``#[cfg(all(test, ...))]`` (inline - ``mod tests { ... }`` blocks, test-only helpers and ``use`` lines); -* files declared as ``#[cfg(test)] mod name;`` and every module below them; +* items gated by a ``cfg`` that implies ``test`` -- ``#[cfg(test)]``, + ``#[cfg(all(unix, test))]`` -- with their other attributes: inline + ``mod tests { ... }`` blocks, test-only helpers, ``use`` lines, struct + fields, enum variants, parameters and match arms; +* files declared as ``#[cfg(test)] mod name;`` (also with ``#[path]``, or + inside a test-only inline module), files that open with + ``#![cfg(test)]``, and every module below them; * files under a ``tests/`` or ``benches/`` directory. Budgets live in ``scripts/loc-budget.json``. Each group names path prefixes @@ -45,12 +49,17 @@ REPO = Path(__file__).resolve().parent.parent BUDGET_FILE = REPO / "scripts" / "loc-budget.json" -CFG_TEST = re.compile(r"#\[cfg\(\s*(?:test|all\(\s*test\b[^\]]*)\)\]") +# `#[cfg(` or `#![cfg(`; implies_test parses the predicate. +CFG_ATTR = re.compile(r"#\s*(!)?\s*\[\s*cfg\s*\(") +CFG_TOKEN = re.compile(r'"(?:\\.|[^"\\])*"|\w+|\S') RAW_STRING = re.compile(r'b?r(#*)"') CHAR_LITERAL = re.compile(r"'(?:\\(?:x[0-9a-fA-F]{2}|u\{[0-9a-fA-F]+\}|.)|[^\\'])'", re.S) -MOD_DECL = re.compile( - r'(?:#\[path\s*=\s*"([^"]+)"\]\s*)?(?:pub(?:\([^)]*\))?\s+)?mod\s+(\w+)\s*;\s*$' +IDENT = re.compile(r"(?:r#)?[A-Za-z_]\w*") +ITEM_KEYWORDS = frozenset( + "async const crate enum extern fn impl let macro_rules mod static struct trait type union unsafe use".split() ) +MOD_ITEM = re.compile(r"(? bool: class FileCount: code: int test: int - test_mods: list[tuple[str, str | None]] = field(default_factory=list) + # (name, #[path] value, enclosing inline modules) for each file module a + # test-only item declares. + test_mods: list[tuple[str, str | None, tuple[str, ...]]] = field(default_factory=list) + # The file opens with an inner `#![cfg(test)]`. + test_file: bool = False + + +def implies_test(pred: str) -> bool: + """Whether a ``cfg`` predicate can only hold when ``test`` is set: + ``test`` itself, an ``all(...)`` with such an argument, or an ``any(...)`` + whose every argument is one. ``not(...)`` never is.""" + toks = CFG_TOKEN.findall(pred) + pos = 0 + + def parse() -> bool: + nonlocal pos + if pos >= len(toks): + return False + name = toks[pos] + pos += 1 + if pos < len(toks) and toks[pos] == "=": + pos += 2 # key = "value" + return False + if pos < len(toks) and toks[pos] == "(": + pos += 1 + args = [] + while pos < len(toks) and toks[pos] != ")": + if toks[pos] == ",": + pos += 1 + continue + args.append(parse()) + pos += 1 + if name == "all": + return any(args) + if name == "any": + return bool(args) and all(args) + return False + return name == "test" + + return parse() and pos == len(toks) + + +def _next_code(src: str, kind: bytearray, j: int) -> int: + n = len(src) + while j < n and (kind[j] == COMMENT or src[j].isspace()): + j += 1 + return j + + +def _match_close(src: str, kind: bytearray, j: int) -> int | None: + """Index of the bracket closing the one at ``j``.""" + depth = 0 + for k in range(j, len(src)): + if kind[k] != CODE: + continue + if src[k] in "{([": + depth += 1 + elif src[k] in "})]": + depth -= 1 + if depth == 0: + return k + return None + + +def _attrs_before(src: str, kind: bytearray, pos: int) -> tuple[int, list[str]]: + """The outer attributes written directly before ``pos``: (start, texts).""" + attrs: list[str] = [] + while True: + q = pos - 1 + while q >= 0 and (kind[q] == COMMENT or src[q].isspace()): + q -= 1 + if q < 0 or src[q] != "]" or kind[q] != CODE: + break + depth, r = 0, q + while r >= 0: + if kind[r] == CODE and src[r] == "]": + depth += 1 + elif kind[r] == CODE and src[r] == "[": + depth -= 1 + if depth == 0: + break + r -= 1 + h = r - 1 + while h >= 0 and src[h].isspace(): + h -= 1 + if r < 0 or h < 0 or src[h] != "#": + break + attrs.append(src[h : q + 1]) + pos = h + return pos, attrs + + +def _starts_item(src: str, kind: bytearray, j: int) -> bool: + """Whether the code at ``j`` is an item or statement, rather than a + struct field, enum variant, parameter or match arm.""" + n = len(src) + while True: + j = _next_code(src, kind, j) + if j < n and src[j] == "#": # another attribute + k = _next_code(src, kind, j + 1) + close = _match_close(src, kind, k) if k < n and src[k] == "[" else None + if close is None: + return True + j = close + 1 + continue + break + if j >= n or src[j] == "{": + return True + m = IDENT.match(src, j) + if not m: + return False + word, after = m.group(0), _next_code(src, kind, m.end()) + if word == "pub": + if after < n and src[after] == "(": + after = _next_code(src, kind, (_match_close(src, kind, after) or after) + 1) + m = IDENT.match(src, after) + if not m: + return True + word, after = m.group(0), _next_code(src, kind, m.end()) + # `pub name: Type` is a field. + return word in ITEM_KEYWORDS or not (src.startswith(":", after) and not src.startswith("::", after)) + return word in ITEM_KEYWORDS or (after < n and src[after] == "!") # or a macro call + + +def _item_end(src: str, kind: bytearray, pos: int) -> int: + """End of the item that starts at ``pos``. + + An item ends at its first top-level ``;``, at the brace closing its first + ``{`` (and a ``;`` right after it), or just before a bracket that closes + the enclosing container. A field, variant, parameter or match arm also + ends at its first top-level ``,``. + """ + n = len(src) + is_field = not _starts_item(src, kind, pos) + depth, angle, in_expr = 0, 0, False + for j in range(pos, n): + if kind[j] != CODE: + continue + ch = src[j] + if ch in "{([": + depth += 1 + elif ch in "})]": + if depth == 0: + return j + depth -= 1 + if depth == 0 and ch == "}": + nxt = _next_code(src, kind, j + 1) + if nxt < n and (src[nxt] == ";" or (is_field and src[nxt] == ",")): + return nxt + 1 + # `Variant { .. } => arm` and `S { .. }.method()` go on. + if not is_field or nxt >= n or src[nxt] not in "=.?": + return j + 1 + elif depth == 0: + if ch == ";": + return j + 1 + if is_field: + prev = src[j - 1] + if ch == ">" and prev == "=": + in_expr, angle = True, 0 # past a match arm's `=>` + elif in_expr: + pass + elif ch == "<": + angle += 1 + elif ch == ">" and prev != "-": + angle = max(0, angle - 1) + if ch == "," and angle == 0: + return j + 1 + return n + + +def _file_mods(src: str, kind: bytearray, lo: int, hi: int) -> list[tuple[str, str | None, tuple[str, ...]]]: + """The ``mod name;`` declarations in ``src[lo:hi]``, each with its + ``#[path]`` and the inline modules around it.""" + out = [] + stack: list[tuple[str, int]] = [] + for m in MOD_ITEM.finditer(src, lo, hi): + at = m.start() + if kind[at] != CODE: + continue + while stack and stack[-1][1] < at: + stack.pop() + name = m.group(1) + if m.group(2) == "{": + close = _match_close(src, kind, m.end() - 1) + stack.append((name, hi if close is None else close)) + continue + _, attrs = _attrs_before(src, kind, at) + paths = [pm.group(1) for pm in map(PATH_ATTR.search, attrs) if pm] + out.append((name, paths[0] if paths else None, tuple(s for s, _ in stack))) + return out def count_source(src: str) -> FileCount: @@ -127,32 +325,29 @@ def count_source(src: str) -> FileCount: kind = classify(src) n = len(src) excluded = bytearray(n) - test_mods: list[tuple[str, str | None]] = [] - for m in CFG_TEST.finditer(src): + test_mods: list[tuple[str, str | None, tuple[str, ...]]] = [] + test_file = False + for m in CFG_ATTR.finditer(src): start = m.start() if kind[start] != CODE or excluded[start]: continue - # The gated item ends at its first top-level `;` or at the brace - # closing its first `{`. - j, depth, end = m.end(), 0, n - while j < n: - if kind[j] == CODE: - ch = src[j] - if ch in "{([": - depth += 1 - elif ch in "})]": - depth -= 1 - if depth == 0 and ch == "}": - end = j + 1 - break - elif ch == ";" and depth == 0: - end = j + 1 - break - j += 1 - item = src[m.end() : end] - decl = MOD_DECL.search(item) - if decl and "{" not in item: - test_mods.append((decl.group(2), decl.group(1))) + close = _match_close(src, kind, m.end() - 1) + if close is None: + continue + attr_end = _next_code(src, kind, close + 1) + if attr_end >= n or src[attr_end] != "]" or not implies_test(src[m.end() : close]): + continue + if m.group(1): + # `#![cfg(test)]` outside any block gates the whole file. + opened = sum(1 if c == "{" else -1 for c, k in zip(src[:start], kind) if k == CODE and c in "{}") + if opened == 0: + test_file = True + excluded = bytearray(b"\x01" * n) + break + continue + end = _item_end(src, kind, attr_end + 1) + start, _ = _attrs_before(src, kind, start) + test_mods.extend(_file_mods(src, kind, start, end)) excluded[start:end] = b"\x01" * (end - start) code_lines: set[int] = set() @@ -163,41 +358,66 @@ def count_source(src: str) -> FileCount: line += 1 elif not ch.isspace() and kind[k] != COMMENT: (test_lines if excluded[k] else code_lines).add(line) - return FileCount(len(code_lines), len(test_lines - code_lines), test_mods) + return FileCount(len(code_lines), len(test_lines - code_lines), test_mods, test_file) -def resolve_module(declaring: str, name: str, path_attr: str | None) -> str | None: - """The file a ``mod name;`` in ``declaring`` loads, if it exists.""" +def is_file(rel: str) -> bool: + return (REPO / rel).is_file() + + +def resolve_module(declaring: str, name: str, path_attr: str | None, inline: tuple[str, ...] = ()) -> str | None: + """The file a ``mod name;`` in ``declaring`` loads, if it exists. + + ``inline`` names the inline ``mod x { ... }`` blocks around the + declaration. Like rustc, an out-of-line ``#[path]`` resolves against the + declaring file's own directory, and one inside inline modules against + their directory. + """ d, base = os.path.split(declaring) owns_dir = base in ("mod.rs", "lib.rs", "main.rs") - sub = d if owns_dir else os.path.join(d, base[:-3]) + sub = os.path.join(d if owns_dir else os.path.join(d, base[:-3]), *inline) if path_attr: - candidates = [os.path.join(sub, path_attr), os.path.join(d, path_attr)] + candidates = [os.path.join(sub if inline else d, path_attr)] else: candidates = [os.path.join(sub, name + ".rs"), os.path.join(sub, name, "mod.rs")] for c in candidates: - if (REPO / c).is_file(): + if is_file(c): return os.path.normpath(c) return None def rust_files() -> list[str]: - out = subprocess.check_output(["git", "ls-files", "*.rs"], cwd=REPO, text=True) - return [ - f - for f in out.split() - if not re.search(r"(^|/)(tests|benches)/", f) - ] + out = subprocess.check_output(["git", "ls-files", "-z", "*.rs"], cwd=REPO, text=True) + return [f for f in out.split("\0") if f and not re.search(r"(^|/)(tests|benches)/", f)] + + +def in_scope(files: list[str], groups: dict) -> list[str]: + """The group files and every file that can declare one of them as a + module: the files beside them and each ancestor directory's module file.""" + member = {f for f in files if any(in_group(f, g) for g in groups.values())} + dirs = {os.path.dirname(f) for f in member} + declarers: set[str] = set() + for d in dirs: + while d: + declarers.update(d + ext for ext in (".rs", "/mod.rs", "/lib.rs", "/main.rs")) + d = os.path.dirname(d) + return [f for f in files if f in member or f in declarers or os.path.dirname(f) in dirs] + +def count_repo(groups: dict | None = None) -> dict[str, int]: + """Implementation lines per tracked Rust file (test-only files count 0). -def count_repo() -> dict[str, int]: - """Implementation lines per tracked Rust file (test-only files count 0).""" + With ``groups``, reads only the files those groups and their module + declarations need. + """ files = rust_files() + if groups is not None: + files = in_scope(files, groups) counts = {f: count_source((REPO / f).read_text(encoding="utf-8", errors="replace")) for f in files} - test_files: set[str] = set() + test_files = {f for f, fc in counts.items() if fc.test_file} for f, fc in counts.items(): - for name, path_attr in fc.test_mods: - target = resolve_module(f, name, path_attr) + for name, path_attr, inline in fc.test_mods: + target = resolve_module(f, name, path_attr, inline) if target: test_files.add(target) # A module below a test-only module is test-only too. @@ -233,7 +453,9 @@ def main(argv: list[str] | None = None) -> int: budget = load_budget() groups = budget["groups"] - per_file = count_repo() + if args.files and args.files not in groups: + ap.error(f"unknown group {args.files!r}; groups: {', '.join(groups)}") + per_file = count_repo(groups) if args.files: group = groups[args.files] @@ -253,7 +475,8 @@ def main(argv: list[str] | None = None) -> int: over.append((name, total, cap)) if args.lower and total < cap: g["max"] = total - print(f"{'all implementation':24s} {sum(per_file.values()):8d}") + union = sum(c for f, c in per_file.items() if any(in_group(f, g) for g in groups.values())) + print(f"{'all groups':24s} {union:8d}") if args.lower: BUDGET_FILE.write_text(json.dumps(budget, indent=2) + "\n") diff --git a/scripts/tests/test_loc_budget.py b/scripts/tests/test_loc_budget.py index f3156b1aa..d5a676d63 100644 --- a/scripts/tests/test_loc_budget.py +++ b/scripts/tests/test_loc_budget.py @@ -80,9 +80,140 @@ def test_test_module_declarations_are_reported(self): "mod shipped;\n" ) fc = count(src) - self.assertEqual(fc.test_mods, [("tests", None), ("x_tests", "x_tests.rs")]) + self.assertEqual(fc.test_mods, [("tests", None, ()), ("x_tests", "x_tests.rs", ())]) self.assertEqual(fc.code, 1) + # Review findings on #2001. + + def test_cfg_test_struct_field_does_not_swallow_following_code(self): + src = ( + "struct S {\n" + " #[cfg(test)]\n" + " probe: HashMap,\n" + " real: u32,\n" + "}\n" + "fn shipped() {}\n" + ) + fc = count(src) + self.assertEqual(fc.code, 4) + self.assertEqual(fc.test, 2) + + def test_cfg_test_pub_field_last_without_comma(self): + src = "struct S {\n real: u32,\n #[cfg(test)]\n pub probe: u32\n}\nfn shipped() {}\n" + self.assertEqual(count(src).code, 4) + + def test_cfg_test_enum_variant_and_match_arm(self): + src = ( + "enum E {\n" + " A,\n" + " #[cfg(test)]\n" + " B { x: u8 },\n" + " C,\n" + "}\n" + "fn f(e: E) -> u8 {\n" + " match e {\n" + " #[cfg(test)]\n" + " E::B { .. } => 1,\n" + " _ => 0,\n" + " }\n" + "}\n" + ) + fc = count(src) + self.assertEqual(fc.code, 9) + self.assertEqual(fc.test, 4) + + def test_cfg_test_fn_param(self): + src = "fn f(\n a: u8,\n #[cfg(test)] b: u8,\n) -> u8 {\n a\n}\nfn g() {}\n" + self.assertEqual(count(src).code, 6) + + def test_cfg_test_braced_use_consumes_its_semicolon(self): + src = "#[cfg(test)]\nuse a::{\n b,\n c,\n};\nfn shipped() {}\n" + fc = count(src) + self.assertEqual(fc.code, 1) + self.assertEqual(fc.test, 5) + + def test_cfg_all_with_test_not_first_and_not_test(self): + src = ( + "#[cfg(all(unix, test))]\nfn a() {}\n" + "#[cfg(all(unix, all(windows, test)))]\nfn b() {}\n" + "#[cfg(all(unix, not(test)))]\nfn shipped() {}\n" + "#[cfg(any(test, test))]\nfn c() {}\n" + ) + self.assertEqual(count(src).code, 2) + + def test_file_level_cfg_test(self): + src = "//! helpers\n#![cfg(test)]\n\nfn a() {}\nmod nested;\n" + fc = count(src) + self.assertTrue(fc.test_file) + self.assertEqual(fc.code, 0) + src = "#![cfg(not(test))]\nfn a() {}\n" + self.assertFalse(count(src).test_file) + self.assertEqual(count(src).code, 2) + + def test_path_attr_after_other_attrs(self): + src = ( + "#[cfg(test)]\n" + '#[path = "x_tests.rs"]\n' + "#[allow(dead_code)]\n" + "mod x_tests;\n" + ) + self.assertEqual(count(src).test_mods, [("x_tests", "x_tests.rs", ())]) + + def test_inline_test_mod_records_nested_file_modules(self): + src = ( + "#[cfg(test)]\n" + "mod support {\n" + " mod fixtures;\n" + " pub mod inner {\n" + ' #[path = "h.rs"]\n' + " mod helpers;\n" + " }\n" + "}\n" + ) + self.assertEqual( + count(src).test_mods, + [("fixtures", None, ("support",)), ("helpers", "h.rs", ("support", "inner"))], + ) + + +class ResolveModuleTest(unittest.TestCase): + def setUp(self): + self.saved = loc_budget.is_file + self.files = set() + loc_budget.is_file = lambda p: p in self.files + + def tearDown(self): + loc_budget.is_file = self.saved + + def test_path_attr_is_relative_to_the_declaring_files_dir(self): + # Rust resolves an out-of-line #[path] against the declaring file's + # directory, even for a non-mod.rs file. + self.files = {"src/a/x_tests.rs", "src/a/x/x_tests.rs"} + self.assertEqual( + loc_budget.resolve_module("src/a/x.rs", "t", "x_tests.rs", ()), "src/a/x_tests.rs" + ) + + def test_inline_module_path(self): + self.files = {"src/a/x/support/fixtures.rs", "src/a/support/fixtures.rs"} + self.assertEqual( + loc_budget.resolve_module("src/a/x.rs", "fixtures", None, ("support",)), + "src/a/x/support/fixtures.rs", + ) + self.assertEqual( + loc_budget.resolve_module("src/a/mod.rs", "fixtures", None, ("support",)), + "src/a/support/fixtures.rs", + ) + + +class RustFilesTest(unittest.TestCase): + def test_paths_with_spaces_survive(self): + saved = loc_budget.subprocess.check_output + loc_budget.subprocess.check_output = lambda *a, **k: "a b/c.rs\0x/tests/t.rs\0d.rs\0" + try: + self.assertEqual(loc_budget.rust_files(), ["a b/c.rs", "d.rs"]) + finally: + loc_budget.subprocess.check_output = saved + class ModeTest(unittest.TestCase): """--check fails over budget, --warn only annotates.""" @@ -94,7 +225,7 @@ def setUp(self): "g": {"paths": ["a/"], "exclude": ["a/skip/"], "max": 10}, } } - loc_budget.count_repo = lambda: {"a/x.rs": 8, "a/y.rs": 5, "a/skip/z.rs": 100, "b/w.rs": 1} + loc_budget.count_repo = lambda groups=None: {"a/x.rs": 8, "a/y.rs": 5, "a/skip/z.rs": 100, "b/w.rs": 1} def tearDown(self): loc_budget.load_budget, loc_budget.count_repo = self.saved @@ -123,6 +254,41 @@ def test_report_mode_never_fails(self): code, out = self.run_main() self.assertEqual(code, 0) self.assertNotIn("::warning", out) + self.assertIn("all groups 13", out) + + def test_files_lists_one_group(self): + code, out = self.run_main("--files", "g") + self.assertEqual(code, 0) + self.assertIn(" 8 a/x.rs", out) + self.assertIn(" 13 total", out) + + def test_files_with_an_unknown_group_is_a_usage_error(self): + import contextlib + import io + + with contextlib.redirect_stderr(io.StringIO()) as err, self.assertRaises(SystemExit) as cm: + loc_budget.main(["--files", "nope"]) + self.assertEqual(cm.exception.code, 2) + self.assertIn("unknown group 'nope'", err.getvalue()) + + +class InScopeTest(unittest.TestCase): + def test_reads_group_files_siblings_and_declarers_only(self): + files = [ + "c/src/lib.rs", + "c/src/a/mod.rs", + "c/src/a/g/mod.rs", + "c/src/a/g/x.rs", + "c/src/a/g/deep/y.rs", + "c/src/a/sib.rs", + "c/src/other/z.rs", + "d/src/lib.rs", + ] + groups = {"g": {"paths": ["c/src/a/g/"], "max": 1}} + self.assertEqual( + loc_budget.in_scope(files, groups), + ["c/src/lib.rs", "c/src/a/mod.rs", "c/src/a/g/mod.rs", "c/src/a/g/x.rs", "c/src/a/g/deep/y.rs"], + ) class BudgetFileTest(unittest.TestCase): From 75d575e6c405f29d4510ae28f07544d3e965155c Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Thu, 5 Nov 2026 23:44:23 -0600 Subject: [PATCH 19/36] Record the 2026-11-05 owner decisions; link the fix PRs Review of #2001: the Decisions table said every read is bounded to the repository root, while G1, J13 and P4 still said to read an ancestor go.work, parent NuGet configs and every include. The owner narrowed the rule on 2026-11-05, and the table now records it with the rest of that day's calls: - the bound applies to workspace runs only; --global / -g / --global-prefix runs are unbounded, and without a git checkout upward lookups behave as before - a marker file above the repository root (below home) makes hosted and vendored refuse with a message naming it - out-of-repository -r includes stay bounded in workspace runs - N11: shrinkwrap.yaml beside pnpm-lock.yaml is shadowed, unless the project pins pnpm 1 / 2, where it governs and vendored refuses - discovery withholds pnpm root and member-lock attestations whenever a pnpm-lock..yaml exists - user patches: the override is --allow-patch-conflicts, and vex never attests under a user patch G1, J9, J13, P4, P9, X12, N1, N2, N4, N5, N11, N12 and N14 name the open PRs that fix them, and the fixes-on-main table gains a PR column. The problem table now counts four checksums.sha256 readers, as J9 does. The stack table splits PR 2 into 2a / 2b with their numbers, and the size budget section says why CI only warns and gives the corrected counts. Co-Authored-By: Claude Opus 5.5 --- docs/design/lock-model-divergences.md | 55 +++++++++++++++------------ docs/design/lock-model-unification.md | 49 +++++++++++++++--------- 2 files changed, 60 insertions(+), 44 deletions(-) diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md index 1fb560ff6..2a2003e2b 100644 --- a/docs/design/lock-model-divergences.md +++ b/docs/design/lock-model-divergences.md @@ -17,12 +17,14 @@ Flags: * **target-specific**: the difference comes from where the entry points (a Socket URL or a committed path) and stays. -## Calls that go against hosted (owner decided 2026-11-04) +## Calls that go against hosted (owner decided 2026-11-04 and 2026-11-05) Owner: the false Maven attestations get fixed, the modes are ground truth over the readers, and the vendored mode's profile, reactor and scala-cli support is back-ported so hosted gains it as the planners merge. P3 and P4 -follow from the install-tree decision. +follow from the install-tree decision. N11 was decided on 2026-11-05. + +"(draft)" after a PR number means that PR is open and not merged yet. | # | Question | Hosted today | Decision | |---|---|---|---| @@ -30,8 +32,8 @@ follow from the install-tree decision. | J3 | Which Maven poms count? | Refuses a reactor root (~8572) | The reactor reader becomes the shared core, so hosted gains reactor support. | | J12 | scala-cli | Guidance only (`patch/redirect/scala_guidance.rs`) | Back-port: hosted wires scala-cli through the merged planner (a same-GAV Socket repository instead of a suffixed version). | | P3 | Is a shadowed `pdm.lock` rewritten? | Only when it governs (`pdm_drives`, ~564) | Rewrite it like every other tree, per the install-tree decision. | -| P4 | Are `-r` includes of `requirements.txt` rewritten? | Root file only (`patch/redirect/requirements.rs` ~208) | Follow includes, per the install-tree decision. | -| N11 | Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` rewritten? | Yes; its all-or-nothing gate covers both files | **Pending.** Proposed: treat it as shadowed (swept for stale identities, never written). pnpm never reads it, and an unparseable debris key in it can refuse the dependency in every lock. | +| P4 | Are `-r` includes of `requirements.txt` rewritten? | Root file only (`patch/redirect/requirements.rs` ~208) | Follow includes, per the install-tree decision. An include outside the repository root stays bounded in workspace runs: never read or rewritten (owner, 2026-11-05; #2025 (draft)). | +| N11 | Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` rewritten? | Yes; its all-or-nothing gate covers both files | **Owner decision (2026-11-05): shadowed.** Swept for stale identities, never written, and out of the all-or-nothing gate. pnpm never reads it, and an unparseable debris key in it can refuse the dependency in every lock. When the project's pnpm evidence pins pnpm 1 / 2, `shrinkwrap.yaml` governs instead and vendored mode refuses. Fixed in #2029 (draft). | ## JVM @@ -43,10 +45,10 @@ follow from the install-tree decision. | J6 | Is sbt resolution evidence required before attesting? | Hosted: no integrity without verified evidence. Vendored: `wired_checked` only checks the owned-file row; discovery's `vendored_check` diagnostics are never consumed. | Hosted. One evidence route (`ResolutionDoc` + `gate::check_new`). | false attestation | | J7 | Another version of a mapped NuGet id is locked | Both writers refuse. Discovery emits a ref per locked version, keyed by raw text, not `normalize_nuget_version`. | Hosted: a diagnostic, no ref | false attestation | | J8 | What counts as "wired" for vendored Maven liveness and GC? | Vendored: substring match on masked text, so a suffix in ``, a profile or `` counts. Discovery and hosted use the scoped `PomScope`. | Scoped reader | | -| J9 | `checksums.sha256` and the `TRUSTED_CHECKSUMS_ON` switch | Four readers with different splitting and switch tests. Hosted's merge silently drops lines it can't split on two spaces. | One reader with the hosted grammar. The merge keeps lines it can't parse. | | +| J9 | `checksums.sha256` and the `TRUSTED_CHECKSUMS_ON` switch | Four readers with different splitting and switch tests. Hosted's merge silently drops lines it can't split on two spaces. | One reader with the hosted grammar. The merge keeps lines it can't parse. Fixed in #2016 (draft). | | | J10 | Gradle Socket index rows | Two parsers. Hosted validates the uuid, URL and hashes; vendored accepts any token and uses a raw `split('\t')`. Vendored checks only root settings; hosted checks every build. | One row grammar, hosted's validation | | | J11 | `verification-metadata.xml` | Two editors, no reader | One editor; a reader for the inventory | | -| J13 | NuGet config scope | Writers read inherited parent configs; discovery doesn't. Discovery refuses a disabled Socket source; the writers don't check ``. | Shared inheritance read. The writers adopt the disabled-source refusal. | | +| J13 | NuGet config scope | Writers read inherited parent configs; discovery doesn't. Discovery refuses a disabled Socket source; the writers don't check ``. | Shared inheritance read, bounded to the repository root in workspace runs; global runs, and runs without a git checkout, read parent configs as before (owner, 2026-11-05; #2025 (draft)). The writers adopt the disabled-source refusal. | | | J14 | NuGet lock enumeration | In-memory views fall back to the root lock. The unwind bypasses the snapshot view (`governed_locks_on_disk`). | One enumeration over `ProjectView` | | | J15 | JVM liveness on a read error | `entry_wired` is `unwrap_or(true)` for `scan`/`vendor`; `vex` uses the checked variant | Fail closed everywhere | false attestation | @@ -59,7 +61,7 @@ follow from the install-tree decision. | P6 | What counts as Socket wiring? | Discovery and the unwind use the host allowlist. The inventory accepts any host. Ledger liveness uses `resolved.contains(uuid)`. `poetry_lock::lock_wiring` uses `contains("/patch/pypi/")`. | The `origin` allowlist everywhere | false attestation (liveness substring) | | P7 | uv workspace roots | Hosted refuses members only. Vendored refuses roots too. | Vendored wires roots, after a test that a member's own `[tool.uv.sources]` can't override the root | | | P8 | `uv.lock` without `pyproject.toml` | Hosted edits the lock; discovery attests it. Vendored refuses. | Vendored proceeds | | -| P9 | uv 0.2 `[[distribution]]` locks | Hosted rewrites. Vendored refuses (no portable path). The unwind refuses to restore them. | Vendored's refusal is target-specific and stays. The unwind must restore what hosted writes. | target-specific | +| P9 | uv 0.2 `[[distribution]]` locks | Hosted rewrites. Vendored refuses (no portable path). The unwind refuses to restore them. | Vendored's refusal is target-specific and stays. The unwind must restore what hosted writes (#2014 (draft)). | target-specific | | P13 | PDM `lock_version` | Writers use an allowlist; discovery a denylist; the inventory checks nothing | The allowlist everywhere | | | P14 | `pipfile-spec` | Writers require 6; discovery and the inventory accept any | 6 everywhere | | @@ -67,32 +69,32 @@ follow from the install-tree decision. | # | Question | Today | Resolution | Flag | |---|---|---|---|---| -| G1 | Which `go.work` and members count? | Discovery: a root `go.work` and in-root members only, ignoring an ancestor `go.work`, `GOWORK` and `-modfile`. Hosted and vendored resolve the workspace go would use (`vendor/go_workspace.rs`). | `go_workspace::load` everywhere | false attestation | +| G1 | Which `go.work` and members count? | Discovery: a root `go.work` and in-root members only, ignoring an ancestor `go.work`, `GOWORK` and `-modfile`. Hosted and vendored resolve the workspace go would use (`vendor/go_workspace.rs`). | `go_workspace::load` everywhere (discovery: #2011 (draft)). In workspace runs it never reads above the repository root, and a `go.work` there makes the modes refuse and discovery withhold; global runs, and runs without a git checkout, find the ancestor `go.work` as before (owner, 2026-11-05; #2025 (draft)). | false attestation | | G16 | Go inventory | Root `go.sum` only, lenient parse, lists `patch.socket.dev/gopatch/` lines as packages | Discovery's model and workspace | | | C2 | Which manifests decide whether a Cargo lock entry is used? | Hosted pins root and member manifests; vendored reads members. Discovery reads only the root `Cargo.toml`. | Members everywhere | false attestation | | C11 | `cargo vendor` / local-registry source replacement | Hosted refuses; vendored and discovery ignore it | Stays | target-specific | | R10 | `gems.rb` / `gems.locked` | Hosted wires the pair bundler loads. Vendored refuses any `gems.rb`. | Vendored supports it | | | R15 | Malformed Gemfile.lock | Discovery reports it unparseable; the inventory lists what parsed | Discovery's rule everywhere | | -| X12 | `composer.lock` with a BOM | Discovery strips it. The inventory drops the file. Vendored refuses to plan, errors on revert, and reports nothing stranded. | Strip one BOM everywhere. Check that GC never trusts the empty "stranded" answer. | | +| X12 | `composer.lock` with a BOM | Discovery strips it. The inventory drops the file. Vendored refuses to plan, errors on revert, and reports nothing stranded. | Strip one BOM everywhere. Check that GC never trusts the empty "stranded" answer. The inventory part in #2017 (draft). | | ## npm family | # | Question | Today | Resolution | Flag | |---|---|---|---|---| -| N1 | pnpm `gitBranchLockfile` with a branch lock present: is the root lock the install tree? | Hosted withholds the root locks and refuses (`hosted/engine.rs` ~930). Vendored refuses. Discovery and the inventory never check the setting. | Hosted | false attestation | -| N2 | npm user patches (#711) and `bun patch` (#367) | Hosted skips that one dependency and records it as refused. Vendored refuses the whole npm v4 lock, or the one bun dependency. Discovery never reads `patchedDependencies` / `"patched"` and attests a pre-#711 hosted pin. | Hosted's per-dependency skip, in discovery too | false attestation | +| N1 | pnpm `gitBranchLockfile` with a branch lock present: is the root lock the install tree? | Hosted withholds the root locks and refuses (`hosted/engine.rs` ~930). Vendored refuses. Discovery and the inventory never check the setting. | Hosted. Discovery withholds pnpm root and member-lock attestations whenever any `pnpm-lock..yaml` exists, whatever the settings or environment say (owner, 2026-11-05; #2010 (draft)). | false attestation | +| N2 | npm user patches (#711) and `bun patch` (#367) | Hosted skips that one dependency and records it as refused. Vendored refuses the whole npm v4 lock, or the one bun dependency. Discovery never reads `patchedDependencies` / `"patched"` and attests a pre-#711 hosted pin. | Hosted's per-dependency skip, in discovery too, for every package manager's user patches (owner decision; #2010 (draft) for npm and bun, #2018 (draft) for pnpm, yarn and vlt). The override is the existing `--allow-patch-conflicts`; `vex` never attests under a user patch, even with it. | false attestation | | N3 | A workspace member's stray npm/bun/vlt lock | Discovery treats it as absent. Hosted and vendored refuse. The inventory reads it as the project's lock. | The inventory ignores it | | -| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by content, not position | | -| N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere | revert could rewrite another project's wiring | +| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by content, not position. Fixed in #2009 (draft). | | +| N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere. The vendored writers, reverters and GC claim only this project's vendor paths in #2025 (draft); the pnpm readers and planner follow in #2009 / #2029 (draft). | revert could rewrite another project's wiring | | N6 | `npm-shrinkwrap.json` and `package-lock.json` | Hosted, vendored and discovery read both, with a contest. The inventory reads the shrinkwrap only. | Both, everywhere | | | N7 | npm lockfileVersion | Hosted rewrites v1 and v4. Vendored accepts v2/v3 only. The inventory ignores v1. Discovery reads v1 and v2+. | Hosted, once vendored `file:` wiring under npm 6 and npm 12.1 v4 has fixtures | | | N8 | npm `file:` directory link identity (#1481) | Hosted and discovery give a versionless link its source directory's `name@version`. Vendored matches the entry's own version, wiring the registry copies with no warning. | Hosted | silent vendored success | | N9 | yarn classic blocks reached only through an alias | Hosted skips them. Vendored rewrites them. Discovery counts an alias block as an install of the real package. | Back-port: hosted rewrites alias blocks (vendored supports what hosted lacks) | | | N10 | Locks of several npm-family package managers in one project | Hosted rewrites every family; discovery reads every family with the contest. The inventory and vendored pick one family (vlt > bun > pnpm > yarn > npm). | Union (separate trees) | | -| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | Pending owner decision (see above) | needs owner | -| N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. | target-specific (vendored) | +| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | Shadowed (owner, 2026-11-05; see above). Fixed in #2029 (draft). | | +| N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. Fixed in #2009 (draft). | target-specific (vendored) | | N13 | Plug'n'Play | Vendored refuses. The hosted berry preflight has no PnP gate. The inventory returns an error, so `scan` sees nothing in a project hosted pinned. | The inventory reads PnP locks. Vendored keeps its refusal. | target-specific (vendored) | -| N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery | target-specific | +| N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery. Fixed in #2009 (draft). | target-specific | | N15 | CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml` | Vendored refuses; the rest handle it | Handle it everywhere | | | N16 | Lock versions | pnpm: vendored accepts only 5.4/6.0/9.0; hosted any but early shrinkwrap. vlt: vendored refuses a lock with no version; hosted rewrites and warns. | Hosted | | | N17 | Bun workspaces, yarn classic offline mirror | Refused differently by the two modes | Stays | target-specific | @@ -103,17 +105,20 @@ follow from the install-tree decision. ## Fixes for false attestations on main -These are fixed directly against `main`, ahead of the stack, one PR each. +These are fixed directly against `main`, ahead of the stack, in their own PRs. The stack rebases over them. -| # | Fix | -|---|---| -| J1, J4 | Maven discovery: a profile declaration or a `.mvn/maven.config` user property that resolves the base version leaves the ref unattested | -| J2, J8, J15 | Vendored Maven liveness: executable classifiers block, "wired" is read from scoped declarations, read errors fail closed | -| J5 | Vendored Gradle liveness reads the lock files | -| J6 | Vendored sbt attestation requires resolution evidence | -| J7 | NuGet discovery: no ref for a version the writers refuse | -| N1, N2, G1, C2, P6 | Next batch: pnpm `gitBranchLockfile`, npm/bun user patches, Go workspaces, Cargo members, the liveness substring match | +| # | Fix | PRs | +|---|---|---| +| J1, J4 | Maven discovery: a profile declaration or a `.mvn/maven.config` user property that resolves the base version leaves the ref unattested | #2005, #2020, #2026 (draft) | +| J2, J8, J15 | Vendored Maven liveness: executable classifiers block, "wired" is read from scoped declarations, read errors fail closed | #2006 (draft) | +| J5 | Vendored Gradle liveness reads the lock files | #2004 (draft) | +| J6 | Vendored sbt attestation requires resolution evidence | #2008, #2030 (draft) | +| J7 | NuGet discovery: no ref for a version the writers refuse | #2007 (draft) | +| N1, N2 | pnpm branch locks and npm/bun user patches in discovery; pnpm, yarn and vlt user patches everywhere | #2010, #2018 (draft) | +| G1 | Go discovery reads the governing `go.work` | #2011 (draft) | +| C2, P6 | Cargo members in discovery; liveness matches uuids by origin, not by substring | #2012, #2019, #2024 (draft) | +| G1, J13, P4, N5 | Reads bounded to the repository root in workspace runs (owner, 2026-11-05) | #2025 (draft) | ## Planner merges (hosted and vendored) diff --git a/docs/design/lock-model-unification.md b/docs/design/lock-model-unification.md index 847dd1492..b14c27ac5 100644 --- a/docs/design/lock-model-unification.md +++ b/docs/design/lock-model-unification.md @@ -1,7 +1,7 @@ # One lock model for every reader Status: in progress (stacked PRs `lock-model/*`). Owner decisions are dated -2026-11-04. +2026-11-04 unless the row says 2026-11-05. Paths below are relative to `crates/socket-patch-core/src/`. @@ -33,7 +33,7 @@ attestation rules) is real policy and stays. | Three Python precedence rules | the inventory's chain (uv by parse success, then poetry or pdm if they have entries, then Pipfile.lock plus requirements), `formats/governing_locks.rs` (uv, poetry, pdm, Pipfile.lock by presence) and discovery (every lock, plus the cross-lock contest). They disagree on four common layouts. | | Second walks of one file | `vendor/npm_origin.rs` walks package-lock `packages` again. Yarn writers call `scan_blocks` directly, and the Berry rewriter uses a third model (`formats/yarn/stanzas.rs`). The vlt writers use a separate line grammar. pnpm's `InstalledPackages` and `vendored_npm_uuids`, and the hosted planner's whole-text `grammar::entries`, skip the env-document filter. | | Second parses of one file | `vex/discover/pypi_locks.rs` parses uv, pylock and script locks itself. The inventory parses them again. `deno.lock` is parsed by `crawlers/deno_crawler.rs` for discovery, the hosted engine and the npm flavor probe. | -| Repeated helper parsers | two Python lock listers, two `-r` include walkers, two PEP 508 direct-reference parsers, five walks of `Cargo.toml` dependency tables, two Rush lock listings, three readers of Maven's `checksums.sha256`, two Gradle Socket-index parsers, two routes to sbt resolution evidence | +| Repeated helper parsers | two Python lock listers, two `-r` include walkers, two PEP 508 direct-reference parsers, five walks of `Cargo.toml` dependency tables, two Rush lock listings, four readers of Maven's `checksums.sha256`, two Gradle Socket-index parsers, two routes to sbt resolution evidence | | Two file-selection layers | the npm flavor router (`lock_inventory/npm_family.rs`) picks one family and one lock. Discovery reads every lock and sweeps the ignored ones with `recognize_ignored`. | The inventory also has gaps. It has no models for NuGet, Maven, Gradle, sbt, @@ -77,8 +77,12 @@ Non-goal: adding new hosted or vendored backends. | Question | Readers before | Decision | |---|---|---| | Is pnpm 11's env-lockfile document (the first YAML document, holding `configDependencies` / `packageManagerDependencies`) part of the project? | The hosted planner and the v9 vendored in-use check read it. `PnpmLock` (inventory and discovery), `InstalledPackages` and the legacy vendored check skip it (#1949). | It counts. pnpm installs config dependencies (into its config store, not `node_modules`), and the hosted planner patches them. The document becomes its own install tree. | -| Does Socket patch a dependency the user patched themselves (npm `patchedDependencies` / `"patched"`, `bun patch`, pnpm `patchedDependencies`, yarn berry `patch:`, vlt patches)? | Hosted skips npm and bun user patches (#711, #367). Vendored refuses them. Discovery attested them until #2010. pnpm, yarn and vlt user patches are not detected anywhere. | **Owner decision:** never, by default, in every mode and every package manager. A Socket pin under a user patch is not attested. Only an explicit user override patches it. | -| May a reader or writer read project files outside the git repository root? | Several do: vendored paths spelled `../.socket/vendor/…`, an ancestor `go.work`, out-of-root `-r` includes, parent NuGet configs. | **Owner decision:** no. Every project-file read is bounded to the git repository root (the project root when there is no repository). Machine-level tool configuration (for example `GOENV`, the user `NuGet.Config`) is listed separately for review. | +| Does Socket patch a dependency the user patched themselves (npm `patchedDependencies` / `"patched"`, `bun patch`, pnpm `patchedDependencies`, yarn berry `patch:`, vlt patches)? | Hosted skips npm and bun user patches (#711, #367). Vendored refuses them. Discovery attested them until #2010. pnpm, yarn and vlt user patches are not detected anywhere. | **Owner decision:** never, by default, in every mode and every package manager. A Socket pin under a user patch is not attested. **2026-11-05:** the only override is the existing `--allow-patch-conflicts`, and `vex` never attests a pin under a user patch, even with it (#2010, #2018). | +| May a reader or writer read project files outside the git repository root? | Several do: vendored paths spelled `../.socket/vendor/…`, an ancestor `go.work`, out-of-root `-r` includes, parent NuGet configs. | **Owner decision:** no. Every project-file read is bounded to the git repository root (the project root when there is no repository). Machine-level tool configuration (for example `GOENV`, the user `NuGet.Config`) is listed separately for review. **2026-11-05:** the bound applies to workspace runs only. Global runs (`--global` / `-g` / `--global-prefix`) read anywhere. Without a git checkout, upward lookups (an ancestor `go.work`, parent NuGet configs, workspace and settings files) behave as before (#2025). | +| In a workspace run, what if a file the tool honors sits above the repository root (an ancestor `pnpm-workspace.yaml`, `.npmrc`, `go.work`, `.cargo/config.toml`)? | Each walk reads it, or stops at a different place. | **Owner decision, 2026-11-05:** such a marker file above the repository root and below the home directory makes hosted and vendored mode refuse with a message naming the file, and readers withhold. Nothing at or above home counts (#2025). | +| Do out-of-repository `-r` includes of `requirements.txt` count? | Hosted rewrites the root file only; the inventory and discovery follow includes wherever they point. | **Owner decision, 2026-11-05:** includes are followed (P4), but an include outside the repository root stays bounded in workspace runs: never read or rewritten (#2025). | +| Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` read (N11)? | Hosted rewrites both under one all-or-nothing gate. Discovery sweeps it as ignored. The inventory reads it as a fallback. Vendored never reads it. | **Owner decision, 2026-11-05:** it is shadowed: swept for stale identities, never rewritten, and outside the gate. If the project's pnpm evidence (install record or `package.json` `packageManager`) pins pnpm 1 or 2, which read only `shrinkwrap.yaml`, then `shrinkwrap.yaml` governs and vendored mode refuses the project (#2029). | +| Does a pnpm branch lock (`pnpm-lock..yaml`) make the root lock unreliable (N1)? | Hosted withholds the root locks and refuses when `gitBranchLockfile` is set. Vendored refuses. Discovery and the inventory never check. | **Owner decision, 2026-11-05:** discovery withholds pnpm root and member-lock attestations whenever any `pnpm-lock..yaml` exists, whatever the settings or environment say (#2010). | ## The model @@ -251,18 +255,24 @@ Deprecation messaging is separate from this work. ## Size budget `scripts/loc-budget.py` counts implementation lines: code outside comments, -outside `#[cfg(test)]` items and outside test-only modules. It compares the -groups in `scripts/loc-budget.json` with their maxima. CI's -`lint-ecosystems` job runs it with `--warn`, which annotates a group over -budget but never fails, so another PR's bug fix in one of these files never -fails on it. There are two groups: -`lock-readers` (`formats/`, `vex/discover/`, `vendor/lock_inventory/`, -`gradle/`, the Python lock helpers in `utils/`, `vendor/npm_origin.rs` and -`crawlers/deno_crawler.rs`) and `lock-writers` (`patch/redirect/`, `hosted/` -and the rest of `vendor/`). At 7bde3ba55 they hold 30,898 and 88,363 lines. -Each PR in this stack -lowers the maxima with `--lower` in the same commit. The NuGet and JVM PRs -add code and raise the readers' maximum by hand, saying why. +outside items whose `cfg` implies `test` and outside test-only modules. It +compares the groups in `scripts/loc-budget.json` with their maxima. There are +two groups: `lock-readers` (`formats/`, `vex/discover/`, +`vendor/lock_inventory/`, `gradle/`, the Python lock helpers in `utils/`, +`vendor/npm_origin.rs` and `crawlers/deno_crawler.rs`) and `lock-writers` +(`patch/redirect/`, `hosted/` and the rest of `vendor/`). At 7bde3ba55 they +hold 30,462 and 88,362 lines. + +CI's `lint-ecosystems` job runs it with `--warn`, which annotates a group +over budget but never fails. That is deliberate. These groups cover most of +the lockfile code, and while the stack is open, bug fixes keep landing on +`main` in the same files (the false-attestation fixes below are a dozen +PRs). A failing check would block each of them on a number this campaign +owns, or push them to raise the maximum by hand, which hides the stack's +progress. The stack is what lowers the numbers: each PR runs `--check` +locally and lowers the maxima with `--lower` in the same commit. The NuGet +and JVM PRs add code and raise the readers' maximum by hand, saying why. Once +the stack lands, `--warn` can become `--check`. ## Stack @@ -278,9 +288,10 @@ A format whose planner merge is large splits into two PRs, readers first. | PR | Branch | Change | |---|---|---| -| 0 | `lock-model/0-design` | This document and the size budget | -| 1 | `lock-model/1-origin` | The `origin` module: discovery's classifier and the redirect's hosted-URL rule, moved verbatim. Discovery and the redirect re-export it. | -| 2 | `lock-model/2-pnpm` | Pilot. One pnpm entry walk for the inventory, discovery, the store crawler, the hosted planner and the vendored in-use check. The env-lockfile document becomes its own tree (see Decisions). | +| 0 | `lock-model/0-design` (#2001) | This document and the size budget | +| 1 | `lock-model/1-origin` (#2002) | The `origin` module: discovery's classifier and the redirect's hosted-URL rule, moved verbatim. Callers import it from `crate::origin`. | +| 2a | `lock-model/2a-pnpm-readers` (#2009) | Pilot, readers. One pnpm entry walk for the inventory, discovery, the store crawler, the hosted planner and the vendored in-use check. The env-lockfile document becomes its own tree (see Decisions). | +| 2b | `lock-model/2b-pnpm-planner` (#2029) | Pilot, planner. One pnpm rewire planner for both modes. `shrinkwrap.yaml` beside `pnpm-lock.yaml` is shadowed (see Decisions). | | 3 | `lock-model/3-npm` | package-lock and shrinkwrap. Folds `vendor/npm_origin.rs`'s second walk into the records. | | 4 | `lock-model/4-yarn` | Classic and Berry. One block model for every reader and writer, `BerryStanzas` included. | | 5 | `lock-model/5-bun-vlt` | `bun.lock`, `bun.lockb` and vlt. The vlt writers move off their line grammar. | From 4b1fc91dd2ec5d7e7412b69827d96f8f72607f17 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 6 Nov 2026 09:24:26 +0000 Subject: [PATCH 20/36] Keep cfg(test) crate-visible fields and const generics from swallowing code loc-budget.py only recognized `pub name: Type` as a field, so a #[cfg(test)] on a `crate name: Type` field or a `const N: usize` generic parameter was read as an item. _item_end then didn't stop at the comma and excluded the rest of the struct or parameter list from the implementation count. Treat `crate` followed by a name like `pub`, and read `const` as a parameter when the attribute follows `<` or `,`, where no item can start. A `crate::path` statement and a plain const item keep their current handling. Reported by Bugbot on #2001. Co-Authored-By: Claude --- scripts/loc-budget.py | 28 +++++++++++++++++++++------- scripts/tests/test_loc_budget.py | 22 ++++++++++++++++++++++ 2 files changed, 43 insertions(+), 7 deletions(-) diff --git a/scripts/loc-budget.py b/scripts/loc-budget.py index b4e024218..7f9b3a0f4 100755 --- a/scripts/loc-budget.py +++ b/scripts/loc-budget.py @@ -220,9 +220,18 @@ def _attrs_before(src: str, kind: bytearray, pos: int) -> tuple[int, list[str]]: return pos, attrs -def _starts_item(src: str, kind: bytearray, j: int) -> bool: +def _prev_code(src: str, kind: bytearray, j: int) -> int: + while j >= 0 and (kind[j] == COMMENT or src[j].isspace()): + j -= 1 + return j + + +def _starts_item(src: str, kind: bytearray, j: int, in_list: bool = False) -> bool: """Whether the code at ``j`` is an item or statement, rather than a - struct field, enum variant, parameter or match arm.""" + struct field, enum variant, parameter, generic parameter or match arm. + + ``in_list`` says the attribute follows a ``<`` or ``,``, where an item + can't start, so ``const N: usize`` there is a const generic parameter.""" n = len(src) while True: j = _next_code(src, kind, j) @@ -240,8 +249,9 @@ def _starts_item(src: str, kind: bytearray, j: int) -> bool: if not m: return False word, after = m.group(0), _next_code(src, kind, m.end()) - if word == "pub": - if after < n and src[after] == "(": + # `crate name: Type` is a crate-visible field, but `crate::f()` is a path. + if word == "pub" or (word == "crate" and IDENT.match(src, after)): + if word == "pub" and after < n and src[after] == "(": after = _next_code(src, kind, (_match_close(src, kind, after) or after) + 1) m = IDENT.match(src, after) if not m: @@ -249,10 +259,12 @@ def _starts_item(src: str, kind: bytearray, j: int) -> bool: word, after = m.group(0), _next_code(src, kind, m.end()) # `pub name: Type` is a field. return word in ITEM_KEYWORDS or not (src.startswith(":", after) and not src.startswith("::", after)) + if word == "const" and in_list: + return False # `const N: usize` in a generic parameter list return word in ITEM_KEYWORDS or (after < n and src[after] == "!") # or a macro call -def _item_end(src: str, kind: bytearray, pos: int) -> int: +def _item_end(src: str, kind: bytearray, pos: int, in_list: bool = False) -> int: """End of the item that starts at ``pos``. An item ends at its first top-level ``;``, at the brace closing its first @@ -261,7 +273,7 @@ def _item_end(src: str, kind: bytearray, pos: int) -> int: ends at its first top-level ``,``. """ n = len(src) - is_field = not _starts_item(src, kind, pos) + is_field = not _starts_item(src, kind, pos, in_list) depth, angle, in_expr = 0, 0, False for j in range(pos, n): if kind[j] != CODE: @@ -345,8 +357,10 @@ def count_source(src: str) -> FileCount: excluded = bytearray(b"\x01" * n) break continue - end = _item_end(src, kind, attr_end + 1) start, _ = _attrs_before(src, kind, start) + prev = _prev_code(src, kind, start - 1) + in_list = prev >= 0 and kind[prev] == CODE and src[prev] in "<," + end = _item_end(src, kind, attr_end + 1, in_list) test_mods.extend(_file_mods(src, kind, start, end)) excluded[start:end] = b"\x01" * (end - start) diff --git a/scripts/tests/test_loc_budget.py b/scripts/tests/test_loc_budget.py index d5a676d63..541552e1c 100644 --- a/scripts/tests/test_loc_budget.py +++ b/scripts/tests/test_loc_budget.py @@ -122,6 +122,28 @@ def test_cfg_test_enum_variant_and_match_arm(self): self.assertEqual(fc.code, 9) self.assertEqual(fc.test, 4) + def test_cfg_test_crate_visible_field(self): + src = "struct S {\n #[cfg(test)]\n crate probe: u32,\n real: u32,\n}\nfn shipped() {}\n" + fc = count(src) + self.assertEqual(fc.code, 4) + self.assertEqual(fc.test, 2) + + def test_cfg_test_const_generic_param(self): + src = "struct S<\n #[cfg(test)] const N: usize,\n T,\n> {\n t: T,\n}\nfn shipped() {}\n" + fc = count(src) + self.assertEqual(fc.code, 6) + self.assertEqual(fc.test, 1) + + def test_cfg_test_const_item_is_still_an_item(self): + src = "#[cfg(test)]\nconst N: u8 = 1;\nfn shipped() {}\n" + fc = count(src) + self.assertEqual(fc.code, 1) + self.assertEqual(fc.test, 2) + + def test_crate_path_statement_is_not_a_field(self): + src = "fn f() {\n #[cfg(test)]\n crate::probe();\n real();\n}\n" + self.assertEqual(count(src).code, 3) + def test_cfg_test_fn_param(self): src = "fn f(\n a: u8,\n #[cfg(test)] b: u8,\n) -> u8 {\n a\n}\nfn g() {}\n" self.assertEqual(count(src).code, 6) From d6ccde3e9300b496febe0f22fde447dce5d626ff Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 6 Nov 2026 12:54:39 +0000 Subject: [PATCH 21/36] Name real symbols and paths in the lock-model design doc Follow-up PRs grep for the names this doc cites. OriginPolicy::hosted_uuid exists nowhere in the tree; the function is DiscoverCtx::hosted_uuid (its memoized fast path is the hosted_hosts cell). The npm flavor router lives at vendor/lock_inventory/npm_family.rs, since doc paths are relative to src/. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_011f9aeNYq2GHyrsNnPBepcC --- docs/design/lock-model-unification.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/design/lock-model-unification.md b/docs/design/lock-model-unification.md index b14c27ac5..f6d8196f2 100644 --- a/docs/design/lock-model-unification.md +++ b/docs/design/lock-model-unification.md @@ -34,7 +34,7 @@ attestation rules) is real policy and stays. | Second walks of one file | `vendor/npm_origin.rs` walks package-lock `packages` again. Yarn writers call `scan_blocks` directly, and the Berry rewriter uses a third model (`formats/yarn/stanzas.rs`). The vlt writers use a separate line grammar. pnpm's `InstalledPackages` and `vendored_npm_uuids`, and the hosted planner's whole-text `grammar::entries`, skip the env-document filter. | | Second parses of one file | `vex/discover/pypi_locks.rs` parses uv, pylock and script locks itself. The inventory parses them again. `deno.lock` is parsed by `crawlers/deno_crawler.rs` for discovery, the hosted engine and the npm flavor probe. | | Repeated helper parsers | two Python lock listers, two `-r` include walkers, two PEP 508 direct-reference parsers, five walks of `Cargo.toml` dependency tables, two Rush lock listings, four readers of Maven's `checksums.sha256`, two Gradle Socket-index parsers, two routes to sbt resolution evidence | -| Two file-selection layers | the npm flavor router (`lock_inventory/npm_family.rs`) picks one family and one lock. Discovery reads every lock and sweeps the ignored ones with `recognize_ignored`. | +| Two file-selection layers | the npm flavor router (`vendor/lock_inventory/npm_family.rs`) picks one family and one lock. Discovery reads every lock and sweeps the ignored ones with `recognize_ignored`. | The inventory also has gaps. It has no models for NuGet, Maven, Gradle, sbt, Hatch or Deno, and it reads no `composer.lock` that starts with a BOM, which @@ -136,7 +136,7 @@ rewritten, from two places: configured origin, the last canonical-uuid path segment) and `SOCKET_PATCH_SERVER_HOST`. The hosted unwind already calls these directly; -* from discovery: `OriginPolicy::hosted_uuid` (the same rule with a memoized +* from discovery: `DiscoverCtx::hosted_uuid` (the same rule with a memoized fast path for plain registry URLs), `vendor_ref`, `vendor_ref_decorated` and `vendor_uuid_dir`, `locate`, `Wired`, and the `socket-patch-` name grammar. From fbe644d8bcfb081da9357e2cd64860cda395362b Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Fri, 6 Nov 2026 09:13:49 -0600 Subject: [PATCH 22/36] Address the review of the origin move - The extractor guard also bans OriginPolicy and crate::origin:: (not bare origin::, which would catch crate::vendor::npm_origin::), and vex::discover no longer re-exports OriginPolicy; mod.rs imports it from crate::origin. - The lock_inventory model guard bans OriginPolicy, crate::origin:: and hosted_uuid(; the formats guard bans OriginPolicy, hosted_uuid( and SOCKET_PATCH_SERVER_HOST. formats may still use origin's pure path grammar (vendor_ref), which applies no host policy. - socket_identities takes &OriginPolicy. The policy memoizes one lowercased host set, shared by hosted_uuid's fast path and a case-insensitive matcher (host_finder), so a discovery run builds the matcher once instead of on every swept file. hosted_uuids_in_text keeps its signature. Answers are unchanged: reqwest already lowercases http(s) hosts. - origin.rs's module doc states one classifier as the goal and names the readers that still classify on their own; it no longer links pub(crate) items (rustdoc private_intra_doc_links). - socket_patch_name_uuid is pub(crate) again; hosted_patch_url_uuids loses its duplicate is_canonical_uuid import. - patch::redirect stops re-exporting hosted_patch_uuid, hosted_patch_url_uuids and SOCKET_PATCH_SERVER_HOST; vlt_heal, upstream, lock_inventory::pypi and discover use crate::origin directly. Extractors keep importing origin helpers through vex::discover's re-export (now with SOCKET_PATCH_SERVER_HOST for pypi_other), as the guard requires. - New test: DiscoverCtx::with_origins(..).hosted_uuid(..) honours a configured origin and only for that run. Budget raised by hand: lock-readers 30540 -> 30544 (the shared host matcher and the re-export note), lock-writers 88313 -> 88315 (rustfmt wraps the upstream origin path). Co-Authored-By: Claude Opus 5.5 --- crates/socket-patch-core/src/formats/mod.rs | 7 +- crates/socket-patch-core/src/origin.rs | 63 +++++++---- .../src/patch/redirect/generation.rs | 2 +- .../src/patch/redirect/mod.rs | 2 - .../src/patch/redirect/upstream/mod.rs | 9 +- .../src/patch/redirect/vlt_heal.rs | 2 +- .../src/vendor/lock_inventory/mod.rs | 5 +- .../src/vendor/lock_inventory/pypi.rs | 4 +- crates/socket-patch-core/src/vendor/path.rs | 2 +- .../socket-patch-core/src/vex/discover/mod.rs | 101 +++++++++++------- .../src/vex/discover/pypi_other.rs | 6 +- scripts/loc-budget.json | 4 +- 12 files changed, 133 insertions(+), 74 deletions(-) diff --git a/crates/socket-patch-core/src/formats/mod.rs b/crates/socket-patch-core/src/formats/mod.rs index 3f8fcb874..8c984979b 100644 --- a/crates/socket-patch-core/src/formats/mod.rs +++ b/crates/socket-patch-core/src/formats/mod.rs @@ -51,14 +51,19 @@ pub use registry::registry; mod architecture_tests { use std::path::Path; - const IMPURE: [&str; 9] = [ + const IMPURE: [&str; 12] = [ "tokio::fs", "std::fs", "read_regular_", "File::open", "OpenOptions", + // The host policy (`crate::origin`). Its pure path grammar + // (`origin::vendor_ref`) is allowed: it applies no host policy. "hosted_patch_uuid", "hosted_patch_url_uuids", + "hosted_uuid(", + "OriginPolicy", + "SOCKET_PATCH_SERVER_HOST", "async fn", ".await", ]; diff --git a/crates/socket-patch-core/src/origin.rs b/crates/socket-patch-core/src/origin.rs index 27714f2d4..6e0c168b3 100644 --- a/crates/socket-patch-core/src/origin.rs +++ b/crates/socket-patch-core/src/origin.rs @@ -1,20 +1,27 @@ //! Socket origin: whether a string a lockfile or package-manager config //! records routes a package to a Socket patch, and to which one. //! -//! This is the one classifier every reader applies: +//! The goal is one classifier that every reader applies. Today discovery and +//! the redirect's hosted unwind apply it, but some readers still classify +//! on their own: the lock inventory's PyPI view +//! (`vendor::lock_inventory::pypi::hosted_pypi_reference`) adds its own +//! url-tail grammar, discovery's PyPI diagnostics (`vex::discover::pypi_other`) +//! compare hosts directly, the upstream restore keeps its own known-origin +//! list, and the inventory matches hosted urls on any host. The lock-model +//! stack moves them here (`docs/design/lock-model-unification.md`). //! //! * **hosted** — [`hosted_patch_uuid`]: an `http(s)` url on Socket's patch //! server or on an operator-configured origin, the patch uuid its last -//! canonical-uuid path segment. [`OriginPolicy::hosted_uuid`] answers the +//! canonical-uuid path segment. `OriginPolicy::hosted_uuid` answers the //! same question with a memoized fast path for the plain registry urls //! every lock is full of; //! * **vendored** — [`vendor_ref`] / [`vendor_ref_decorated`] / //! [`vendor_uuid_dir`]: a path that STARTS at this project's //! `.socket/vendor///`, its leaf traversal-safe; -//! * **names** — [`socket_patch_name_uuid`]: the `socket-patch-` +//! * **names** — `socket_patch_name_uuid`: the `socket-patch-` //! registry, repository and source names. //! -//! [`OriginPolicy::locate`] computes the hosted and vendored answers for one +//! `OriginPolicy::locate` computes the hosted and vendored answers for one //! string at once. //! //! The format models ([`crate::formats`]) stay pure: they hand out the raw @@ -74,7 +81,6 @@ pub fn hosted_patch_uuid(url: &str, extra_origins: &[String]) -> Option /// uses it to RECOGNIZE every Socket identity a lockfile mentions, including /// the malformed or rejected shapes whose "last uuid" is not a patch. pub fn hosted_patch_url_uuids(url: &str, extra_origins: &[String]) -> Option> { - use crate::patch::path_safety::is_canonical_uuid; use crate::utils::purl::percent_decode_purl_component; let unescaped = url.trim().replace("\\/", "/"); @@ -174,8 +180,11 @@ fn plain_url_domain(url: &str) -> Option { #[derive(Debug)] pub(crate) struct OriginPolicy<'a> { origins: &'a [String], - /// The hosts a Socket-hosted url can name (see [`Self::hosted_uuid`]). + /// The hosts a Socket-hosted url can name, lowercased (see + /// [`Self::hosts`]). hosts: OnceLock>, + /// [`Self::hosts`] as an ASCII-case-insensitive matcher. + host_finder: OnceLock, } impl<'a> OriginPolicy<'a> { @@ -183,6 +192,7 @@ impl<'a> OriginPolicy<'a> { OriginPolicy { origins, hosts: OnceLock::new(), + host_finder: OnceLock::new(), } } @@ -191,6 +201,33 @@ impl<'a> OriginPolicy<'a> { self.origins } + /// The hosts a Socket-hosted url can name: Socket's patch server and + /// each configured origin's host, lowercased. Built once per policy. + fn hosts(&self) -> &BTreeSet { + self.hosts.get_or_init(|| { + let mut hosts = BTreeSet::from([SOCKET_PATCH_SERVER_HOST.to_string()]); + hosts.extend(self.origins.iter().filter_map(|o| { + reqwest::Url::parse(o.trim()) + .ok()? + .host_str() + .map(str::to_ascii_lowercase) + })); + hosts + }) + } + + /// A matcher for any of [`Self::hosts`] in any ASCII case: the cheap + /// pre-filter of discovery's identity sweep (only a token naming an + /// accepted host can be ours). Built once per policy. + pub(crate) fn host_finder(&self) -> &aho_corasick::AhoCorasick { + self.host_finder.get_or_init(|| { + aho_corasick::AhoCorasick::builder() + .ascii_case_insensitive(true) + .build(self.hosts()) + .expect("literal host patterns") + }) + } + /// The patch uuid of a Socket-HOSTED url, or `None` for anything else /// (see [`hosted_patch_uuid`] for the accepted /// spellings and the host allowlist). @@ -200,17 +237,7 @@ impl<'a> OriginPolicy<'a> { /// here. pub(crate) fn hosted_uuid(&self, url: &str) -> Option { if let Some(host) = plain_url_domain(url) { - let hosts = self.hosts.get_or_init(|| { - let mut hosts = BTreeSet::from([SOCKET_PATCH_SERVER_HOST.to_string()]); - hosts.extend(self.origins.iter().filter_map(|o| { - reqwest::Url::parse(o.trim()) - .ok()? - .host_str() - .map(str::to_string) - })); - hosts - }); - if !hosts.contains(&host) { + if !self.hosts().contains(&host) { return None; } } @@ -241,7 +268,7 @@ impl<'a> OriginPolicy<'a> { /// maven ``, nuget ``) or, with /// `vendored = true`, `socket-patch-vendor-` (maven's vendored repo /// id). Exact grammar only — a suffix or a non-canonical uuid is not ours. -pub fn socket_patch_name_uuid(name: &str, vendored: bool) -> Option { +pub(crate) fn socket_patch_name_uuid(name: &str, vendored: bool) -> Option { crate::patch::redirect::generation::pin_name_uuid(name.trim(), vendored).map(str::to_string) } diff --git a/crates/socket-patch-core/src/patch/redirect/generation.rs b/crates/socket-patch-core/src/patch/redirect/generation.rs index a1889db08..bf28a3274 100644 --- a/crates/socket-patch-core/src/patch/redirect/generation.rs +++ b/crates/socket-patch-core/src/patch/redirect/generation.rs @@ -51,7 +51,7 @@ pub(crate) fn hosted_pin_name(uuid: &str) -> String { /// `socket-patch-vendor-` (maven's vendored repository id). /// No trimming: the rewriter must never treat a user's padded pin as its /// own, while lockfile discovery trims at its call site -/// (`vex::discover::socket_patch_name_uuid`). +/// (`origin::socket_patch_name_uuid`). pub(crate) fn pin_name_uuid(name: &str, vendored: bool) -> Option<&str> { let prefix = if vendored { VENDOR_PIN_NAME_PREFIX diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 706b6eb21..f28fa9f14 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -7606,8 +7606,6 @@ pub fn grant_token_path_segment(url: &str, patch_uuid: &str) -> Option { (!token.is_empty()).then(|| token.to_string()) } -pub use crate::origin::{hosted_patch_url_uuids, hosted_patch_uuid, SOCKET_PATCH_SERVER_HOST}; - /// A dep's Socket index URL as a regex source with the per-request rotating /// segments (grant token, patch uuid) wildcarded — an exact-URL pattern /// misses the URL a previous run wrote under an older grant. The grant token diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs b/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs index 1b1e54f55..4ab24aa93 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs @@ -476,8 +476,11 @@ pub fn foreign_dep_origins<'a>( dep_origins(deps) .into_iter() .filter(|origin| { - let known = std::iter::once(format!("https://{}", super::SOCKET_PATCH_SERVER_HOST)) - .chain(configured.iter().cloned()); + let known = std::iter::once(format!( + "https://{}", + crate::origin::SOCKET_PATCH_SERVER_HOST + )) + .chain(configured.iter().cloned()); !known .into_iter() .any(|k| dep_origins_of_url(&k).as_deref() == Some(origin.as_str())) @@ -768,7 +771,7 @@ impl Ctx<'_> { /// The hosted patch uuid `url` names, under the same host allowlist /// discovery applies. pub(crate) fn hosted_uuid(&self, url: &str) -> Option { - super::hosted_patch_uuid(url, self.origins) + crate::origin::hosted_patch_uuid(url, self.origins) } } diff --git a/crates/socket-patch-core/src/patch/redirect/vlt_heal.rs b/crates/socket-patch-core/src/patch/redirect/vlt_heal.rs index 13a28b124..dd5b82513 100644 --- a/crates/socket-patch-core/src/patch/redirect/vlt_heal.rs +++ b/crates/socket-patch-core/src/patch/redirect/vlt_heal.rs @@ -14,9 +14,9 @@ use std::path::{Path, PathBuf}; use serde_json::{Map, Value}; -use super::hosted_patch_uuid; use crate::constants::npm_family::{VLT_HIDDEN_LOCK_REL, VLT_STORE_DIR}; use crate::manifest::schema::PatchRecord; +use crate::origin::hosted_patch_uuid; use crate::patch::apply::{is_safe_relative_subpath, normalize_file_path}; use crate::patch::file_hash::compute_file_git_sha256; use crate::patch::package::read_archive_bytes_to_map_strict; diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/mod.rs b/crates/socket-patch-core/src/vendor/lock_inventory/mod.rs index 3d4bc58fb..7dc3dda28 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/mod.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/mod.rs @@ -402,7 +402,7 @@ mod architecture_tests { const PURE_READER: &str = "// ── pure reader ──"; /// Anything that reads content, does I/O, or applies the hosted-origin /// policy. - const IMPURE: [&str; 9] = [ + const IMPURE: [&str; 12] = [ "tokio::fs", "std::fs", "read_regular_", @@ -410,6 +410,9 @@ mod architecture_tests { "OpenOptions", "hosted_patch_uuid", "hosted_patch_url_uuids", + "hosted_uuid(", + "OriginPolicy", + "crate::origin::", "async fn", ".await", ]; diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs b/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs index b1822066a..2d9636f7f 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs @@ -173,12 +173,12 @@ pub(crate) fn hosted_artifact_url(url: &str) -> Result/// /// /` tail is matched from the END, so a path-prefixed /// `--patch-server-url` deployment and a hosted sdist are recognized too. pub(crate) fn hosted_pypi_reference(url: &str, origins: &[String]) -> Option { - crate::patch::redirect::hosted_patch_url_uuids(url, origins)?; + crate::origin::hosted_patch_url_uuids(url, origins)?; hosted_artifact_url(url).ok().filter(|coords| { coords .uuid_level diff --git a/crates/socket-patch-core/src/vendor/path.rs b/crates/socket-patch-core/src/vendor/path.rs index 993d8b5d3..86c643154 100644 --- a/crates/socket-patch-core/src/vendor/path.rs +++ b/crates/socket-patch-core/src/vendor/path.rs @@ -289,7 +289,7 @@ fn split_nuget_leaf(stem: &str) -> Option<(&str, &str)> { /// /// The leaf must be CLEAN (no lock-format suffix such as yarn's `#` or /// berry's `::locator=…`) and pypi dist names come back un-canonicalized; -/// `vex::discover::vendored_leaf_purl` strips the former and canonicalizes +/// `origin::vendored_leaf_purl` strips the former and canonicalizes /// the latter for lockfile-discovered references. pub(crate) fn leaf_to_purl(eco: &str, leaf: &str) -> Option { match eco { diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 5c08758c3..fcefec8b5 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -78,7 +78,7 @@ //! panic, never `unwrap` on file data, bound every recursion. //! 4. **Hosted identity** comes ONLY from [`DiscoverCtx::hosted_uuid`] (the //! Socket host allowlist + last-canonical-uuid-segment rule of -//! [`crate::patch::redirect::hosted_patch_uuid`]) — except identities +//! [`crate::origin::hosted_patch_uuid`]) — except identities //! that are not URLs: Go's `patch.socket.dev/gopatch/` module path //! ([`crate::vendor::go_mod_edit::HOSTED_GO_MODULE_PREFIX`]) and the //! `socket-patch-` registry / repository / source names @@ -197,9 +197,14 @@ use crate::vendor::lock_inventory::{ use crate::vendor::path::{ecosystem_dir_for_purl, ECOSYSTEM_DIRS, VENDOR_DIR}; use crate::vendor::state::VendorEntry; +use crate::origin::OriginPolicy; +// The origin helpers the extractors use, re-exported so they import every +// identity helper from here (the architecture guard below bans +// `crate::origin::` in extractor files: hosted identity comes only from +// `DiscoverCtx::hosted_uuid`). pub(crate) use crate::origin::{ mentions_vendor_dir, names_vendor_dir, npm_vendored_tarball_names, root_anchored_spelling, - socket_patch_name_uuid, LocateOpts, Located, OriginPolicy, Wired, + socket_patch_name_uuid, LocateOpts, Located, Wired, SOCKET_PATCH_SERVER_HOST, }; pub use crate::origin::{ strip_leaf_suffix, vendor_ref, vendor_ref_decorated, vendor_uuid_dir, vendored_leaf_purl, @@ -1578,8 +1583,9 @@ async fn ignore_member_stray_locks(ctx: &mut DiscoverCtx<'_>, out: &mut Discover }; for rel in &stray.ignored { let mentions = match ctx.view.read_bytes(rel).await { - Ok(bytes) => !socket_identities(&String::from_utf8_lossy(&bytes), ctx.origin.origins()) - .is_empty(), + Ok(bytes) => { + !socket_identities(&String::from_utf8_lossy(&bytes), &ctx.origin).is_empty() + } Err(_) => false, }; ctx.recognize_ignored(rel).await; @@ -1691,7 +1697,7 @@ impl<'a> DiscoverCtx<'a> { /// Record every Socket identity `text` (the content of root-relative /// `rel`) mentions — see [`socket_identities`]. fn recognize_text(&self, rel: &str, text: &str) { - let found = socket_identities(text, self.origin.origins()); + let found = socket_identities(text, &self.origin); if found.is_empty() { return; } @@ -1903,7 +1909,7 @@ impl<'a> DiscoverCtx<'a> { /// ids, `§` in legacy ids), since a vlt node key may be its only mention; /// * [`WiringMode::Hosted`]: every canonical-uuid path segment of a url on /// an accepted patch-server origin -/// ([`crate::patch::redirect::hosted_patch_url_uuids`]: plain, +/// ([`crate::origin::hosted_patch_url_uuids`]: plain, /// `sparse+` / `registry+` prefixed, or a wholly percent-encoded berry /// `__archiveUrl`), and every canonical-uuid segment of a /// `patch.socket.dev/gopatch/…` Go module path. @@ -1917,9 +1923,10 @@ impl<'a> DiscoverCtx<'a> { /// /// Memoized by content ([`IdentityMemo`]): one scan runs several /// discoveries over mostly byte-identical lockfiles, and this sweep is the -/// largest per-file cost after parsing. -fn socket_identities(text: &str, origins: &[String]) -> BTreeSet<(String, WiringMode)> { - IdentityMemo::sweep(text, origins) +/// largest per-file cost after parsing. The host matcher is the policy's +/// own ([`OriginPolicy::host_finder`]), built once per run. +fn socket_identities(text: &str, policy: &OriginPolicy<'_>) -> BTreeSet<(String, WiringMode)> { + IdentityMemo::sweep(text, policy) } /// The process-wide memo behind [`socket_identities`]: the sweep's result @@ -1947,12 +1954,12 @@ impl IdentityMemo { ENTRIES.get_or_init(|| Mutex::new(std::collections::VecDeque::new())) } - fn sweep(text: &str, origins: &[String]) -> IdentitySet { + fn sweep(text: &str, policy: &OriginPolicy<'_>) -> IdentitySet { if text.len() < Self::MIN_LEN { - return socket_identities_inner(text, origins); + return socket_identities_inner(text, policy); } use sha2::Digest as _; - let mut origin_key = origins.to_vec(); + let mut origin_key = policy.origins().to_vec(); origin_key.sort(); origin_key.dedup(); let key: IdentityKey = (sha2::Sha256::digest(text.as_bytes()).into(), origin_key); @@ -1964,7 +1971,7 @@ impl IdentityMemo { if let Some((_, found)) = lock().iter().find(|(k, _)| *k == key) { return found.clone(); } - let found = socket_identities_inner(text, origins); + let found = socket_identities_inner(text, policy); let mut entries = lock(); if !entries.iter().any(|(k, _)| *k == key) { if entries.len() == Self::CAPACITY { @@ -1980,14 +1987,17 @@ impl IdentityMemo { /// rules as discovery's sweep (the upstream restore's "does anything in /// this file still need the hosted-side setting" probe). pub(crate) fn hosted_uuids_in_text(text: &str, origins: &[String]) -> BTreeSet { - socket_identities(text, origins) + socket_identities(text, &OriginPolicy::new(origins)) .into_iter() .filter(|(_, mode)| *mode == WiringMode::Hosted) .map(|(uuid, _)| uuid) .collect() } -fn socket_identities_inner(text: &str, origins: &[String]) -> BTreeSet<(String, WiringMode)> { +fn socket_identities_inner( + text: &str, + policy: &OriginPolicy<'_>, +) -> BTreeSet<(String, WiringMode)> { use aho_corasick::AhoCorasick; use std::borrow::Cow; @@ -2048,17 +2058,7 @@ fn socket_identities_inner(text: &str, origins: &[String]) -> BTreeSet<(String, // Cheap pre-filter: only a token naming an accepted host (any ASCII // case) can be ours. - let mut hosts = vec![crate::patch::redirect::SOCKET_PATCH_SERVER_HOST.to_string()]; - hosts.extend(origins.iter().filter_map(|o| { - reqwest::Url::parse(o.trim()) - .ok()? - .host_str() - .map(str::to_ascii_lowercase) - })); - let host_finder = AhoCorasick::builder() - .ascii_case_insensitive(true) - .build(&hosts) - .expect("literal host patterns"); + let host_finder = policy.host_finder(); static SEPARATORS: std::sync::OnceLock = std::sync::OnceLock::new(); let separators = SEPARATORS.get_or_init(|| { AhoCorasick::builder() @@ -2074,7 +2074,7 @@ fn socket_identities_inner(text: &str, origins: &[String]) -> BTreeSet<(String, if !host_finder.is_match(&bytes[start..end]) { continue; } - let uuids = crate::patch::redirect::hosted_patch_url_uuids(&norm[start..end], origins); + let uuids = crate::origin::hosted_patch_url_uuids(&norm[start..end], policy.origins()); for uuid in uuids.into_iter().flatten() { found.insert((uuid, WiringMode::Hosted)); } @@ -4156,15 +4156,16 @@ mod tests { fn socket_identities_recognize_every_spelling_and_nothing_else() { let a = UUID_A; let t = TOKEN; + let none = OriginPolicy::new(&[]); let hosted = |text: &str| { - socket_identities(text, &[]) + socket_identities(text, &none) .into_iter() .filter(|(_, m)| *m == WiringMode::Hosted) .map(|(u, _)| u) .collect::>() }; let vendored = |text: &str| { - socket_identities(text, &[]) + socket_identities(text, &none) .into_iter() .filter(|(_, m)| *m == WiringMode::Vendored) .map(|(u, _)| u) @@ -4241,17 +4242,20 @@ mod tests { ".socket/vendor/npm/not-a-uuid/x.tgz".to_string(), "https://patch.socket.dev/patch/npm/x/1.0.0/tok/placeholder/x.tgz".to_string(), ] { - assert!(socket_identities(&text, &[]).is_empty(), "{text}"); + assert!(socket_identities(&text, &none).is_empty(), "{text}"); } // A configured patch-server origin counts only when configured. let staging = format!("\"resolved\": \"http://127.0.0.1:4545/patch/npm/x/1/{t}/{a}/x.tgz\""); - assert!(socket_identities(&staging, &[]).is_empty()); + assert!(socket_identities(&staging, &none).is_empty()); assert_eq!( - socket_identities(&staging, &["http://127.0.0.1:4545".to_string()]) - .into_iter() - .map(|(u, _)| u) - .collect::>(), + socket_identities( + &staging, + &OriginPolicy::new(&["http://127.0.0.1:4545".to_string()]) + ) + .into_iter() + .map(|(u, _)| u) + .collect::>(), both ); } @@ -4273,7 +4277,7 @@ mod tests { }; let staging = ["http://127.0.0.1:4545".to_string()]; let uuids = |text: &str, origins: &[String]| { - socket_identities(text, origins) + socket_identities(text, &OriginPolicy::new(origins)) .into_iter() .map(|(u, m)| (u, m == WiringMode::Hosted)) .collect::>() @@ -4301,9 +4305,26 @@ mod tests { ]; assert_eq!(uuids(&lock(a), &twice), uuids(&lock(a), &staging)); assert_eq!( - socket_identities(&lock(a), &staging), - socket_identities_inner(&lock(a), &staging) + socket_identities(&lock(a), &OriginPolicy::new(&staging)), + socket_identities_inner(&lock(a), &OriginPolicy::new(&staging)) + ); + } + + /// The ctx classifies under the origins it was built with: a url on a + /// configured `--patch-server-url` origin is hosted for that run only + /// (the classifier itself is tested in `crate::origin`). + #[test] + fn ctx_hosted_uuid_honours_the_runs_configured_origins() { + let root = Path::new("/nonexistent"); + let staging = ["http://127.0.0.1:4545".to_string()]; + let url = format!("http://127.0.0.1:4545/patch/npm/x/1.0.0/{TOKEN}/{UUID_A}/x-1.0.0.tgz"); + let ctx = DiscoverCtx::with_origins(root, &staging); + assert_eq!(ctx.hosted_uuid(&url).as_deref(), Some(UUID_A)); + assert_eq!( + ctx.hosted_uuid("https://registry.npmjs.org/x/-/x-1.0.0.tgz"), + None ); + assert_eq!(DiscoverCtx::with_origins(root, &[]).hosted_uuid(&url), None); } /// The claim API: a recognized uuid is decided by the refs alone (live @@ -4502,6 +4523,10 @@ mod tests { "OpenOptions", "hosted_patch_uuid(", "hosted_patch_url_uuids(", + // The policy itself, and the origin module (its helpers come + // re-exported from this module; `npm_origin::` is unrelated). + "OriginPolicy", + "crate::origin::", // The file-reading entry points of the shared readers an // extractor imports (the lock inventory, the writers' config // readers): each reads content outside the recognizing ctx. diff --git a/crates/socket-patch-core/src/vex/discover/pypi_other.rs b/crates/socket-patch-core/src/vex/discover/pypi_other.rs index 9a724b3e6..313ec2f6f 100644 --- a/crates/socket-patch-core/src/vex/discover/pypi_other.rs +++ b/crates/socket-patch-core/src/vex/discover/pypi_other.rs @@ -116,7 +116,7 @@ use toml_edit::DocumentMut; use super::{ mentions_vendor_dir, pypi_purl, toml_or_diag, vendored_leaf_purl, DiscoverCtx, Discovery, LocateOpts, PatchedRef, TomlDiag, VendorRef, Wired, DIAG_LOCKFILE_UNPARSEABLE, - DIAG_LOCKFILE_UNREADABLE, DIAG_REF_INVALID, + DIAG_LOCKFILE_UNREADABLE, DIAG_REF_INVALID, SOCKET_PATCH_SERVER_HOST, }; use crate::crawlers::python_crawler::canonicalize_pypi_name; use crate::utils::digest::sha256_hex; @@ -572,9 +572,7 @@ fn classify(ctx: &DiscoverCtx<'_>, location: &str, uri: bool) -> Result/ path" )); } - if reqwest::Url::parse(location) - .is_ok_and(|u| u.host_str() == Some(crate::patch::redirect::SOCKET_PATCH_SERVER_HOST)) - { + if reqwest::Url::parse(location).is_ok_and(|u| u.host_str() == Some(SOCKET_PATCH_SERVER_HOST)) { return Err(format!( "{location:?} is a Socket patch-server url that names no patch uuid" )); diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json index bfe5d936b..272b903fa 100644 --- a/scripts/loc-budget.json +++ b/scripts/loc-budget.json @@ -17,7 +17,7 @@ "crates/socket-patch-core/src/utils/pipenv.rs", "crates/socket-patch-core/src/crawlers/deno_crawler.rs" ], - "max": 30540 + "max": 30544 }, "lock-writers": { "paths": [ @@ -30,7 +30,7 @@ "crates/socket-patch-core/src/vendor/npm_origin.rs", "crates/socket-patch-core/src/vendor/test_support" ], - "max": 88313 + "max": 88315 } } } From 64dfe54da26a67b0d7c77b83d891eef05f4a8ff3 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 7 Nov 2026 04:25:43 +0000 Subject: [PATCH 23/36] Test the pnpm takeover gate with a mixed-ending lock The refusals test merged in from main (#2015) expected a pure CRLF pnpm lock to be refused with vendor_lockfile_crlf_unsupported. This branch teaches the pnpm rewriter to wire pure CRLF locks byte-faithfully, so that refusal now applies only to locks that mix CRLF and LF. Give the test a mixed-ending lock so it still checks that a hosted pin's takeover meets the pnpm lock-text gates. Co-Authored-By: Claude --- crates/socket-patch-core/src/vendor/refusals.rs | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/crates/socket-patch-core/src/vendor/refusals.rs b/crates/socket-patch-core/src/vendor/refusals.rs index acd529e27..34be6f8b5 100644 --- a/crates/socket-patch-core/src/vendor/refusals.rs +++ b/crates/socket-patch-core/src/vendor/refusals.rs @@ -882,6 +882,8 @@ mod tests { /// A hosted pnpm pin's takeover meets the pnpm lock-text gates /// (#853); a ledger entry already at the selected uuid is in sync. + /// The lock mixes CRLF and LF endings, which the pnpm rewriter + /// cannot write back byte-faithfully (a pure CRLF lock is wired). #[tokio::test] async fn a_hosted_pin_meets_the_pnpm_gates_before_its_takeover() { let tmp = tempfile::tempdir().unwrap(); @@ -892,7 +894,7 @@ mod tests { .unwrap(); std::fs::write( tmp.path().join("pnpm-lock.yaml"), - HOSTED_V9.replace('\n', "\r\n"), + HOSTED_V9.replacen('\n', "\r\n", 1), ) .unwrap(); let pin = HostedPin { From ff603e27554d1e18b9f2076cd0cba8780eec0323 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 7 Nov 2026 07:47:27 +0000 Subject: [PATCH 24/36] Gate the inventory's Rush check on is_rush, as discovery does The npm-family inventory took rush.json as present via view.exists, so a directory, an implied in-memory directory or a link to a directory named rush.json counted as Rush there while discovery's is_rush (a regular file) said no. Under pnpm lockfile=false the inventory then kept the hosted pin's integrity in a root lock pnpm never reads, while discovery withheld it. Use the shared is_rush gate at both call sites. Co-Authored-By: Claude --- .../src/vendor/lock_inventory/npm_family.rs | 8 ++++---- .../src/vendor/lock_inventory/tests.rs | 11 +++++++++++ 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs b/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs index f8bc82c6a..00a8b3f6b 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs @@ -19,7 +19,7 @@ use super::dedup_prefer_integrity; use super::npm::inventory_package_lock_in; use super::pnpm::{ inventory_pnpm_lock_in, inventory_pnpm_lock_rel_in, inventory_pnpm_member_locks_in, - inventory_rush_pnpm_locks_in, + inventory_rush_pnpm_locks_in, is_rush, }; use super::view::ProjectView; use super::vlt::inventory_vlt_in; @@ -52,7 +52,7 @@ pub(crate) async fn inventory_npm_lock( project_root: &Path, ) -> Result)>, UnsupportedNpmLayout> { let view = ProjectView::Disk(project_root); - let rush = view.exists("rush.json").await; + let rush = is_rush(view).await; inventory_npm_lock_raw_in(&view, rush) .await .map(|found| found.map(|(flavor, raw)| (flavor, dedup_prefer_integrity(raw)))) @@ -77,7 +77,7 @@ pub(crate) async fn inventory_npm_trees( pub(super) async fn inventory_npm_trees_raw_in( view: &ProjectView<'_>, ) -> Result, UnsupportedNpmLayout> { - let rush = view.exists("rush.json").await; + let rush = is_rush(*view).await; let mut out = inventory_npm_lock_raw_in(view, rush) .await? .map(|(_, entries)| entries) @@ -90,7 +90,7 @@ pub(super) async fn inventory_npm_trees_raw_in( /// The routed lock's guarded instances ([`inventory_npm_lock`] before its /// collapse). With pnpm's `lockfile` setting off ([`lockfile_disabled`]; -/// not under Rush, `rush` being whether `rush.json` exists, which runs +/// not under Rush, `rush` being [`is_rush`], the gate discovery shares, which runs /// pnpm with settings of its own, as in lockfile discovery's /// `extract_pnpm`), pnpm neither reads nor writes the root or member /// locks, so a Socket-hosted pin in them is no lockfile basis. Discovery diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs index 72aaa9d4a..9d7692ee9 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs @@ -926,6 +926,17 @@ async fn pnpm_lockfile_off_withholds_hosted_pins() { integrity(&entries, "left-pad"), LockIntegrity::Sri("sha512-PATCHED==".into()) ); + + // A directory named rush.json is not Rush (the gate discovery shares, + // `is_rush`), so the root lock still loses the pin. + let tmp = tempfile::tempdir().unwrap(); + write_nested(tmp.path(), "rush.json/placeholder", "").await; + write(tmp.path(), ".npmrc", "lockfile=false\n").await; + write(tmp.path(), "pnpm-lock.yaml", &lock).await; + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); + assert_eq!(integrity(&entries, "left-pad"), LockIntegrity::None); + let (_, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + assert_eq!(integrity(&entries, "left-pad"), LockIntegrity::None); } /// The inventory twin of discovery's From 2b4475e67601971ca4ca3a61240e3cb57a9ffede Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 7 Nov 2026 09:16:34 +0000 Subject: [PATCH 25/36] Read package.json beside a root pnpm lock pair in hosted mode The hosted pnpm planner picks which of a root pnpm-lock.yaml / shrinkwrap.yaml pair to rewrite from the pnpm major the project pins: node_modules/.modules.yaml, else package.json's packageManager. The hosted engine only read package.json beside an npm or yarn lock, so on a fresh clone that pins pnpm 1 or 2 in packageManager (no install record) hosted rewrote the stray pnpm-lock.yaml while pnpm installs from shrinkwrap.yaml. Vendored mode and discovery read the pin from disk and already chose shrinkwrap.yaml. Read the manifest as advisory input whenever both root pnpm locks are present, so all three agree. Addresses Bugbot 4500674838 on #2029. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_011f9aePC6sLFkqRM2XPWq4P --- crates/socket-patch-core/src/hosted/engine.rs | 45 ++++++++++++++++++- 1 file changed, 43 insertions(+), 2 deletions(-) diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index 13ae7e4d4..d41b0412f 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -621,10 +621,17 @@ pub async fn read_candidate_files( // and yarn classic rewriters read it as advisory input only: no // rewriter edits it, so a link or an unreadable in-memory entry is left // out (the rewriters then keep their conservative reading) rather than - // refused. + // refused. Beside a root `pnpm-lock.yaml` / `shrinkwrap.yaml` pair its + // `packageManager` pin decides which of the two pnpm reads + // (`formats::pnpm::plan::planned_locks`), as vendored mode and + // discovery read it, when no `node_modules/.modules.yaml` records the + // installed major (a fresh clone). } else if candidates.iter().any(|c| c.dep.ecosystem == "npm") && (NPM_LOCKS.iter().any(|lock| out.files.contains_key(*lock)) - || out.files.contains_key("yarn.lock")) + || out.files.contains_key("yarn.lock") + || ROOT_PNPM_LOCKS + .iter() + .all(|lock| out.files.contains_key(*lock))) { let rel = crate::hosted::memory::select::NPM_MANIFEST_REL; if let Some(text) = read_advisory(view, unreadable, rel).await { @@ -4332,6 +4339,40 @@ mod tests { ); } + #[tokio::test] + async fn a_pnpm_lock_pair_reads_the_package_manager_pin() { + // pnpm 2 pinned in package.json, no install record: pnpm reads + // shrinkwrap.yaml, so the planner must see the pin to leave the + // stray pnpm-lock.yaml alone. + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + let manifest = r#"{"name":"app","packageManager":"pnpm@2.25.7"}"#; + std::fs::write(root.join("package.json"), manifest).unwrap(); + std::fs::write(root.join("pnpm-lock.yaml"), "lockfileVersion: 5.4\n").unwrap(); + std::fs::write( + root.join("shrinkwrap.yaml"), + "shrinkwrapMinorVersion: 9\nshrinkwrapVersion: 3\n", + ) + .unwrap(); + let candidates = vec![left_pad_candidate()]; + let read = + read_candidate_files(&ProjectView::Disk(root), &BTreeSet::new(), &candidates).await; + assert_eq!( + read.files.get("package.json").map(String::as_str), + Some(manifest) + ); + assert_eq!( + crate::formats::pnpm::plan::planned_locks(&read.files), + vec!["shrinkwrap.yaml"] + ); + + // One pnpm lock alone governs whatever the pin: no manifest read. + std::fs::remove_file(root.join("shrinkwrap.yaml")).unwrap(); + let read = + read_candidate_files(&ProjectView::Disk(root), &BTreeSet::new(), &candidates).await; + assert!(!read.files.contains_key("package.json")); + } + #[tokio::test] async fn issue_490_the_root_manifest_overrides_reach_the_npm_rewriter() { let redirected = |done: &Rewritten| { From f5215a436a547346a4f32f4e9434032697a04b2c Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 9 Nov 2026 01:44:43 +0000 Subject: [PATCH 26/36] Carry main's new pnpm marks onto per-tree UnwiredCopy scopes After #2002 squash-merged, merge its final head and then main into this branch. Main's #2010 added UnwiredCopy marks scoped by file only, while this PR scopes them by (file, TreePart). A user-patched mark now scopes to exactly its ref's tree; a pnpm branch-lock mark covers both trees of the lock, because a branch lock replaces the whole file, its env document included (fail closed, as main intended for every ref of the lock). Co-Authored-By: Claude --- .../socket-patch-core/src/vex/discover/npm.rs | 59 +++++++++++-------- 1 file changed, 34 insertions(+), 25 deletions(-) diff --git a/crates/socket-patch-core/src/vex/discover/npm.rs b/crates/socket-patch-core/src/vex/discover/npm.rs index 3e37aa2b5..9ce85bdfa 100644 --- a/crates/socket-patch-core/src/vex/discover/npm.rs +++ b/crates/socket-patch-core/src/vex/discover/npm.rs @@ -340,7 +340,7 @@ pub(super) fn mark_user_patched( return; }; out.unwired_copy(UnwiredCopy { - scope: Some(r.source_file.clone()), + scope: Some((r.source_file.clone(), r.part)), target: CopyTarget::Ref { purl: r.purl.clone(), uuid: r.uuid.clone(), @@ -865,19 +865,26 @@ async fn extract_pnpm(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { /// be listed (`why`), so a member's `pnpm-lock..yaml` cannot be /// ruled out. See [`unattest_git_branch_lock`]. fn unattest_unlisted_branch_locks(file: &str, why: &str, out: &mut Discovery) { - out.unwired_copy(UnwiredCopy { - scope: Some(PathBuf::from(file)), - target: CopyTarget::Any, - file: PathBuf::from(file), - detail: format!( - "its pin is in {file}, but a pnpm branch lock cannot be ruled out: {why}; with \ - `gitBranchLockfile` on, pnpm installs a git branch from its own \ - `pnpm-lock..yaml` rather than {file}" - ), - kind: UnattestedKind::PnpmGitBranchLock, - }); + for part in BRANCH_LOCK_PARTS { + out.unwired_copy(UnwiredCopy { + scope: Some((PathBuf::from(file), part)), + target: CopyTarget::Any, + file: PathBuf::from(file), + detail: format!( + "its pin is in {file}, but a pnpm branch lock cannot be ruled out: {why}; with \ + `gitBranchLockfile` on, pnpm installs a git branch from its own \ + `pnpm-lock..yaml` rather than {file}" + ), + kind: UnattestedKind::PnpmGitBranchLock, + }); + } } +/// Every install tree of a pnpm lock: a branch lock replaces the whole +/// file, its env lockfile document included, so a branch-lock mark covers +/// the refs of both documents. +const BRANCH_LOCK_PARTS: [TreePart; 2] = [TreePart::File, TreePart::PnpmEnv]; + /// Mark every ref of the pnpm lock `file` [`UnattestedKind::PnpmGitBranchLock`] /// because `branch_locks` (root-relative `pnpm-lock..yaml` files) /// exist: with `gitBranchLockfile` on pnpm installs a branch from its own @@ -896,19 +903,21 @@ fn unattest_git_branch_lock(file: &str, branch_locks: &[String], out: &mut Disco .find(|lock| Path::new(lock.as_str()).parent() == Some(dir)) .or(branch_locks.first()) .map_or(file, String::as_str); - out.unwired_copy(UnwiredCopy { - scope: Some(PathBuf::from(file)), - target: CopyTarget::Any, - file: PathBuf::from(shown), - detail: format!( - "its pin is in {file} while the pnpm branch lock(s) {} exist: with \ - `gitBranchLockfile` on, pnpm installs a git branch from its own \ - `pnpm-lock..yaml` rather than {file}, and a branch lock's presence \ - withholds the statement whatever the setting says", - branch_locks.join(", ") - ), - kind: UnattestedKind::PnpmGitBranchLock, - }); + for part in BRANCH_LOCK_PARTS { + out.unwired_copy(UnwiredCopy { + scope: Some((PathBuf::from(file), part)), + target: CopyTarget::Any, + file: PathBuf::from(shown), + detail: format!( + "its pin is in {file} while the pnpm branch lock(s) {} exist: with \ + `gitBranchLockfile` on, pnpm installs a git branch from its own \ + `pnpm-lock..yaml` rather than {file}, and a branch lock's presence \ + withholds the statement whatever the setting says", + branch_locks.join(", ") + ), + kind: UnattestedKind::PnpmGitBranchLock, + }); + } } /// One pnpm lock at root-relative `file` (see [`extract_pnpm`]). From 622e9c92246902ad9b7bb6041e5baced6ea710d2 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 9 Nov 2026 03:37:37 +0000 Subject: [PATCH 27/36] Drop a pnpm env twin when its project ref is withheld drop_env_twins compared refs only within one vec, so when the project document's ref moved to shadowed or rewirable (a file: copy or another lock contesting it), its env-document twin with the same (file, purl, uuid, mode) stayed in refs and was attested in the project ref's place. Build the project set across refs, shadowed and rewirable before dropping twins from each. Co-Authored-By: Claude --- .../socket-patch-core/src/vex/discover/mod.rs | 54 ++++++++++--------- .../socket-patch-core/src/vex/discover/npm.rs | 32 +++++++++++ 2 files changed, 62 insertions(+), 24 deletions(-) diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 576147ba1..95d2df5e3 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -1431,29 +1431,35 @@ impl Discovery { /// file's project document also wires (the same `(purl, uuid, mode)`): /// one wiring of the package in that file, whose project instance is /// the one an installed `node_modules` copy is verified against, even - /// when the two pins differ. - fn drop_env_twins(refs: &mut Vec) { - let project: HashSet<(PathBuf, String, String, WiringMode)> = refs - .iter() - .filter(|r| r.part == TreePart::File) - .map(|r| { - ( - r.source_file.clone(), - r.purl.clone(), - r.uuid.clone(), - r.mode, - ) - }) - .collect(); - refs.retain(|r| { - r.part == TreePart::File - || !project.contains(&( - r.source_file.clone(), - r.purl.clone(), - r.uuid.clone(), - r.mode, - )) - }); + /// when the two pins differ. The project twin counts wherever it ended + /// up (`refs`, `shadowed` or `rewirable`), so a withheld project ref + /// never leaves its env twin attested in its place. + fn drop_env_twins(&mut self) { + let project: HashSet<(PathBuf, String, String, WiringMode)> = + [&self.refs, &self.shadowed, &self.rewirable] + .into_iter() + .flatten() + .filter(|r| r.part == TreePart::File) + .map(|r| { + ( + r.source_file.clone(), + r.purl.clone(), + r.uuid.clone(), + r.mode, + ) + }) + .collect(); + for refs in [&mut self.refs, &mut self.shadowed, &mut self.rewirable] { + refs.retain(|r| { + r.part == TreePart::File + || !project.contains(&( + r.source_file.clone(), + r.purl.clone(), + r.uuid.clone(), + r.mode, + )) + }); + } } /// The install tree `file` belongs to: its tree's root file, or itself. @@ -1621,8 +1627,8 @@ impl Discovery { self.unattested.dedup(); self.contested.sort(); self.contested.dedup(); + self.drop_env_twins(); for refs in [&mut self.refs, &mut self.shadowed, &mut self.rewirable] { - Self::drop_env_twins(refs); refs.sort_by(|a, b| { (&a.source_file, &a.purl, &a.uuid, a.mode, a.part).cmp(&( &b.source_file, diff --git a/crates/socket-patch-core/src/vex/discover/npm.rs b/crates/socket-patch-core/src/vex/discover/npm.rs index 9ce85bdfa..396f143bb 100644 --- a/crates/socket-patch-core/src/vex/discover/npm.rs +++ b/crates/socket-patch-core/src/vex/discover/npm.rs @@ -3434,6 +3434,38 @@ mod tests { Some(LockIntegrity::Sri(SRI.into())) ); + // Wired in both documents, the project's ref withheld (a `file:` + // copy of the release installs beside it): its env twin is dropped + // too, never attested in the project ref's place. + let p = Project::new(); + p.write( + "pnpm-lock.yaml", + format!( + "{}{}", + env(&env_wired), + project(&format!("{wired}{file_copy}")) + ), + ); + p.write( + "forks/left-pad/package.json", + r#"{"name":"left-pad","version":"1.3.0"}"#, + ); + let out = p.discover().await; + assert!(out.refs.is_empty(), "{:#?}", out.refs); + assert!( + out.shadowed + .iter() + .chain(&out.rewirable) + .all(|r| r.part == TreePart::File), + "{:#?} {:#?}", + out.shadowed, + out.rewirable + ); + assert_ne!( + out.hosted_claim("pkg:npm/left-pad@1.3.0", UUID_A), + Some(true) + ); + // The env document's registry entry contests no other lock's ref. let npm_lock = format!( r#"{{"name":"m","lockfileVersion":3,"packages":{{"":{{"name":"m"}}, From 868d135df220e337e80e3b33f9291a33f70c7e63 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Mon, 9 Nov 2026 15:21:55 -0500 Subject: [PATCH 28/36] Withhold a pnpm env ref the project document resolves elsewhere Owner decision on #2009: a config-dependency (env lockfile document) ref is not attested while the project document of the same lock resolves the same name@version from elsewhere. A VEX statement covers the release, not one install tree, so attesting it would claim the release is fixed while node_modules holds upstream. TreePart::withheld_by makes the project tree's copies reach env refs of the same file, through the existing machinery: - a registry entry moves the env ref to `rewirable` (a re-run rewires that entry, and the project ref then replaces the env twin); - a `file:` copy (and any project-tree unpatched copy) shadows it; - a bundled copy marks it Unattested (BundledCopy), like a project ref. The branch-lock marks no longer need a separate env-scoped copy: the project-scoped one now covers env refs. pnpm_env_document_is_its_own_ install_tree's first loop now expects withholding, with registry-only, file:-only and bundled-only cases plus a different-version control. Co-Authored-By: Claude Opus 5.5 --- crates/socket-patch-cli/CLI_CONTRACT.md | 2 +- .../socket-patch-core/src/vex/discover/mod.rs | 128 +++++++--- .../socket-patch-core/src/vex/discover/npm.rs | 220 ++++++++++++------ 3 files changed, 248 insertions(+), 102 deletions(-) diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 75b31e8a3..97eb0e82e 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -528,7 +528,7 @@ Discovery is read-only, never touches the network, and never fails the run: a ma | Ecosystem | Files read | Hosted reference | Vendored reference | Hosted pin (`integrity_required`) | |---|---|---|---|---| | npm | `package-lock.json` and `npm-shrinkwrap.json` (both when both exist) | `resolved` on the patch host (`packages` in v2/v3; `dependencies` only in v1; `link` entries skipped, and so are `inBundle` / `bundled` entries a dependency bundles (a copy the root project's own `bundleDependencies` covers, and the dependencies npm installs beneath it, has no parent tarball and is rewired like any entry, #1618; a copy beneath a root-bundled package that declares its own bundle stays skipped, since the lock cannot tell which bundle carries it), and so is any entry npm installs from a git, URL or `file:` spec or from a dependency's own `npm-shrinkwrap.json` (beneath a `hasShrinkwrap: true` package, which npm 7–11 install from that shrinkwrap; `redirect_npm_shrinkwrapped_instance_skipped` / `vendor_shrinkwrapped_instance_skipped`), together with every ref for the same `name@version`) | `resolved: file:.socket/vendor/npm//-.tgz` | `integrity`, required | -| pnpm | `pnpm-lock.yaml` (every `lockfileVersion`); `shrinkwrap.yaml` only when there is no `pnpm-lock.yaml`; with `rush.json`, `common/config/rush/pnpm-lock.yaml` + `common/config/subspaces/*/pnpm-lock.yaml` | `packages:` `resolution.tarball` on the patch host | `file:.socket/vendor/npm/…` tarball + key | `integrity`, required. With pnpm's `lockfile` setting off (#1074, outside Rush) pnpm reads no lock: every hosted reference in the root and member locks keeps no pin, and one `patched_ref_unattributable` per lock names them; vendored references are unaffected. | +| pnpm | `pnpm-lock.yaml` (every `lockfileVersion`); `shrinkwrap.yaml` only when there is no `pnpm-lock.yaml`; with `rush.json`, `common/config/rush/pnpm-lock.yaml` + `common/config/subspaces/*/pnpm-lock.yaml` | `packages:` `resolution.tarball` on the patch host | `file:.socket/vendor/npm/…` tarball + key | `integrity`, required. With pnpm's `lockfile` setting off (#1074, outside Rush) pnpm reads no lock: every hosted reference in the root and member locks keeps no pin, and one `patched_ref_unattributable` per lock names them; vendored references are unaffected. pnpm 11+'s env lockfile document (config dependencies, the pinned package manager) is read as its own install tree: its references neither contest nor are contested by another lock, and its entries never contest the project document's references. One of its references is never attested while the project document of the same lock resolves the same `name@version` from elsewhere, because a VEX statement covers the release, not one install tree: a registry entry or a `file:` copy withholds it with `patched_ref_unattributable` (re-running `scan` rewires a registry entry), and a bundled copy omits it like any pnpm bundled copy (`vex_pnpm_bundled_copy`). | | yarn | `yarn.lock` (classic and berry) | classic `resolved`; berry entry keyed and resolved `@` + root `package.json` `resolutions` `"@npm:": ""` (older releases: `resolution: …::__archiveUrl=`) | classic `resolved "file:./.socket/vendor/npm/…#"`; berry `file:` entry **plus** a root `package.json` `resolutions` mapping onto the same artifact (without it the entry is orphaned: diagnosed, no ref) | classic `integrity` / `#sha1`, berry `checksum`, required | | bun | `bun.lock`; `bun.lockb` only when there is no `bun.lock` (bun reads exactly one) | URL tuple / binary remote-tarball resolution; version from the URL leaf. A user URL or `file:` tarball tuple / record whose `-.tgz` leaf names the same `name@version` is installed from its own spec, so every ref for that `name@version` is dropped | `.socket/vendor/npm//-.tgz` tuple / local-tarball resolution | `sha512-…`, required. A 2-tuple that Bun < 1.3.10 re-saved without its digest is still a reference, but it attests only from an installed tree. | | vlt | `vlt-lock.json` (lockfileVersion absent, `0` or `1`; a BOM-prefixed, unparseable, non-object or other-version lock is not read: diagnosed, no ref). `vlt.json` (read only for its `modifiers`) and `node_modules/.vlt-lock.json` are never wiring. | a registry node (any segment) whose slot [3] is a `/patch/npm/…` URL on the patch host with the leaf `-.tgz` of its DepID's `name@version`, whose embedded `/` path (when present) is that `name@version`, and slot [1] == name; version from the DepID | a `file` node `.socket/vendor/npm//-/node_modules/` (or a user-installed `-.tgz`) with slot [1] == name; version from the path. A same-`name@version` registry node, or a diagnosed Socket-shaped one, beside it is diagnosed, no ref. | slot [2] `sha512-…`, required (a hosted node without one is no reference). A same-`name@version` node on another registry, or a diagnosed Socket-shaped one, keeps the reference but withholds the lockfile basis: only an installed tree whose every store copy verifies attests. So does a lock some vlt release discards, the conditions of `redirect_vlt_lockfile_version_missing`, `redirect_vlt_old_lockfile_ignored` and `redirect_vlt_scalar_registry_ignored` (which in-run `--vex` withholds too): every hosted reference in it keeps no pin, and one `patched_ref_unattributable` names them. | diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 95d2df5e3..e0c737bda 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -259,16 +259,41 @@ pub enum TreePart { /// pnpm 11+'s env lockfile document (config dependencies, the pinned /// package manager), which installs apart from the project's /// `node_modules`: its own tree (owner decision, lock-model divergence - /// N4). Its refs and the project document's entries of the same lock - /// never contest each other ([`Discovery::contest_within_locks`], - /// [`UnwiredCopy`] scopes), its wiring never exempts the project - /// document in [`Discovery::contest_across_locks`], and that contest - /// skips it both ways: no other lock decides a config dependency, and a - /// config dependency decides no other lock's ref. Within the document - /// the same-lock rules apply. + /// N4). Its wiring never exempts the project document in + /// [`Discovery::contest_across_locks`], and that contest skips it both + /// ways: no other lock decides a config dependency, and a config + /// dependency decides no other lock's ref. Within the document the + /// same-lock rules apply. Its entries never contest the project + /// document's refs, but the project document's copies DO withhold its + /// refs ([`TreePart::withheld_by`], owner decision on #2009). PnpmEnv, } +impl TreePart { + /// Whether a copy of a package installed in tree `copy` of a ref's own + /// file withholds that ref (`self`): a copy in the same tree does, and + /// so does a pnpm lock's project document copy for a ref in the env + /// document of the same lock. A VEX statement covers the release, not + /// one install tree, so attesting a config dependency's wiring while + /// the project's `node_modules` installs the same `name@version` from + /// the registry, a `file:` copy or a bundled copy would claim the + /// release is fixed where it is not (owner decision on #2009). The + /// reverse does not hold: the env document's copies install into + /// pnpm's config store only and withhold no project ref. + pub(crate) fn withheld_by(self, copy: TreePart) -> bool { + self == copy || (self == TreePart::PnpmEnv && copy == TreePart::File) + } + + /// How diagnostics name this tree of root-relative `file`: + /// ` (env lockfile)` for the env document, else `file`. + pub(crate) fn label(self, file: &str) -> String { + match self { + TreePart::File => file.to_string(), + TreePart::PnpmEnv => format!("{file} (env lockfile)"), + } + } +} + impl From for TreePart { fn from(document: crate::formats::pnpm::PnpmDocument) -> Self { match document { @@ -766,8 +791,8 @@ pub enum CopyTarget { /// extractor has run ([`Discovery::unattest_unwired_copies`]). #[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)] pub struct UnwiredCopy { - /// Only refs of this root-relative file, in this tree of it; `None` = - /// refs of any file. + /// Only refs of this root-relative file, in a tree this tree of it + /// withholds ([`TreePart::withheld_by`]); `None` = refs of any file. pub(crate) scope: Option<(PathBuf, TreePart)>, pub(crate) target: CopyTarget, /// Root-relative file that shows the copy. @@ -782,7 +807,7 @@ impl UnwiredCopy { if self .scope .as_ref() - .is_some_and(|(f, part)| *f != r.source_file || *part != r.part) + .is_some_and(|(f, part)| *f != r.source_file || !r.part.withheld_by(*part)) { return false; } @@ -847,8 +872,10 @@ pub struct Discovery { /// another entry of the same lock (an `npm:` alias added after the /// pin), the npm twin lock, another manager's lock /// ([`Discovery::contest_across_locks`]), npm 6's legacy mirror, a Bun - /// registry entry. Each is diagnosed [`DIAG_REF_UNATTRIBUTABLE`] and is - /// neither attested nor unwound by the management commands, but it is + /// registry entry, the project document's registry entry beside a pnpm + /// env-lockfile ref ([`Discovery::contest_within_locks`]). Each is + /// diagnosed [`DIAG_REF_UNATTRIBUTABLE`] and is neither attested nor + /// unwound by the management commands, but it is /// still a pin a live lock records: the rollout's recorded view counts /// it ([`HostedPin::recorded`]), so the re-scan that rewires the other /// copy is not capped as NEW (#1195). Validated and sorted like `refs`. @@ -1238,9 +1265,10 @@ impl Discovery { let purls: BTreeSet<&str> = self .refs .iter() - // A file-level copy: no extractor records one in a pnpm - // env lockfile, whose refs are a tree of their own. - .filter(|r| r.source_file == u.file && r.part == TreePart::File) + // A file-level copy, in the project tree: it withholds a + // pnpm env-lockfile ref of the same file too + // ([`TreePart::withheld_by`]). + .filter(|r| r.source_file == u.file && r.part.withheld_by(TreePart::File)) .filter(|r| match &u.version { Some(v) => r.purl.rsplit_once('@').is_some_and(|(_, rv)| rv == v), None => true, @@ -1272,9 +1300,16 @@ impl Discovery { /// that copy whatever the wiring does, so the ref is diagnosed /// ([`DIAG_REF_UNATTRIBUTABLE`], naming the entry) and not emitted, but /// [`Discovery::shadowed`]. Its uuid stays recognized (rule 11). + /// + /// A pnpm env-lockfile ref ([`TreePart::PnpmEnv`]) is withheld by the + /// project document's copies too ([`TreePart::withheld_by`]): a `file:` + /// copy shadows it as above, and a registry entry of the same release + /// moves it to [`Discovery::rewirable`] (a re-run rewires that entry, + /// and the project ref then stands in its place, see + /// [`Discovery::drop_env_twins`]). fn contest_within_locks(&mut self) { self.expand_unidentified_copies(); - if self.unpatched_copies.is_empty() { + if self.unpatched_copies.is_empty() && self.refs.iter().all(|r| r.part == TreePart::File) { return; } let refs = std::mem::take(&mut self.refs); @@ -1282,26 +1317,51 @@ impl Discovery { let copy = self .unpatched_copies .iter() - .find(|c| c.purl == r.purl && c.file == r.source_file && c.part == r.part) + .find(|c| c.purl == r.purl && c.file == r.source_file && r.part.withheld_by(c.part)) .cloned(); - match copy { - Some(c) => { - let file = r.source_file.to_string_lossy().into_owned(); - self.diag( - DIAG_REF_UNATTRIBUTABLE, - &file, - format!( - "{file}: {} is wired to Socket patch {} but lock entry `{}` {}; \ - that copy stays UNPATCHED and nothing is attested", - r.purl, r.uuid, c.key, c.how - ), - ); - // Withheld, not lost: rollback / remove / the takeover - // still unwind the wiring (#828). - self.shadow(r); - } - None => self.refs.push(r), + let file = r.source_file.to_string_lossy().into_owned(); + let label = r.part.label(&file); + if let Some(c) = copy { + let entry = if c.part == r.part { + "" + } else { + "the project document's " + }; + self.diag( + DIAG_REF_UNATTRIBUTABLE, + &file, + format!( + "{label}: {} is wired to Socket patch {} but {entry}lock entry `{}` {}; \ + that copy stays UNPATCHED and nothing is attested", + r.purl, r.uuid, c.key, c.how + ), + ); + // Withheld, not lost: rollback / remove / the takeover + // still unwind the wiring (#828). + self.shadow(r); + continue; + } + let registry = r.part == TreePart::PnpmEnv + && self.elsewhere.iter().any(|e| { + e.file == r.source_file && e.part == TreePart::File && e.covers(&r.purl) + }); + if registry { + self.diag( + DIAG_REF_UNATTRIBUTABLE, + &file, + format!( + "{label}: {} is wired to Socket patch {}, but the project document of \ + the same lock resolves that version from elsewhere (not a Socket \ + patch); a VEX statement covers the release, not one install tree, so \ + the patch is not attested while `node_modules` installs the unpatched \ + copy — re-run `socket-patch scan` / `vendor` to rewire it too", + r.purl, r.uuid + ), + ); + self.withhold_rewirable(r); + continue; } + self.refs.push(r); } } diff --git a/crates/socket-patch-core/src/vex/discover/npm.rs b/crates/socket-patch-core/src/vex/discover/npm.rs index ef8f2daa0..0bedf0ce4 100644 --- a/crates/socket-patch-core/src/vex/discover/npm.rs +++ b/crates/socket-patch-core/src/vex/discover/npm.rs @@ -741,7 +741,11 @@ fn entry_ref( /// (config dependencies, the pinned package manager; identified by its /// importers, [`PnpmLock::packages`]) is an install tree of its own /// ([`TreePart::PnpmEnv`]): its refs are emitted, and they never contest -/// or are contested by the project document or another lock. +/// or are contested by another lock, nor contest the project document's. +/// The project document's registry entry, `file:` copy or bundled copy of +/// the same `name@version` withholds an env ref, though (owner decision on +/// #2009, [`TreePart::withheld_by`]): a VEX statement covers the release, +/// not one install tree. /// /// With pnpm's `gitBranchLockfile` on and a `pnpm-lock..yaml` /// present (#556), every ref of the root and member locks is marked @@ -874,26 +878,19 @@ async fn extract_pnpm(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { /// be listed (`why`), so a member's `pnpm-lock..yaml` cannot be /// ruled out. See [`unattest_git_branch_lock`]. fn unattest_unlisted_branch_locks(file: &str, why: &str, out: &mut Discovery) { - for part in BRANCH_LOCK_PARTS { - out.unwired_copy(UnwiredCopy { - scope: Some((PathBuf::from(file), part)), - target: CopyTarget::Any, - file: PathBuf::from(file), - detail: format!( - "its pin is in {file}, but a pnpm branch lock cannot be ruled out: {why}; with \ - `gitBranchLockfile` on, pnpm installs a git branch from its own \ - `pnpm-lock..yaml` rather than {file}" - ), - kind: UnattestedKind::PnpmGitBranchLock, - }); - } + out.unwired_copy(UnwiredCopy { + scope: Some((PathBuf::from(file), TreePart::File)), + target: CopyTarget::Any, + file: PathBuf::from(file), + detail: format!( + "its pin is in {file}, but a pnpm branch lock cannot be ruled out: {why}; with \ + `gitBranchLockfile` on, pnpm installs a git branch from its own \ + `pnpm-lock..yaml` rather than {file}" + ), + kind: UnattestedKind::PnpmGitBranchLock, + }); } -/// Every install tree of a pnpm lock: a branch lock replaces the whole -/// file, its env lockfile document included, so a branch-lock mark covers -/// the refs of both documents. -const BRANCH_LOCK_PARTS: [TreePart; 2] = [TreePart::File, TreePart::PnpmEnv]; - /// Mark every ref of the pnpm lock `file` [`UnattestedKind::PnpmGitBranchLock`] /// because `branch_locks` (root-relative `pnpm-lock..yaml` files) /// exist: with `gitBranchLockfile` on pnpm installs a branch from its own @@ -904,7 +901,9 @@ const BRANCH_LOCK_PARTS: [TreePart; 2] = [TreePart::File, TreePart::PnpmEnv]; /// the wiring in `file` is intact (rollback, remove and list still manage /// it, and the ledgers keep it live), and no `scan` / `vendor` re-run can /// rewire the branch lock. The mark names the branch lock in `file`'s own -/// directory when there is one. +/// directory when there is one. A branch lock replaces the whole file, so +/// the mark is the project tree's, which covers the env lockfile +/// document's refs too ([`TreePart::withheld_by`]). fn unattest_git_branch_lock(file: &str, branch_locks: &[String], out: &mut Discovery) { let dir = Path::new(file).parent().unwrap_or(Path::new("")); let shown = branch_locks @@ -912,21 +911,19 @@ fn unattest_git_branch_lock(file: &str, branch_locks: &[String], out: &mut Disco .find(|lock| Path::new(lock.as_str()).parent() == Some(dir)) .or(branch_locks.first()) .map_or(file, String::as_str); - for part in BRANCH_LOCK_PARTS { - out.unwired_copy(UnwiredCopy { - scope: Some((PathBuf::from(file), part)), - target: CopyTarget::Any, - file: PathBuf::from(shown), - detail: format!( - "its pin is in {file} while the pnpm branch lock(s) {} exist: with \ - `gitBranchLockfile` on, pnpm installs a git branch from its own \ - `pnpm-lock..yaml` rather than {file}, and a branch lock's presence \ - withholds the statement whatever the setting says", - branch_locks.join(", ") - ), - kind: UnattestedKind::PnpmGitBranchLock, - }); - } + out.unwired_copy(UnwiredCopy { + scope: Some((PathBuf::from(file), TreePart::File)), + target: CopyTarget::Any, + file: PathBuf::from(shown), + detail: format!( + "its pin is in {file} while the pnpm branch lock(s) {} exist: with \ + `gitBranchLockfile` on, pnpm installs a git branch from its own \ + `pnpm-lock..yaml` rather than {file}, and a branch lock's presence \ + withholds the statement whatever the setting says", + branch_locks.join(", ") + ), + kind: UnattestedKind::PnpmGitBranchLock, + }); } /// One pnpm lock at root-relative `file` (see [`extract_pnpm`]). @@ -953,8 +950,9 @@ async fn extract_pnpm_lock( return; } // Each document is its own install tree: pnpm 11+'s env lockfile is - // read as [`TreePart::PnpmEnv`], so its refs and the project document's - // entries never contest each other. + // read as [`TreePart::PnpmEnv`], so its entries never contest the + // project document's refs (the project document's copies do withhold + // its refs, [`TreePart::withheld_by`]). for document in lock.documents() { let tree = PnpmTree::of(file, document); extract_pnpm_document(ctx, &tree, &lock, document, disabled, out).await; @@ -973,11 +971,11 @@ struct PnpmTree<'a> { impl<'a> PnpmTree<'a> { fn of(file: &'a str, document: PnpmDocument) -> Self { let part = TreePart::from(document); - let label = match part { - TreePart::File => file.to_string(), - TreePart::PnpmEnv => format!("{file} (env lockfile)"), - }; - PnpmTree { file, part, label } + PnpmTree { + file, + part, + label: part.label(file), + } } } @@ -3411,9 +3409,12 @@ mod tests { /// pnpm 11+'s env lockfile document is read as an install tree of its /// own (owner decision, lock-model divergence N4). Before, discovery /// skipped it (#1949), so a Socket wiring of a config dependency was - /// no ref although the hosted planner writes it. Now it is a ref; the - /// project document's registry entry, `file:` copy and bundled copy of - /// the same release never contest it (and the reverse); its wiring does + /// no ref although the hosted planner writes it. Now it is a ref, but + /// one the project document of the same lock withholds when it resolves + /// the same release from the registry (rewirable), a `file:` copy + /// (shadowed) or a bundled copy (unattested): a VEX statement covers the + /// release, not one tree (owner decision on #2009). The env document's + /// entries never contest the project document's refs; its wiring does /// not exempt the project document from contesting another lock; and /// it contests no other lock's ref. #[tokio::test] @@ -3440,12 +3441,31 @@ mod tests { let bundle = " host-pkg@1.0.0:\n resolution: {integrity: sha512-HOST==}\n \ bundledDependencies: true\n\n"; - // Wired in the env document, the project resolving the same release - // from the registry, from a `file:` copy, or bundling it: a ref. - for (case, project_packages) in [ - ("registry", registry.to_string()), - ("file: copy", format!("{registry}{file_copy}")), - ("bundled copy", format!("{registry}{bundle}")), + // Wired in the env document, the project document resolving the + // same release elsewhere: never attested. A registry entry a + // re-run can rewire withholds it as rewirable, a `file:` copy + // shadows it, and a bundled copy (whose version the lock does not + // record) leaves it a ref marked unattested. + #[derive(Debug, PartialEq)] + enum Withheld { + Rewirable, + Shadowed, + Unattested, + } + for (case, project_packages, how) in [ + ("registry", registry.to_string(), Withheld::Rewirable), + ("file: copy", file_copy.to_string(), Withheld::Shadowed), + ( + "registry + file: copy", + format!("{registry}{file_copy}"), + Withheld::Shadowed, + ), + ("bundled copy", bundle.to_string(), Withheld::Unattested), + ( + "registry + bundled copy", + format!("{registry}{bundle}"), + Withheld::Rewirable, + ), ] { for crlf in [false, true] { let p = Project::new(); @@ -3461,25 +3481,85 @@ mod tests { r#"{"name":"left-pad","version":"1.3.0"}"#, ); let out = p.discover().await; - assert_refs(&out, &hosted); + let case = format!("{case} (crlf: {crlf})"); + let withheld = match how { + Withheld::Rewirable => &out.rewirable, + Withheld::Shadowed => &out.shadowed, + Withheld::Unattested => &out.refs, + }; + assert_eq!(withheld.len(), 1, "{case}: {out:#?}"); // The real lock path, with the tree carried beside it // (code-review finding 6: no pseudo path). assert_eq!( - out.refs[0].source_file, - std::path::Path::new("pnpm-lock.yaml") - ); - assert_eq!(out.refs[0].part, TreePart::PnpmEnv, "{case}"); - assert!(out.unattested.is_empty(), "{case}: {:#?}", out.unattested); - assert!(out.shadowed.is_empty(), "{case}: {:#?}", out.shadowed); - assert!(out.contested.is_empty(), "{case}: {:#?}", out.contested); - assert_eq!( - out.hosted_claim("pkg:npm/left-pad@1.3.0", UUID_A), - Some(true), + ( + withheld[0].purl.as_str(), + withheld[0].source_file.as_path(), + withheld[0].part + ), + ( + "pkg:npm/left-pad@1.3.0", + std::path::Path::new("pnpm-lock.yaml"), + TreePart::PnpmEnv + ), "{case}" ); + assert!(out.contested.is_empty(), "{case}: {:#?}", out.contested); + if how == Withheld::Unattested { + // The ref stays (no re-run clears a bundled copy), but + // the VEX plan omits it. + assert_eq!(out.unattested.len(), 1, "{case}: {:#?}", out.unattested); + let u = &out.unattested[0]; + assert_eq!( + (u.purl.as_str(), u.uuid.as_str(), u.kind), + ( + "pkg:npm/left-pad@1.3.0", + UUID_A, + UnattestedKind::BundledCopy + ), + "{case}" + ); + } else { + assert!(out.refs.is_empty(), "{case}: {:#?}", out.refs); + assert_eq!( + out.hosted_claim("pkg:npm/left-pad@1.3.0", UUID_A), + Some(false), + "{case}" + ); + let diag = out + .diagnostics + .iter() + .find(|d| d.code == DIAG_REF_UNATTRIBUTABLE) + .unwrap_or_else(|| panic!("{case}: {:#?}", out.diagnostics)); + assert!( + diag.detail.starts_with("pnpm-lock.yaml (env lockfile): ") + && diag.detail.contains("project document"), + "{case}: {}", + diag.detail + ); + } } } + // Wired in the env document, the project document resolving + // another version from the registry: a ref, attested. + let p = Project::new(); + p.write( + "pnpm-lock.yaml", + format!( + "{}{}", + env(&wired), + project(®istry.replace("1.3.0", "1.2.0")) + ), + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert_eq!(out.refs[0].part, TreePart::PnpmEnv); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + assert_eq!( + out.hosted_claim("pkg:npm/left-pad@1.3.0", UUID_A), + Some(true) + ); + // Wired in the project document, the env document resolving the // same release from the registry: a ref. let p = Project::new(); @@ -3557,8 +3637,8 @@ mod tests { assert!(out.contested.is_empty(), "{:#?}", out.contested); // The env document's wiring does not exempt the project document, - // whose registry entry still contests package-lock.json's ref; the - // env document's own ref stands. + // whose registry entry still contests package-lock.json's ref (and + // withholds the env document's own ref). let p = Project::new(); p.write("package-lock.json", &npm_lock); p.write( @@ -3566,10 +3646,16 @@ mod tests { format!("{}{}", env(&wired), project(registry)), ); let out = p.discover().await; - assert_refs(&out, &hosted); + assert!(out.refs.is_empty(), "{:#?}", out.refs); assert_eq!( - out.refs[0].source_file, - std::path::Path::new("pnpm-lock.yaml") + out.rewirable + .iter() + .map(|r| (r.source_file.as_path(), r.part)) + .collect::>(), + [ + (std::path::Path::new("package-lock.json"), TreePart::File), + (std::path::Path::new("pnpm-lock.yaml"), TreePart::PnpmEnv), + ] ); assert_eq!(out.contested.len(), 1, "{:#?}", out.contested); let c = &out.contested[0]; From a85794538f64326ee678c430fbc6d0be5a24eaaf Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Mon, 9 Nov 2026 15:22:02 -0500 Subject: [PATCH 29/36] Note the env-ref withholding in the N4 divergence row Co-Authored-By: Claude Opus 5.5 --- docs/design/lock-model-divergences.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md index f6cb67c08..6e5648482 100644 --- a/docs/design/lock-model-divergences.md +++ b/docs/design/lock-model-divergences.md @@ -84,7 +84,7 @@ follow from the install-tree decision. N11 was decided on 2026-11-05. | N1 | pnpm `gitBranchLockfile` with a branch lock present: is the root lock the install tree? | Hosted withholds the root locks and refuses (`hosted/engine.rs` ~930). Vendored refuses. Discovery and the inventory never check the setting. | Hosted. Discovery withholds pnpm root and member-lock attestations whenever any `pnpm-lock..yaml` exists, whatever the settings or environment say (owner, 2026-11-05; #2010 (draft)). | false attestation | | N2 | npm user patches (#711) and `bun patch` (#367) | Hosted skips that one dependency and records it as refused. Vendored refuses the whole npm v4 lock, or the one bun dependency. Discovery never reads `patchedDependencies` / `"patched"` and attests a pre-#711 hosted pin. | Hosted's per-dependency skip, in discovery too, for every package manager's user patches (owner decision; #2010 (draft) for npm and bun, #2018 (draft) for pnpm, yarn and vlt). The override is the existing `--allow-patch-conflicts`; `vex` never attests under a user patch, even with it. | false attestation | | N3 | A workspace member's stray npm/bun/vlt lock | Discovery treats it as absent. Hosted and vendored refuse. The inventory reads it as the project's lock. | The inventory ignores it | | -| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by content, not position. **Resolved in PR 2a (#2009)**: every reader reads both documents through `PnpmLock`, each entry tagged by document; discovery records carry the env document's install tree (`TreePart::PnpmEnv`) beside the real lock path, and that tree neither contests nor is contested by the project document or another lock; the env document is the FIRST document when its importers carry only `configDependencies` / `packageManagerDependencies` (pnpm's own position rule), and every other document is read as the project. | | +| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by content, not position. **Resolved in PR 2a (#2009)**: every reader reads both documents through `PnpmLock`, each entry tagged by document; discovery records carry the env document's install tree (`TreePart::PnpmEnv`) beside the real lock path, and that tree neither contests nor is contested by another lock, and never contests the project document; the project document's registry entry, `file:` copy or bundled copy of the same release does withhold an env ref (owner decision on #2009: a VEX statement covers the release, not one tree); the env document is the FIRST document when its importers carry only `configDependencies` / `packageManagerDependencies` (pnpm's own position rule), and every other document is read as the project. | | | N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere. The vendored writers, reverters and GC claim only this project's vendor paths in #2025 (draft). pnpm part **resolved in PR 2a (#2009)**: the hosted pnpm "vendored?" check reads the rekeyed packages entry with `origin::vendor_ref`; the planner follows in #2029 (draft). | revert could rewrite another project's wiring | | N6 | `npm-shrinkwrap.json` and `package-lock.json` | Hosted, vendored and discovery read both, with a contest. The inventory reads the shrinkwrap only. | Both, everywhere | | | N7 | npm lockfileVersion | Hosted rewrites v1 and v4. Vendored accepts v2/v3 only. The inventory ignores v1. Discovery reads v1 and v2+. | Hosted, once vendored `file:` wiring under npm 6 and npm 12.1 v4 has fixtures | | From a673e1b75d9ac7c61130ec40198be751e3c39de6 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Mon, 9 Nov 2026 15:26:03 -0500 Subject: [PATCH 30/36] Name a project copy that withholds a pnpm config-dependency pin The hosted gate's copy check compared the pin's tree with the copy's, so a config-dependency (env document) pin withheld by the project document's `file:` copy was neither named with redirect_copy_unpatched nor kept out of the in-run VEX. It now asks TreePart::withheld_by, the rule discovery withholds with. Co-Authored-By: Claude Opus 5.5 --- .../tests/in_process_redirect_pnpm.rs | 78 +++++++++++++++++++ crates/socket-patch-core/src/hosted/engine.rs | 2 +- 2 files changed, 79 insertions(+), 1 deletion(-) diff --git a/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs b/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs index fb7f1cbc7..e912bcbe7 100644 --- a/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs +++ b/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs @@ -3012,6 +3012,84 @@ async fn hosted_pnpm_file_tarball_copy_is_named() { .await; } +/// #2009: the registry release is a pnpm config dependency (the env +/// lockfile document) and the project document installs only a `file:` +/// copy of the same `name@version`. The scan pins the config dependency, +/// and names the project's copy with `redirect_copy_unpatched`, since a +/// VEX statement covers the release, not one install tree. Before, the +/// env pin's copy check compared trees, so the copy went unnamed. +#[tokio::test] +#[serial] +async fn hosted_pnpm_config_dependency_names_the_projects_file_copy() { + let server = MockServer::start().await; + mock_discovery(&server).await; + mock_reference(&server).await; + + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + let copy_key = format!("file:forks/{NAME}"); + write_file_copy_project( + root, + ©_key, + &format!(" resolution: {{directory: forks/{NAME}, type: directory}}\n"), + &format!("file:../../forks/{NAME}"), + ); + std::fs::remove_dir_all(root.join("packages").join("a")).unwrap(); + std::fs::write( + root.join("pnpm-lock.yaml"), + format!( + "--- +lockfileVersion: '9.0' + +importers: + + .: + configDependencies: + {NAME}: + specifier: {VERSION} + version: {VERSION} + +packages: + + {NAME}@{VERSION}: + resolution: {{integrity: {UPSTREAM_SHA512}}} + +--- +lockfileVersion: '9.0' + +importers: + .: {{}} + packages/b: + dependencies: + {NAME}: + specifier: file:../../forks/{NAME} + version: {copy_key} + +packages: + {NAME}@{copy_key}: + resolution: {{directory: forks/{NAME}, type: directory}} + +snapshots: + {NAME}@{copy_key}: {{}} +" + ), + ) + .unwrap(); + + let (code, doc) = run_hosted_json(root, &server.uri()); + assert_eq!(code, Some(0), "{doc}"); + let lock = std::fs::read_to_string(root.join("pnpm-lock.yaml")).unwrap(); + assert!( + lock.contains(&format!("tarball: {HOSTED_URL}")), + "the config dependency is pinned: {lock}" + ); + let text = doc.to_string(); + assert!( + text.contains("redirect_copy_unpatched") && text.contains(&format!("`{NAME}@{copy_key}`")), + "the project's copy is named: {doc:#}" + ); +} + /// The #935 workspace: member `a` depends on the registry release, member /// `b` on the `file:` copy `copy_key` (spelled `spec` in its manifest). fn write_file_copy_project(root: &Path, copy_key: &str, copy_entry: &str, spec: &str) { diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index 60fb83af9..946667b78 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -2203,7 +2203,7 @@ async fn unattributed_pins( for c in discovery.unpatched_copies.iter().filter(|c| { c.purl == r.purl && c.file == r.source_file - && c.part == r.part + && r.part.withheld_by(c.part) && matches!( c.file.file_name().and_then(|n| n.to_str()), Some("pnpm-lock.yaml" | "shrinkwrap.yaml" | "yarn.lock") From a303dd69789f9750033f1cb912cfe3d10faca437 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Mon, 9 Nov 2026 15:41:26 -0500 Subject: [PATCH 31/36] Address the second code review of the pnpm reader PR - Drop a pnpm env twin before any contest, so it leaves no Unattested mark (keyed by purl and uuid, which withheld the project ref) and no diagnostic behind. - Restore a hosted pin in the env lockfile document bare: pnpm 11+ records config dependencies without `tarball:` whatever lockfileIncludeTarballUrl says, so rollback no longer writes the project document's policy into it. - The Rush reader no longer stats rush.json again; its one caller does. - Divergences row N4: documents are identified by position and content. Co-Authored-By: Claude Opus 5.5 --- .../tests/in_process_redirect.rs | 47 +++++++++++++++++++ .../src/patch/redirect/upstream/npm.rs | 20 ++++++-- .../src/vendor/lock_inventory/pnpm.rs | 9 ++-- .../socket-patch-core/src/vex/discover/mod.rs | 11 +++-- .../socket-patch-core/src/vex/discover/npm.rs | 25 ++++++++++ docs/design/lock-model-divergences.md | 2 +- 6 files changed, 100 insertions(+), 14 deletions(-) diff --git a/crates/socket-patch-cli/tests/in_process_redirect.rs b/crates/socket-patch-cli/tests/in_process_redirect.rs index 152fefb22..e134c95a3 100644 --- a/crates/socket-patch-cli/tests/in_process_redirect.rs +++ b/crates/socket-patch-cli/tests/in_process_redirect.rs @@ -6075,6 +6075,53 @@ async fn pnpm_rollback_keeps_tarball_under_workspace_include_tarball_url() { assert_eq!(pnpm_pin_and_rollback(tmp.path(), &server), pristine); } +/// #2009 review: pnpm 11+ records a config dependency bare in the env +/// lockfile document even under `lockfileIncludeTarballUrl`, so a rollback +/// of a pin in both documents gives the project entry its `tarball:` back +/// and leaves the env entry bare. It used to write the project document's +/// `tarball:` into the env entry too, a lock pnpm never writes. +#[tokio::test] +#[serial] +async fn pnpm_rollback_keeps_the_env_document_bare_under_include_tarball_url() { + let server = MockServer::start().await; + mock_discovery(&server).await; + mock_reference(&server).await; + mock_view(&server).await; + mock_npm_registry(&server, "sha512-UPSTREAMupstream==", None).await; + let tarball = format!( + "{}/npm-registry/{NAME}/-/{NAME}-{VERSION}.tgz", + server.uri() + ); + + let tmp = tempfile::tempdir().unwrap(); + let project = write_pnpm_tarball_project(tmp.path(), &tarball); + let pristine = format!( + "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n \ + {NAME}:\n specifier: {VERSION}\n version: {VERSION}\n\npackages:\n\n \ + {NAME}@{VERSION}:\n resolution: {{integrity: sha512-UPSTREAMupstream==}}\n\n\ + snapshots:\n\n {NAME}@{VERSION}: {{}}\n\n---\n{project}" + ); + std::fs::write(tmp.path().join("pnpm-lock.yaml"), &pristine).unwrap(); + std::fs::write( + tmp.path().join("pnpm-workspace.yaml"), + "packages:\n - '.'\nlockfileIncludeTarballUrl: true\n", + ) + .unwrap(); + let pinned_twice = |root: &Path| { + let lock = std::fs::read_to_string(root.join("pnpm-lock.yaml")).unwrap(); + lock.matches("patch.test").count() + }; + let env = run_redirect_subprocess(tmp.path(), &server.uri()); + assert_eq!(vlt_hosted_common::redirected(&env), 1, "{env:#}"); + assert_eq!(pinned_twice(tmp.path()), 2, "both documents are pinned"); + let (code, env) = rollback_json(tmp.path(), &server); + assert_eq!(code, Some(0), "rollback: {env:#}"); + assert_eq!( + std::fs::read_to_string(tmp.path().join("pnpm-lock.yaml")).unwrap(), + pristine + ); +} + /// #557, the other case pnpm records `tarball:` in: the registry's /// `dist.tarball` is not the conventional URL pnpm would derive, so pnpm /// keeps it even without the setting. diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs index 03268939e..79a4ef416 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs @@ -1965,10 +1965,22 @@ pub(crate) async fn restore_pnpm_locks( let mut splices: Vec<(std::ops::Range, String)> = Vec::new(); // (uuid, name@version whose `tarball:` only the guessed setting decided) let mut handled: Vec<(String, Option)> = Vec::new(); + // pnpm 11+ records the env lockfile document's resolutions bare + // whatever `lockfileIncludeTarballUrl` says (see + // `pnpm_tarball_policy`): only a URL it cannot derive gets + // `tarball:` there. + let env_spans: Vec> = pnpm::documents(&text) + .into_iter() + .filter(|(_, document)| *document == pnpm::PnpmDocument::Env) + .map(|(span, _)| span) + .collect(); for entry in pnpm::entries(&text) { let Some(resolution) = pnpm::resolution(&entry) else { continue; }; + let in_env = env_spans + .iter() + .any(|span| span.contains(&resolution.range.start)); let Some((_, uuid, name, version)) = hits.iter().find(|(r, ..)| *r == resolution.range) else { continue; @@ -1990,13 +2002,15 @@ pub(crate) async fn restore_pnpm_locks( // pnpm records `tarball:` under lockfileIncludeTarballUrl and for // a URL it cannot derive from the registry (#557). let derived = policy.derives_tarball(name, version, &dist.tarball, found.from_project); - let restored = if policy.always || !derived { + let restored = if (policy.always && !in_env) || !derived { resolution.rewrite(integrity, &dist.tarball) } else { resolution.restore(integrity) }; - // A URL pnpm records anyway is no guess. - let guessed = (policy.guess.is_some() && derived).then(|| format!("{name}@{version}")); + // A URL pnpm records anyway is no guess, and neither is an env + // document entry. + let guessed = + (policy.guess.is_some() && derived && !in_env).then(|| format!("{name}@{version}")); splices.push((resolution.range.clone(), restored)); handled.push((uuid.clone(), guessed)); } diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs b/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs index 99dc87c36..6b5a06653 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs @@ -122,15 +122,12 @@ fn pnpm_lock_text_inventory(text: &str) -> Option> { /// the source lock into common/temp and runs pnpm there. /// /// Called (via `npm_family::inventory_npm_trees_raw_in`) only when -/// `rush.json` is present, which that caller checks once through the -/// view; the locks are read beside whatever lock the router picks. Outside -/// a Rush monorepo ([`is_rush`]) it reads nothing. The locks are the ones +/// `rush.json` is present ([`is_rush`]), which that caller checks once +/// through the view, so this reader does not stat it again; the locks are +/// read beside whatever lock the router picks. The locks are the ones /// lockfile discovery reads ([`rush_lock_rels`]), each read through the /// view. Missing files/dirs are skipped fail-soft. pub(super) async fn inventory_rush_pnpm_locks_in(view: &ProjectView<'_>) -> Vec { - if !is_rush(*view).await { - return Vec::new(); - } let mut out = Vec::new(); for rel in rush_lock_rels(*view).await { if let Some(entries) = inventory_pnpm_lock_rel_in(view, &rel).await { diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index e0c737bda..2154feb56 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -1308,6 +1308,7 @@ impl Discovery { /// and the project ref then stands in its place, see /// [`Discovery::drop_env_twins`]). fn contest_within_locks(&mut self) { + self.drop_env_twins(); self.expand_unidentified_copies(); if self.unpatched_copies.is_empty() && self.refs.iter().all(|r| r.part == TreePart::File) { return; @@ -1491,9 +1492,12 @@ impl Discovery { /// file's project document also wires (the same `(purl, uuid, mode)`): /// one wiring of the package in that file, whose project instance is /// the one an installed `node_modules` copy is verified against, even - /// when the two pins differ. The project twin counts wherever it ended - /// up (`refs`, `shadowed` or `rewirable`), so a withheld project ref - /// never leaves its env twin attested in its place. + /// when the two pins differ. Runs before any contest + /// ([`Discovery::contest_within_locks`]), so the twin leaves no + /// diagnostic or [`Unattested`] mark (keyed by `(purl, uuid)`, which + /// would withhold the project ref) behind, and a withheld project ref + /// never leaves it attested in its place. The project twin counts + /// wherever an extractor put it (`refs`, `shadowed` or `rewirable`). fn drop_env_twins(&mut self) { let project: HashSet<(PathBuf, String, String, WiringMode)> = [&self.refs, &self.shadowed, &self.rewirable] @@ -1687,7 +1691,6 @@ impl Discovery { self.unattested.dedup(); self.contested.sort(); self.contested.dedup(); - self.drop_env_twins(); for refs in [&mut self.refs, &mut self.shadowed, &mut self.rewirable] { refs.sort_by(|a, b| { (&a.source_file, &a.purl, &a.uuid, a.mode, a.part).cmp(&( diff --git a/crates/socket-patch-core/src/vex/discover/npm.rs b/crates/socket-patch-core/src/vex/discover/npm.rs index 0bedf0ce4..0d88d2e7d 100644 --- a/crates/socket-patch-core/src/vex/discover/npm.rs +++ b/crates/socket-patch-core/src/vex/discover/npm.rs @@ -3620,6 +3620,31 @@ mod tests { out.hosted_claim("pkg:npm/left-pad@1.3.0", UUID_A), Some(true) ); + // The dropped twin leaves no diagnostic of its own behind. + assert!( + out.diagnostics + .iter() + .all(|d| !d.detail.contains("(env lockfile)")), + "{:#?}", + out.diagnostics + ); + + // Wired in both documents, the env document bundling a copy: its + // twin is dropped before any mark, so the project ref (whose tree + // the env document's copies never reach) is attested. + let p = Project::new(); + p.write( + "pnpm-lock.yaml", + format!( + "{}{}", + env(&format!("{env_wired}{bundle}")), + project(&wired) + ), + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert_eq!(out.refs[0].part, TreePart::File); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); // The env document's registry entry contests no other lock's ref. let npm_lock = format!( diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md index 6e5648482..867f00ea4 100644 --- a/docs/design/lock-model-divergences.md +++ b/docs/design/lock-model-divergences.md @@ -84,7 +84,7 @@ follow from the install-tree decision. N11 was decided on 2026-11-05. | N1 | pnpm `gitBranchLockfile` with a branch lock present: is the root lock the install tree? | Hosted withholds the root locks and refuses (`hosted/engine.rs` ~930). Vendored refuses. Discovery and the inventory never check the setting. | Hosted. Discovery withholds pnpm root and member-lock attestations whenever any `pnpm-lock..yaml` exists, whatever the settings or environment say (owner, 2026-11-05; #2010 (draft)). | false attestation | | N2 | npm user patches (#711) and `bun patch` (#367) | Hosted skips that one dependency and records it as refused. Vendored refuses the whole npm v4 lock, or the one bun dependency. Discovery never reads `patchedDependencies` / `"patched"` and attests a pre-#711 hosted pin. | Hosted's per-dependency skip, in discovery too, for every package manager's user patches (owner decision; #2010 (draft) for npm and bun, #2018 (draft) for pnpm, yarn and vlt). The override is the existing `--allow-patch-conflicts`; `vex` never attests under a user patch, even with it. | false attestation | | N3 | A workspace member's stray npm/bun/vlt lock | Discovery treats it as absent. Hosted and vendored refuse. The inventory reads it as the project's lock. | The inventory ignores it | | -| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by content, not position. **Resolved in PR 2a (#2009)**: every reader reads both documents through `PnpmLock`, each entry tagged by document; discovery records carry the env document's install tree (`TreePart::PnpmEnv`) beside the real lock path, and that tree neither contests nor is contested by another lock, and never contests the project document; the project document's registry entry, `file:` copy or bundled copy of the same release does withhold an env ref (owner decision on #2009: a VEX statement covers the release, not one tree); the env document is the FIRST document when its importers carry only `configDependencies` / `packageManagerDependencies` (pnpm's own position rule), and every other document is read as the project. | | +| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by position and content (the env document is the first one, and only when its importers say so). **Resolved in PR 2a (#2009)**: every reader reads both documents through `PnpmLock`, each entry tagged by document; discovery records carry the env document's install tree (`TreePart::PnpmEnv`) beside the real lock path, and that tree neither contests nor is contested by another lock, and never contests the project document; the project document's registry entry, `file:` copy or bundled copy of the same release does withhold an env ref (owner decision on #2009: a VEX statement covers the release, not one tree); the env document is the FIRST document when its importers carry only `configDependencies` / `packageManagerDependencies` (pnpm's own position rule), and every other document is read as the project. | | | N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere. The vendored writers, reverters and GC claim only this project's vendor paths in #2025 (draft). pnpm part **resolved in PR 2a (#2009)**: the hosted pnpm "vendored?" check reads the rekeyed packages entry with `origin::vendor_ref`; the planner follows in #2029 (draft). | revert could rewrite another project's wiring | | N6 | `npm-shrinkwrap.json` and `package-lock.json` | Hosted, vendored and discovery read both, with a contest. The inventory reads the shrinkwrap only. | Both, everywhere | | | N7 | npm lockfileVersion | Hosted rewrites v1 and v4. Vendored accepts v2/v3 only. The inventory ignores v1. Discovery reads v1 and v2+. | Hosted, once vendored `file:` wiring under npm 6 and npm 12.1 v4 has fixtures | | From ac8e9e09ff650940dc55010d3fc0d12b9259cbe1 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Mon, 9 Nov 2026 15:47:10 -0500 Subject: [PATCH 32/36] Pin the shadowed early shrinkwrap in pnpm_hosted early_pnpm1_refuses_non_durable_shrinkwrap_across_all_locks asserted the pre-N11 rule, where pnpm 1's early shrinkwrap refused the dependency even beside a pnpm-lock.yaml. Beside one it is shadowed now, so it refuses nothing and only pnpm-lock.yaml is rewritten; a lone one still refuses. The test failed on this branch since the N11 change. Co-Authored-By: Claude Opus 5.5 --- crates/socket-patch-core/tests/pnpm_hosted.rs | 28 +++++++++++++++---- 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/crates/socket-patch-core/tests/pnpm_hosted.rs b/crates/socket-patch-core/tests/pnpm_hosted.rs index 33a07f3a2..0b237e870 100644 --- a/crates/socket-patch-core/tests/pnpm_hosted.rs +++ b/crates/socket-patch-core/tests/pnpm_hosted.rs @@ -134,19 +134,35 @@ fn mixed_newlines_preserve_unrelated_bytes() { roundtrip("pnpm-lock.yaml", lock, &dep("left-pad"), 1); } +/// pnpm 1's early shrinkwrap (`shrinkwrapVersion: 3`) is no durable +/// lock: a lone one refuses the dependency. Beside a `pnpm-lock.yaml` it +/// is shadowed (N11: pnpm >= 3 never reads it), so it refuses nothing and +/// only the governing lock is rewritten. #[test] -fn early_pnpm1_refuses_non_durable_shrinkwrap_across_all_locks() { +fn early_pnpm1_shrinkwrap_refuses_only_when_it_governs() { let old = "shrinkwrapVersion: 3\npackages:\n /left-pad/1.3.0:\n resolution:\n integrity: sha1-UPSTREAM=\n"; let newer = "lockfileVersion: '9.0'\npackages:\n left-pad@1.3.0:\n resolution: {integrity: sha512-UPSTREAM==}\n"; - let files = BTreeMap::from([ - ("shrinkwrap.yaml".into(), old.into()), - ("pnpm-lock.yaml".into(), newer.into()), - ]); - let result = rewrite_registry_redirect(&files, &[dep("left-pad")]); + let alone = BTreeMap::from([("shrinkwrap.yaml".into(), old.into())]); + let result = rewrite_registry_redirect(&alone, &[dep("left-pad")]); assert!(result.files.is_empty() && result.edits.is_empty()); assert!(result.refused_pnpm_uuids.contains("patch-id")); assert_eq!( result.warnings[0].code, "redirect_pnpm_legacy_lockfile_unsupported" ); + + let both = BTreeMap::from([ + ("shrinkwrap.yaml".into(), old.into()), + ("pnpm-lock.yaml".into(), newer.into()), + ]); + let result = rewrite_registry_redirect(&both, &[dep("left-pad")]); + assert!( + result.refused_pnpm_uuids.is_empty(), + "{:?}", + result.warnings + ); + assert_eq!( + result.files.keys().map(String::as_str).collect::>(), + ["pnpm-lock.yaml"] + ); } From 8b59a5a894f6e059161141b119dfc4dea3e83fdb Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Mon, 9 Nov 2026 16:09:30 -0500 Subject: [PATCH 33/36] Address the second code review of the pnpm planner PR - The hosted confirmation probe skips a pnpm lock pnpm does not read (N11): a stale Socket URL in a shadowed shrinkwrap.yaml confirmed the dependency, which the attribution gate then dropped as redirect_unattributable. - The pnpm trust/heal post-pass takes the governing root lock from planned_locks, so under a pnpm 1 / 2 pin a stray v9 pnpm-lock.yaml no longer plans a trustLockfile write. - Mixed line endings get their own error type instead of overloading io::ErrorKind::Unsupported, which an OS ENOTSUP read also returns; read_pnpm_text normalizes through utils::line_endings::to_lf. - The pnpm pin is read only when both pnpm-lock.yaml and shrinkwrap.yaml exist (planned_locks, vendoring, discovery). - Docs: divergences N16 names the two checks; read_project refuses mixed endings, not CRLF. Co-Authored-By: Claude Opus 5.5 --- .../src/formats/pnpm/plan.rs | 14 ++- crates/socket-patch-core/src/hosted/engine.rs | 96 ++++++++++++++++++- .../socket-patch-core/src/vendor/pnpm_lock.rs | 49 +++++++--- .../socket-patch-core/src/vex/discover/npm.rs | 6 +- docs/design/lock-model-divergences.md | 2 +- 5 files changed, 148 insertions(+), 19 deletions(-) diff --git a/crates/socket-patch-core/src/formats/pnpm/plan.rs b/crates/socket-patch-core/src/formats/pnpm/plan.rs index 34d5fb051..82224fa44 100644 --- a/crates/socket-patch-core/src/formats/pnpm/plan.rs +++ b/crates/socket-patch-core/src/formats/pnpm/plan.rs @@ -100,13 +100,19 @@ pub(crate) fn governing_pnpm_lock( pub(crate) fn planned_locks(files: &BTreeMap) -> Vec<&String> { let planned = |key: &&String| { let leaf = key.rsplit('/').next().unwrap_or(key); + if !matches!(leaf, PNPM_LOCK | PNPM_SHRINKWRAP_LEGACY) { + return false; + } // `dir` keeps its trailing `/` (empty at the root). let in_dir = |rel: &str| format!("{}{rel}", &key[..key.len() - leaf.len()]); - let pinned = pinned_pnpm_major(|rel| files.get(&in_dir(rel)).cloned()); let has = |rel: &str| files.contains_key(&in_dir(rel)); - matches!(leaf, PNPM_LOCK | PNPM_SHRINKWRAP_LEGACY) - && governing_pnpm_lock(has(PNPM_LOCK), has(PNPM_SHRINKWRAP_LEGACY), pinned) - == Some(leaf) + let both = (has(PNPM_LOCK), has(PNPM_SHRINKWRAP_LEGACY)); + // The pin decides only between the two, so it is read only then. + let pinned = match both { + (true, true) => pinned_pnpm_major(|rel| files.get(&in_dir(rel)).cloned()), + _ => None, + }; + governing_pnpm_lock(both.0, both.1, pinned) == Some(leaf) }; files.keys().filter(planned).collect() } diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index dc4542846..b5042092c 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -2693,8 +2693,19 @@ fn pnpm_trust( .count(); // The locks pnpm installs the project from: the root lock, plus each // member's own under `sharedWorkspaceLockfile: false` (#492). A Rush - // lock is never one: rush installs it from common/temp. - let governing: Vec<&str> = std::iter::once("pnpm-lock.yaml") + // lock is never one: rush installs it from common/temp. The root lock + // is the one pnpm reads (N11): `shrinkwrap.yaml` under a pnpm 1 / 2 + // pin, so a stray `pnpm-lock.yaml` never drives the heal or the trust + // config. + let planned = crate::formats::pnpm::plan::planned_locks(files); + let root_lock = [ + crate::constants::npm_family::PNPM_LOCK, + crate::constants::npm_family::PNPM_SHRINKWRAP_LEGACY, + ] + .into_iter() + .find(|lock| planned.iter().any(|key| key.as_str() == *lock)) + .unwrap_or(crate::constants::npm_family::PNPM_LOCK); + let governing: Vec<&str> = std::iter::once(root_lock) .chain(member_lock_keys.iter().map(String::as_str)) .filter(|key| !rush_lock_keys.iter().any(|rush| rush == key)) .collect(); @@ -3334,9 +3345,24 @@ fn confirm( name.ends_with("/pom.xml") || maven_reactor_refused && (name == "pom.xml" || name.starts_with(".mvn/")) }; + // A pnpm lock pnpm does not read (a `shrinkwrap.yaml` beside a + // `pnpm-lock.yaml`, or the reverse under a pnpm 1 / 2 pin, N11) is + // never rewritten, so a Socket URL an earlier run left in it pins + // nothing either. + let planned_pnpm = crate::formats::pnpm::plan::planned_locks(files); + let is_shadowed_pnpm = |name: &str| { + matches!( + name.rsplit('/').next(), + Some( + crate::constants::npm_family::PNPM_LOCK + | crate::constants::npm_family::PNPM_SHRINKWRAP_LEGACY + ) + ) && !planned_pnpm.iter().any(|planned| planned.as_str() == name) + }; let final_texts: Vec<(&str, &String)> = files .iter() .filter(|(name, _)| !(pdm_inactive && name.as_str() == "pdm.lock")) + .filter(|(name, _)| !is_shadowed_pnpm(name)) .filter(|(name, _)| !is_npm_manifest(name)) .filter(|(name, _)| !is_unpinned_maven(name)) .filter(|(name, _)| !crate::patch::redirect::sbt::is_synthetic_key(name)) @@ -4579,6 +4605,72 @@ mod tests { ); } + /// N11 review: under a pnpm 1 / 2 pin `shrinkwrap.yaml` governs, so a + /// stray v9 `pnpm-lock.yaml` an earlier run redirected is no heal root + /// and plans no `trustLockfile` write for a lock pnpm never reads. + #[tokio::test] + async fn a_shadowed_pnpm_lock_is_no_heal_root() { + let redirected = LEFT_PAD_V9_LOCK.replace( + "resolution: {integrity: sha512-UPSTREAM==}", + &format!( + "resolution: {{integrity: sha512-PATCHED==, tarball: {}}}", + left_pad_url() + ), + ); + let mut p = MemoryProject::new(); + p.insert_text("pnpm-lock.yaml", redirected.as_str()); + p.insert_text( + "shrinkwrap.yaml", + "shrinkwrapMinorVersion: 9\nshrinkwrapVersion: 3\n", + ); + p.insert_text( + "package.json", + r#"{"name":"app","packageManager":"pnpm@2.25.7"}"#, + ); + let (_, done) = npm_rewrite(&ProjectView::Memory(&p), &BTreeSet::new()).await; + assert!( + !done.rewrite.files.contains_key(PNPM_WORKSPACE_REL), + "{:?}", + done.rewrite.files.keys() + ); + + // Without the pin, pnpm-lock.yaml governs and the heal plans it. + p.insert_text("package.json", r#"{"name":"app"}"#); + let (_, done) = npm_rewrite(&ProjectView::Memory(&p), &BTreeSet::new()).await; + assert!( + done.rewrite.files.contains_key(PNPM_WORKSPACE_REL), + "{:?}", + done.rewrite.files.keys() + ); + } + + /// N11 review: a shadowed `shrinkwrap.yaml` still carrying an earlier + /// run's Socket URL confirms nothing when the governing + /// `pnpm-lock.yaml` no longer pins the package. It used to confirm by + /// substring, and only the attribution gate then dropped the dependency + /// as `redirect_unattributable`, blaming the project's lockfiles. + #[tokio::test] + async fn a_shadowed_pnpm_lock_confirms_nothing() { + let mut p = MemoryProject::new(); + p.insert_text( + "pnpm-lock.yaml", + "lockfileVersion: '9.0'\n\npackages:\n other@1.0.0:\n \ + resolution: {integrity: sha512-OTHER==}\n", + ); + p.insert_text( + "shrinkwrap.yaml", + format!( + "shrinkwrapVersion: 3\nshrinkwrapMinorVersion: 9\npackages:\n /left-pad/1.3.0:\n \ + resolution:\n integrity: sha512-PATCHED==\n tarball: {}\n", + left_pad_url() + ) + .as_str(), + ); + let (_, done) = npm_rewrite(&ProjectView::Memory(&p), &BTreeSet::new()).await; + assert!(done.confirmed.is_empty(), "{:?}", done.confirmed); + assert!(done.unattributed.is_empty(), "{:?}", done.unattributed); + } + #[tokio::test] async fn a_pnpm_lock_pair_reads_the_package_manager_pin() { // pnpm 2 pinned in package.json, no install record: pnpm reads diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock.rs b/crates/socket-patch-core/src/vendor/pnpm_lock.rs index 032a5ac56..437134a06 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock.rs @@ -550,7 +550,8 @@ pub(super) async fn git_branch_lock_refusal( } /// Read the pair, refusing (before any write) a file that is missing, -/// unreadable, not the shape the surgery has fixtures for, or CRLF. +/// unreadable, not the shape the surgery has fixtures for, or that mixes +/// CRLF and LF line endings ([`read_pnpm_text`]). async fn read_project( project_root: &Path, dialect: PnpmDialect, @@ -584,8 +585,12 @@ async fn read_project( // pnpm 1 / 2 read only `shrinkwrap.yaml`, whose grammar the rekey has no // fixture for: there `pnpm-lock.yaml` is stray, and wiring it would // patch nothing pnpm installs (N11, the choice hosted and discovery make). + // The pin matters only beside a shrinkwrap, so it is read only then. let shrinkwrap = view.exists_no_follow(PNPM_SHRINKWRAP_LEGACY).await; - let pinned = crate::utils::pnpm_workspace::pinned_pnpm_major_in(&view).await; + let pinned = match shrinkwrap { + true => crate::utils::pnpm_workspace::pinned_pnpm_major_in(&view).await, + false => None, + }; if governing_pnpm_lock(true, shrinkwrap, pinned) == Some(PNPM_SHRINKWRAP_LEGACY) { let why = "the project pins pnpm 1 / 2, which installs from shrinkwrap.yaml and ignores \ pnpm-lock.yaml; vendored mode does not write that grammar — upgrade pnpm \ @@ -703,29 +708,51 @@ async fn read_project( /// endings it is written back with. A CRLF file (a Windows autocrlf /// checkout) is edited like its LF twin and written back CRLF, as hosted /// mode's byte splices keep it (divergences N15). A file that mixes endings -/// is an `Unsupported` error, which no read of a regular file returns: no -/// one ending writes it back byte-faithfully, so vendoring refuses it and -/// the revert fails before any write. +/// is a [`MixedLineEndings`] error ([`is_mixed_line_endings`]): no one +/// ending writes it back byte-faithfully, so vendoring refuses it and the +/// revert fails before any write. async fn read_pnpm_text(root: &Path, file: &str) -> std::io::Result<(String, LineEndings)> { let text = read_regular_to_string(&root.join(file)).await?; match LineEndings::of(&text) { LineEndings::Mixed => Err(std::io::Error::new( - std::io::ErrorKind::Unsupported, - format!( + std::io::ErrorKind::InvalidData, + MixedLineEndings(format!( "{file} mixes CRLF and LF line endings, which this rewriter cannot write back \ byte-faithfully — normalize the file to one line ending and retry" - ), + )), + )), + LineEndings::Crlf => Ok(( + crate::utils::line_endings::to_lf(&text).into_owned(), + LineEndings::Crlf, )), - LineEndings::Crlf => Ok((text.replace("\r\n", "\n"), LineEndings::Crlf)), eol => Ok((text, eol)), } } +/// [`read_pnpm_text`]'s error for a file that mixes line endings, told +/// apart from an OS read error by its type, not its `io::ErrorKind`. +#[derive(Debug)] +struct MixedLineEndings(String); + +impl std::fmt::Display for MixedLineEndings { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.0) + } +} + +impl std::error::Error for MixedLineEndings {} + +/// Whether `e` is [`read_pnpm_text`]'s mixed-line-ending refusal. +fn is_mixed_line_endings(e: &std::io::Error) -> bool { + e.get_ref() + .is_some_and(|inner| inner.is::()) +} + /// The vendoring refusal for a [`read_pnpm_text`] error other than a /// missing file: `vendor_lockfile_crlf_unsupported` for mixed endings, else /// `vendor_lockfile_missing` with `remedy`. fn read_refusal(file: &str, e: std::io::Error, remedy: &str) -> Box { - Box::new(match e.kind() == std::io::ErrorKind::Unsupported { + Box::new(match is_mixed_line_endings(&e) { true => refused("vendor_lockfile_crlf_unsupported", e.to_string()), false => refused( "vendor_lockfile_missing", @@ -1014,7 +1041,7 @@ pub(super) async fn revert_pnpm_dialect( .flatten() .filter_map(|r| r.as_ref().err()) { - if e.kind() == std::io::ErrorKind::Unsupported { + if is_mixed_line_endings(e) { return RevertOutcome::failed(format!("{e}; nothing was reverted")); } } diff --git a/crates/socket-patch-core/src/vex/discover/npm.rs b/crates/socket-patch-core/src/vex/discover/npm.rs index cb55e2fc9..40e8ee247 100644 --- a/crates/socket-patch-core/src/vex/discover/npm.rs +++ b/crates/socket-patch-core/src/vex/discover/npm.rs @@ -812,9 +812,13 @@ async fn extract_pnpm(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { // the other one: the patches it still names are recognized as NOT // wired (module docs, rule 11), so a ledger record its stale Socket // url alone keeps "mentioned" is dead. - let pinned = crate::utils::pnpm_workspace::pinned_pnpm_major_in(&ctx.view).await; let (lock, shrinkwrap) = (PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY); let both = (ctx.exists(lock).await, ctx.exists(shrinkwrap).await); + // The pin decides only between the two, so it is read only then. + let pinned = match both { + (true, true) => crate::utils::pnpm_workspace::pinned_pnpm_major_in(&ctx.view).await, + _ => None, + }; if let Some(governing) = governing_pnpm_lock(both.0, both.1, pinned) { extract_pnpm_lock(ctx, governing, disabled, out).await; read.push(governing.to_string()); diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md index 67052efda..f9619b895 100644 --- a/docs/design/lock-model-divergences.md +++ b/docs/design/lock-model-divergences.md @@ -96,7 +96,7 @@ follow from the install-tree decision. N11 was decided on 2026-11-05. | N13 | Plug'n'Play | Vendored refuses. The hosted berry preflight has no PnP gate. The inventory returns an error, so `scan` sees nothing in a project hosted pinned. | The inventory reads PnP locks. Vendored keeps its refusal. | target-specific (vendored) | | N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery. **Resolved in PR 2a (#2009)**: under the setting (not under Rush), the inventory keeps a Socket-hosted pnpm entry but drops its integrity, as discovery withholds the lock pin. | target-specific | | N15 | CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml` | Vendored refuses; the rest handle it | Handle it everywhere. **Resolved in PR 2b (#2029)**: the vendored planner and reverter edit a CRLF lock or workspace file like its LF twin and write it back CRLF. Only a file mixing CRLF and LF is still refused (`vendor_lockfile_crlf_unsupported`), since no single ending writes it back byte-faithfully. | | -| N16 | Lock versions | pnpm: vendored accepts only 5.4/6.0/9.0; hosted any but early shrinkwrap. vlt: vendored refuses a lock with no version; hosted rewrites and warns. | Hosted. **pnpm, PR 2b (#2029)**: one table, `plan::Target::check_lock`. Vendored keeps its 5.4/6.0/9.0 range: no other version has a captured pnpm `file:` override shape that proves its rekey, so the other versions stay refused, target-specifically. vlt is unchanged. | target-specific (pnpm vendored) | +| N16 | Lock versions | pnpm: vendored accepts only 5.4/6.0/9.0; hosted any but early shrinkwrap. vlt: vendored refuses a lock with no version; hosted rewrites and warns. | Hosted. **pnpm, PR 2b (#2029)**: two target-specific checks: hosted refuses only pnpm 1's early shrinkwrap (`plan::hosted_refuses_lock`), and the vendored dialects route by `formats::pnpm::sniff_lock_grammar` (`PnpmDialect::check_lock`). Vendored keeps its 5.4/6.0/9.0 range: no other version has a captured pnpm `file:` override shape that proves its rekey, so the other versions stay refused, target-specifically. vlt is unchanged. | target-specific (pnpm vendored) | | N17 | Bun workspaces, yarn classic offline mirror | Refused differently by the two modes | Stays | target-specific | | N18 | An unparseable npm lock | Hosted warns and skips. Vendored refuses the primary lock. Discovery diagnoses. The inventory returns nothing silently. | Discovery's diagnostic, surfaced by every consumer | | | N19 | yarn.lock with a stray top-level line | Discovery rejects the file; hosted and the inventory scan its blocks | Discovery's rule | | From ff4c8db587024dc578aabe263b9ba9dee9baeafd Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Mon, 9 Nov 2026 16:36:48 -0500 Subject: [PATCH 34/36] Simplify the folded pnpm reader and planner Behavior-preserving cleanups over the folded lock-model 2 diff: - The pnpm architecture test reads production code through the shared vendor::test_support::production_sources instead of its own lexer. - The hosted unwind walks PnpmLock::packages once (document tags and resolutions included) instead of two raw entry walks plus its own env-span tagging, so it leaves the raw-walk allowlist. - One PnpmDialect::entry_in_use backs both dialects' in-use checks (memoized; the legacy one no longer parses the whole lock per entry). - plan::shadowed_locks names the pnpm locks pnpm does not read, for the engine's confirmation probe and trust/heal root; planned_locks shares its per-key rule. - Cheaper hot paths: the store crawler's key-only walk skips document tagging; grammar::suffix no longer allocates; drop_env_twins returns early with no env ref; the inventory reads the lockfile setting only when a hosted pin exists; PnpmLock::documents needs no sort. - Comments and the divergences doc state current behavior instead of review history; shared test fixtures (env document head, Rush consts); three vendored helpers made private. Co-Authored-By: Claude Opus 5.5 --- .../tests/in_process_redirect.rs | 9 +- .../tests/in_process_redirect_pnpm.rs | 3 +- .../tests/in_process_vendor_pnpm_takeover.rs | 3 +- .../src/formats/pnpm/grammar.rs | 6 +- .../socket-patch-core/src/formats/pnpm/mod.rs | 243 ++---------------- .../src/formats/pnpm/plan.rs | 60 +++-- crates/socket-patch-core/src/hosted/engine.rs | 38 +-- .../src/patch/redirect/mod.rs | 10 +- .../src/patch/redirect/upstream/npm.rs | 30 +-- .../src/vendor/lock_inventory/npm_family.rs | 29 +-- .../src/vendor/lock_inventory/tests.rs | 19 +- .../socket-patch-core/src/vendor/pnpm_lock.rs | 68 ++--- .../src/vendor/pnpm_lock_legacy.rs | 9 +- .../socket-patch-core/src/vex/discover/mod.rs | 14 +- .../socket-patch-core/src/vex/discover/npm.rs | 19 +- docs/design/lock-model-divergences.md | 16 +- 16 files changed, 195 insertions(+), 381 deletions(-) diff --git a/crates/socket-patch-cli/tests/in_process_redirect.rs b/crates/socket-patch-cli/tests/in_process_redirect.rs index e134c95a3..c9ee39e95 100644 --- a/crates/socket-patch-cli/tests/in_process_redirect.rs +++ b/crates/socket-patch-cli/tests/in_process_redirect.rs @@ -6075,11 +6075,10 @@ async fn pnpm_rollback_keeps_tarball_under_workspace_include_tarball_url() { assert_eq!(pnpm_pin_and_rollback(tmp.path(), &server), pristine); } -/// #2009 review: pnpm 11+ records a config dependency bare in the env -/// lockfile document even under `lockfileIncludeTarballUrl`, so a rollback -/// of a pin in both documents gives the project entry its `tarball:` back -/// and leaves the env entry bare. It used to write the project document's -/// `tarball:` into the env entry too, a lock pnpm never writes. +/// pnpm 11+ records a config dependency bare in the env lockfile document +/// even under `lockfileIncludeTarballUrl`, so a rollback of a pin in both +/// documents gives the project entry its `tarball:` back and leaves the env +/// entry bare. #[tokio::test] #[serial] async fn pnpm_rollback_keeps_the_env_document_bare_under_include_tarball_url() { diff --git a/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs b/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs index e912bcbe7..d224537b0 100644 --- a/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs +++ b/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs @@ -3016,8 +3016,7 @@ async fn hosted_pnpm_file_tarball_copy_is_named() { /// lockfile document) and the project document installs only a `file:` /// copy of the same `name@version`. The scan pins the config dependency, /// and names the project's copy with `redirect_copy_unpatched`, since a -/// VEX statement covers the release, not one install tree. Before, the -/// env pin's copy check compared trees, so the copy went unnamed. +/// VEX statement covers the release, not one install tree. #[tokio::test] #[serial] async fn hosted_pnpm_config_dependency_names_the_projects_file_copy() { diff --git a/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs b/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs index 60ff577bd..cb18531e0 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs @@ -533,8 +533,7 @@ async fn scan_vendored_over_hosted_pnpm_plain_dep_still_takes_over() { /// Lock-model 2b (divergence N15): a CRLF `pnpm-lock.yaml` (a /// `core.autocrlf` checkout) takes over like its LF twin, and the vendored -/// lock keeps its CRLF endings. It used to be refused -/// (`vendor_lockfile_crlf_unsupported`) and keep the hosted pin. +/// lock keeps its CRLF endings. #[tokio::test(flavor = "multi_thread")] async fn scan_vendored_over_hosted_pnpm_crlf_lock_takes_over_and_stays_crlf() { let server = MockServer::start().await; diff --git a/crates/socket-patch-core/src/formats/pnpm/grammar.rs b/crates/socket-patch-core/src/formats/pnpm/grammar.rs index 9f7ce6e4e..df81b1879 100644 --- a/crates/socket-patch-core/src/formats/pnpm/grammar.rs +++ b/crates/socket-patch-core/src/formats/pnpm/grammar.rs @@ -242,9 +242,11 @@ pub(crate) fn entry_bundled<'a>(entry: &Entry<'a>) -> Option> { pub(crate) fn suffix<'a>(key: &'a str, name: &str, version: &str) -> Option<&'a str> { let key = unquote(key); let key = key.strip_prefix('/').unwrap_or(key); + // `name@version` or `name/version`, without allocating either. let suffix = key - .strip_prefix(&format!("{name}@{version}")) - .or_else(|| key.strip_prefix(&format!("{name}/{version}")))?; + .strip_prefix(name)? + .strip_prefix(['@', '/'])? + .strip_prefix(version)?; if suffix .chars() .next() diff --git a/crates/socket-patch-core/src/formats/pnpm/mod.rs b/crates/socket-patch-core/src/formats/pnpm/mod.rs index bfe6451a3..177fe4581 100644 --- a/crates/socket-patch-core/src/formats/pnpm/mod.rs +++ b/crates/socket-patch-core/src/formats/pnpm/mod.rs @@ -90,7 +90,10 @@ impl<'a> PnpmPackage<'a> { /// vendored, hosted, directory, git); each consumer applies its own key /// and resolution rules. fn pnpm_packages(text: &str, resolve: Resolve) -> Vec> { - let documents = grammar::documents(text); + let documents = match resolve { + Resolve::Yes => grammar::documents(text), + Resolve::No => Vec::new(), + }; grammar::entries(text) .into_iter() .map(|entry| { @@ -118,8 +121,9 @@ fn pnpm_packages(text: &str, resolve: Resolve) -> Vec> { #[derive(Clone, Copy)] enum Resolve { Yes, - /// Keys and bodies only: the store crawler's hot path - /// ([`InstalledPackages::from_lock_text`]) never reads a resolution. + /// Keys and bodies only, every entry tagged `Project`: the store + /// crawler's hot path ([`InstalledPackages::from_lock_text`]) reads + /// neither a resolution nor a document. No, } @@ -436,10 +440,10 @@ impl<'t> PnpmLock<'t> { /// The document kinds the lock has entries in, project first. pub(crate) fn documents(&self) -> Vec { - let mut kinds: Vec = self.packages.iter().map(|p| p.document).collect(); - kinds.sort(); - kinds.dedup(); - kinds + [PnpmDocument::Project, PnpmDocument::Env] + .into_iter() + .filter(|kind| self.packages.iter().any(|p| p.document == *kind)) + .collect() } fn has_packages_section(&self) -> bool { @@ -660,6 +664,10 @@ pub(crate) fn vendored_npm_uuids(text: &str) -> HashSet { mod tests { use super::*; + /// pnpm 11's env lockfile document for config dependency + /// `cfg-dep@2.0.0`, up to its packages entry's key line. + const ENV_DOC_HEAD: &str = "lockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n cfg-dep:\n specifier: 2.0.0\n version: 2.0.0\n\npackages:\n\n cfg-dep@2.0.0:\n"; + /// #1197: the current lockfile's packages, by name and version, in the /// shapes real pnpm 7.33.7 / 8.15.9 / 10–12 write after an upgrade /// (`left-pad` 1.3.0 → 1.2.0), plus non-registry entries. @@ -789,7 +797,7 @@ mod tests { #[test] fn env_document_is_read_as_its_own_tree() { let env = format!( - "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n cfg-dep:\n specifier: 2.0.0\n version: 2.0.0\n\npackages:\n\n cfg-dep@2.0.0:\n resolution: {{integrity: sha512-c}}\n\n b@file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz:\n resolution: {{integrity: sha512-x, tarball: file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz}}\n version: 1.0.0\n\n" + "---\n{ENV_DOC_HEAD} resolution: {{integrity: sha512-c}}\n\n b@file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz:\n resolution: {{integrity: sha512-x, tarball: file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz}}\n version: 1.0.0\n\n" ); let project = "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n dependencies:\n left-pad:\n specifier: 1.3.0\n version: 1.3.0\n\npackages:\n\n left-pad@1.3.0:\n resolution: {integrity: sha512-x}\n"; for text in [ @@ -860,10 +868,10 @@ mod tests { /// the FIRST one, when its importers say so, as pnpm and the vendored /// planners' `split_project_document` read it: an env-shaped LATER /// document is a project one, so it cannot opt out of the project - /// tree's contests (code-review finding 2). + /// tree's contests. #[test] fn documents_are_identified_by_position_and_content() { - let env = "lockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n cfg-dep:\n specifier: 2.0.0\n version: 2.0.0\n\npackages:\n\n cfg-dep@2.0.0:\n resolution: {integrity: sha512-c}\n"; + let env = &format!("{ENV_DOC_HEAD} resolution: {{integrity: sha512-c}}\n"); let project = |dep: &str| { format!( "lockfileVersion: '9.0'\n\nimporters:\n\n .:\n dependencies:\n {dep}:\n specifier: 1.0.0\n version: 1.0.0\n\npackages:\n\n {dep}@1.0.0:\n resolution: {{integrity: sha512-x}}\n" @@ -887,8 +895,7 @@ mod tests { own(&[("a@1.0.0", Project), ("cfg-dep@2.0.0", Project)]) ); assert!(!PnpmLock::parse(&swapped).documents().contains(&Env)); - // A first document that is not the env document is a project one, - // and its entries are no longer skipped. + // A first document that is not the env document is a project one. let two = format!("---\n{}---\n{}", project("a"), project("b")); assert_eq!( kinds(&two), @@ -913,12 +920,12 @@ mod tests { } } - /// An entry is tagged by its KEY line (code-review finding 5): a key + /// An entry is tagged by its KEY line: a key /// whose body is empty — the last line of the env document, right /// before `---` — has its body offset on the separator, in no document. #[test] fn an_entry_with_an_empty_body_keeps_its_document() { - let env = "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n cfg-dep:\n specifier: 2.0.0\n version: 2.0.0\n\npackages:\n\n cfg-dep@2.0.0:\n"; + let env = format!("---\n{ENV_DOC_HEAD}"); let project = "---\nlockfileVersion: '9.0'\n\npackages:\n\n a@1.0.0:\n resolution: {integrity: sha512-x}\n\n b@1.0.0:"; let text = format!("{env}{project}"); for text in [text.clone(), text.replace('\n', "\r\n")] { @@ -1095,16 +1102,10 @@ mod tests { /// a new reader in the same file still fails. #[cfg(test)] mod architecture_tests { - use std::path::{Path, PathBuf}; + use crate::vendor::test_support::production_sources::production_sources; /// `(root-relative file, uses, why)`. const WRITERS: &[(&str, usize, &str)] = &[ - ( - "patch/redirect/upstream/npm.rs", - 2, - "hosted unwind (`restore_pnpm_locks`): splices each pin's \ - resolution byte range back to the registry spelling", - ), ( "vendor/pnpm_lock.rs", 3, @@ -1139,171 +1140,18 @@ mod architecture_tests { && (code.contains("bounds(") || code.contains("SectionIndex::build("))) } - /// The lines of `text` outside `#[cfg(test)]` / `#[cfg(all(test, …))]` - /// items: an out-of-line `mod name;` declaration drops that line only, - /// an inline item (`mod tests { … }`, a helper `fn`) drops through its - /// closing brace, so production code after either is still scanned. - fn production(text: &str) -> Vec<&str> { - let lines: Vec<&str> = text.split('\n').collect(); - let deltas = brace_deltas(text); - let mut out = Vec::new(); - let mut i = 0; - while i < lines.len() { - let attr = lines[i].trim_start(); - if !(attr.starts_with("#[cfg(test)]") || attr.starts_with("#[cfg(all(test")) { - out.push(lines[i]); - i += 1; - continue; - } - // Skip the gated item's other attributes, then the item. - i += 1; - while i < lines.len() && lines[i].trim_start().starts_with("#[") { - i += 1; - } - let mut depth = 0i64; - let mut opened = false; - while i < lines.len() { - let (open, close) = deltas[i]; - let line = lines[i]; - i += 1; - depth += open - close; - opened |= open > 0; - if (!opened && line.trim_end().ends_with(';')) || (opened && depth <= 0) { - break; - } - } - } - out - } - - /// Per line of `text` (split on `\n`), the `{` and `}` in code: outside - /// comments and string, raw string and char literals, which may span - /// lines. - fn brace_deltas(text: &str) -> Vec<(i64, i64)> { - let chars: Vec = text.chars().collect(); - let mut out = vec![(0i64, 0i64)]; - let mut i = 0; - let ident = |c: char| c.is_alphanumeric() || c == '_'; - // Advance past `n` chars, opening a new line entry at each `\n`. - let step = |out: &mut Vec<(i64, i64)>, i: &mut usize, n: usize| { - for _ in 0..n { - if chars.get(*i) == Some(&'\n') { - out.push((0, 0)); - } - *i += 1; - } - }; - while i < chars.len() { - let c = chars[i]; - let next = chars.get(i + 1).copied(); - if c == '/' && next == Some('/') { - while i < chars.len() && chars[i] != '\n' { - i += 1; - } - } else if c == '/' && next == Some('*') { - let mut depth = 0; - while i < chars.len() { - if chars[i] == '/' && chars.get(i + 1) == Some(&'*') { - depth += 1; - step(&mut out, &mut i, 2); - } else if chars[i] == '*' && chars.get(i + 1) == Some(&'/') { - depth -= 1; - step(&mut out, &mut i, 2); - if depth == 0 { - break; - } - } else { - step(&mut out, &mut i, 1); - } - } - } else if c == 'r' - && matches!(next, Some('"' | '#')) - && (i == 0 || !ident(chars[i - 1]) || chars[i - 1] == 'b') - { - let hashes = chars[i + 1..].iter().take_while(|&&h| h == '#').count(); - if chars.get(i + 1 + hashes) != Some(&'"') { - step(&mut out, &mut i, 1); - continue; - } - step(&mut out, &mut i, 2 + hashes); - while i < chars.len() { - let closes = chars[i] == '"' - && chars[i + 1..] - .iter() - .take(hashes) - .filter(|&&h| h == '#') - .count() - == hashes; - if closes { - step(&mut out, &mut i, 1 + hashes); - break; - } - step(&mut out, &mut i, 1); - } - } else if c == '"' { - step(&mut out, &mut i, 1); - while i < chars.len() && chars[i] != '"' { - let n = if chars[i] == '\\' { 2 } else { 1 }; - step(&mut out, &mut i, n); - } - step(&mut out, &mut i, 1); - } else if c == '\'' && next == Some('\\') { - // An escaped char literal (`'\''`, `'\u{7b}'`). - step(&mut out, &mut i, 2); - while i < chars.len() && chars[i] != '\'' { - step(&mut out, &mut i, 1); - } - step(&mut out, &mut i, 1); - } else if c == '\'' && chars.get(i + 2) == Some(&'\'') { - step(&mut out, &mut i, 3); - } else { - let last = out.last_mut().expect("one entry per line"); - match c { - '{' => last.0 += 1, - '}' => last.1 += 1, - _ => {} - } - step(&mut out, &mut i, 1); - } - } - out - } - - fn rs_files(dir: &Path, out: &mut Vec) { - for entry in std::fs::read_dir(dir).expect("read src dir") { - let path = entry.expect("dir entry").path(); - if path.is_dir() { - rs_files(&path, out); - } else if path.extension().is_some_and(|e| e == "rs") { - out.push(path); - } - } - } - #[test] fn pnpm_entries_are_read_through_the_model() { - let src = Path::new(env!("CARGO_MANIFEST_DIR")).join("src"); - let mut files = Vec::new(); - rs_files(&src, &mut files); + let src = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src"); let mut found: Vec<(String, usize)> = Vec::new(); - for path in files { - let rel = path - .strip_prefix(&src) - .unwrap() - .to_string_lossy() - .replace('\\', "/"); - if rel.starts_with("formats/pnpm/") || rel.ends_with("tests.rs") { + // Test-only files and items are left out by the shared reader. + for file in production_sources(&src) { + if file.rel.starts_with("formats/pnpm/") { continue; } - let text = std::fs::read_to_string(&path) - .unwrap() - .replace("\r\n", "\n"); - let uses = production(&text) - .into_iter() - .filter(|l| raw_walk(l)) - .count(); + let uses = file.text.lines().filter(|l| raw_walk(l)).count(); if uses > 0 { - found.push((rel, uses)); + found.push((file.rel, uses)); } } found.sort(); @@ -1331,39 +1179,4 @@ mod architecture_tests { assert!(!raw_walk(" lock.get(\"packages\")")); assert!(!raw_walk(" for p in PnpmLock::parse(t).packages() {")); } - - /// Code-review finding 1: an out-of-line test module declared near the - /// top of a file (`crawlers/npm_crawler.rs`, `patch/redirect/mod.rs`, - /// `vendor/mod.rs` do) used to end the scan there, so a raw walk deep - /// in the file went unseen. Only the gated items are dropped now. - #[test] - fn production_drops_only_test_gated_items() { - let src = "use a;\n#[cfg(test)]\nmod oracle;\n#[cfg(all(test, unix))]\n#[path = \"x.rs\"]\nmod x_tests;\n\ - #[cfg(test)]\nfn helper() {\n let s = \"}\";\n let c = '{';\n if true { pnpm::entries(t); }\n}\n\ - fn reader<'a>(t: &'a str) {\n for e in pnpm::entries(t) {}\n}\n\ - #[cfg(test)]\nmod tests {\n fn t() { pnpm::entries(\"{\"); }\n \ - const S: &str = \"a {\n{ b\";\n const R: &str = r#\"{\"{\n\"#;\n /* { */\n}\n\ - fn after() { let x = line == \"packages:\"; }\n"; - let kept = production(src); - assert_eq!(kept.iter().filter(|l| raw_walk(l)).count(), 2, "{kept:#?}"); - assert!(kept.iter().all(|l| !l.contains("mod oracle"))); - assert!(kept.iter().any(|l| l.contains("fn after()"))); - assert!(kept.iter().all(|l| !l.contains("fn helper"))); - // The real files that declare test modules early keep their late - // production code. - let src = Path::new(env!("CARGO_MANIFEST_DIR")).join("src"); - for (rel, late) in [ - ("crawlers/npm_crawler.rs", "fn is_safe_npm_component("), - ("patch/redirect/mod.rs", "fn rewrite_golang_target("), - ("vendor/mod.rs", "pub async fn vendored_purl_keys("), - ] { - let text = std::fs::read_to_string(src.join(rel)) - .unwrap() - .replace("\r\n", "\n"); - assert!( - production(&text).iter().any(|l| l.starts_with(late)), - "{rel}: production code before its test module was not scanned" - ); - } - } } diff --git a/crates/socket-patch-core/src/formats/pnpm/plan.rs b/crates/socket-patch-core/src/formats/pnpm/plan.rs index 82224fa44..ed7ee2998 100644 --- a/crates/socket-patch-core/src/formats/pnpm/plan.rs +++ b/crates/socket-patch-core/src/formats/pnpm/plan.rs @@ -98,23 +98,44 @@ pub(crate) fn governing_pnpm_lock( /// rewrites, in key order: per directory, the [`governing_pnpm_lock`], read /// with the evidence `files` holds for that directory. pub(crate) fn planned_locks(files: &BTreeMap) -> Vec<&String> { - let planned = |key: &&String| { - let leaf = key.rsplit('/').next().unwrap_or(key); - if !matches!(leaf, PNPM_LOCK | PNPM_SHRINKWRAP_LEGACY) { - return false; - } - // `dir` keeps its trailing `/` (empty at the root). - let in_dir = |rel: &str| format!("{}{rel}", &key[..key.len() - leaf.len()]); - let has = |rel: &str| files.contains_key(&in_dir(rel)); - let both = (has(PNPM_LOCK), has(PNPM_SHRINKWRAP_LEGACY)); - // The pin decides only between the two, so it is read only then. - let pinned = match both { - (true, true) => pinned_pnpm_major(|rel| files.get(&in_dir(rel)).cloned()), - _ => None, - }; - governing_pnpm_lock(both.0, both.1, pinned) == Some(leaf) + files.keys().filter(|key| governs(files, key)).collect() +} + +/// The pnpm locks of `files` pnpm does not read (the complement of +/// [`planned_locks`]): never rewritten, and a Socket URL left in one pins +/// nothing. +pub(crate) fn shadowed_locks(files: &BTreeMap) -> Vec<&String> { + files + .keys() + .filter(|key| is_pnpm_lock_key(key) && !governs(files, key)) + .collect() +} + +/// Whether root-relative `key` names a `pnpm-lock.yaml` or `shrinkwrap.yaml`. +fn is_pnpm_lock_key(key: &str) -> bool { + matches!( + key.rsplit('/').next(), + Some(PNPM_LOCK | PNPM_SHRINKWRAP_LEGACY) + ) +} + +/// Whether the pnpm lock `key` is its directory's [`governing_pnpm_lock`], +/// read with the evidence `files` holds for that directory. +fn governs(files: &BTreeMap, key: &str) -> bool { + if !is_pnpm_lock_key(key) { + return false; + } + let leaf = key.rsplit('/').next().unwrap_or(key); + // `dir` keeps its trailing `/` (empty at the root). + let in_dir = |rel: &str| format!("{}{rel}", &key[..key.len() - leaf.len()]); + let has = |rel: &str| files.contains_key(&in_dir(rel)); + let both = (has(PNPM_LOCK), has(PNPM_SHRINKWRAP_LEGACY)); + // The pin decides only between the two, so it is read only then. + let pinned = match both { + (true, true) => pinned_pnpm_major(|rel| files.get(&in_dir(rel)).cloned()), + _ => None, }; - files.keys().filter(planned).collect() + governing_pnpm_lock(both.0, both.1, pinned) == Some(leaf) } /// How one `packages:` / `snapshots:` key names a patched release. @@ -199,10 +220,9 @@ mod tests { ); } - /// N11 (owner decision): of a directory's `pnpm-lock.yaml` and - /// `shrinkwrap.yaml`, the one the pinned pnpm major reads governs and - /// the other is shadowed (review finding: pnpm 1 / 2 read only the - /// shrinkwrap). + /// N11: of a directory's `pnpm-lock.yaml` and `shrinkwrap.yaml`, the + /// one the pinned pnpm major reads governs and the other is shadowed + /// (pnpm 1 / 2 read only the shrinkwrap). #[test] fn the_pinned_pnpm_major_picks_the_governing_lock() { // No pin: `pnpm-lock.yaml` governs a pair, and each lone lock itself. diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index 670c3191f..0c7dba9ab 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -23,7 +23,7 @@ use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; use serde::{Deserialize, Serialize}; use crate::api::types::PackageVendorResult; -use crate::constants::npm_family::VLT_HIDDEN_LOCK_REL; +use crate::constants::npm_family::{PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY, VLT_HIDDEN_LOCK_REL}; use crate::formats::cargo::source_replacement::CratesIoSource; use crate::formats::governing_locks::PYPI_REQUIREMENTS; use crate::patch::redirect::npmrc::{ @@ -2706,14 +2706,11 @@ fn pnpm_trust( // is the one pnpm reads (N11): `shrinkwrap.yaml` under a pnpm 1 / 2 // pin, so a stray `pnpm-lock.yaml` never drives the heal or the trust // config. - let planned = crate::formats::pnpm::plan::planned_locks(files); - let root_lock = [ - crate::constants::npm_family::PNPM_LOCK, - crate::constants::npm_family::PNPM_SHRINKWRAP_LEGACY, - ] - .into_iter() - .find(|lock| planned.iter().any(|key| key.as_str() == *lock)) - .unwrap_or(crate::constants::npm_family::PNPM_LOCK); + let shadowed = crate::formats::pnpm::plan::shadowed_locks(files); + let root_lock = match shadowed.iter().any(|key| key.as_str() == PNPM_LOCK) { + true => PNPM_SHRINKWRAP_LEGACY, + false => PNPM_LOCK, + }; let governing: Vec<&str> = std::iter::once(root_lock) .chain(member_lock_keys.iter().map(String::as_str)) .filter(|key| !rush_lock_keys.iter().any(|rush| rush == key)) @@ -3358,20 +3355,11 @@ fn confirm( // `pnpm-lock.yaml`, or the reverse under a pnpm 1 / 2 pin, N11) is // never rewritten, so a Socket URL an earlier run left in it pins // nothing either. - let planned_pnpm = crate::formats::pnpm::plan::planned_locks(files); - let is_shadowed_pnpm = |name: &str| { - matches!( - name.rsplit('/').next(), - Some( - crate::constants::npm_family::PNPM_LOCK - | crate::constants::npm_family::PNPM_SHRINKWRAP_LEGACY - ) - ) && !planned_pnpm.iter().any(|planned| planned.as_str() == name) - }; + let shadowed_pnpm = crate::formats::pnpm::plan::shadowed_locks(files); let final_texts: Vec<(&str, &String)> = files .iter() .filter(|(name, _)| !(pdm_inactive && name.as_str() == "pdm.lock")) - .filter(|(name, _)| !is_shadowed_pnpm(name)) + .filter(|(name, _)| !shadowed_pnpm.contains(name)) .filter(|(name, _)| !is_npm_manifest(name)) .filter(|(name, _)| !is_unpinned_maven(name)) .filter(|(name, _)| !crate::patch::redirect::sbt::is_synthetic_key(name)) @@ -4614,7 +4602,7 @@ mod tests { ); } - /// N11 review: under a pnpm 1 / 2 pin `shrinkwrap.yaml` governs, so a + /// N11: under a pnpm 1 / 2 pin `shrinkwrap.yaml` governs, so a /// stray v9 `pnpm-lock.yaml` an earlier run redirected is no heal root /// and plans no `trustLockfile` write for a lock pnpm never reads. #[tokio::test] @@ -4653,11 +4641,9 @@ mod tests { ); } - /// N11 review: a shadowed `shrinkwrap.yaml` still carrying an earlier - /// run's Socket URL confirms nothing when the governing - /// `pnpm-lock.yaml` no longer pins the package. It used to confirm by - /// substring, and only the attribution gate then dropped the dependency - /// as `redirect_unattributable`, blaming the project's lockfiles. + /// N11: a shadowed `shrinkwrap.yaml` still carrying an earlier run's + /// Socket URL confirms nothing (and so is no `redirect_unattributable` + /// skip) when the governing `pnpm-lock.yaml` does not pin the package. #[tokio::test] async fn a_shadowed_pnpm_lock_confirms_nothing() { let mut p = MemoryProject::new(); diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 23b292b3b..2dd3d5c86 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -28635,8 +28635,8 @@ packages: legacy_overrides_only, "redirect_pnpm_entry_not_found", ), - // Code-review finding 3: another version vendored is not this - // one (the dep's version is simply not locked). + // Another version vendored is not this one (the dep's version + // is simply not locked). ( "another version vendored", v9(&format!(".socket/vendor/npm/{uuid}/left-pad-1.2.0.tgz")) @@ -28646,9 +28646,7 @@ packages: // Lock-model 2b: the hint reads the key with the planning core's // `match_key`, the rule the vendored planner rekeys by, so the // artifact leaf names the release and the `version:` line is not - // consulted. Before, a rekey without a `version:` line was "not - // found" and one whose leaf names another release but whose - // `version:` line matched was "vendored". + // consulted. ( "a rekey without a version line", v9(&format!(".socket/vendor/npm/{uuid}/left-pad-1.3.0.tgz")) @@ -28675,7 +28673,7 @@ packages: /// only the `pnpm-lock.yaml`, so the hosted planner never rewrites the /// shrinkwrap and leaves it out of the all-or-nothing gate: debris in it /// (a key the splice cannot repoint, or pnpm 1's early shrinkwrap head) - /// no longer refuses the dependency in every lock. A `shrinkwrap.yaml` + /// refuses nothing. A `shrinkwrap.yaml` /// that is the only pnpm lock is still rewritten. #[test] fn pnpm_shrinkwrap_beside_a_pnpm_lock_is_shadowed() { diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs index 79a4ef416..745840c0a 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs @@ -1894,7 +1894,7 @@ pub(crate) async fn restore_pnpm_locks( files: &[String], ctx: &Ctx<'_>, ) -> FormatResult { - use crate::formats::pnpm::grammar as pnpm; + use crate::formats::pnpm::{grammar as pnpm, PnpmDocument, PnpmLock}; let mut result = FormatResult::default(); let pins = by_uuid(pins); @@ -1904,8 +1904,9 @@ pub(crate) async fn restore_pnpm_locks( }; // (resolution range, uuid, name, version, rebuilt-without-integrity) let mut hits: Vec<(std::ops::Range, String, String, String)> = Vec::new(); - for entry in pnpm::entries(&text) { - let Some(resolution) = pnpm::resolution(&entry) else { + let lock = PnpmLock::parse(&text); + for package in lock.packages() { + let (entry, Some(resolution)) = (&package.entry, &package.resolution) else { continue; }; let Some(uuid) = resolution.tarball().and_then(|t| ctx.hosted_uuid(t)) else { @@ -1965,22 +1966,15 @@ pub(crate) async fn restore_pnpm_locks( let mut splices: Vec<(std::ops::Range, String)> = Vec::new(); // (uuid, name@version whose `tarball:` only the guessed setting decided) let mut handled: Vec<(String, Option)> = Vec::new(); - // pnpm 11+ records the env lockfile document's resolutions bare - // whatever `lockfileIncludeTarballUrl` says (see - // `pnpm_tarball_policy`): only a URL it cannot derive gets - // `tarball:` there. - let env_spans: Vec> = pnpm::documents(&text) - .into_iter() - .filter(|(_, document)| *document == pnpm::PnpmDocument::Env) - .map(|(span, _)| span) - .collect(); - for entry in pnpm::entries(&text) { - let Some(resolution) = pnpm::resolution(&entry) else { + for package in lock.packages() { + let Some(resolution) = &package.resolution else { continue; }; - let in_env = env_spans - .iter() - .any(|span| span.contains(&resolution.range.start)); + // pnpm 11+ records the env lockfile document's resolutions bare + // whatever `lockfileIncludeTarballUrl` says (see + // `pnpm_tarball_policy`): only a URL it cannot derive gets + // `tarball:` there. + let in_env = package.document == PnpmDocument::Env; let Some((_, uuid, name, version)) = hits.iter().find(|(r, ..)| *r == resolution.range) else { continue; @@ -3866,7 +3860,7 @@ mod tests { let marker_only = format!("---\n{}", lock("")); assert!(include(&marker_only, None, Some(RC_ON), None)); // Neither is an env-shaped document AFTER the project one: pnpm - // writes the env document first (code-review finding 2). + // writes the env document first. let env_last = format!("---\n{}{env}", lock("")); assert_eq!(super::pnpm_settings_major(&pinned_only, None), Some(11)); assert_eq!(super::pnpm_settings_major(&env_last, None), None); diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs b/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs index 00a8b3f6b..e438d7499 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs @@ -106,26 +106,25 @@ async fn inventory_npm_lock_raw_in( ) -> Result)>, UnsupportedNpmLayout> { let mut found = inventory_npm_router_lock_raw_in(view).await?; if let Some((NpmLockFlavor::Pnpm | NpmLockFlavor::PnpmLegacy, entries)) = &mut found { - if !rush && lockfile_disabled(view, false).await.is_some() { - withhold_hosted_pins(entries); + // The settings are read only for a lock that pins something. + if !rush && entries.iter().any(is_hosted) && lockfile_disabled(view, false).await.is_some() + { + for entry in entries.iter_mut().filter(|e| is_hosted(e)) { + entry.integrity = LockIntegrity::None; + } } } Ok(found) } -/// Drop the integrity of every entry resolving from Socket's patch server -/// (see [`inventory_npm_lock_raw_in`]). -fn withhold_hosted_pins(entries: &mut [LockfileEntry]) { - for entry in entries { - if entry - .resolved - .as_deref() - .and_then(|url| hosted_patch_uuid(url, &[])) - .is_some() - { - entry.integrity = LockIntegrity::None; - } - } +/// Whether `entry` resolves from Socket's patch server (see +/// [`inventory_npm_lock_raw_in`]). +fn is_hosted(entry: &LockfileEntry) -> bool { + entry + .resolved + .as_deref() + .and_then(|url| hosted_patch_uuid(url, &[])) + .is_some() } /// The lock the npm flavor router picks, with its fallbacks (see diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs index 40d2a55b5..f02cc40fb 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs @@ -753,23 +753,22 @@ packages: async fn rush_json_without_any_lock_yields_none() { // rush.json but no common/subspace lock at all: nothing to inventory. let tmp = tempfile::tempdir().unwrap(); - write(tmp.path(), "rush.json", r#"{"rushVersion":"5.0.0"}"#).await; + write(tmp.path(), "rush.json", RUSH_JSON).await; assert!(inventory_npm_lock(tmp.path()).await.unwrap().is_none()); assert!(inventory_npm_trees(tmp.path()).await.unwrap().is_empty()); } /// Lock-model divergence N12: Rush's locks are their own install trees, /// read whenever `rush.json` is present — beside a root lock too, as -/// hosted mode and lockfile discovery read them. Before, a root lock -/// shadowed them (this test pinned "the root lock wins"). +/// hosted mode and lockfile discovery read them. #[tokio::test] async fn rush_locks_are_read_beside_the_root_lock() { let tmp = tempfile::tempdir().unwrap(); - write(tmp.path(), "rush.json", r#"{"rushVersion":"5.0.0"}"#).await; + write(tmp.path(), "rush.json", RUSH_JSON).await; write(tmp.path(), "pnpm-lock.yaml", PNPM_LOCK).await; write_nested( tmp.path(), - "common/config/rush/pnpm-lock.yaml", + RUSH_COMMON_LOCK, "lockfileVersion: '9.0' packages: @@ -781,7 +780,7 @@ packages: .await; // The routed lock keeps its flavor and holds only its own entries - // (code-review finding 9: Rush's are no flavor's). + // (Rush's are no flavor's). let (flavor, routed) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); assert_eq!(flavor, NpmLockFlavor::Pnpm); assert!(!routed.iter().any(|e| e.name == "only-in-common")); @@ -822,7 +821,7 @@ packages: ); } -/// Code-review finding 4: the env document's registry instance of a +/// The env document's registry instance of a /// release comes first in the lock and may carry the only verifier (the /// project's hosted instance loses its pin under `lockfile=false`), but a /// config dependency never stands in for the project's own copy: the @@ -873,7 +872,7 @@ async fn env_document_instances_never_displace_project_instances() { /// basis. Lockfile discovery keeps the ref and withholds its pin; the /// inventory keeps the entry (and its hosted url) and drops its /// integrity, so nothing fetches or verifies against it. Registry entries -/// keep theirs. Before, the inventory ignored the setting. Rush runs pnpm +/// keep theirs. Rush runs pnpm /// with settings of its own, so its locks are untouched (as in discovery). #[tokio::test] async fn pnpm_lockfile_off_withholds_hosted_pins() { @@ -918,9 +917,9 @@ async fn pnpm_lockfile_off_withholds_hosted_pins() { // Rush: the setting is not Rush's, so its lock keeps the pin. let tmp = tempfile::tempdir().unwrap(); - write(tmp.path(), "rush.json", r#"{"rushVersion":"5.0.0"}"#).await; + write(tmp.path(), "rush.json", RUSH_JSON).await; write(tmp.path(), ".npmrc", "lockfile=false\n").await; - write_nested(tmp.path(), "common/config/rush/pnpm-lock.yaml", &lock).await; + write_nested(tmp.path(), RUSH_COMMON_LOCK, &lock).await; let entries = inventory_npm_trees(tmp.path()).await.unwrap(); assert_eq!( integrity(&entries, "left-pad"), diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock.rs b/crates/socket-patch-core/src/vendor/pnpm_lock.rs index 437134a06..b9bbe5654 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock.rs @@ -893,18 +893,29 @@ pub(super) async fn preflight_packages_dialect( /// `None`: cannot determine (missing/unreadable/unsupported lock) — /// callers must keep the entry, fail-safe. pub async fn pnpm_entry_in_use(entry: &VendorEntry, project_root: &Path) -> Option { - let text = read_regular_to_string(&project_root.join(PNPM_LOCK)) - .await - .ok()?; - if PnpmDialect::V9.check_lock(&text).is_err() { - return None; - } - // Every `packages:`/`snapshots:` block key resolving into - // `.socket/vendor/npm//` — the format model's one walk - // ([`vendored_npm_uuids`], CRLF read like LF), collected once per lock - // bytes: a revert pass probes once per ledger entry. - let vendored = IN_USE_MEMO.parse_infallible(text.as_bytes(), || vendored_npm_uuids(&text)); - Some(vendored.contains(&entry.uuid)) + PnpmDialect::V9.entry_in_use(entry, project_root).await +} + +impl PnpmDialect { + /// Whether `entry`'s artifact is still consumed by a lock this dialect + /// reads: `None` when the lock is missing, unreadable or another + /// dialect's (callers keep the entry, fail-safe). + pub(super) async fn entry_in_use( + self, + entry: &VendorEntry, + project_root: &Path, + ) -> Option { + let text = read_regular_to_string(&project_root.join(PNPM_LOCK)) + .await + .ok()?; + self.check_lock(&text).ok()?; + // Every `packages:`/`snapshots:` block key resolving into + // `.socket/vendor/npm//` — the format model's one walk + // ([`vendored_npm_uuids`], CRLF read like LF), collected once per + // lock bytes: a revert pass probes once per ledger entry. + let vendored = IN_USE_MEMO.parse_infallible(text.as_bytes(), || vendored_npm_uuids(&text)); + Some(vendored.contains(&entry.uuid)) + } } /// FAIL-CLOSED revert guard for a ledger entry with NO wiring records, @@ -3850,11 +3861,7 @@ fn dep_map_refs(block: &[String]) -> Vec<(usize, &str, &str, &str)> { /// unwound on its own) — keeps that live value, so the lock stays /// resolvable whichever entry is unwound first. With `live == new` this is /// `original` verbatim. -pub(super) fn merge_live_dep_refs( - original: &[String], - new: &[String], - live: &[String], -) -> Vec { +fn merge_live_dep_refs(original: &[String], new: &[String], live: &[String]) -> Vec { let values = |block| -> HashMap<&str, &str> { dep_map_refs(block) .into_iter() @@ -3876,7 +3883,7 @@ pub(super) fn merge_live_dep_refs( /// Are two blocks equal up to their dependency-ref VALUES (what /// [`merge_live_dep_refs`] may change)? -pub(super) fn same_modulo_dep_refs(a: &[String], b: &[String]) -> bool { +fn same_modulo_dep_refs(a: &[String], b: &[String]) -> bool { let (refs_a, refs_b) = (dep_map_refs(a), dep_map_refs(b)); a.len() == b.len() && refs_a.len() == refs_b.len() @@ -3986,7 +3993,7 @@ pub(super) fn lines_value(lines: &[String]) -> Value { Value::Array(lines.iter().map(|l| Value::String(l.clone())).collect()) } -pub(super) fn value_lines(v: &Value) -> Option> { +fn value_lines(v: &Value) -> Option> { v.as_array().map(|a| { a.iter() .filter_map(Value::as_str) @@ -5792,7 +5799,7 @@ snapshots: /// Divergences N15: a CRLF pnpm-lock.yaml (a Windows autocrlf checkout) /// is wired like its LF twin and keeps its CRLF endings, as hosted /// mode's byte splices keep them, and the revert restores the CRLF - /// bytes. Vendoring used to refuse it (`vendor_lockfile_crlf_unsupported`). + /// bytes. #[tokio::test] async fn crlf_lock_is_wired_like_its_lf_twin_and_stays_crlf() { let crlf_lock = P1_BEFORE_LOCK.replace('\n', "\r\n"); @@ -6124,9 +6131,8 @@ snapshots: } /// N15 on pnpm 11's two-document lock: vendored, then checked out with - /// autocrlf (`---\r` separators), the lock reverts to its CRLF bytes. - /// Review finding: the revert's document check read the raw CRLF text, - /// refused `---\r` as a marker line, and failed every such revert. + /// autocrlf (`---\r` separators), the lock reverts to its CRLF bytes: + /// the document check reads the LF form, so `---\r` is a separator. #[tokio::test] async fn crlf_two_document_lock_reverts() { let before = two_doc(ENV_DOC_CONFIG_DEP, P1_BEFORE_LOCK); @@ -6141,10 +6147,9 @@ snapshots: assert_eq!(fx.read(PNPM_LOCK).await, before.replace('\n', "\r\n")); } - /// Review finding: a revert met a lock or workspace file that MIXES CRLF - /// and LF and wrote it back all-LF. It now fails before any write, dry - /// run included, as vendoring refuses such a file: every surface and the - /// artifact stay as they are. + /// A revert over a lock or workspace file that MIXES CRLF and LF fails + /// before any write, dry run included, as vendoring refuses such a + /// file: every surface and the artifact stay as they are. #[tokio::test] async fn mixed_line_ending_revert_fails_before_any_write() { for file in [PNPM_LOCK, PNPM_WORKSPACE] { @@ -6578,7 +6583,7 @@ snapshots: // Future versions: the fix is an allowlisted-emitting pnpm or a // newer build, never "pnpm >= 9" (the user's pnpm already is). - // An unparseable major (review finding) is not "old" either. + // An unparseable major is not "old" either. for head in [ "lockfileVersion: '9.1'\n", "lockfileVersion: '10.0'\n", @@ -7452,7 +7457,7 @@ snapshots: /// N15 for pnpm-workspace.yaml: a CRLF file (Windows autocrlf checkout) /// is read like its LF twin, so the existing `overrides:` section is found /// and extended (never a DUPLICATE section, which pnpm rejects), and the - /// file keeps its CRLF endings. Vendoring used to refuse the file. + /// file keeps its CRLF endings. #[tokio::test] async fn crlf_workspace_file_gains_the_override_in_its_own_section() { let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; @@ -9969,9 +9974,8 @@ snapshots: // Lock-model 2b (divergence N5, vendored planner): the planner reads // its own rekeyed keys with the planning core's `match_key` // (`origin::vendor_ref`, root-anchored), as hosted mode and discovery - // do. Before, `parse_vendor_path` found the anchor anywhere, so a - // sibling checkout's `../.socket/vendor/…` rekey was "ours" and got - // rekeyed into this project's artifact. + // do, so a sibling checkout's `../.socket/vendor/…` rekey is not + // ours. let art = format!(".socket/vendor/npm/{UUID}/left-pad-1.3.0.tgz"); let target = |key: &str| target_key(key, "left-pad", "1.3.0"); assert_eq!(target(&format!("left-pad@file:{art}")), Some(true)); diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs b/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs index 0cbf3e33e..acc5ee6e7 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs @@ -59,7 +59,6 @@ use serde_json::Value; use crate::manifest::schema::PatchRecord; use crate::patch::apply::PatchSources; -use crate::utils::fs::read_regular_to_string; use super::common::refused; use super::path::parse_vendor_path_or_machine_absolute; @@ -76,7 +75,7 @@ use crate::formats::pnpm::lines::{ next_block, parse_key_line, section_bounds, split_lines, unquote_value, yaml_key, yaml_key_like, yaml_value, }; -use crate::formats::pnpm::{PnpmLock, PnpmLockGrammar}; +use crate::formats::pnpm::PnpmLockGrammar; /// The [`VendorEntry::flavor`] string legacy wirings are stamped with. /// Distinct from the v9 backend's `"pnpm"` so an older binary (which has no @@ -330,11 +329,7 @@ pub(crate) async fn preflight_packages( /// (the `overrides:` declaration alone never counts); `None` when /// undeterminable — callers keep the entry, fail-safe. pub async fn pnpm_legacy_entry_in_use(entry: &VendorEntry, project_root: &Path) -> Option { - let text = read_regular_to_string(&project_root.join(PNPM_LOCK)) - .await - .ok()?; - PnpmDialect::Legacy.check_lock(&text).ok()?; - Some(PnpmLock::parse(&text).vendored_in_use(&entry.uuid)) + PnpmDialect::Legacy.entry_in_use(entry, project_root).await } impl LegacyLock { diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 2154feb56..73899292f 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -1320,9 +1320,9 @@ impl Discovery { .iter() .find(|c| c.purl == r.purl && c.file == r.source_file && r.part.withheld_by(c.part)) .cloned(); - let file = r.source_file.to_string_lossy().into_owned(); - let label = r.part.label(&file); if let Some(c) = copy { + let file = r.source_file.to_string_lossy().into_owned(); + let label = r.part.label(&file); let entry = if c.part == r.part { "" } else { @@ -1347,6 +1347,8 @@ impl Discovery { e.file == r.source_file && e.part == TreePart::File && e.covers(&r.purl) }); if registry { + let file = r.source_file.to_string_lossy().into_owned(); + let label = r.part.label(&file); self.diag( DIAG_REF_UNATTRIBUTABLE, &file, @@ -1499,6 +1501,14 @@ impl Discovery { /// never leaves it attested in its place. The project twin counts /// wherever an extractor put it (`refs`, `shadowed` or `rewirable`). fn drop_env_twins(&mut self) { + let lists = [&self.refs, &self.shadowed, &self.rewirable]; + if !lists + .into_iter() + .flatten() + .any(|r| r.part == TreePart::PnpmEnv) + { + return; + } let project: HashSet<(PathBuf, String, String, WiringMode)> = [&self.refs, &self.shadowed, &self.rewirable] .into_iter() diff --git a/crates/socket-patch-core/src/vex/discover/npm.rs b/crates/socket-patch-core/src/vex/discover/npm.rs index 40e8ee247..d8e8efcce 100644 --- a/crates/socket-patch-core/src/vex/discover/npm.rs +++ b/crates/socket-patch-core/src/vex/discover/npm.rs @@ -3415,13 +3415,11 @@ mod tests { } /// pnpm 11+'s env lockfile document is read as an install tree of its - /// own (owner decision, lock-model divergence N4). Before, discovery - /// skipped it (#1949), so a Socket wiring of a config dependency was - /// no ref although the hosted planner writes it. Now it is a ref, but - /// one the project document of the same lock withholds when it resolves - /// the same release from the registry (rewirable), a `file:` copy - /// (shadowed) or a bundled copy (unattested): a VEX statement covers the - /// release, not one tree (owner decision on #2009). The env document's + /// own (divergence N4): a Socket wiring of a config dependency is a + /// ref, but one the project document of the same lock withholds when it + /// resolves the same release from the registry (rewirable), a `file:` + /// copy (shadowed) or a bundled copy (unattested): a VEX statement + /// covers the release, not one tree (owner decision). The env document's /// entries never contest the project document's refs; its wiring does /// not exempt the project document from contesting another lock; and /// it contests no other lock's ref. @@ -3496,8 +3494,7 @@ mod tests { Withheld::Unattested => &out.refs, }; assert_eq!(withheld.len(), 1, "{case}: {out:#?}"); - // The real lock path, with the tree carried beside it - // (code-review finding 6: no pseudo path). + // The real lock path, with the tree carried beside it. assert_eq!( ( withheld[0].purl.as_str(), @@ -3581,7 +3578,7 @@ mod tests { assert!(out.contested.is_empty(), "{:#?}", out.contested); // Wired in both documents, the pins differing: ONE ref, the project - // document's (code-review finding 6: no twin refs of one file). + // document's. let env_wired = wired.replace(SRI, "sha512-ENVPIN=="); let p = Project::new(); p.write( @@ -4326,7 +4323,7 @@ mod tests { out.recognized_files(UUID_A, WiringMode::Hosted), vec![std::path::Path::new("shrinkwrap.yaml")] ); - // N11 review finding: a project that pins pnpm 1 / 2 installs from + // N11: a project that pins pnpm 1 / 2 installs from // `shrinkwrap.yaml`, so there it is the lock that attests and the // stray `pnpm-lock.yaml` is the shadowed one, as the hosted planner // picks them (`plan::governing_pnpm_lock`). diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md index f9619b895..e79113a10 100644 --- a/docs/design/lock-model-divergences.md +++ b/docs/design/lock-model-divergences.md @@ -33,7 +33,7 @@ follow from the install-tree decision. N11 was decided on 2026-11-05. | J12 | scala-cli | Guidance only (`patch/redirect/scala_guidance.rs`) | Back-port: hosted wires scala-cli through the merged planner (a same-GAV Socket repository instead of a suffixed version). | | P3 | Is a shadowed `pdm.lock` rewritten? | Only when it governs (`pdm_drives`, ~564) | Rewrite it like every other tree, per the install-tree decision. | | P4 | Are `-r` includes of `requirements.txt` rewritten? | Root file only (`patch/redirect/requirements.rs` ~208) | Follow includes, per the install-tree decision. An include outside the repository root stays bounded in workspace runs: never read or rewritten (owner, 2026-11-05; #2025 (draft)). | -| N11 | Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` rewritten? | Yes; its all-or-nothing gate covers both files | **Resolved (owner: shadowed) in PR 2b (#2029).** Swept for stale identities, never written, and out of the all-or-nothing gate (`formats::pnpm::plan::governing_pnpm_lock`, which hosted and discovery share). When the project pins pnpm 1 / 2 (install record or package.json `packageManager`), which reads only `shrinkwrap.yaml`, the stray `pnpm-lock.yaml` is the shadowed one. pnpm never reads it, and an unparseable debris key in it used to refuse the dependency in every lock. A `shrinkwrap.yaml` that is the only pnpm lock is rewritten as before. | +| N11 | Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` rewritten? | Yes; its all-or-nothing gate covers both files | **Resolved (owner: shadowed) in #2009**; see N11 under npm. | ## JVM @@ -84,19 +84,19 @@ follow from the install-tree decision. N11 was decided on 2026-11-05. | N1 | pnpm `gitBranchLockfile` with a branch lock present: is the root lock the install tree? | Hosted withholds the root locks and refuses (`hosted/engine.rs` ~930). Vendored refuses. Discovery and the inventory never check the setting. | Hosted. Discovery withholds pnpm root and member-lock attestations whenever any `pnpm-lock..yaml` exists, whatever the settings or environment say (owner, 2026-11-05; #2010 (draft)). | false attestation | | N2 | npm user patches (#711) and `bun patch` (#367) | Hosted skips that one dependency and records it as refused. Vendored refuses the whole npm v4 lock, or the one bun dependency. Discovery never reads `patchedDependencies` / `"patched"` and attests a pre-#711 hosted pin. | Hosted's per-dependency skip, in discovery too, for every package manager's user patches (owner decision; #2010 (draft) for npm and bun, #2018 (draft) for pnpm, yarn and vlt). The override is the existing `--allow-patch-conflicts`; `vex` never attests under a user patch, even with it. | false attestation | | N3 | A workspace member's stray npm/bun/vlt lock | Discovery treats it as absent. Hosted and vendored refuse. The inventory reads it as the project's lock. | The inventory ignores it | | -| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by position and content (the env document is the first one, and only when its importers say so). **Resolved in PR 2a (#2009)**: every reader reads both documents through `PnpmLock`, each entry tagged by document; discovery records carry the env document's install tree (`TreePart::PnpmEnv`) beside the real lock path, and that tree neither contests nor is contested by another lock, and never contests the project document; the project document's registry entry, `file:` copy or bundled copy of the same release does withhold an env ref (owner decision on #2009: a VEX statement covers the release, not one tree); the env document is the FIRST document when its importers carry only `configDependencies` / `packageManagerDependencies` (pnpm's own position rule), and every other document is read as the project. | | -| N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere. The vendored writers, reverters and GC claim only this project's vendor paths in #2025 (draft). pnpm part **resolved in PR 2a (#2009)**: the hosted pnpm "vendored?" check reads the rekeyed packages entry with `origin::vendor_ref`. **PR 2b (#2029)**: both modes read pnpm keys with one rule, `formats::pnpm::plan::match_key` (`origin::vendor_ref` plus the artifact leaf), so the vendored planner no longer takes a sibling checkout's rekey for its own. The vendored reverter still finds its wiring with `parse_vendor_path` (values such as pnpm 7/8's absolute specifier need it). | revert could rewrite another project's wiring | +| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree. **Resolved in #2009**: both documents are read through `PnpmLock`; the env document (the first, when its importers carry only `configDependencies` / `packageManagerDependencies`) is its own install tree (`TreePart::PnpmEnv`, see its docs), and the project document's copies of a release withhold an env ref of it (owner decision: a VEX statement covers the release, not one tree). | | +| N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere. The vendored writers, reverters and GC claim only this project's vendor paths in #2025 (draft). pnpm part **resolved in #2009**: both modes read pnpm keys with one rule, `formats::pnpm::plan::match_key` (`origin::vendor_ref` plus the artifact leaf), so neither takes a sibling checkout's rekey for its own. The vendored reverter still finds its wiring with `parse_vendor_path` (values such as pnpm 7/8's absolute specifier need it). | revert could rewrite another project's wiring | | N6 | `npm-shrinkwrap.json` and `package-lock.json` | Hosted, vendored and discovery read both, with a contest. The inventory reads the shrinkwrap only. | Both, everywhere | | | N7 | npm lockfileVersion | Hosted rewrites v1 and v4. Vendored accepts v2/v3 only. The inventory ignores v1. Discovery reads v1 and v2+. | Hosted, once vendored `file:` wiring under npm 6 and npm 12.1 v4 has fixtures | | | N8 | npm `file:` directory link identity (#1481) | Hosted and discovery give a versionless link its source directory's `name@version`. Vendored matches the entry's own version, wiring the registry copies with no warning. | Hosted | silent vendored success | | N9 | yarn classic blocks reached only through an alias | Hosted skips them. Vendored rewrites them. Discovery counts an alias block as an install of the real package. | Back-port: hosted rewrites alias blocks (vendored supports what hosted lacks) | | | N10 | Locks of several npm-family package managers in one project | Hosted rewrites every family; discovery reads every family with the contest. The inventory and vendored pick one family (vlt > bun > pnpm > yarn > npm). | Union (separate trees) | | -| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | **Resolved (owner: shadowed) in PR 2b (#2029)**: hosted never rewrites it and leaves it out of the gate; discovery keeps sweeping it; vendored never reads it. A pnpm 1 / 2 pin flips the pair: `shrinkwrap.yaml` governs, `pnpm-lock.yaml` is shadowed, and vendored mode refuses the project (`vendor_lockfile_version_unsupported`). A lone `shrinkwrap.yaml` keeps today's behavior. | | -| N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. **Resolved in PR 2a (#2009)** (the inventory reads them whenever `rush.json` exists). | target-specific (vendored) | +| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | **Resolved (owner: shadowed) in #2009**: hosted never rewrites it and leaves it out of the gate; discovery keeps sweeping it; vendored never reads it. A pnpm 1 / 2 pin flips the pair: `shrinkwrap.yaml` governs, `pnpm-lock.yaml` is shadowed, and vendored mode refuses the project (`vendor_lockfile_version_unsupported`). A lone `shrinkwrap.yaml` keeps today's behavior. | | +| N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. **Resolved in #2009** (the inventory reads them whenever `rush.json` exists). | target-specific (vendored) | | N13 | Plug'n'Play | Vendored refuses. The hosted berry preflight has no PnP gate. The inventory returns an error, so `scan` sees nothing in a project hosted pinned. | The inventory reads PnP locks. Vendored keeps its refusal. | target-specific (vendored) | -| N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery. **Resolved in PR 2a (#2009)**: under the setting (not under Rush), the inventory keeps a Socket-hosted pnpm entry but drops its integrity, as discovery withholds the lock pin. | target-specific | -| N15 | CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml` | Vendored refuses; the rest handle it | Handle it everywhere. **Resolved in PR 2b (#2029)**: the vendored planner and reverter edit a CRLF lock or workspace file like its LF twin and write it back CRLF. Only a file mixing CRLF and LF is still refused (`vendor_lockfile_crlf_unsupported`), since no single ending writes it back byte-faithfully. | | -| N16 | Lock versions | pnpm: vendored accepts only 5.4/6.0/9.0; hosted any but early shrinkwrap. vlt: vendored refuses a lock with no version; hosted rewrites and warns. | Hosted. **pnpm, PR 2b (#2029)**: two target-specific checks: hosted refuses only pnpm 1's early shrinkwrap (`plan::hosted_refuses_lock`), and the vendored dialects route by `formats::pnpm::sniff_lock_grammar` (`PnpmDialect::check_lock`). Vendored keeps its 5.4/6.0/9.0 range: no other version has a captured pnpm `file:` override shape that proves its rekey, so the other versions stay refused, target-specifically. vlt is unchanged. | target-specific (pnpm vendored) | +| N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery. **Resolved in #2009**: under the setting (not under Rush), the inventory keeps a Socket-hosted pnpm entry but drops its integrity, as discovery withholds the lock pin. | target-specific | +| N15 | CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml` | Vendored refuses; the rest handle it | Handle it everywhere. **Resolved in #2009**: the vendored planner and reverter edit a CRLF lock or workspace file like its LF twin and write it back CRLF. Only a file mixing CRLF and LF is still refused (`vendor_lockfile_crlf_unsupported`), since no single ending writes it back byte-faithfully. | | +| N16 | Lock versions | pnpm: vendored accepts only 5.4/6.0/9.0; hosted any but early shrinkwrap. vlt: vendored refuses a lock with no version; hosted rewrites and warns. | Hosted. **pnpm, #2009**: two target-specific checks: hosted refuses only pnpm 1's early shrinkwrap (`plan::hosted_refuses_lock`), and the vendored dialects route by `formats::pnpm::sniff_lock_grammar` (`PnpmDialect::check_lock`). Vendored keeps its 5.4/6.0/9.0 range: no other version has a captured pnpm `file:` override shape that proves its rekey, so the other versions stay refused, target-specifically. vlt is unchanged. | target-specific (pnpm vendored) | | N17 | Bun workspaces, yarn classic offline mirror | Refused differently by the two modes | Stays | target-specific | | N18 | An unparseable npm lock | Hosted warns and skips. Vendored refuses the primary lock. Discovery diagnoses. The inventory returns nothing silently. | Discovery's diagnostic, surfaced by every consumer | | | N19 | yarn.lock with a stray top-level line | Discovery rejects the file; hosted and the inventory scan its blocks | Discovery's rule | | From aa9f5c988c8487647a65e8b2ec02fe12826d7cdd Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Mon, 9 Nov 2026 16:47:34 -0500 Subject: [PATCH 35/36] Address the code review of the simplify pass - plan::root_lock picks the root's governing pnpm lock directly: the simplify pass derived it from the shadowed set, which named a missing pnpm-lock.yaml when shrinkwrap.yaml is the only lock. One pnpm_lock_leaf helper backs planned_locks, shadowed_locks and governs. - Remove PnpmLock::vendored_in_use and its cache, which no production code calls since both dialects use PnpmDialect::entry_in_use; the tests read vendored_npm_uuids directly. - parse_keys documents that it leaves every entry tagged Project, and PnpmLock::documents keeps collecting the tags actually present. - drop_env_twins reuses its list array; the unification doc notes that #2029 was folded into #2009. Co-Authored-By: Claude Opus 5.5 --- .../socket-patch-core/src/formats/pnpm/mod.rs | 52 +++++++------------ .../src/formats/pnpm/plan.rs | 40 ++++++++++---- crates/socket-patch-core/src/hosted/engine.rs | 9 +--- .../socket-patch-core/src/vex/discover/mod.rs | 30 +++++------ docs/design/lock-model-unification.md | 4 +- 5 files changed, 67 insertions(+), 68 deletions(-) diff --git a/crates/socket-patch-core/src/formats/pnpm/mod.rs b/crates/socket-patch-core/src/formats/pnpm/mod.rs index 177fe4581..1cc44d1f2 100644 --- a/crates/socket-patch-core/src/formats/pnpm/mod.rs +++ b/crates/socket-patch-core/src/formats/pnpm/mod.rs @@ -13,8 +13,8 @@ //! tarball, the raw material lockfile discovery (`vex::discover::npm`) //! classifies as hosted / vendored refs, and [`PnpmLock::wired_integrity`] //! repair's trust anchor for a vendored artifact; -//! * [`PnpmLock::vendored_in_use`] — whether a vendored artifact is still -//! consumed (both vendored backends' revert guards and ledger liveness); +//! * [`vendored_npm_uuids`] — the vendored artifacts a lock still consumes +//! (both vendored backends' revert guards and ledger liveness); //! * [`plan_hosted`] — the hosted planner (the redirect rewriter's pnpm //! leg: resolution splices over the whole lock set); //! * the vendored planners (`vendor::pnpm_lock` for lockfileVersion 9.0, @@ -41,7 +41,6 @@ pub(crate) use grammar::{ pub(crate) use hosted::plan_hosted; use std::collections::HashSet; -use std::sync::OnceLock; use crate::constants::npm_family::PNPM_LOCK; use crate::formats::text::strip_bom; @@ -380,9 +379,6 @@ pub struct PnpmLock<'t> { text: &'t str, /// Every document's entries, with offsets into `text`. packages: Vec>, - /// [`vendored_npm_uuids`] of the whole lock, computed on first use (the - /// store crawler never asks). - vendored: OnceLock>, } /// One `packages:` entry whose resolution names a tarball — a candidate @@ -400,18 +396,17 @@ impl<'t> PnpmLock<'t> { PnpmLock { text, packages: pnpm_packages(text, Resolve::Yes), - vendored: OnceLock::new(), } } /// [`PnpmLock::parse`] without the resolution maps (every - /// [`PnpmPackage::resolution`] is `None`), for a reader of keys and - /// entry fields only. + /// [`PnpmPackage::resolution`] is `None`) or document tags (every + /// entry is [`PnpmDocument::Project`]), for a reader of keys and entry + /// fields only: its document queries mean nothing. fn parse_keys(text: &'t str) -> Self { PnpmLock { text, packages: pnpm_packages(text, Resolve::No), - vendored: OnceLock::new(), } } @@ -440,10 +435,10 @@ impl<'t> PnpmLock<'t> { /// The document kinds the lock has entries in, project first. pub(crate) fn documents(&self) -> Vec { - [PnpmDocument::Project, PnpmDocument::Env] - .into_iter() - .filter(|kind| self.packages.iter().any(|p| p.document == *kind)) - .collect() + let mut kinds: Vec = self.packages.iter().map(|p| p.document).collect(); + kinds.sort(); + kinds.dedup(); + kinds } fn has_packages_section(&self) -> bool { @@ -520,19 +515,6 @@ impl<'t> PnpmLock<'t> { .find_map(|r| r.integrity.filter(|sri| is_sri_pin(sri))) .map(str::to_string) } - - /// Is the vendored npm artifact of patch `uuid` still consumed by this - /// lock? `true` when a `packages:` / `snapshots:` block is keyed by it - /// ([`vendored_npm_uuid`] — v9's `name@file:` and legacy's bare `file:` - /// keys alike); `false` when the lock carries none (the `overrides:` - /// declaration alone never counts: pnpm keeps it mirrored from - /// package.json even when nothing matches it). CRLF locks read like LF - /// ones. - pub fn vendored_in_use(&self, uuid: &str) -> bool { - self.vendored - .get_or_init(|| vendored_npm_uuids(self.text)) - .contains(uuid) - } } /// The packages a pnpm "current lockfile" — the `lock.yaml` pnpm writes in @@ -638,7 +620,9 @@ impl InstalledPackages { /// into `.socket/vendor/npm//` — the block-key grammar the vendored /// planners splice with ([`lines::parse_key_line`] at two-space indent), /// read in one walk with each line's `\r` dropped, so a CRLF lock (a -/// Windows autocrlf checkout) answers like its LF twin. +/// Windows autocrlf checkout) answers like its LF twin. An `overrides:` +/// declaration alone never counts: pnpm keeps it mirrored from +/// package.json even when nothing matches it. pub(crate) fn vendored_npm_uuids(text: &str) -> HashSet { let mut out = HashSet::new(); let mut in_section = false; @@ -822,7 +806,7 @@ mod tests { ); assert!(lock.resolves("left-pad", "1.3.0")); assert!(lock.resolves("cfg-dep", "2.0.0")); - assert!(lock.vendored_in_use(UUID)); + assert!(vendored_npm_uuids(&text).contains(UUID)); let tagged: Vec<_> = lock .packages() .iter() @@ -947,7 +931,7 @@ mod tests { } #[test] - fn vendored_in_use_reads_v9_and_legacy_keys_and_crlf() { + fn vendored_npm_uuids_read_v9_and_legacy_keys_and_crlf() { let v9 = format!( "lockfileVersion: '9.0'\n\npackages:\n\n a@file:.socket/vendor/npm/{UUID}/a-1.0.0.tgz:\n resolution: {{integrity: sha512-x, tarball: file:.socket/vendor/npm/{UUID}/a-1.0.0.tgz}}\n version: 1.0.0\n" ); @@ -958,12 +942,12 @@ mod tests { "lockfileVersion: '9.0'\n\nsnapshots:\n\n a@file:.socket/vendor/npm/{UUID}/a-1.0.0.tgz: {{}}\n" ); for text in [&v9, &legacy, &snapshot] { - assert!(PnpmLock::parse(text).vendored_in_use(UUID), "{text}"); + assert!(vendored_npm_uuids(text).contains(UUID), "{text}"); let other = "22222222-2222-4222-8222-222222222222"; - assert!(!PnpmLock::parse(text).vendored_in_use(other)); + assert!(!vendored_npm_uuids(text).contains(other)); let crlf = text.replace('\n', "\r\n"); assert!( - PnpmLock::parse(&crlf).vendored_in_use(UUID), + vendored_npm_uuids(&crlf).contains(UUID), "CRLF reads like LF" ); } @@ -971,7 +955,7 @@ mod tests { let overrides = format!( "lockfileVersion: '9.0'\n\noverrides:\n a@1.0.0: file:.socket/vendor/npm/{UUID}/a-1.0.0.tgz\n" ); - assert!(!PnpmLock::parse(&overrides).vendored_in_use(UUID)); + assert!(!vendored_npm_uuids(&overrides).contains(UUID)); } #[test] diff --git a/crates/socket-patch-core/src/formats/pnpm/plan.rs b/crates/socket-patch-core/src/formats/pnpm/plan.rs index ed7ee2998..b3fd7c6b4 100644 --- a/crates/socket-patch-core/src/formats/pnpm/plan.rs +++ b/crates/socket-patch-core/src/formats/pnpm/plan.rs @@ -107,25 +107,33 @@ pub(crate) fn planned_locks(files: &BTreeMap) -> Vec<&String> { pub(crate) fn shadowed_locks(files: &BTreeMap) -> Vec<&String> { files .keys() - .filter(|key| is_pnpm_lock_key(key) && !governs(files, key)) + .filter(|key| pnpm_lock_leaf(key).is_some() && !governs(files, key)) .collect() } -/// Whether root-relative `key` names a `pnpm-lock.yaml` or `shrinkwrap.yaml`. -fn is_pnpm_lock_key(key: &str) -> bool { - matches!( - key.rsplit('/').next(), - Some(PNPM_LOCK | PNPM_SHRINKWRAP_LEGACY) - ) +/// The root directory's governing pnpm lock ([`governing_pnpm_lock`]), +/// `pnpm-lock.yaml` when `files` holds neither. +pub(crate) fn root_lock(files: &BTreeMap) -> &'static str { + [PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY] + .into_iter() + .find(|lock| files.contains_key(*lock) && governs(files, lock)) + .unwrap_or(PNPM_LOCK) +} + +/// The leaf of root-relative `key` when it names a `pnpm-lock.yaml` or +/// `shrinkwrap.yaml`. +fn pnpm_lock_leaf(key: &str) -> Option<&str> { + key.rsplit('/') + .next() + .filter(|leaf| matches!(*leaf, PNPM_LOCK | PNPM_SHRINKWRAP_LEGACY)) } /// Whether the pnpm lock `key` is its directory's [`governing_pnpm_lock`], /// read with the evidence `files` holds for that directory. fn governs(files: &BTreeMap, key: &str) -> bool { - if !is_pnpm_lock_key(key) { + let Some(leaf) = pnpm_lock_leaf(key) else { return false; - } - let leaf = key.rsplit('/').next().unwrap_or(key); + }; // `dir` keeps its trailing `/` (empty at the root). let in_dir = |rel: &str| format!("{}{rel}", &key[..key.len() - leaf.len()]); let has = |rel: &str| files.contains_key(&in_dir(rel)); @@ -288,6 +296,18 @@ mod tests { planned(&record).last().map(String::as_str), Some("shrinkwrap.yaml") ); + + // The root's governing lock, and the shadowed rest. + assert_eq!(root_lock(&files(None)), PNPM_LOCK); + assert_eq!(root_lock(&files(Some("2.25.7"))), PNPM_SHRINKWRAP_LEGACY); + assert_eq!( + shadowed_locks(&files(Some("2.25.7"))), + ["a/shrinkwrap.yaml", "pnpm-lock.yaml"] + ); + let lone: BTreeMap = + [(PNPM_SHRINKWRAP_LEGACY.to_string(), String::new())].into(); + assert_eq!(root_lock(&lone), PNPM_SHRINKWRAP_LEGACY); + assert_eq!(root_lock(&BTreeMap::new()), PNPM_LOCK); } #[test] diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index 0c7dba9ab..845e45f23 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -23,7 +23,7 @@ use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; use serde::{Deserialize, Serialize}; use crate::api::types::PackageVendorResult; -use crate::constants::npm_family::{PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY, VLT_HIDDEN_LOCK_REL}; +use crate::constants::npm_family::VLT_HIDDEN_LOCK_REL; use crate::formats::cargo::source_replacement::CratesIoSource; use crate::formats::governing_locks::PYPI_REQUIREMENTS; use crate::patch::redirect::npmrc::{ @@ -2706,12 +2706,7 @@ fn pnpm_trust( // is the one pnpm reads (N11): `shrinkwrap.yaml` under a pnpm 1 / 2 // pin, so a stray `pnpm-lock.yaml` never drives the heal or the trust // config. - let shadowed = crate::formats::pnpm::plan::shadowed_locks(files); - let root_lock = match shadowed.iter().any(|key| key.as_str() == PNPM_LOCK) { - true => PNPM_SHRINKWRAP_LEGACY, - false => PNPM_LOCK, - }; - let governing: Vec<&str> = std::iter::once(root_lock) + let governing: Vec<&str> = std::iter::once(crate::formats::pnpm::plan::root_lock(files)) .chain(member_lock_keys.iter().map(String::as_str)) .filter(|key| !rush_lock_keys.iter().any(|rush| rush == key)) .collect(); diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 73899292f..917c8795e 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -1503,26 +1503,26 @@ impl Discovery { fn drop_env_twins(&mut self) { let lists = [&self.refs, &self.shadowed, &self.rewirable]; if !lists - .into_iter() + .iter() + .copied() .flatten() .any(|r| r.part == TreePart::PnpmEnv) { return; } - let project: HashSet<(PathBuf, String, String, WiringMode)> = - [&self.refs, &self.shadowed, &self.rewirable] - .into_iter() - .flatten() - .filter(|r| r.part == TreePart::File) - .map(|r| { - ( - r.source_file.clone(), - r.purl.clone(), - r.uuid.clone(), - r.mode, - ) - }) - .collect(); + let project: HashSet<(PathBuf, String, String, WiringMode)> = lists + .into_iter() + .flatten() + .filter(|r| r.part == TreePart::File) + .map(|r| { + ( + r.source_file.clone(), + r.purl.clone(), + r.uuid.clone(), + r.mode, + ) + }) + .collect(); for refs in [&mut self.refs, &mut self.shadowed, &mut self.rewirable] { refs.retain(|r| { r.part == TreePart::File diff --git a/docs/design/lock-model-unification.md b/docs/design/lock-model-unification.md index f6d8196f2..448294e08 100644 --- a/docs/design/lock-model-unification.md +++ b/docs/design/lock-model-unification.md @@ -81,7 +81,7 @@ Non-goal: adding new hosted or vendored backends. | May a reader or writer read project files outside the git repository root? | Several do: vendored paths spelled `../.socket/vendor/…`, an ancestor `go.work`, out-of-root `-r` includes, parent NuGet configs. | **Owner decision:** no. Every project-file read is bounded to the git repository root (the project root when there is no repository). Machine-level tool configuration (for example `GOENV`, the user `NuGet.Config`) is listed separately for review. **2026-11-05:** the bound applies to workspace runs only. Global runs (`--global` / `-g` / `--global-prefix`) read anywhere. Without a git checkout, upward lookups (an ancestor `go.work`, parent NuGet configs, workspace and settings files) behave as before (#2025). | | In a workspace run, what if a file the tool honors sits above the repository root (an ancestor `pnpm-workspace.yaml`, `.npmrc`, `go.work`, `.cargo/config.toml`)? | Each walk reads it, or stops at a different place. | **Owner decision, 2026-11-05:** such a marker file above the repository root and below the home directory makes hosted and vendored mode refuse with a message naming the file, and readers withhold. Nothing at or above home counts (#2025). | | Do out-of-repository `-r` includes of `requirements.txt` count? | Hosted rewrites the root file only; the inventory and discovery follow includes wherever they point. | **Owner decision, 2026-11-05:** includes are followed (P4), but an include outside the repository root stays bounded in workspace runs: never read or rewritten (#2025). | -| Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` read (N11)? | Hosted rewrites both under one all-or-nothing gate. Discovery sweeps it as ignored. The inventory reads it as a fallback. Vendored never reads it. | **Owner decision, 2026-11-05:** it is shadowed: swept for stale identities, never rewritten, and outside the gate. If the project's pnpm evidence (install record or `package.json` `packageManager`) pins pnpm 1 or 2, which read only `shrinkwrap.yaml`, then `shrinkwrap.yaml` governs and vendored mode refuses the project (#2029). | +| Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` read (N11)? | Hosted rewrites both under one all-or-nothing gate. Discovery sweeps it as ignored. The inventory reads it as a fallback. Vendored never reads it. | **Owner decision, 2026-11-05:** it is shadowed: swept for stale identities, never rewritten, and outside the gate. If the project's pnpm evidence (install record or `package.json` `packageManager`) pins pnpm 1 or 2, which read only `shrinkwrap.yaml`, then `shrinkwrap.yaml` governs and vendored mode refuses the project (#2009). | | Does a pnpm branch lock (`pnpm-lock..yaml`) make the root lock unreliable (N1)? | Hosted withholds the root locks and refuses when `gitBranchLockfile` is set. Vendored refuses. Discovery and the inventory never check. | **Owner decision, 2026-11-05:** discovery withholds pnpm root and member-lock attestations whenever any `pnpm-lock..yaml` exists, whatever the settings or environment say (#2010). | ## The model @@ -291,7 +291,7 @@ A format whose planner merge is large splits into two PRs, readers first. | 0 | `lock-model/0-design` (#2001) | This document and the size budget | | 1 | `lock-model/1-origin` (#2002) | The `origin` module: discovery's classifier and the redirect's hosted-URL rule, moved verbatim. Callers import it from `crate::origin`. | | 2a | `lock-model/2a-pnpm-readers` (#2009) | Pilot, readers. One pnpm entry walk for the inventory, discovery, the store crawler, the hosted planner and the vendored in-use check. The env-lockfile document becomes its own tree (see Decisions). | -| 2b | `lock-model/2b-pnpm-planner` (#2029) | Pilot, planner. One pnpm rewire planner for both modes. `shrinkwrap.yaml` beside `pnpm-lock.yaml` is shadowed (see Decisions). | +| 2b | `lock-model/2b-pnpm-planner` (#2029, folded into #2009) | Pilot, planner. One pnpm rewire planner for both modes. `shrinkwrap.yaml` beside `pnpm-lock.yaml` is shadowed (see Decisions). | | 3 | `lock-model/3-npm` | package-lock and shrinkwrap. Folds `vendor/npm_origin.rs`'s second walk into the records. | | 4 | `lock-model/4-yarn` | Classic and Berry. One block model for every reader and writer, `BerryStanzas` included. | | 5 | `lock-model/5-bun-vlt` | `bun.lock`, `bun.lockb` and vlt. The vlt writers move off their line grammar. | From 9bbb4b797753526d6ae30a0709a48e12528a85ce Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Mon, 9 Nov 2026 17:22:03 -0500 Subject: [PATCH 36/36] Merge main: scope #2018's user-patched marks per install tree #2018's user_patched extractor built UnwiredCopy marks with the old file-only scope; #2009 scopes them by (file, TreePart). Each mark is for exactly one ref, so it takes that ref's own (source_file, part), as npm::mark_user_patched does. A pnpm env-lockfile ref of a user-patched package is marked too: the index is per package, so a missed attestation is the safe side. Co-Authored-By: Claude Opus 5.5 --- .../src/vex/discover/user_patched.rs | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/crates/socket-patch-core/src/vex/discover/user_patched.rs b/crates/socket-patch-core/src/vex/discover/user_patched.rs index f418476db..7fd7fcede 100644 --- a/crates/socket-patch-core/src/vex/discover/user_patched.rs +++ b/crates/socket-patch-core/src/vex/discover/user_patched.rs @@ -75,10 +75,12 @@ pub(crate) async fn extract(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { if index.is_empty() { return; } - // Each mark is an `UnwiredCopy` for exactly that ref, as the npm and - // Bun extractors' (`npm::mark_user_patched`): it lands only on a ref - // that survives every contest, and a ref one of those already covers - // keeps that first reason. + // Each mark is an `UnwiredCopy` for exactly that ref, in its own tree + // of its file, as the npm and Bun extractors' (`npm::mark_user_patched`): + // it lands only on a ref that survives every contest, and a ref one of + // those already covers keeps that first reason. A pnpm env-lockfile ref + // (`TreePart::PnpmEnv`) of a user-patched package is marked too: the + // index is per package, and a missed attestation is the safe side. let marks: Vec = out .refs .iter() @@ -97,7 +99,7 @@ pub(crate) async fn extract(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { sources.join(", ") ); Some(UnwiredCopy { - scope: Some(r.source_file.clone()), + scope: Some((r.source_file.clone(), r.part)), target: CopyTarget::Ref { purl: r.purl.clone(), uuid: r.uuid.clone(),