diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index c9f242c3a..62f0bdc28 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -306,7 +306,7 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc **pnpm per-branch locks (#556)**: with `gitBranchLockfile` on (a top-level `gitBranchLockfile: true` in the root `pnpm-workspace.yaml`; when the YAML leaves the key unset, `git-branch-lockfile=true` in the root `.npmrc`, as pnpm 10 and older read it; on disk also `pnpm_config_git_branch_lockfile` (pnpm 11 and newer) or `npm_config_git_branch_lockfile` (pnpm 10 and older; either when the major is unknown, read as for the `lockfile` setting above) set to `true`; a workspace member with no `pnpm-workspace.yaml` of its own reads the governing ancestor's file and the `.npmrc` beside it), pnpm installs a git branch from its own `pnpm-lock..yaml` whenever that lock exists, in every directory it installs a lock from: the root, and under `sharedWorkspaceLockfile: false` each member's (#492). When the setting is on AND at least one such `pnpm-lock..yaml` exists (at the root, or in a member that installs from its own lock), hosted mode leaves every root pnpm lock (`pnpm-lock.yaml`, `shrinkwrap.yaml`) and `pnpm-workspace.yaml` untouched (every member lock too), confirms no npm pin from them and warns `redirect_pnpm_git_branch_lockfile` (in place of `redirect_pnpm_no_lockfile` when the branch lock is the only one); the detail names the setting and the branch lock(s), and the remedy is to turn `gitBranchLockfile` off and fold the branch locks into `pnpm-lock.yaml` with `pnpm install --merge-git-branch-lockfiles`, then re-run. A vendored→hosted takeover of a pnpm-vendored purl is refused with the same code (the vendored wiring and ledger stay byte-for-byte). Vendored mode refuses the same projects with `vendor_pnpm_git_branch_lockfile`, before any write and in the pre-flight. The setting with no branch lock (main, or a branch whose dependencies never changed) installs from `pnpm-lock.yaml`, which both modes wire as usual; a stray branch lock with the setting off is ignored. The in-memory engine fetches root `pnpm-lock.*.yaml` names presence-only and refuses the same way. `vex` omits every pin of those pnpm locks (`vex_pnpm_git_branch_lockfile`) whenever such a branch lock exists, whatever the setting says (fail-closed; read from the project's files only, not the environment); see **Unattested references**. -`redirect_pnpm_no_lockfile` names pnpm when installer markers exist without a lock; `redirect_pnpm_entry_vendored` identifies a vendored entry instead of reporting it missing. Supported `shrinkwrap.yaml` files are writable lockfiles, not read-only markers. +`redirect_pnpm_no_lockfile` names pnpm when installer markers exist without a lock; `redirect_pnpm_entry_vendored` identifies a vendored entry instead of reporting it missing. Supported `shrinkwrap.yaml` files are writable lockfiles, not read-only markers, when no `pnpm-lock.yaml` sits beside them. pnpm >= 3 never reads a `shrinkwrap.yaml` it has a `pnpm-lock.yaml` for, so that one is shadowed (unless the project pins pnpm 1 or 2 in `node_modules/.modules.yaml` or package.json's `packageManager`: those read only `shrinkwrap.yaml`, so there the stray `pnpm-lock.yaml` is the shadowed one): hosted mode never rewrites it and leaves it out of the all-or-nothing gate, so an entry it cannot repoint (or pnpm 1's early shrinkwrap head) refuses nothing, and `vex` only sweeps it for stale Socket identities. Vendored mode never writes it either, and refuses a project that pins pnpm 1 or 2 beside a `shrinkwrap.yaml` (`vendor_lockfile_version_unsupported`: it does not write the shrinkwrap grammar, and the stray `pnpm-lock.yaml` would patch nothing pnpm installs). **vlt hosted-mode contract**: `scan` / `get --mode hosted` rewrite, in `vlt-lock.json`, every default-registry node of a granted `name@version` (the `''` / `npm` segment or a URL segment equal to the lock's scalar `registry`, both DepID grammars, every peer and modifier variant): slot [2] becomes the granted sha512 and slot [3] the hosted URL (appended to a 3-tuple); the DepID, flags and trailing slots, the line ending and every other byte stay. `options` is never edited and `vlt.json` is only read. A lock with another `lockfileVersion` (decided on the raw JSON token), a BOM, a non-object body or a `nodes` section outside vlt's one-node-per-line layout refuses the whole lock (`redirect_vlt_lock_unsupported`). **Confirmation**: vlt drives when its install state (`node_modules/.vlt-lock.json` or `node_modules/.vlt/`) is present or no other npm-family lock is; then only `vlt-lock.json` confirms a uuid. Otherwise every lock is rewritten, `redirect_vlt_sibling_lockfiles` warns, and the other locks' rules confirm, including a dep `vlt-lock.json` merely does not wire (`redirect_vlt_entry_not_found`, `redirect_vlt_entry_vendored`, or `redirect_vlt_custom_registry_skipped` alone). Whichever lock drives, a dep the vlt rewriter refuses (`redirect_vlt_missing_sha512`, `redirect_vlt_unsupported_lock_key`) is never confirmed by any lock, although a sibling lock may already carry its rewritten URL. **Artifacts**: the scan never downloads the hosted artifacts; the patch server sends them unencoded (`Cache-Control: no-transform`), and vlt verifies slot [2] against the bytes it receives at install time. **Heal**: stale installed copies of Socket-owned nodes are removed so the next `vlt install` extracts the patched bytes, and `rollback` / `remove` do the same for the registry bytes (`--no-vlt-install-cleanup` keeps them; optional dependencies' copies are always kept); `redirect_vlt_reinstall_required` says what happened and what to run. The same-run `--vex` never attests a vlt package whose installed copy is stale or unchecked, whose lock a vlt release may ignore (`redirect_vlt_lockfile_version_missing`, `redirect_vlt_old_lockfile_ignored`, `redirect_vlt_scalar_registry_ignored`), or which also resolves from a non-default registry (`redirect_vlt_custom_registry_skipped`). `vlt.json` or vlt install state without `vlt-lock.json` warns `redirect_vlt_no_lockfile` instead of `redirect_npm_no_lockfile`. `rollback` / `remove` restore each hosted node's slots [2] and [3] from the version document of the registry the node resolves against (slot [3] is its `dist.tarball`, as vlt writes it; `upstream_registry_fallback` when that registry can't be read), following the lock's own slot-[3] convention (see "Hosted unwind coverage"). Tested releases: `docs/testing/vlt-compatibility.md`. @@ -562,7 +562,7 @@ Discovery is read-only, never touches the network, and never fails the run: a ma | Ecosystem | Files read | Hosted reference | Vendored reference | Hosted pin (`integrity_required`) | |---|---|---|---|---| | npm | `package-lock.json` and `npm-shrinkwrap.json` (both when both exist) | `resolved` on the patch host (`packages` in v2/v3; `dependencies` only in v1; `link` entries skipped, and so are `inBundle` / `bundled` entries a dependency bundles (a copy the root project's own `bundleDependencies` covers, and the dependencies npm installs beneath it, has no parent tarball and is rewired like any entry, #1618; a copy beneath a root-bundled package that declares its own bundle stays skipped, since the lock cannot tell which bundle carries it), and so is any entry npm installs from a git, URL or `file:` spec or from a dependency's own `npm-shrinkwrap.json` (beneath a `hasShrinkwrap: true` package, which npm 7–11 install from that shrinkwrap; `redirect_npm_shrinkwrapped_instance_skipped` / `vendor_shrinkwrapped_instance_skipped`), together with every ref for the same `name@version`) | `resolved: file:.socket/vendor/npm//-.tgz` | `integrity`, required | -| pnpm | `pnpm-lock.yaml` (every `lockfileVersion`); `shrinkwrap.yaml` only when there is no `pnpm-lock.yaml`; with `rush.json`, `common/config/rush/pnpm-lock.yaml` + `common/config/subspaces/*/pnpm-lock.yaml` | `packages:` `resolution.tarball` on the patch host | `file:.socket/vendor/npm/…` tarball + key | `integrity`, required. With pnpm's `lockfile` setting off (#1074, outside Rush) pnpm reads no lock: every hosted reference in the root and member locks keeps no pin, and one `patched_ref_unattributable` per lock names them; vendored references are unaffected. | +| pnpm | `pnpm-lock.yaml` (every `lockfileVersion`); `shrinkwrap.yaml` only when there is no `pnpm-lock.yaml`; with `rush.json`, `common/config/rush/pnpm-lock.yaml` + `common/config/subspaces/*/pnpm-lock.yaml` | `packages:` `resolution.tarball` on the patch host | `file:.socket/vendor/npm/…` tarball + key | `integrity`, required. With pnpm's `lockfile` setting off (#1074, outside Rush) pnpm reads no lock: every hosted reference in the root and member locks keeps no pin, and one `patched_ref_unattributable` per lock names them; vendored references are unaffected. pnpm 11+'s env lockfile document (config dependencies, the pinned package manager) is read as its own install tree: its references neither contest nor are contested by another lock, and its entries never contest the project document's references. One of its references is never attested while the project document of the same lock resolves the same `name@version` from elsewhere, because a VEX statement covers the release, not one install tree: a registry entry or a `file:` copy withholds it with `patched_ref_unattributable` (re-running `scan` rewires a registry entry), and a bundled copy omits it like any pnpm bundled copy (`vex_pnpm_bundled_copy`). | | yarn | `yarn.lock` (classic and berry) | classic `resolved`; berry entry keyed and resolved `@` + root `package.json` `resolutions` `"@npm:": ""` (older releases: `resolution: …::__archiveUrl=`) | classic `resolved "file:./.socket/vendor/npm/…#"`; berry `file:` entry **plus** a root `package.json` `resolutions` mapping onto the same artifact (without it the entry is orphaned: diagnosed, no ref) | classic `integrity` / `#sha1`, berry `checksum`, required | | bun | `bun.lock`; `bun.lockb` only when there is no `bun.lock` (bun reads exactly one) | URL tuple / binary remote-tarball resolution; version from the URL leaf. A user URL or `file:` tarball tuple / record whose `-.tgz` leaf names the same `name@version` is installed from its own spec, so every ref for that `name@version` is dropped | `.socket/vendor/npm//-.tgz` tuple / local-tarball resolution | `sha512-…`, required. A 2-tuple that Bun < 1.3.10 re-saved without its digest is still a reference, but it attests only from an installed tree. | | vlt | `vlt-lock.json` (lockfileVersion absent, `0` or `1`; a BOM-prefixed, unparseable, non-object or other-version lock is not read: diagnosed, no ref). `vlt.json` (read only for its `modifiers`) and `node_modules/.vlt-lock.json` are never wiring. | a registry node (any segment) whose slot [3] is a `/patch/npm/…` URL on the patch host with the leaf `-.tgz` of its DepID's `name@version`, whose embedded `/` path (when present) is that `name@version`, and slot [1] == name; version from the DepID | a `file` node `.socket/vendor/npm//-/node_modules/` (or a user-installed `-.tgz`) with slot [1] == name; version from the path. A same-`name@version` registry node, or a diagnosed Socket-shaped one, beside it is diagnosed, no ref. | slot [2] `sha512-…`, required (a hosted node without one is no reference). A same-`name@version` node on another registry, or a diagnosed Socket-shaped one, keeps the reference but withholds the lockfile basis: only an installed tree whose every store copy verifies attests. So does a lock some vlt release discards, the conditions of `redirect_vlt_lockfile_version_missing`, `redirect_vlt_old_lockfile_ignored` and `redirect_vlt_scalar_registry_ignored` (which in-run `--vex` withholds too): every hosted reference in it keeps no pin, and one `patched_ref_unattributable` names them. | diff --git a/crates/socket-patch-cli/src/commands/scan/discovery.rs b/crates/socket-patch-cli/src/commands/scan/discovery.rs index e0c691cc6..5508ce78e 100644 --- a/crates/socket-patch-cli/src/commands/scan/discovery.rs +++ b/crates/socket-patch-cli/src/commands/scan/discovery.rs @@ -722,6 +722,7 @@ mod tests { resolved: None, integrity: LockIntegrity::None, source_kind: SourceKind::Unspecified, + pnpm_env: false, } } diff --git a/crates/socket-patch-cli/tests/in_process_redirect.rs b/crates/socket-patch-cli/tests/in_process_redirect.rs index b83195572..f7e407dcb 100644 --- a/crates/socket-patch-cli/tests/in_process_redirect.rs +++ b/crates/socket-patch-cli/tests/in_process_redirect.rs @@ -6133,6 +6133,52 @@ async fn pnpm_rollback_keeps_tarball_under_workspace_include_tarball_url() { assert_eq!(pnpm_pin_and_rollback(tmp.path(), &server), pristine); } +/// pnpm 11+ records a config dependency bare in the env lockfile document +/// even under `lockfileIncludeTarballUrl`, so a rollback of a pin in both +/// documents gives the project entry its `tarball:` back and leaves the env +/// entry bare. +#[tokio::test] +#[serial] +async fn pnpm_rollback_keeps_the_env_document_bare_under_include_tarball_url() { + let server = MockServer::start().await; + mock_discovery(&server).await; + mock_reference(&server).await; + mock_view(&server).await; + mock_npm_registry(&server, "sha512-UPSTREAMupstream==", None).await; + let tarball = format!( + "{}/npm-registry/{NAME}/-/{NAME}-{VERSION}.tgz", + server.uri() + ); + + let tmp = tempfile::tempdir().unwrap(); + let project = write_pnpm_tarball_project(tmp.path(), &tarball); + let pristine = format!( + "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n \ + {NAME}:\n specifier: {VERSION}\n version: {VERSION}\n\npackages:\n\n \ + {NAME}@{VERSION}:\n resolution: {{integrity: sha512-UPSTREAMupstream==}}\n\n\ + snapshots:\n\n {NAME}@{VERSION}: {{}}\n\n---\n{project}" + ); + std::fs::write(tmp.path().join("pnpm-lock.yaml"), &pristine).unwrap(); + std::fs::write( + tmp.path().join("pnpm-workspace.yaml"), + "packages:\n - '.'\nlockfileIncludeTarballUrl: true\n", + ) + .unwrap(); + let pinned_twice = |root: &Path| { + let lock = std::fs::read_to_string(root.join("pnpm-lock.yaml")).unwrap(); + lock.matches("patch.test").count() + }; + let env = run_redirect_subprocess(tmp.path(), &server.uri()); + assert_eq!(vlt_hosted_common::redirected(&env), 1, "{env:#}"); + assert_eq!(pinned_twice(tmp.path()), 2, "both documents are pinned"); + let (code, env) = rollback_json(tmp.path(), &server); + assert_eq!(code, Some(0), "rollback: {env:#}"); + assert_eq!( + std::fs::read_to_string(tmp.path().join("pnpm-lock.yaml")).unwrap(), + pristine + ); +} + /// #557, the other case pnpm records `tarball:` in: the registry's /// `dist.tarball` is not the conventional URL pnpm would derive, so pnpm /// keeps it even without the setting. diff --git a/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs b/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs index 4abbd02c3..acb33dd60 100644 --- a/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs +++ b/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs @@ -3012,6 +3012,83 @@ async fn hosted_pnpm_file_tarball_copy_is_named() { .await; } +/// #2009: the registry release is a pnpm config dependency (the env +/// lockfile document) and the project document installs only a `file:` +/// copy of the same `name@version`. The scan pins the config dependency, +/// and names the project's copy with `redirect_copy_unpatched`, since a +/// VEX statement covers the release, not one install tree. +#[tokio::test] +#[serial] +async fn hosted_pnpm_config_dependency_names_the_projects_file_copy() { + let server = MockServer::start().await; + mock_discovery(&server).await; + mock_reference(&server).await; + + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + let copy_key = format!("file:forks/{NAME}"); + write_file_copy_project( + root, + ©_key, + &format!(" resolution: {{directory: forks/{NAME}, type: directory}}\n"), + &format!("file:../../forks/{NAME}"), + ); + std::fs::remove_dir_all(root.join("packages").join("a")).unwrap(); + std::fs::write( + root.join("pnpm-lock.yaml"), + format!( + "--- +lockfileVersion: '9.0' + +importers: + + .: + configDependencies: + {NAME}: + specifier: {VERSION} + version: {VERSION} + +packages: + + {NAME}@{VERSION}: + resolution: {{integrity: {UPSTREAM_SHA512}}} + +--- +lockfileVersion: '9.0' + +importers: + .: {{}} + packages/b: + dependencies: + {NAME}: + specifier: file:../../forks/{NAME} + version: {copy_key} + +packages: + {NAME}@{copy_key}: + resolution: {{directory: forks/{NAME}, type: directory}} + +snapshots: + {NAME}@{copy_key}: {{}} +" + ), + ) + .unwrap(); + + let (code, doc) = run_hosted_json(root, &server.uri()); + assert_eq!(code, Some(0), "{doc}"); + let lock = std::fs::read_to_string(root.join("pnpm-lock.yaml")).unwrap(); + assert!( + lock.contains(&format!("tarball: {HOSTED_URL}")), + "the config dependency is pinned: {lock}" + ); + let text = doc.to_string(); + assert!( + text.contains("redirect_copy_unpatched") && text.contains(&format!("`{NAME}@{copy_key}`")), + "the project's copy is named: {doc:#}" + ); +} + /// The #935 workspace: member `a` depends on the registry release, member /// `b` on the `file:` copy `copy_key` (spelled `spec` in its manifest). fn write_file_copy_project(root: &Path, copy_key: &str, copy_entry: &str, spec: &str) { diff --git a/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs b/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs index 7d4bcc49d..cb18531e0 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs @@ -1,7 +1,7 @@ //! Hermetic hosted → vendored takeover tests through the built binary for //! pnpm projects whose shape the pnpm vendored backend refuses although //! hosted mode accepts it (#853): a `catalog:` dependency -//! (`vendor_lock_entry_unsupported`), a CRLF `pnpm-lock.yaml` +//! (`vendor_lock_entry_unsupported`), a `pnpm-lock.yaml` mixing CRLF and LF //! (`vendor_lockfile_crlf_unsupported`) and a conflicting user override //! (a range) in `pnpm-workspace.yaml` (`vendor_override_conflict`). //! @@ -54,6 +54,8 @@ enum Shape { Catalog, /// A plain dependency whose lock is converted to CRLF after hosting. Crlf, + /// A plain dependency whose lock mixes CRLF and LF after hosting. + MixedEol, /// A plain dependency plus a conflicting user /// `overrides: { left-pad: ^1.3.0 }` in pnpm-workspace.yaml. Override, @@ -90,7 +92,7 @@ fn write_pnpm_project(root: &Path, shape: Shape) { Shape::Catalog => format!("packages:\n - .\ncatalog:\n {NAME}: {VERSION}\n"), Shape::Override => format!("overrides:\n {NAME}: ^{VERSION}\n"), Shape::ExactPin => format!("overrides:\n {NAME}: {VERSION}\n"), - Shape::Crlf | Shape::Plain | Shape::TwoDocument => String::new(), + Shape::Crlf | Shape::MixedEol | Shape::Plain | Shape::TwoDocument => String::new(), }; if !workspace.is_empty() { std::fs::write(root.join("pnpm-workspace.yaml"), workspace).unwrap(); @@ -117,7 +119,7 @@ fn write_pnpm_project(root: &Path, shape: Shape) { )), Shape::Override => lock.push_str(&format!("overrides:\n {NAME}: ^{VERSION}\n\n")), Shape::ExactPin => lock.push_str(&format!("overrides:\n {NAME}: {VERSION}\n\n")), - Shape::Crlf | Shape::Plain | Shape::TwoDocument => {} + Shape::Crlf | Shape::MixedEol | Shape::Plain | Shape::TwoDocument => {} } let specifier = match shape { Shape::Catalog => "'catalog:'".to_string(), @@ -305,7 +307,8 @@ fn snapshot(root: &Path) -> Vec<(&'static str, Option>)> { /// The hosted project: pristine lock, then a real `scan --mode hosted` /// (and, for [`Shape::Crlf`], the lock converted to CRLF as a -/// `core.autocrlf` checkout would leave it). Returns the hosted wiring. +/// `core.autocrlf` checkout would leave it; for [`Shape::MixedEol`], its +/// first line only). Returns the hosted wiring. fn host_project(root: &Path, api: &str, shape: Shape) -> Vec<(&'static str, Option>)> { write_pnpm_project(root, shape); let (code, env) = run_mode(root, api, "scan", "hosted", &[]); @@ -316,8 +319,10 @@ fn host_project(root: &Path, api: &str, shape: Shape) -> Vec<(&'static str, Opti lock.contains(HOSTED_URL), "hosted mode must pin the {shape:?} lock entry:\n{lock}\n{env:#}" ); - if matches!(shape, Shape::Crlf) { - std::fs::write(&lock_path, lock.replace('\n', "\r\n")).unwrap(); + match shape { + Shape::Crlf => std::fs::write(&lock_path, lock.replace('\n', "\r\n")).unwrap(), + Shape::MixedEol => std::fs::write(&lock_path, lock.replacen('\n', "\r\n", 1)).unwrap(), + _ => {} } snapshot(root) } @@ -402,17 +407,21 @@ async fn get_vendored_over_hosted_pnpm_catalog_dep_keeps_the_hosted_pin() { refused_takeover_keeps_hosted_pin(Shape::Catalog, "get", "vendor_lock_entry_unsupported").await; } -/// #853: a CRLF `pnpm-lock.yaml` (a `core.autocrlf` checkout). +/// #853: a `pnpm-lock.yaml` mixing CRLF and LF. (A CRLF lock, a +/// `core.autocrlf` checkout, takes over since lock-model 2b: see +/// [`scan_vendored_over_hosted_pnpm_crlf_lock_takes_over_and_stays_crlf`].) #[tokio::test(flavor = "multi_thread")] -async fn scan_vendored_over_hosted_pnpm_crlf_lock_keeps_the_hosted_pin() { - refused_takeover_keeps_hosted_pin(Shape::Crlf, "scan", "vendor_lockfile_crlf_unsupported") +async fn scan_vendored_over_hosted_pnpm_mixed_eol_lock_keeps_the_hosted_pin() { + refused_takeover_keeps_hosted_pin(Shape::MixedEol, "scan", "vendor_lockfile_crlf_unsupported") .await; } -/// #853: `get --mode vendored` over a CRLF lock (a project-level refusal). +/// #853: `get --mode vendored` over a mixed-ending lock (a project-level +/// refusal). #[tokio::test(flavor = "multi_thread")] -async fn get_vendored_over_hosted_pnpm_crlf_lock_keeps_the_hosted_pin() { - refused_takeover_keeps_hosted_pin(Shape::Crlf, "get", "vendor_lockfile_crlf_unsupported").await; +async fn get_vendored_over_hosted_pnpm_mixed_eol_lock_keeps_the_hosted_pin() { + refused_takeover_keeps_hosted_pin(Shape::MixedEol, "get", "vendor_lockfile_crlf_unsupported") + .await; } /// #853: a conflicting user override (a range) in `pnpm-workspace.yaml`. @@ -522,6 +531,31 @@ async fn scan_vendored_over_hosted_pnpm_plain_dep_still_takes_over() { ); } +/// Lock-model 2b (divergence N15): a CRLF `pnpm-lock.yaml` (a +/// `core.autocrlf` checkout) takes over like its LF twin, and the vendored +/// lock keeps its CRLF endings. +#[tokio::test(flavor = "multi_thread")] +async fn scan_vendored_over_hosted_pnpm_crlf_lock_takes_over_and_stays_crlf() { + let server = MockServer::start().await; + mock_api(&server).await; + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + host_project(root, &server.uri(), Shape::Crlf); + + let (exit, env) = run_mode(root, &server.uri(), "scan", "vendored", &[]); + assert_eq!(exit, 0, "the CRLF takeover must succeed: {env:#}"); + let lock = std::fs::read_to_string(root.join("pnpm-lock.yaml")).unwrap(); + assert!(!lock.contains(HOSTED_URL), "{lock}"); + assert!( + lock.contains(&format!(".socket/vendor/npm/{UUID}/")), + "the lock must point at the vendored artifact:\n{lock}" + ); + assert!( + !lock.replace("\r\n", "").contains('\n'), + "the lock keeps its CRLF endings: {lock:?}" + ); +} + /// #854: a user exact pin equal to the vendored version in /// `pnpm-workspace.yaml` (the pnpm 10.5+/11/12 override map) is taken over /// like the same pin in package.json: the workspace value becomes the diff --git a/crates/socket-patch-core/src/formats/cargo/mod.rs b/crates/socket-patch-core/src/formats/cargo/mod.rs index 081b7f230..459533241 100644 --- a/crates/socket-patch-core/src/formats/cargo/mod.rs +++ b/crates/socket-patch-core/src/formats/cargo/mod.rs @@ -493,6 +493,7 @@ impl CargoLock { out.push(LockfileEntry { ecosystem: "cargo", source_kind, + pnpm_env: false, purl, name: pkg.name.clone(), version, diff --git a/crates/socket-patch-core/src/formats/composer/mod.rs b/crates/socket-patch-core/src/formats/composer/mod.rs index 8aa1a6aad..435820ef5 100644 --- a/crates/socket-patch-core/src/formats/composer/mod.rs +++ b/crates/socket-patch-core/src/formats/composer/mod.rs @@ -188,6 +188,7 @@ impl<'a> ComposerLock<'a> { out.push(LockfileEntry { ecosystem: "composer", source_kind: SourceKind::Unspecified, + pnpm_env: false, name, version, purl, diff --git a/crates/socket-patch-core/src/formats/gem/mod.rs b/crates/socket-patch-core/src/formats/gem/mod.rs index e041581ba..09e40090c 100644 --- a/crates/socket-patch-core/src/formats/gem/mod.rs +++ b/crates/socket-patch-core/src/formats/gem/mod.rs @@ -290,6 +290,7 @@ impl<'t> GemfileLock<'t> { out.push(LockfileEntry { ecosystem: "gem", source_kind: SourceKind::Unspecified, + pnpm_env: false, purl, resolved, name: name.to_string(), diff --git a/crates/socket-patch-core/src/formats/pnpm/grammar.rs b/crates/socket-patch-core/src/formats/pnpm/grammar.rs index 6030b57bf..df81b1879 100644 --- a/crates/socket-patch-core/src/formats/pnpm/grammar.rs +++ b/crates/socket-patch-core/src/formats/pnpm/grammar.rs @@ -1,28 +1,17 @@ //! The `packages:` entry grammar: read pnpm's package blocks without //! reserializing unrelated YAML. The old shrinkwrap and lockfile 5.1/5.2 //! formats use block resolutions; newer locks use flow mappings. Peer -//! suffixes are identities, not part of the version. The hosted planner -//! ([`super::hosted`]), the lock inventory and lockfile discovery all read -//! entries through this one walk. +//! suffixes are identities, not part of the version. Every reader reads +//! entries through [`super::PnpmLock`], which wraps this walk; the writers +//! that splice byte ranges (the hosted planner, [`super::hosted`], and the +//! hosted unwind) call it directly (see `architecture_tests` in +//! `formats/pnpm/mod.rs`). use std::ops::Range; use crate::formats::text::strip_bom; -/// Early pnpm 1 writes shrinkwrapVersion 3 without a minor version and -/// unconditionally drops registry tarball URLs on install. Its frozen flag -/// cannot preserve this redirect (verified with pnpm 1.0.0). -pub(crate) fn unsupported_early_shrinkwrap(content: &str) -> bool { - let content = strip_bom(content); - let version = content - .lines() - .find_map(|line| line.strip_prefix("shrinkwrapVersion:")); - let minor = content - .lines() - .find_map(|line| line.strip_prefix("shrinkwrapMinorVersion:")); - version.is_some_and(|v| v.trim().trim_matches(['\'', '"']) == "3") - && !minor.is_some_and(|v| v.trim().parse::().is_ok_and(|v| v > 0)) -} +use super::lines::ENV_DOC_IMPORTER_FIELDS; pub(crate) struct Entry<'a> { pub key: &'a str, @@ -88,24 +77,96 @@ pub(crate) fn is_pnpm_lock_text(text: &str) -> bool { .any(|line| line.starts_with("lockfileVersion:") || line.starts_with("shrinkwrapVersion:")) } -/// The main document of a pnpm lock: the text after its last column-0 -/// `---` marker, else all of it. pnpm 11+ writes the env lockfile -/// (`configDependencies` / `packageManagerDependencies`, whose resolutions -/// never carry `tarball:`) as a first document ahead of the project lock. -pub(crate) fn main_document(content: &str) -> &str { +/// Which document of a pnpm lock a span belongs to ([`documents`]). +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum PnpmDocument { + /// The project lock: what `pnpm install` links into `node_modules`. + Project, + /// pnpm 11+'s env lockfile: the `configDependencies` / + /// `packageManagerDependencies` pnpm installs into its config store + /// and its own package-manager install, not into `node_modules`. Its + /// own install tree (docs/design/lock-model-unification.md, Decisions). + Env, +} + +/// The documents of a pnpm lock, in file order: the byte range of each +/// (separator lines excluded, a leading BOM inside the first) and what it +/// is. A text without a column-0 `---` line is one project document. A +/// blank document (the empty span before a leading `---`) is dropped. +/// +/// pnpm >= 11 writes the env lockfile as the FIRST document, ahead of the +/// project lock, and reads it from there (as the vendored planners' +/// `lines::split_project_document` does). So the env document is told by +/// position AND content: in a lock of two or more documents, the first one +/// is [`PnpmDocument::Env`] when its `importers:` carry only +/// [`ENV_DOC_IMPORTER_FIELDS`] (at least one). Every other document is +/// [`PnpmDocument::Project`], an env-shaped later one included: no +/// document can opt out of the project tree's contests, and a crafted lock +/// cannot hide entries by adding documents, since each one is read. CRLF +/// reads like LF. +pub(crate) fn documents(content: &str) -> Vec<(Range, PnpmDocument)> { + let mut spans = Vec::new(); let mut start = 0; let mut offset = 0; - for line in content.split_inclusive('\n') { - offset += line.len(); + for (i, line) in content.split_inclusive('\n').enumerate() { let text = line.trim_end_matches(['\r', '\n']); + let text = if i == 0 { strip_bom(text) } else { text }; if text .strip_prefix("---") .is_some_and(|rest| rest.is_empty() || rest.starts_with([' ', '\t'])) { - start = offset; + spans.push(start..offset); + start = offset + line.len(); } + offset += line.len(); + } + spans.push(start..content.len()); + spans.retain(|span| { + strip_bom(&content[span.clone()]).lines().any(|line| { + let line = line.trim(); + !line.is_empty() && !line.starts_with('#') + }) + }); + if spans.is_empty() { + return vec![(0..content.len(), PnpmDocument::Project)]; } - &content[start..] + let several = spans.len() > 1; + spans + .into_iter() + .enumerate() + .map(|(i, span)| { + let kind = if several && i == 0 && is_env_document(&content[span.clone()]) { + PnpmDocument::Env + } else { + PnpmDocument::Project + }; + (span, kind) + }) + .collect() +} + +/// Whether one document's `importers:` carry only the env lockfile's +/// fields ([`ENV_DOC_IMPORTER_FIELDS`]), at least one of them. +fn is_env_document(doc: &str) -> bool { + let mut in_importers = false; + let mut env_fields = 0usize; + for line in strip_bom(doc).split('\n') { + let line = line.strip_suffix('\r').unwrap_or(line); + if !line.is_empty() && !line.starts_with([' ', '#']) { + in_importers = line == "importers:"; + continue; + } + if !in_importers { + continue; + } + if let Some((field, _, _)) = super::lines::parse_key_line(line, 4) { + if !ENV_DOC_IMPORTER_FIELDS.contains(&field) { + return false; + } + env_fields += 1; + } + } + env_fields > 0 } /// The value of the entry-level (four-space) `field:` line of a packages @@ -181,9 +242,11 @@ pub(crate) fn entry_bundled<'a>(entry: &Entry<'a>) -> Option> { pub(crate) fn suffix<'a>(key: &'a str, name: &str, version: &str) -> Option<&'a str> { let key = unquote(key); let key = key.strip_prefix('/').unwrap_or(key); + // `name@version` or `name/version`, without allocating either. let suffix = key - .strip_prefix(&format!("{name}@{version}")) - .or_else(|| key.strip_prefix(&format!("{name}/{version}")))?; + .strip_prefix(name)? + .strip_prefix(['@', '/'])? + .strip_prefix(version)?; if suffix .chars() .next() @@ -194,23 +257,6 @@ pub(crate) fn suffix<'a>(key: &'a str, name: &str, version: &str) -> Option<&'a Some(suffix) } -pub(crate) fn supported_suffix(suffix: &str) -> bool { - if suffix.is_empty() || suffix.starts_with('_') { - return true; - } - let mut depth = 0usize; - for c in suffix.chars() { - match c { - '(' => depth += 1, - ')' if depth > 0 => depth -= 1, - ')' => return false, - _ if depth == 0 => return false, - _ => {} - } - } - depth == 0 -} - pub(crate) struct Resolution<'a> { pub range: Range, pub fields: Vec<(&'a str, &'a str)>, diff --git a/crates/socket-patch-core/src/formats/pnpm/hosted.rs b/crates/socket-patch-core/src/formats/pnpm/hosted.rs index c491bd22c..54908d9dc 100644 --- a/crates/socket-patch-core/src/formats/pnpm/hosted.rs +++ b/crates/socket-patch-core/src/formats/pnpm/hosted.rs @@ -4,13 +4,14 @@ //! rewriter's pnpm leg). use std::borrow::Cow; -use std::collections::BTreeMap; +use std::collections::{BTreeMap, HashSet}; use serde_json::Value; use crate::patch::redirect::{full_name, DepOverride, FileEdit, RewriteResult, RewriteWarning}; use super::grammar as pnpm; +use super::plan::{hosted_refuses_lock, match_key, planned_locks, KeyMatch, Target}; /// Whether `entry` resolves to exactly `artifact_url` — the per-instance /// residual-gate predicate. @@ -42,7 +43,7 @@ struct PnpmLockState<'f> { /// Entry indices sorted by normalized (unquoted, `/`-stripped) key. sorted: Vec, pending: Vec<(std::ops::Range, String)>, - spliced: std::collections::HashSet, + spliced: HashSet, changed: bool, } @@ -51,7 +52,7 @@ impl<'f> PnpmLockState<'f> { let mut state = PnpmLockState { path, text: Cow::Borrowed(text), - early_shrinkwrap: pnpm::unsupported_early_shrinkwrap(text), + early_shrinkwrap: hosted_refuses_lock(text), entries: Vec::new(), sorted: Vec::new(), pending: Vec::new(), @@ -96,33 +97,37 @@ impl<'f> PnpmLockState<'f> { } /// Entries whose key names `fname@version` (any suffix), in file order — - /// the same set a full [`pnpm::suffix`] scan of the logical lock yields. + /// the same set a full [`match_key`] scan of the logical lock yields. fn hits(&mut self, fname: &str, version: &str) -> Vec { - let hits = self.lookup(fname, version); + let hits = self.lookup(fname, version, false); if hits.iter().any(|i| self.spliced.contains(i)) { self.materialize(); - return self.lookup(fname, version); + return self.lookup(fname, version, false); } hits } - fn lookup(&self, fname: &str, version: &str) -> Vec { + /// The entries [`match_key`] matches as a registry instance or, with + /// `rekeyed`, as a vendored rekey, found by their key prefixes. + fn lookup(&self, fname: &str, version: &str, rekeyed: bool) -> Vec { + let prefixes = match rekeyed { + false => [format!("{fname}@{version}"), format!("{fname}/{version}")], + true => [format!("{fname}@file:"), "file:".to_string()], + }; let mut out = Vec::new(); - for sep in ['@', '/'] { - let prefix = format!("{fname}{sep}{version}"); + for prefix in &prefixes { let start = self .sorted .partition_point(|&i| self.norm_key(i) < prefix.as_str()); - out.extend( - self.sorted[start..] - .iter() - .take_while(|&&i| self.norm_key(i).starts_with(prefix.as_str())) - .copied(), - ); + let rest = self.sorted[start..].iter(); + out.extend(rest.take_while(|&&i| self.norm_key(i).starts_with(prefix.as_str()))); } out.sort_unstable(); out.dedup(); - out.retain(|&i| pnpm::suffix(self.entry(i).key, fname, version).is_some()); + out.retain(|&i| { + match_key(self.entry(i).key, fname, version) + .is_some_and(|m| (m == KeyMatch::Rekeyed) == rekeyed) + }); out } @@ -175,17 +180,11 @@ pub(crate) fn plan_hosted( let npm: Vec<&DepOverride> = overrides.iter().filter(|o| o.ecosystem == "npm").collect(); // A pnpm lock lives at the project root or at any nested path (e.g. Rush // repos keep them under `common/config/rush/`); every such files-map key - // is rewritten under the same grammar. Deterministic order: BTreeMap - // iterates keys sorted, so goldens are stable across every lock in the set. - let lock_keys: Vec<&String> = files - .keys() - .filter(|k| { - matches!( - k.rsplit('/').next(), - Some("pnpm-lock.yaml" | "shrinkwrap.yaml") - ) - }) - .collect(); + // is rewritten under the same grammar, except the one of a + // `pnpm-lock.yaml` / `shrinkwrap.yaml` pair pnpm never reads + // ([`planned_locks`]). Deterministic order: + // BTreeMap iterates keys sorted, so goldens are stable across every lock. + let lock_keys = planned_locks(files); if npm.is_empty() || lock_keys.is_empty() { return; } @@ -252,7 +251,7 @@ pub(crate) fn plan_hosted( let entry = lock.entry(i); let suffix = pnpm::suffix(entry.key, &fname, &dep.version) .expect("hits only holds entries naming this dep"); - let resolution = if pnpm::supported_suffix(suffix) { + let resolution = if Target::Hosted.repoints(suffix) { pnpm::resolution(&entry) } else { None @@ -338,44 +337,16 @@ pub(crate) fn plan_hosted( // is named as such: `socket-patch vendor` removes the registry // resolution this grammar looks for (v9 respells the packages key // `@file:.socket/vendor/…`; v5/v6 rekey it to a bare `file:` - // key but keep the `@: file:…` overrides line), so - // the generic not-locked wording would send users on a wild-goose - // `pnpm install` when the real path is a mode switch. Fail-closed - // either way: nothing is rewritten for the dep. + // key with a `name:` line), so the generic not-locked wording would + // send users on a wild-goose `pnpm install` when the real path is a + // mode switch. Fail-closed either way: nothing is rewritten for the + // dep. if !matched_any { - let v9_vendored_key = format!("{fname}@file:"); - let override_key = format!("{fname}@{}", dep.version); - // Scanned over the post-splice text, so fold pending splices in. - for lock in locks.iter_mut() { - lock.materialize(); - } - let vendored = locks.iter().any(|lock| { - lock.text.lines().any(|line| { - let t = line.trim_start(); - let t = t.strip_prefix('\'').unwrap_or(t); - // v9 packages/snapshots key (leading `/` in v6 spelling). - // The vendor backend always writes the RELATIVE - // `file:.socket/vendor/…` spelling here, so anchoring on - // it keeps a user's own `file:` dep of the same name - // from being misreported as vendored. - let key = t.strip_prefix('/').unwrap_or(t); - if key - .strip_prefix(&v9_vendored_key) - .is_some_and(|rest| rest.starts_with(".socket/vendor/")) - { - return true; - } - // overrides / root-dep line: `@: file:…` - // (pnpm <=8 absolutizes the value, so only the - // `.socket/vendor/` tail is stable enough to match). - t.strip_prefix(&override_key) - .map(|rest| rest.strip_prefix('\'').unwrap_or(rest)) - .and_then(|rest| rest.strip_prefix(':')) - .is_some_and(|rest| { - rest.contains("file:") && rest.contains(".socket/vendor/") - }) - }) - }); + // A rekey ([`KeyMatch::Rekeyed`]) is read from `text` without + // folding `pending` in: a resolution splice never touches a key. + let vendored = locks + .iter() + .any(|lock| !lock.lookup(&fname, &dep.version, true).is_empty()); if vendored { result.warnings.push(RewriteWarning { code: "redirect_pnpm_entry_vendored".into(), diff --git a/crates/socket-patch-core/src/formats/pnpm/lines.rs b/crates/socket-patch-core/src/formats/pnpm/lines.rs index 0badfcb72..3899f56ea 100644 --- a/crates/socket-patch-core/src/formats/pnpm/lines.rs +++ b/crates/socket-patch-core/src/formats/pnpm/lines.rs @@ -26,7 +26,8 @@ pub(crate) fn section_bounds(lines: &[String], name: &str) -> Option<(usize, usi /// The importer dependency fields pnpm writes into the env document of a /// two-document lock. -const ENV_DOC_IMPORTER_FIELDS: [&str; 2] = ["configDependencies", "packageManagerDependencies"]; +pub(crate) const ENV_DOC_IMPORTER_FIELDS: [&str; 2] = + ["configDependencies", "packageManagerDependencies"]; /// Split a lock into `(prefix, project document)`, `prefix + project == /// text`. pnpm >= 11 writes `pnpm-lock.yaml` as two YAML documents when the diff --git a/crates/socket-patch-core/src/formats/pnpm/mod.rs b/crates/socket-patch-core/src/formats/pnpm/mod.rs index 4abd400ba..1cc44d1f2 100644 --- a/crates/socket-patch-core/src/formats/pnpm/mod.rs +++ b/crates/socket-patch-core/src/formats/pnpm/mod.rs @@ -13,8 +13,8 @@ //! tarball, the raw material lockfile discovery (`vex::discover::npm`) //! classifies as hosted / vendored refs, and [`PnpmLock::wired_integrity`] //! repair's trust anchor for a vendored artifact; -//! * [`PnpmLock::vendored_in_use`] — whether a vendored artifact is still -//! consumed (both vendored backends' revert guards and ledger liveness); +//! * [`vendored_npm_uuids`] — the vendored artifacts a lock still consumes +//! (both vendored backends' revert guards and ledger liveness); //! * [`plan_hosted`] — the hosted planner (the redirect rewriter's pnpm //! leg: resolution splices over the whole lock set); //! * the vendored planners (`vendor::pnpm_lock` for lockfileVersion 9.0, @@ -28,11 +28,13 @@ pub(crate) mod grammar; pub(crate) mod hosted; pub(crate) mod lines; pub(crate) mod modules_yaml; +pub(crate) mod plan; mod version; pub(crate) mod workspace; pub use version::{matches_file_id_build_keys, root_only_workspace_breaks_add}; +pub use grammar::PnpmDocument; pub(crate) use grammar::{ entry_bundled, entry_field, is_pnpm_lock_text, Bundled, Entry, Resolution, }; @@ -58,6 +60,9 @@ pub(crate) struct PnpmPackage<'a> { /// The entry's `resolution:` map; `None` when it has none or the /// grammar refuses it (duplicate keys, nested values, aliases). pub(crate) resolution: Option>, + /// The lock document the entry is in ([`grammar::documents`]): the + /// project lock or pnpm 11+'s env lockfile, each its own install tree. + pub(crate) document: PnpmDocument, } impl<'a> PnpmPackage<'a> { @@ -78,28 +83,56 @@ impl<'a> PnpmPackage<'a> { } } -/// Every `packages:` entry of a pnpm lock text, in lock order — the ONE -/// entry walk the inventory, lockfile discovery and repair share. Every -/// entry is returned (registry, rekeyed vendored, hosted, directory, git); -/// each consumer applies its own key and resolution rules. -pub(crate) fn pnpm_packages(text: &str) -> Vec> { +/// Every `packages:` entry of a pnpm lock text, in lock order, each tagged +/// with its document — the ONE entry walk every reader shares (through +/// [`PnpmLock::packages`]). Every entry is returned (registry, rekeyed +/// vendored, hosted, directory, git); each consumer applies its own key +/// and resolution rules. +fn pnpm_packages(text: &str, resolve: Resolve) -> Vec> { + let documents = match resolve { + Resolve::Yes => grammar::documents(text), + Resolve::No => Vec::new(), + }; grammar::entries(text) .into_iter() - .map(|entry| PnpmPackage { - key: grammar::unquote(entry.key.trim()), - resolution: grammar::resolution(&entry), - entry, + .map(|entry| { + // Tag by the KEY line: an entry with an empty body (its key the + // last line of a document) has its body offset on the next + // document's `---` line, or at the end of the text. + let key_at = entry.key.as_ptr() as usize - text.as_ptr() as usize; + PnpmPackage { + key: grammar::unquote(entry.key.trim()), + resolution: match resolve { + Resolve::Yes => grammar::resolution(&entry), + Resolve::No => None, + }, + document: documents + .iter() + .find(|(span, _)| span.contains(&key_at)) + .map_or(PnpmDocument::Project, |(_, kind)| *kind), + entry, + } }) .collect() } +/// Whether [`pnpm_packages`] parses each entry's `resolution:` map. +#[derive(Clone, Copy)] +enum Resolve { + Yes, + /// Keys and bodies only, every entry tagged `Project`: the store + /// crawler's hot path ([`InstalledPackages::from_lock_text`]) reads + /// neither a resolution nor a document. + No, +} + /// The `(integrity, tarball)` of one recorded `packages:` block — a /// vendored planner's wiring fragment (its key line and body, as /// recorded), each unquoted; `None` unless the fragment is exactly one /// entry with a flat resolution. pub(crate) fn fragment_resolution(block: &[String]) -> Option<(Option, Option)> { let text = format!("packages:\n{}\n", block.join("\n")); - let packages = pnpm_packages(&text); + let packages = pnpm_packages(&text, Resolve::Yes); let [package] = packages.as_slice() else { return None; }; @@ -257,28 +290,31 @@ fn head_lock_version(text: &str) -> Option { /// looping them back to "re-lock with pnpm >= 9" would hand them the lock /// they have. pub fn sniff_lock_grammar(text: &str) -> Result { - match head_lock_version(text).as_deref() { + let version = head_lock_version(text); + match version.as_deref() { Some("9.0") => Ok(PnpmLockGrammar::V9), Some("5.4") => Ok(PnpmLockGrammar::V54), Some("6.0") => Ok(PnpmLockGrammar::V60), - Some(v) => { - let major = v.split('.').next().and_then(|m| m.parse::().ok()); - Err(match major { - Some(m) if m < 9 => format!( - "{PNPM_LOCK} has lockfileVersion {v}; supported versions are 5.4 \ - (pnpm 7), 6.0 (pnpm 8), and 9.0 (pnpm >= 9) — re-lock with pnpm >= 9" - ), - _ => format!( - "{PNPM_LOCK} has lockfileVersion {v}; this socket-patch build supports \ + v => { + // An old (or missing) version is fixed by upgrading pnpm; a newer or + // unreadable one means pnpm outgrew this build. + let major = v.map(|v| v.split('.').next().and_then(|m| m.parse::().ok())); + let old = matches!(major, None | Some(Some(..9))); + let v = v.map_or("no lockfileVersion in its head".into(), |v| { + format!("lockfileVersion {v}") + }); + Err(match old { + false => format!( + "{PNPM_LOCK} has {v}; this socket-patch build supports \ lockfileVersions 5.4, 6.0, and 9.0 — re-lock with a pnpm release that \ emits one of them, or update socket-patch" ), + true => format!( + "{PNPM_LOCK} has {v}; supported versions are 5.4 (pnpm 7), 6.0 \ + (pnpm 8), and 9.0 (pnpm >= 9) — re-lock with pnpm >= 9" + ), }) } - None => Err(format!( - "{PNPM_LOCK} has no lockfileVersion in its head; supported versions are 5.4, \ - 6.0, and 9.0 — re-lock with pnpm >= 9" - )), } } @@ -328,56 +364,21 @@ pub fn is_shrinkwrap_lock(text: &str) -> bool { .any(|line| line.starts_with("shrinkwrapVersion:")) } -/// The lockfileVersion the v9 vendored planner splices. -const V9_LOCK_VERSION: &str = "9.0"; - -/// `lockfileVersion: '9.0'` head check (accept pnpm's single quotes plus -/// double-quoted/bare spellings) — the v9 vendored planner's own guard. -/// The flavor router sniffs with [`sniff_lock_grammar`] instead, whose -/// allowlist also routes the legacy 5.4/6.0 grammars to their planner; -/// this check only fires if a non-9.0 lock reaches the v9 planner directly. -pub(crate) fn check_v9_lock_version(text: &str) -> Result<(), String> { - match head_lock_version(text) { - Some(v) if v == V9_LOCK_VERSION => Ok(()), - Some(v) => { - // The remedy must point the right way: 5.x (pnpm 7) / 6.x - // (pnpm 8) locks predate the v9 grammar and upgrading pnpm - // re-locks them, but a HIGHER version means the user's pnpm - // already outgrew this build — telling them "re-lock with - // pnpm >= 9" would loop them back to the lock they have. - let major = v.split('.').next().and_then(|m| m.parse::().ok()); - Err(match major { - Some(m) if m < 9 => format!( - "{PNPM_LOCK} has lockfileVersion {v}; only {V9_LOCK_VERSION} is \ - supported — re-lock with pnpm >= 9" - ), - _ => format!( - "{PNPM_LOCK} has lockfileVersion {v}; this socket-patch build supports \ - lockfileVersion {V9_LOCK_VERSION} — re-lock with a pnpm release \ - that emits it, or update socket-patch" - ), - }) - } - None => Err(format!( - "{PNPM_LOCK} has no lockfileVersion in its head; only \ - {V9_LOCK_VERSION} is supported — re-lock with pnpm >= 9" - )), - } -} - // ── the model ── /// One pnpm lock, parsed once (see the module docs). +/// +/// Every document is read. pnpm 11+ writes its env lockfile +/// (`configDependencies`, `packageManagerDependencies`) as a first +/// document; its entries are installed too (into pnpm's config store, not +/// `node_modules`), so they are part of the lock, tagged +/// [`PnpmDocument::Env`]: a reader that keeps install trees apart +/// (lockfile discovery's contests) reads the env document as a tree of its +/// own ([`PnpmPackage::document`]). pub struct PnpmLock<'t> { text: &'t str, - /// The project document ([`grammar::main_document`]): pnpm 11+ writes - /// its env lockfile (`configDependencies`, `packageManagerDependencies`) - /// as a first document whose packages the install never links (#1949). - doc: &'t str, - /// Entries of the project document only, with offsets into `text`. + /// Every document's entries, with offsets into `text`. packages: Vec>, - /// [`vendored_npm_uuids`] of the project document. - vendored: HashSet, } /// One `packages:` entry whose resolution names a tarball — a candidate @@ -392,18 +393,20 @@ pub(crate) struct PnpmTarballRef<'p> { impl<'t> PnpmLock<'t> { /// Parse a lock text (any content: a non-pnpm text has no entries). pub fn parse(text: &'t str) -> Self { - let doc = grammar::main_document(strip_bom(text)); - // `doc` is a suffix of `text`: parse the whole text so offsets stay - // relative to it, and keep the entries that start in `doc`. - let doc_start = text.len() - doc.len(); PnpmLock { text, - doc, - packages: pnpm_packages(text) - .into_iter() - .filter(|p| p.entry.offset >= doc_start) - .collect(), - vendored: vendored_npm_uuids(doc), + packages: pnpm_packages(text, Resolve::Yes), + } + } + + /// [`PnpmLock::parse`] without the resolution maps (every + /// [`PnpmPackage::resolution`] is `None`) or document tags (every + /// entry is [`PnpmDocument::Project`]), for a reader of keys and entry + /// fields only: its document queries mean nothing. + fn parse_keys(text: &'t str) -> Self { + PnpmLock { + text, + packages: pnpm_packages(text, Resolve::No), } } @@ -416,23 +419,40 @@ impl<'t> PnpmLock<'t> { is_pnpm_lock_text(self.text) } - /// Every `packages:` entry, in lock order. + /// Every `packages:` entry of every document, in lock order, each + /// tagged with its [`PnpmDocument`]. pub(crate) fn packages(&self) -> &[PnpmPackage<'t>] { &self.packages } + /// The entries of one document kind, in lock order. + pub(crate) fn packages_of( + &self, + document: PnpmDocument, + ) -> impl Iterator> { + self.packages.iter().filter(move |p| p.document == document) + } + + /// The document kinds the lock has entries in, project first. + pub(crate) fn documents(&self) -> Vec { + let mut kinds: Vec = self.packages.iter().map(|p| p.document).collect(); + kinds.sort(); + kinds.dedup(); + kinds + } + fn has_packages_section(&self) -> bool { - self.doc + self.text .lines() .any(|l| l.trim_end_matches('\r') == "packages:") } /// The registry inventory: one entry per plain-registry packages key - /// (every key generation, [`pnpm_registry_key`]), flow and block - /// resolutions, CRLF included. A resolution the grammar refuses leaves - /// the entry listed without a verifier; our own vendored tarballs are - /// not registry dependencies and are dropped. `None` when the lock has - /// no `packages:` section. + /// (every key generation, [`pnpm_registry_key`]) of every document, + /// flow and block resolutions, CRLF included. A resolution the grammar + /// refuses leaves the entry listed without a verifier; our own vendored + /// tarballs are not registry dependencies and are dropped. `None` when + /// the lock has no `packages:` section. pub fn entries(&self) -> Option> { if !self.has_packages_section() { return None; @@ -455,12 +475,10 @@ impl<'t> PnpmLock<'t> { { continue; } - out.push(LockfileEntry::npm( - name, - version, - tarball.and_then(http_url), - integrity, - )); + let mut entry = + LockfileEntry::npm(name, version, tarball.and_then(http_url), integrity); + entry.pnpm_env = package.document == PnpmDocument::Env; + out.push(entry); } Some(out) } @@ -497,17 +515,6 @@ impl<'t> PnpmLock<'t> { .find_map(|r| r.integrity.filter(|sri| is_sri_pin(sri))) .map(str::to_string) } - - /// Is the vendored npm artifact of patch `uuid` still consumed by this - /// lock? `true` when a `packages:` / `snapshots:` block is keyed by it - /// ([`vendored_npm_uuid`] — v9's `name@file:` and legacy's bare `file:` - /// keys alike); `false` when the lock carries none (the `overrides:` - /// declaration alone never counts: pnpm keeps it mirrored from - /// package.json even when nothing matches it). CRLF locks read like LF - /// ones. - pub fn vendored_in_use(&self, uuid: &str) -> bool { - self.vendored.contains(uuid) - } } /// The packages a pnpm "current lockfile" — the `lock.yaml` pnpm writes in @@ -526,54 +533,42 @@ pub(crate) struct InstalledPackages { impl InstalledPackages { /// Read a current lockfile in any lock generation (v5.4 `/name/1.0.0`, /// v6 `/name@1.0.0`, v9 `name@1.0.0` keys; a non-registry entry by its - /// `name:` / `version:` fields, or a v9 key's name). `None` when the - /// text is not a pnpm lock, or holds an entry whose package cannot be - /// named — the caller must then keep every store entry. + /// `name:` / `version:` fields, the last of each winning, or a v9 key's + /// name), through [`PnpmLock`]'s entry walk. `None` when the text is not + /// a pnpm lock, or holds an entry whose package cannot be named — the + /// caller must then keep every store entry. + /// + /// Every document counts. pnpm writes the current lockfile as the + /// project lock alone (the env lockfile lives only in `pnpm-lock.yaml`, + /// and config dependencies install into `node_modules/.pnpm-config`, not + /// the virtual store), so a second document is not expected here; if + /// one appears, its entries keep their store entries too, the side that + /// never hides an installed package. pub(crate) fn from_lock_text(text: &str) -> Option { if !grammar::is_pnpm_lock_text(text) { return None; } - // One line walk (the scan reads this on every run, for every - // package of the install): two-space `packages:` keys and their - // four-space `name:` / `version:` fields, nothing else. + // Keys and `name:` / `version:` lines only: the scan reads this on + // every run, for every package of the install. + let lock = PnpmLock::parse_keys(text); let mut out = Self::default(); - let mut entries = 0usize; - let mut in_packages = false; - let mut current: Option<(&str, Option<&str>, Option<&str>)> = None; - let mut finish = |entry: Option<(&str, Option<&str>, Option<&str>)>, out: &mut Self| { - let Some((key, field_name, field_version)) = entry else { - return Some(()); - }; - entries += 1; - out.insert_entry(key, field_name, field_version) - }; - for raw in grammar::main_document(strip_bom(text)).split('\n') { - let line = raw.strip_suffix('\r').unwrap_or(raw); - if !line.is_empty() && !line.starts_with([' ', '#']) { - finish(current.take(), &mut out)?; - in_packages = line == "packages:"; - continue; - } - if !in_packages { - continue; - } - if let Some((key, _, _)) = lines::parse_key_line(line, 2) { - finish(current.take(), &mut out)?; - current = Some((key, None, None)); - } else if let (Some(entry), Some((field, _, value))) = - (current.as_mut(), lines::parse_key_line(line, 4)) - { - let value = Some(grammar::unquote(value.trim())).filter(|v| !v.is_empty()); - match field { - "name" => entry.1 = value, - "version" => entry.2 = value, - _ => {} + for package in lock.packages() { + let (mut name, mut version) = (None, None); + for line in package.entry.body.split('\n') { + let line = line.strip_suffix('\r').unwrap_or(line); + if let Some((field, _, value)) = lines::parse_key_line(line, 4) { + let value = Some(grammar::unquote(value.trim())).filter(|v| !v.is_empty()); + match field { + "name" => name = value, + "version" => version = value, + _ => {} + } } } + out.insert_entry(package.key, name, version)?; } - finish(current.take(), &mut out)?; // A lock listing no package at all says nothing about the store. - (entries > 0).then_some(out) + (!lock.packages().is_empty()).then_some(out) } /// Record one `packages:` entry; `None` when its package cannot be @@ -625,7 +620,9 @@ impl InstalledPackages { /// into `.socket/vendor/npm//` — the block-key grammar the vendored /// planners splice with ([`lines::parse_key_line`] at two-space indent), /// read in one walk with each line's `\r` dropped, so a CRLF lock (a -/// Windows autocrlf checkout) answers like its LF twin. +/// Windows autocrlf checkout) answers like its LF twin. An `overrides:` +/// declaration alone never counts: pnpm keeps it mirrored from +/// package.json even when nothing matches it. pub(crate) fn vendored_npm_uuids(text: &str) -> HashSet { let mut out = HashSet::new(); let mut in_section = false; @@ -651,6 +648,10 @@ pub(crate) fn vendored_npm_uuids(text: &str) -> HashSet { mod tests { use super::*; + /// pnpm 11's env lockfile document for config dependency + /// `cfg-dep@2.0.0`, up to its packages entry's key line. + const ENV_DOC_HEAD: &str = "lockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n cfg-dep:\n specifier: 2.0.0\n version: 2.0.0\n\npackages:\n\n cfg-dep@2.0.0:\n"; + /// #1197: the current lockfile's packages, by name and version, in the /// shapes real pnpm 7.33.7 / 8.15.9 / 10–12 write after an upgrade /// (`left-pad` 1.3.0 → 1.2.0), plus non-registry entries. @@ -770,14 +771,17 @@ mod tests { assert!(!PnpmLock::parse(importers).resolves("left-pad", "1.3.0")); } - /// #1949: a pnpm 11+ lock's env document (config dependencies the - /// install never links) is not part of the inventory: `entries()`, - /// `resolves()`, the vendored uuids and `InstalledPackages` all read - /// the project document alone, and agree with each other. + /// pnpm 11+'s env document is part of the lock (owner decision, + /// lock-model divergence N4; this reverses #1949's skip): pnpm installs + /// its config dependencies, and the hosted planner and the v9 vendored + /// in-use check already read it. `entries()`, `resolves()`, the + /// vendored uuids and `InstalledPackages` all read both documents and + /// agree with each other; every entry is tagged with its document, so + /// discovery keeps the env document a tree of its own. #[test] - fn env_document_is_not_in_the_inventory() { + fn env_document_is_read_as_its_own_tree() { let env = format!( - "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n cfg-dep:\n specifier: 2.0.0\n version: 2.0.0\n\npackages:\n\n cfg-dep@2.0.0:\n resolution: {{integrity: sha512-c}}\n\n b@file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz:\n resolution: {{integrity: sha512-x, tarball: file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz}}\n version: 1.0.0\n\n" + "---\n{ENV_DOC_HEAD} resolution: {{integrity: sha512-c}}\n\n b@file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz:\n resolution: {{integrity: sha512-x, tarball: file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz}}\n version: 1.0.0\n\n" ); let project = "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n dependencies:\n left-pad:\n specifier: 1.3.0\n version: 1.3.0\n\npackages:\n\n left-pad@1.3.0:\n resolution: {integrity: sha512-x}\n"; for text in [ @@ -794,17 +798,44 @@ mod tests { .collect(); assert_eq!( names, - [("left-pad".to_string(), "1.3.0".to_string())], + [ + ("cfg-dep".to_string(), "2.0.0".to_string()), + ("left-pad".to_string(), "1.3.0".to_string()) + ], "{text:?}" ); assert!(lock.resolves("left-pad", "1.3.0")); - assert!(!lock.resolves("cfg-dep", "2.0.0")); - assert!(!lock.vendored_in_use(UUID)); - assert!(lock.packages().iter().all(|p| p.entry.offset >= env.len())); + assert!(lock.resolves("cfg-dep", "2.0.0")); + assert!(vendored_npm_uuids(&text).contains(UUID)); + let tagged: Vec<_> = lock + .packages() + .iter() + .map(|p| (p.key, p.document)) + .collect(); + assert_eq!( + tagged, + [ + ("cfg-dep@2.0.0", PnpmDocument::Env), + ( + format!("b@file:.socket/vendor/npm/{UUID}/b-1.0.0.tgz").as_str(), + PnpmDocument::Env + ), + ("left-pad@1.3.0", PnpmDocument::Project), + ], + "{text:?}" + ); + assert_eq!(lock.documents(), [PnpmDocument::Project, PnpmDocument::Env]); let installed = InstalledPackages::from_lock_text(&text).unwrap(); assert!(installed.contains("left-pad", "1.3.0")); - assert!(!installed.contains("cfg-dep", "2.0.0")); + assert!(installed.contains("cfg-dep", "2.0.0")); + assert!(installed.contains("b", "1.0.0")); } + // One document: the project, whatever its importers say. + let lone = env.trim_start_matches("---\n"); + assert!(PnpmLock::parse(lone) + .packages() + .iter() + .all(|p| p.document == PnpmDocument::Project)); // Offsets still index the whole text. let text = format!("{env}{project}"); let lock = PnpmLock::parse(&text); @@ -815,8 +846,92 @@ mod tests { ); } + /// N4: before, the readers took the text after the LAST `---` as the + /// project, so a lock with an extra document chose its own "project" + /// and hid the rest; now every document is read. The env document is + /// the FIRST one, when its importers say so, as pnpm and the vendored + /// planners' `split_project_document` read it: an env-shaped LATER + /// document is a project one, so it cannot opt out of the project + /// tree's contests. #[test] - fn vendored_in_use_reads_v9_and_legacy_keys_and_crlf() { + fn documents_are_identified_by_position_and_content() { + let env = &format!("{ENV_DOC_HEAD} resolution: {{integrity: sha512-c}}\n"); + let project = |dep: &str| { + format!( + "lockfileVersion: '9.0'\n\nimporters:\n\n .:\n dependencies:\n {dep}:\n specifier: 1.0.0\n version: 1.0.0\n\npackages:\n\n {dep}@1.0.0:\n resolution: {{integrity: sha512-x}}\n" + ) + }; + let kinds = |text: &str| -> Vec<(String, PnpmDocument)> { + PnpmLock::parse(text) + .packages() + .iter() + .map(|p| (p.key.to_string(), p.document)) + .collect() + }; + let own = |pairs: &[(&str, PnpmDocument)]| -> Vec<(String, PnpmDocument)> { + pairs.iter().map(|(k, d)| (k.to_string(), *d)).collect() + }; + use PnpmDocument::{Env, Project}; + // An env-shaped document after the project is a project one. + let swapped = format!("---\n{}---\n{env}", project("a")); + assert_eq!( + kinds(&swapped), + own(&[("a@1.0.0", Project), ("cfg-dep@2.0.0", Project)]) + ); + assert!(!PnpmLock::parse(&swapped).documents().contains(&Env)); + // A first document that is not the env document is a project one. + let two = format!("---\n{}---\n{}", project("a"), project("b")); + assert_eq!( + kinds(&two), + own(&[("a@1.0.0", Project), ("b@1.0.0", Project)]) + ); + // A third document is read too, CRLF and a BOM included. + let three = format!("---\n{env}---\n{}---\n{}", project("a"), project("b")); + for text in [ + three.clone(), + three.replace('\n', "\r\n"), + format!("\u{feff}{three}"), + ] { + assert_eq!( + kinds(&text), + own(&[ + ("cfg-dep@2.0.0", Env), + ("a@1.0.0", Project), + ("b@1.0.0", Project) + ]), + "{text:?}" + ); + } + } + + /// An entry is tagged by its KEY line: a key + /// whose body is empty — the last line of the env document, right + /// before `---` — has its body offset on the separator, in no document. + #[test] + fn an_entry_with_an_empty_body_keeps_its_document() { + let env = format!("---\n{ENV_DOC_HEAD}"); + let project = "---\nlockfileVersion: '9.0'\n\npackages:\n\n a@1.0.0:\n resolution: {integrity: sha512-x}\n\n b@1.0.0:"; + let text = format!("{env}{project}"); + for text in [text.clone(), text.replace('\n', "\r\n")] { + let tagged: Vec<_> = PnpmLock::parse(&text) + .packages() + .iter() + .map(|p| (p.key, p.document)) + .collect(); + assert_eq!( + tagged, + [ + ("cfg-dep@2.0.0", PnpmDocument::Env), + ("a@1.0.0", PnpmDocument::Project), + ("b@1.0.0", PnpmDocument::Project), + ], + "{text:?}" + ); + } + } + + #[test] + fn vendored_npm_uuids_read_v9_and_legacy_keys_and_crlf() { let v9 = format!( "lockfileVersion: '9.0'\n\npackages:\n\n a@file:.socket/vendor/npm/{UUID}/a-1.0.0.tgz:\n resolution: {{integrity: sha512-x, tarball: file:.socket/vendor/npm/{UUID}/a-1.0.0.tgz}}\n version: 1.0.0\n" ); @@ -827,12 +942,12 @@ mod tests { "lockfileVersion: '9.0'\n\nsnapshots:\n\n a@file:.socket/vendor/npm/{UUID}/a-1.0.0.tgz: {{}}\n" ); for text in [&v9, &legacy, &snapshot] { - assert!(PnpmLock::parse(text).vendored_in_use(UUID), "{text}"); + assert!(vendored_npm_uuids(text).contains(UUID), "{text}"); let other = "22222222-2222-4222-8222-222222222222"; - assert!(!PnpmLock::parse(text).vendored_in_use(other)); + assert!(!vendored_npm_uuids(text).contains(other)); let crlf = text.replace('\n', "\r\n"); assert!( - PnpmLock::parse(&crlf).vendored_in_use(UUID), + vendored_npm_uuids(&crlf).contains(UUID), "CRLF reads like LF" ); } @@ -840,7 +955,7 @@ mod tests { let overrides = format!( "lockfileVersion: '9.0'\n\noverrides:\n a@1.0.0: file:.socket/vendor/npm/{UUID}/a-1.0.0.tgz\n" ); - assert!(!PnpmLock::parse(&overrides).vendored_in_use(UUID)); + assert!(!vendored_npm_uuids(&overrides).contains(UUID)); } #[test] @@ -891,11 +1006,6 @@ mod tests { may_need_store_flag(plain), "{plain}" ); - assert_eq!( - check_v9_lock_version(&bom), - check_v9_lock_version(plain), - "{plain}" - ); let keys = |text: &str| { PnpmLock::parse(text).entries().map(|e| { e.into_iter() @@ -912,9 +1022,7 @@ mod tests { assert_eq!(lock_version_major(&format!("\u{feff}{v9}")), Some(9)); assert!(may_need_store_flag(&format!("\u{feff}{v52}"))); assert!(may_need_store_flag(&format!("\u{feff}{shrinkwrap}"))); - assert!(grammar::unsupported_early_shrinkwrap( - "\u{feff}shrinkwrapVersion: 3\n" - )); + assert!(plan::hosted_refuses_lock("\u{feff}shrinkwrapVersion: 3\n")); // Exactly one BOM is encoding; a second one is content, as for pnpm. assert!(!is_pnpm_lock_text( "\u{feff}\u{feff}lockfileVersion: '9.0'\n" @@ -969,3 +1077,90 @@ mod tests { assert!(!may_need_store_flag("packages: {}\n")); } } + +/// ARCHITECTURE GUARD: outside `formats/pnpm`, pnpm lock entries are READ +/// through [`PnpmLock`] (one walk, every document tagged), never through +/// the grammar's raw entry walk or a hand-rolled `packages:` line walk. +/// Writers that splice by byte range or line position keep their own +/// walk; each such use is allowlisted below with a label and a count, so +/// a new reader in the same file still fails. +#[cfg(test)] +mod architecture_tests { + use crate::vendor::test_support::production_sources::production_sources; + + /// `(root-relative file, uses, why)`. + const WRITERS: &[(&str, usize, &str)] = &[ + ( + "vendor/pnpm_lock.rs", + 3, + "vendored planner (both dialects): the shared lookup and rekey \ + over line blocks (`lock_has_target_package`, `edit_packages`, \ + `LockIndex`), keys read with `plan::match_key`", + ), + ( + "vendor/pnpm_lock_legacy.rs", + 4, + "5.4 / 6.0 dialect: its ref gate, sorted rekey, dep-ref edit and \ + revert over line blocks", + ), + ]; + + /// Whether one production line reads pnpm entries outside the model. + fn raw_walk(line: &str) -> bool { + let code = line.trim_start(); + if code.starts_with("//") { + return false; + } + [ + "grammar::entries(", + "pnpm::entries(", + "pnpm_grammar::entries(", + "pnpm_packages(", + "== \"packages:\"", + ] + .iter() + .any(|needle| code.contains(needle)) + || (code.contains("\"packages\")") + && (code.contains("bounds(") || code.contains("SectionIndex::build("))) + } + + #[test] + fn pnpm_entries_are_read_through_the_model() { + let src = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src"); + let mut found: Vec<(String, usize)> = Vec::new(); + // Test-only files and items are left out by the shared reader. + for file in production_sources(&src) { + if file.rel.starts_with("formats/pnpm/") { + continue; + } + let uses = file.text.lines().filter(|l| raw_walk(l)).count(); + if uses > 0 { + found.push((file.rel, uses)); + } + } + found.sort(); + let mut allowed: Vec<(String, usize)> = WRITERS + .iter() + .map(|(rel, uses, _)| (rel.to_string(), *uses)) + .collect(); + allowed.sort(); + assert_eq!( + found, allowed, + "pnpm lock entries read outside formats/pnpm: read them through \ + PnpmLock::packages (a reader), or allowlist a writer that needs \ + byte or line positions in WRITERS with its reason" + ); + } + + #[test] + fn raw_walk_needles() { + assert!(raw_walk(" for entry in pnpm::entries(&text) {")); + assert!(raw_walk(" in_packages = line == \"packages:\";")); + assert!(raw_walk( + " let (s, e) = section_bounds(lines, \"packages\").unwrap();" + )); + assert!(!raw_walk(" // pnpm::entries(&text)")); + assert!(!raw_walk(" lock.get(\"packages\")")); + assert!(!raw_walk(" for p in PnpmLock::parse(t).packages() {")); + } +} diff --git a/crates/socket-patch-core/src/formats/pnpm/plan.rs b/crates/socket-patch-core/src/formats/pnpm/plan.rs new file mode 100644 index 000000000..b3fd7c6b4 --- /dev/null +++ b/crates/socket-patch-core/src/formats/pnpm/plan.rs @@ -0,0 +1,344 @@ +//! The pnpm planning core both modes share. Hosted ([`super::hosted`]) and +//! vendored (`vendor::pnpm_lock`, `vendor::pnpm_lock_legacy`) mode differ +//! only in where a rewired entry points ([`Target`]); this module owns what +//! they agree on: +//! +//! * which lock versions each target's edit has fixtures for +//! ([`hosted_refuses_lock`]; vendored takes the grammars its rekey was +//! captured against, [`super::sniff_lock_grammar`], and refuses the rest, +//! target-specifically: divergences N16); +//! * how a lock key names a patched `name@version` ([`match_key`]): the +//! registry entry with its peer suffix, or the entry a vendored rewire +//! already rekeyed; +//! * the all-or-nothing gate: an instance the target's edit cannot repoint +//! ([`Target::repoints`]) refuses the release in every lock, so no lock +//! is left half-rewired. +//! +//! The edits themselves stay with the targets: hosted splices each +//! instance's `resolution:`; vendored rekeys the entry to its `file:` +//! artifact and mirrors the override pnpm cross-checks. + +use super::grammar; + +use std::collections::BTreeMap; + +use crate::constants::npm_family::{PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY}; +use crate::formats::text::strip_bom; +use crate::utils::pnpm_workspace::pinned_pnpm_major; +use crate::vendor::npm_common::tgz_rel_leaf; + +/// Where a rewired entry points. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Target { + /// A Socket-hosted tarball URL (`scan --mode hosted`): the instance's + /// `resolution:` is spliced in place. + Hosted, + /// A committed `.socket/vendor/npm//` tarball (`vendor`): + /// the entry is rekeyed to `file:`. + Vendored, +} + +/// Whether the hosted splice refuses a lock: pnpm 1's early shrinkwrap +/// (shrinkwrapVersion 3 without a minor version), whose installers drop +/// registry tarball URLs whatever the frozen flag says (verified with pnpm +/// 1.0.0). Hosted takes every other lock version (divergences N16). +pub(crate) fn hosted_refuses_lock(text: &str) -> bool { + let field = |name: &str| strip_bom(text).lines().find_map(|l| l.strip_prefix(name)); + field("shrinkwrapVersion:").is_some_and(|v| v.trim().trim_matches(['\'', '"']) == "3") + && !field("shrinkwrapMinorVersion:") + .is_some_and(|v| v.trim().parse::().is_ok_and(|v| v > 0)) +} + +impl Target { + /// Whether this target's edit repoints a registry instance whose key + /// carries `suffix` after `name@version`. + pub(crate) fn repoints(self, suffix: &str) -> bool { + match self { + // The rekey moves the plain entry only: a peer-suffixed rekey + // has no fixture and would dangle. + Target::Vendored => suffix.is_empty(), + // The splice repoints every peer context it can parse: none, + // pnpm 5's `_peer@1`, or balanced `(peer@1)` groups. + Target::Hosted if suffix.is_empty() || suffix.starts_with('_') => true, + Target::Hosted => { + let mut depth = 0usize; + for c in suffix.chars() { + match c { + '(' => depth += 1, + ')' if depth > 0 => depth -= 1, + _ if depth == 0 => return false, + _ => {} + } + } + depth == 0 + } + } + } +} + +/// The lock pnpm reads in a directory holding `pnpm-lock.yaml` (`lock`) +/// and/or a pnpm <= 2 `shrinkwrap.yaml` (`shrinkwrap`), given the pnpm major +/// the project pins ([`pinned_pnpm_major`]); the other one is SHADOWED +/// (owner decision on divergence N11): never rewritten, never part of the +/// all-or-nothing gate (debris in it cannot refuse a dependency), only swept +/// for stale Socket identities by lockfile discovery. pnpm 1 / 2 read only +/// `shrinkwrap.yaml`; pnpm >= 3 read only `pnpm-lock.yaml`. With no pin, +/// `pnpm-lock.yaml` governs (pnpm <= 2 never writes one). +pub(crate) fn governing_pnpm_lock( + lock: bool, + shrinkwrap: bool, + pinned_major: Option, +) -> Option<&'static str> { + let pnpm_1_or_2 = pinned_major.is_some_and(|major| major <= 2); + let governs = shrinkwrap && (!lock || pnpm_1_or_2); + (governs.then_some(PNPM_SHRINKWRAP_LEGACY)).or(lock.then_some(PNPM_LOCK)) +} + +/// The pnpm locks of a hosted run's `files` (root-relative paths) a planner +/// rewrites, in key order: per directory, the [`governing_pnpm_lock`], read +/// with the evidence `files` holds for that directory. +pub(crate) fn planned_locks(files: &BTreeMap) -> Vec<&String> { + files.keys().filter(|key| governs(files, key)).collect() +} + +/// The pnpm locks of `files` pnpm does not read (the complement of +/// [`planned_locks`]): never rewritten, and a Socket URL left in one pins +/// nothing. +pub(crate) fn shadowed_locks(files: &BTreeMap) -> Vec<&String> { + files + .keys() + .filter(|key| pnpm_lock_leaf(key).is_some() && !governs(files, key)) + .collect() +} + +/// The root directory's governing pnpm lock ([`governing_pnpm_lock`]), +/// `pnpm-lock.yaml` when `files` holds neither. +pub(crate) fn root_lock(files: &BTreeMap) -> &'static str { + [PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY] + .into_iter() + .find(|lock| files.contains_key(*lock) && governs(files, lock)) + .unwrap_or(PNPM_LOCK) +} + +/// The leaf of root-relative `key` when it names a `pnpm-lock.yaml` or +/// `shrinkwrap.yaml`. +fn pnpm_lock_leaf(key: &str) -> Option<&str> { + key.rsplit('/') + .next() + .filter(|leaf| matches!(*leaf, PNPM_LOCK | PNPM_SHRINKWRAP_LEGACY)) +} + +/// Whether the pnpm lock `key` is its directory's [`governing_pnpm_lock`], +/// read with the evidence `files` holds for that directory. +fn governs(files: &BTreeMap, key: &str) -> bool { + let Some(leaf) = pnpm_lock_leaf(key) else { + return false; + }; + // `dir` keeps its trailing `/` (empty at the root). + let in_dir = |rel: &str| format!("{}{rel}", &key[..key.len() - leaf.len()]); + let has = |rel: &str| files.contains_key(&in_dir(rel)); + let both = (has(PNPM_LOCK), has(PNPM_SHRINKWRAP_LEGACY)); + // The pin decides only between the two, so it is read only then. + let pinned = match both { + (true, true) => pinned_pnpm_major(|rel| files.get(&in_dir(rel)).cloned()), + _ => None, + }; + governing_pnpm_lock(both.0, both.1, pinned) == Some(leaf) +} + +/// How one `packages:` / `snapshots:` key names a patched release. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum KeyMatch<'k> { + /// The registry entry, with whatever follows `name@version` (empty, or + /// a peer suffix such as `(react@18.2.0)` / `_react@18.2.0`). + Registry { suffix: &'k str }, + /// The entry a vendored rewire rekeyed: v9's `name@file:` or + /// legacy's bare `file:`, where the artifact is this + /// project's vendored tarball of this release + /// ([`crate::origin::vendor_ref`]: root-anchored, so a sibling + /// checkout's `../.socket/vendor/…` is not ours; any uuid). + Rekeyed, +} + +/// How `key` names `name@version`, if it does (see [`KeyMatch`]). Keys in +/// every grammar's spelling (`name@1.0.0`, `/name@1.0.0`, `/name/1.0.0`, +/// quoted) read alike. +pub(crate) fn match_key<'k>(key: &'k str, name: &str, version: &str) -> Option> { + if let Some(suffix) = grammar::suffix(key, name, version) { + return Some(KeyMatch::Registry { suffix }); + } + let key = grammar::unquote(key); + let path = key + .strip_prefix(name) + .and_then(|rest| rest.strip_prefix("@file:")) + .or_else(|| key.strip_prefix("file:"))?; + // The leaf binding matters: a project can vendor one package at several + // versions, and an edit must never take a sibling version's entry. + crate::origin::vendor_ref(path) + .filter(|v| v.eco == "npm" && v.leaf == tgz_rel_leaf(name, version)) + .map(|_| KeyMatch::Rekeyed) +} + +#[cfg(test)] +mod tests { + use super::super::{sniff_lock_grammar, PnpmLockGrammar}; + use super::*; + + const ART: &str = ".socket/vendor/npm/0b8d2c3e-4f5a-4b6c-8d7e-9f0a1b2c3d4e/left-pad-1.3.0.tgz"; + + #[test] + fn keys_of_every_grammar_name_the_release() { + for key in [ + "left-pad@1.3.0", + "/left-pad@1.3.0", + "/left-pad/1.3.0", + "'left-pad@1.3.0'", + ] { + assert_eq!( + match_key(key, "left-pad", "1.3.0"), + Some(KeyMatch::Registry { suffix: "" }), + "{key}" + ); + } + assert_eq!( + match_key("left-pad@1.3.0(react@18.2.0)", "left-pad", "1.3.0"), + Some(KeyMatch::Registry { + suffix: "(react@18.2.0)" + }) + ); + assert_eq!(match_key("left-pad@1.3.01", "left-pad", "1.3.0"), None); + assert_eq!( + match_key(&format!("left-pad@file:{ART}"), "left-pad", "1.3.0"), + Some(KeyMatch::Rekeyed) + ); + assert_eq!( + match_key(&format!("file:{ART}"), "left-pad", "1.3.0"), + Some(KeyMatch::Rekeyed) + ); + // A sibling version's artifact, another package's key, and a + // sibling checkout's path are not this release's rekey. + assert_eq!(match_key(&format!("file:{ART}"), "left-pad", "1.3.1"), None); + assert_eq!( + match_key(&format!("is-odd@file:{ART}"), "left-pad", "1.3.0"), + None + ); + assert_eq!( + match_key(&format!("left-pad@file:../{ART}"), "left-pad", "1.3.0"), + None + ); + } + + /// N11: of a directory's `pnpm-lock.yaml` and `shrinkwrap.yaml`, the + /// one the pinned pnpm major reads governs and the other is shadowed + /// (pnpm 1 / 2 read only the shrinkwrap). + #[test] + fn the_pinned_pnpm_major_picks_the_governing_lock() { + // No pin: `pnpm-lock.yaml` governs a pair, and each lone lock itself. + assert_eq!(governing_pnpm_lock(true, true, None), Some(PNPM_LOCK)); + assert_eq!( + governing_pnpm_lock(false, true, None), + Some(PNPM_SHRINKWRAP_LEGACY) + ); + assert_eq!(governing_pnpm_lock(false, false, Some(2)), None); + // pnpm 1 / 2 read only `shrinkwrap.yaml`; pnpm >= 3 only the lock. + assert_eq!( + governing_pnpm_lock(true, true, Some(2)), + Some(PNPM_SHRINKWRAP_LEGACY) + ); + assert_eq!(governing_pnpm_lock(true, true, Some(3)), Some(PNPM_LOCK)); + assert_eq!(governing_pnpm_lock(true, false, Some(1)), Some(PNPM_LOCK)); + + let files = |pin: Option<&str>| { + let mut files: BTreeMap = [ + "a/pnpm-lock.yaml", + "a/shrinkwrap.yaml", + "b/shrinkwrap.yaml", + "pnpm-lock.yaml", + "shrinkwrap.yaml", + ] + .into_iter() + .map(|k| (k.to_string(), String::new())) + .collect(); + if let Some(pin) = pin { + let pkg = format!(r#"{{"packageManager": "pnpm@{pin}"}}"#); + files.insert("package.json".into(), pkg); + } + files + }; + let planned = |files: &BTreeMap| { + planned_locks(files) + .into_iter() + .cloned() + .collect::>() + }; + // No evidence, or pnpm >= 3: each `pnpm-lock.yaml` shadows its + // sibling shrinkwrap. + for pin in [None, Some("9.15.0")] { + assert_eq!( + planned(&files(pin)), + ["a/pnpm-lock.yaml", "b/shrinkwrap.yaml", "pnpm-lock.yaml"], + "{pin:?}" + ); + } + // pnpm 2 pinned at the root: there `shrinkwrap.yaml` governs and the + // stray `pnpm-lock.yaml` is shadowed; `a/` has no evidence of its own. + assert_eq!( + planned(&files(Some("2.25.7"))), + ["a/pnpm-lock.yaml", "b/shrinkwrap.yaml", "shrinkwrap.yaml"] + ); + // The install record outranks package.json. + let mut record = files(Some("9.15.0")); + record.insert( + "node_modules/.modules.yaml".into(), + "packageManager: pnpm@2.25.7\n".into(), + ); + assert_eq!( + planned(&record).last().map(String::as_str), + Some("shrinkwrap.yaml") + ); + + // The root's governing lock, and the shadowed rest. + assert_eq!(root_lock(&files(None)), PNPM_LOCK); + assert_eq!(root_lock(&files(Some("2.25.7"))), PNPM_SHRINKWRAP_LEGACY); + assert_eq!( + shadowed_locks(&files(Some("2.25.7"))), + ["a/shrinkwrap.yaml", "pnpm-lock.yaml"] + ); + let lone: BTreeMap = + [(PNPM_SHRINKWRAP_LEGACY.to_string(), String::new())].into(); + assert_eq!(root_lock(&lone), PNPM_SHRINKWRAP_LEGACY); + assert_eq!(root_lock(&BTreeMap::new()), PNPM_LOCK); + } + + #[test] + fn each_target_repoints_what_its_edit_has_fixtures_for() { + assert!(Target::Hosted.repoints("")); + assert!(Target::Hosted.repoints("(react@18.2.0)")); + assert!(Target::Hosted.repoints("_react@18.2.0")); + assert!(!Target::Hosted.repoints("(react@18.2.0")); + assert!(Target::Vendored.repoints("")); + assert!(!Target::Vendored.repoints("(react@18.2.0)")); + assert!(!Target::Vendored.repoints("_react@18.2.0")); + } + + /// Divergence N16: hosted reads every lock but early shrinkwrap, while + /// vendored keeps its refusal for every version its rekey was not + /// captured against (target-specific). + #[test] + fn lock_versions_each_target_rewrites() { + for (head, vendored) in [ + ("lockfileVersion: '9.0'\n", Some(PnpmLockGrammar::V9)), + ("lockfileVersion: '6.0'\n", Some(PnpmLockGrammar::V60)), + ("lockfileVersion: 5.4\n", Some(PnpmLockGrammar::V54)), + ("lockfileVersion: 5.3\n", None), + ("lockfileVersion: 5.1\n", None), + ("lockfileVersion: '6.1'\n", None), + ("lockfileVersion: '10.0'\n", None), + ("shrinkwrapVersion: 3\nshrinkwrapMinorVersion: 9\n", None), + ] { + assert!(!hosted_refuses_lock(head), "{head}"); + assert_eq!(sniff_lock_grammar(head).ok(), vendored, "{head}"); + } + assert!(hosted_refuses_lock("shrinkwrapVersion: 3\n")); + } +} diff --git a/crates/socket-patch-core/src/formats/pnpm/workspace.rs b/crates/socket-patch-core/src/formats/pnpm/workspace.rs index 5c0d12b7f..63b83c9fd 100644 --- a/crates/socket-patch-core/src/formats/pnpm/workspace.rs +++ b/crates/socket-patch-core/src/formats/pnpm/workspace.rs @@ -202,11 +202,17 @@ pub(crate) fn lock_has_member_importers>(lines: &[S]) -> bool { } /// [`lock_has_member_importers`] of a whole `pnpm-lock.yaml` text: its -/// project document alone ([`super::grammar::main_document`]), so importers +/// project documents alone ([`super::grammar::documents`]), so importers /// of a pnpm 11+ env document never count. pub(crate) fn lock_text_has_member_importers(text: &str) -> bool { - let doc = super::grammar::main_document(strip_bom(text)); - lock_has_member_importers(&doc.split('\n').collect::>()) + use super::grammar::{documents, PnpmDocument}; + documents(text) + .into_iter() + .filter(|(_, kind)| *kind == PnpmDocument::Project) + .any(|(span, _)| { + let doc = strip_bom(&text[span]); + lock_has_member_importers(&doc.split('\n').collect::>()) + }) } /// `text` with the contiguous run of lines `block` removed, when that diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index d682993d0..18bd851f0 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -872,10 +872,17 @@ pub async fn read_candidate_files( // and yarn classic rewriters read it as advisory input only: no // rewriter edits it, so a link or an unreadable in-memory entry is left // out (the rewriters then keep their conservative reading) rather than - // refused. + // refused. Beside a root `pnpm-lock.yaml` / `shrinkwrap.yaml` pair its + // `packageManager` pin decides which of the two pnpm reads + // (`formats::pnpm::plan::planned_locks`), as vendored mode and + // discovery read it, when no `node_modules/.modules.yaml` records the + // installed major (a fresh clone). } else if candidates.iter().any(|c| c.dep.ecosystem == "npm") && (NPM_LOCKS.iter().any(|lock| out.files.contains_key(*lock)) - || out.files.contains_key("yarn.lock")) + || out.files.contains_key("yarn.lock") + || ROOT_PNPM_LOCKS + .iter() + .all(|lock| out.files.contains_key(*lock))) { let rel = crate::hosted::memory::select::NPM_MANIFEST_REL; if let Some(text) = read_advisory(view, unreadable, rel).await { @@ -2415,6 +2422,7 @@ async fn unattributed_pins( for c in discovery.unpatched_copies.iter().filter(|c| { c.purl == r.purl && c.file == r.source_file + && r.part.withheld_by(c.part) && matches!( c.file.file_name().and_then(|n| n.to_str()), Some("pnpm-lock.yaml" | "shrinkwrap.yaml" | "yarn.lock") @@ -2897,8 +2905,11 @@ fn pnpm_trust( .count(); // The locks pnpm installs the project from: the root lock, plus each // member's own under `sharedWorkspaceLockfile: false` (#492). A Rush - // lock is never one: rush installs it from common/temp. - let governing: Vec<&str> = std::iter::once("pnpm-lock.yaml") + // lock is never one: rush installs it from common/temp. The root lock + // is the one pnpm reads (N11): `shrinkwrap.yaml` under a pnpm 1 / 2 + // pin, so a stray `pnpm-lock.yaml` never drives the heal or the trust + // config. + let governing: Vec<&str> = std::iter::once(crate::formats::pnpm::plan::root_lock(files)) .chain(member_lock_keys.iter().map(String::as_str)) .filter(|key| !rush_lock_keys.iter().any(|rush| rush == key)) .collect(); @@ -3538,9 +3549,15 @@ fn confirm( name.ends_with("/pom.xml") || maven_reactor_refused && (name == "pom.xml" || name.starts_with(".mvn/")) }; + // A pnpm lock pnpm does not read (a `shrinkwrap.yaml` beside a + // `pnpm-lock.yaml`, or the reverse under a pnpm 1 / 2 pin, N11) is + // never rewritten, so a Socket URL an earlier run left in it pins + // nothing either. + let shadowed_pnpm = crate::formats::pnpm::plan::shadowed_locks(files); let final_texts: Vec<(&str, &String)> = files .iter() .filter(|(name, _)| !(pdm_inactive && name.as_str() == "pdm.lock")) + .filter(|(name, _)| !shadowed_pnpm.contains(name)) .filter(|(name, _)| !is_npm_manifest(name)) .filter(|(name, _)| !is_unpinned_maven(name)) .filter(|(name, _)| !crate::patch::redirect::sbt::is_synthetic_key(name)) @@ -4783,6 +4800,104 @@ mod tests { ); } + /// N11: under a pnpm 1 / 2 pin `shrinkwrap.yaml` governs, so a + /// stray v9 `pnpm-lock.yaml` an earlier run redirected is no heal root + /// and plans no `trustLockfile` write for a lock pnpm never reads. + #[tokio::test] + async fn a_shadowed_pnpm_lock_is_no_heal_root() { + let redirected = LEFT_PAD_V9_LOCK.replace( + "resolution: {integrity: sha512-UPSTREAM==}", + &format!( + "resolution: {{integrity: sha512-PATCHED==, tarball: {}}}", + left_pad_url() + ), + ); + let mut p = MemoryProject::new(); + p.insert_text("pnpm-lock.yaml", redirected.as_str()); + p.insert_text( + "shrinkwrap.yaml", + "shrinkwrapMinorVersion: 9\nshrinkwrapVersion: 3\n", + ); + p.insert_text( + "package.json", + r#"{"name":"app","packageManager":"pnpm@2.25.7"}"#, + ); + let (_, done) = npm_rewrite(&ProjectView::Memory(&p), &BTreeSet::new()).await; + assert!( + !done.rewrite.files.contains_key(PNPM_WORKSPACE_REL), + "{:?}", + done.rewrite.files.keys() + ); + + // Without the pin, pnpm-lock.yaml governs and the heal plans it. + p.insert_text("package.json", r#"{"name":"app"}"#); + let (_, done) = npm_rewrite(&ProjectView::Memory(&p), &BTreeSet::new()).await; + assert!( + done.rewrite.files.contains_key(PNPM_WORKSPACE_REL), + "{:?}", + done.rewrite.files.keys() + ); + } + + /// N11: a shadowed `shrinkwrap.yaml` still carrying an earlier run's + /// Socket URL confirms nothing (and so is no `redirect_unattributable` + /// skip) when the governing `pnpm-lock.yaml` does not pin the package. + #[tokio::test] + async fn a_shadowed_pnpm_lock_confirms_nothing() { + let mut p = MemoryProject::new(); + p.insert_text( + "pnpm-lock.yaml", + "lockfileVersion: '9.0'\n\npackages:\n other@1.0.0:\n \ + resolution: {integrity: sha512-OTHER==}\n", + ); + p.insert_text( + "shrinkwrap.yaml", + format!( + "shrinkwrapVersion: 3\nshrinkwrapMinorVersion: 9\npackages:\n /left-pad/1.3.0:\n \ + resolution:\n integrity: sha512-PATCHED==\n tarball: {}\n", + left_pad_url() + ) + .as_str(), + ); + let (_, done) = npm_rewrite(&ProjectView::Memory(&p), &BTreeSet::new()).await; + assert!(done.confirmed.is_empty(), "{:?}", done.confirmed); + assert!(done.unattributed.is_empty(), "{:?}", done.unattributed); + } + + #[tokio::test] + async fn a_pnpm_lock_pair_reads_the_package_manager_pin() { + // pnpm 2 pinned in package.json, no install record: pnpm reads + // shrinkwrap.yaml, so the planner must see the pin to leave the + // stray pnpm-lock.yaml alone. + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + let manifest = r#"{"name":"app","packageManager":"pnpm@2.25.7"}"#; + std::fs::write(root.join("package.json"), manifest).unwrap(); + std::fs::write(root.join("pnpm-lock.yaml"), "lockfileVersion: 5.4\n").unwrap(); + std::fs::write( + root.join("shrinkwrap.yaml"), + "shrinkwrapMinorVersion: 9\nshrinkwrapVersion: 3\n", + ) + .unwrap(); + let candidates = vec![left_pad_candidate()]; + let read = + read_candidate_files(&ProjectView::Disk(root), &BTreeSet::new(), &candidates).await; + assert_eq!( + read.files.get("package.json").map(String::as_str), + Some(manifest) + ); + assert_eq!( + crate::formats::pnpm::plan::planned_locks(&read.files), + vec!["shrinkwrap.yaml"] + ); + + // One pnpm lock alone governs whatever the pin: no manifest read. + std::fs::remove_file(root.join("shrinkwrap.yaml")).unwrap(); + let read = + read_candidate_files(&ProjectView::Disk(root), &BTreeSet::new(), &candidates).await; + assert!(!read.files.contains_key("package.json")); + } + #[tokio::test] async fn issue_490_the_root_manifest_overrides_reach_the_npm_rewriter() { let redirected = |done: &Rewritten| { diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 91f3a625c..2dd3d5c86 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -28578,6 +28578,186 @@ packages: ); } + /// Lock-model divergence N5 (pnpm): "is this dep vendored?" reads the + /// rekeyed packages entry with the origin classifier + /// (`origin::vendor_ref`), the rule discovery reads vendored pnpm keys + /// with, instead of a substring test over every line. A path that is + /// not this project's vendored npm artifact (no canonical uuid dir, + /// another ecosystem's dir) and an `overrides:` line with no rekeyed + /// entry behind it are the generic entry-not-found now; the backends' + /// real v9 and legacy shapes stay vendored (see + /// `pnpm_vendored_entry_is_named_vendored_not_entry_not_found`). + #[test] + fn pnpm_vendored_diagnosis_reads_the_rekeyed_entry_through_origin() { + let ovr = npm_override( + "left-pad", + "1.3.0", + "http://patch.test/left-pad-1.3.0.tgz", + "sha512-PATCHED==", + ); + let uuid = "1a2b3c4d-5e6f-4a1b-8c2d-0123456789ab"; + let v9 = |path: &str| { + format!( + "lockfileVersion: '9.0'\n\npackages:\n\n left-pad@file:{path}:\n \ + resolution: {{integrity: sha512-VENDORED==, tarball: file:{path}}}\n \ + version: 1.3.0\n" + ) + }; + let legacy_overrides_only = format!( + "lockfileVersion: '6.0'\n\noverrides:\n left-pad@1.3.0: \ + file:/abs/project/.socket/vendor/npm/{uuid}/left-pad-1.3.0.tgz\n\npackages:\n\n \ + /other@1.0.0:\n resolution: {{integrity: sha512-OTHER==}}\n dev: false\n" + ); + let codes = |lock: &str| { + let files = BTreeMap::from([("pnpm-lock.yaml".to_string(), lock.to_string())]); + let r = rewrite_registry_redirect(&files, std::slice::from_ref(&ovr)); + assert!(r.files.is_empty() && r.edits.is_empty(), "{lock}"); + r.warnings.into_iter().map(|w| w.code).collect::>() + }; + for (case, lock, want) in [ + ( + "the backend's v9 shape", + v9(&format!(".socket/vendor/npm/{uuid}/left-pad-1.3.0.tgz")), + "redirect_pnpm_entry_vendored", + ), + ( + "no uuid dir", + v9(".socket/vendor/npm/left-pad-1.3.0.tgz"), + "redirect_pnpm_entry_not_found", + ), + ( + "another ecosystem's dir", + v9(&format!(".socket/vendor/cargo/{uuid}/left-pad-1.3.0.tgz")), + "redirect_pnpm_entry_not_found", + ), + ( + "an overrides line alone", + legacy_overrides_only, + "redirect_pnpm_entry_not_found", + ), + // Another version vendored is not this one (the dep's version + // is simply not locked). + ( + "another version vendored", + v9(&format!(".socket/vendor/npm/{uuid}/left-pad-1.2.0.tgz")) + .replace("version: 1.3.0", "version: 1.2.0"), + "redirect_pnpm_entry_not_found", + ), + // Lock-model 2b: the hint reads the key with the planning core's + // `match_key`, the rule the vendored planner rekeys by, so the + // artifact leaf names the release and the `version:` line is not + // consulted. + ( + "a rekey without a version line", + v9(&format!(".socket/vendor/npm/{uuid}/left-pad-1.3.0.tgz")) + .replace(" version: 1.3.0\n", ""), + "redirect_pnpm_entry_vendored", + ), + ( + "another release's artifact under this version line", + v9(&format!(".socket/vendor/npm/{uuid}/left-pad-1.2.0.tgz")), + "redirect_pnpm_entry_not_found", + ), + ( + "a sibling checkout's artifact", + v9(&format!("../.socket/vendor/npm/{uuid}/left-pad-1.3.0.tgz")), + "redirect_pnpm_entry_not_found", + ), + ] { + assert_eq!(codes(&lock), vec![want], "{case}"); + } + } + + /// Divergence N11 (owner decision, lock-model 2b): a pnpm <= 2 + /// `shrinkwrap.yaml` beside a `pnpm-lock.yaml` is shadowed. pnpm reads + /// only the `pnpm-lock.yaml`, so the hosted planner never rewrites the + /// shrinkwrap and leaves it out of the all-or-nothing gate: debris in it + /// (a key the splice cannot repoint, or pnpm 1's early shrinkwrap head) + /// refuses nothing. A `shrinkwrap.yaml` + /// that is the only pnpm lock is still rewritten. + #[test] + fn pnpm_shrinkwrap_beside_a_pnpm_lock_is_shadowed() { + let url = "http://patch.test/left-pad-1.3.0.tgz"; + let ovr = npm_override("left-pad", "1.3.0", url, "sha512-PATCHED=="); + let lock = "lockfileVersion: '9.0'\n\npackages:\n\n left-pad@1.3.0:\n \ + resolution: {integrity: sha512-OLD==}\n"; + let shrinkwrap = |head: &str, key: &str| { + format!( + "{head}\n\npackages:\n\n {key}:\n resolution: {{integrity: sha512-OLD==}}\n" + ) + }; + let supported = "shrinkwrapVersion: 3\nshrinkwrapMinorVersion: 9"; + let run = |files: &[(&str, String)]| { + let files: BTreeMap = files + .iter() + .map(|(k, v)| (k.to_string(), v.clone())) + .collect(); + rewrite_registry_redirect(&files, std::slice::from_ref(&ovr)) + }; + for (case, shrink) in [ + ("a plain entry", shrinkwrap(supported, "/left-pad/1.3.0")), + ( + "an unparseable key", + shrinkwrap(supported, "/left-pad/1.3.0)("), + ), + ( + "an early shrinkwrap", + shrinkwrap("shrinkwrapVersion: 3", "/left-pad/1.3.0"), + ), + ] { + let r = run(&[ + ("pnpm-lock.yaml", lock.to_string()), + ("shrinkwrap.yaml", shrink), + ]); + assert_eq!( + r.files.keys().collect::>(), + ["pnpm-lock.yaml"], + "{case}: only the governing lock is written: {:?}", + r.warnings + ); + assert!(r.files["pnpm-lock.yaml"].contains(url), "{case}"); + assert!(r.warnings.is_empty(), "{case}: {:?}", r.warnings); + assert!(r.refused_pnpm_uuids.is_empty(), "{case}"); + assert!(r.edits.iter().all(|e| e.path == "pnpm-lock.yaml"), "{case}"); + } + // A nested `shrinkwrap.yaml` is shadowed only by its own sibling. + let r = run(&[ + ("pnpm-lock.yaml", lock.to_string()), + ( + "sub/shrinkwrap.yaml", + shrinkwrap(supported, "/left-pad/1.3.0"), + ), + ]); + assert!( + r.files.contains_key("sub/shrinkwrap.yaml"), + "{:?}", + r.warnings + ); + // A project pinning pnpm 2 installs from `shrinkwrap.yaml` (review + // finding): it is rewritten and the stray `pnpm-lock.yaml` is the + // shadowed one, so debris in it refuses nothing. + let pnpm2 = r#"{"packageManager": "pnpm@2.25.7"}"#.to_string(); + let r = run(&[ + ("package.json", pnpm2), + ( + "pnpm-lock.yaml", + lock.replace("left-pad@1.3.0:", "left-pad@1.3.0)(:"), + ), + ("shrinkwrap.yaml", shrinkwrap(supported, "/left-pad/1.3.0")), + ]); + assert_eq!( + r.files.keys().collect::>(), + ["shrinkwrap.yaml"], + "{:?}", + r.warnings + ); + assert!(r.files["shrinkwrap.yaml"].contains(url)); + assert!(r.refused_pnpm_uuids.is_empty(), "{:?}", r.warnings); + // The only pnpm lock: rewritten as before. + let r = run(&[("shrinkwrap.yaml", shrinkwrap(supported, "/left-pad/1.3.0"))]); + assert!(r.files["shrinkwrap.yaml"].contains(url), "{:?}", r.warnings); + } + /// A classic-lock block whose first line is not a `key:` line /// (hand-mangled) is skipped without derailing the rewrite of the real /// entry — and survives byte-identically. diff --git a/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs b/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs index 2d026a253..8a00efcb7 100644 --- a/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs +++ b/crates/socket-patch-core/src/patch/redirect/pnpm_equivalence_tests.rs @@ -232,7 +232,7 @@ fn indexed_pnpm_rewrite_matches_golden_on_depscan_sized_lock_set() { }; let root_extra = vec![ format!( - " vendored-pkg@file:.socket/vendor/npm/vendored-pkg-1.0.0.tgz:\n resolution: {{tarball: file:.socket/vendor/npm/vendored-pkg-1.0.0.tgz}}\n" + " vendored-pkg@file:.socket/vendor/npm/0a1b2c3d-4e5f-4a6b-8c7d-8e9f0a1b2c3d/vendored-pkg-1.0.0.tgz:\n resolution: {{tarball: file:.socket/vendor/npm/0a1b2c3d-4e5f-4a6b-8c7d-8e9f0a1b2c3d/vendored-pkg-1.0.0.tgz}}\n version: 1.0.0\n" ), // A second copy of an entry that already points at the hosted // artifact (a re-run): rebuilt == original, no edit. diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs index 709fb0da1..745840c0a 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs @@ -1299,15 +1299,14 @@ fn pnpm_registry_is_own(registry: Option<&str>) -> bool { /// env lockfile document ahead of the project lock (only pnpm >= 11 /// writes one, so its `.npmrc` pnpm settings are ignored). pub(crate) fn pnpm_settings_major(text: &str, pm_major: Option) -> Option { - use crate::formats::pnpm::grammar::{is_pnpm_lock_text, main_document}; + use crate::formats::pnpm::grammar::{documents, PnpmDocument}; use crate::formats::pnpm::lock_version_major; let legacy_lock = lock_version_major(text).is_some_and(|major| major < 9) || text .lines() .any(|line| line.starts_with("shrinkwrapVersion:")); - let main = main_document(text); - let env_document = is_pnpm_lock_text(&text[..text.len() - main.len()]); + let env_document = documents(text).iter().any(|(_, k)| *k == PnpmDocument::Env); pm_major .or(legacy_lock.then_some(8)) .or(env_document.then_some(11)) @@ -1527,17 +1526,17 @@ fn pnpm_include_tarball( /// URL can still appear with it off, under a second registry). With no /// bare one, a tarball pnpm could have derived proves it on. `None`: no /// such resolution (an unconventional URL is recorded either way). Only -/// the main document counts: pnpm 11+'s env lockfile document records its +/// the project documents count: pnpm 11+'s env lockfile document records its /// resolutions bare whatever the setting. fn pnpm_lock_tarball_evidence( text: &str, registry: impl Fn(&str) -> Option, is_hosted: impl Fn(&str) -> bool, ) -> Option { - use crate::formats::pnpm::{classify_pnpm_key, grammar::main_document, pnpm_packages, PnpmKey}; + use crate::formats::pnpm::{classify_pnpm_key, PnpmDocument, PnpmKey, PnpmLock}; let mut derived = false; - for package in pnpm_packages(main_document(text)) { + for package in PnpmLock::parse(text).packages_of(PnpmDocument::Project) { let PnpmKey::Registry { name, version } = classify_pnpm_key(package.key) else { continue; }; @@ -1895,7 +1894,7 @@ pub(crate) async fn restore_pnpm_locks( files: &[String], ctx: &Ctx<'_>, ) -> FormatResult { - use crate::formats::pnpm::grammar as pnpm; + use crate::formats::pnpm::{grammar as pnpm, PnpmDocument, PnpmLock}; let mut result = FormatResult::default(); let pins = by_uuid(pins); @@ -1905,8 +1904,9 @@ pub(crate) async fn restore_pnpm_locks( }; // (resolution range, uuid, name, version, rebuilt-without-integrity) let mut hits: Vec<(std::ops::Range, String, String, String)> = Vec::new(); - for entry in pnpm::entries(&text) { - let Some(resolution) = pnpm::resolution(&entry) else { + let lock = PnpmLock::parse(&text); + for package in lock.packages() { + let (entry, Some(resolution)) = (&package.entry, &package.resolution) else { continue; }; let Some(uuid) = resolution.tarball().and_then(|t| ctx.hosted_uuid(t)) else { @@ -1966,10 +1966,15 @@ pub(crate) async fn restore_pnpm_locks( let mut splices: Vec<(std::ops::Range, String)> = Vec::new(); // (uuid, name@version whose `tarball:` only the guessed setting decided) let mut handled: Vec<(String, Option)> = Vec::new(); - for entry in pnpm::entries(&text) { - let Some(resolution) = pnpm::resolution(&entry) else { + for package in lock.packages() { + let Some(resolution) = &package.resolution else { continue; }; + // pnpm 11+ records the env lockfile document's resolutions bare + // whatever `lockfileIncludeTarballUrl` says (see + // `pnpm_tarball_policy`): only a URL it cannot derive gets + // `tarball:` there. + let in_env = package.document == PnpmDocument::Env; let Some((_, uuid, name, version)) = hits.iter().find(|(r, ..)| *r == resolution.range) else { continue; @@ -1991,13 +1996,15 @@ pub(crate) async fn restore_pnpm_locks( // pnpm records `tarball:` under lockfileIncludeTarballUrl and for // a URL it cannot derive from the registry (#557). let derived = policy.derives_tarball(name, version, &dist.tarball, found.from_project); - let restored = if policy.always || !derived { + let restored = if (policy.always && !in_env) || !derived { resolution.rewrite(integrity, &dist.tarball) } else { resolution.restore(integrity) }; - // A URL pnpm records anyway is no guess. - let guessed = (policy.guess.is_some() && derived).then(|| format!("{name}@{version}")); + // A URL pnpm records anyway is no guess, and neither is an env + // document entry. + let guessed = + (policy.guess.is_some() && derived && !in_env).then(|| format!("{name}@{version}")); splices.push((resolution.range.clone(), restored)); handled.push((uuid.clone(), guessed)); } @@ -3852,6 +3859,13 @@ mod tests { // A lone leading `---` marker is no env document. let marker_only = format!("---\n{}", lock("")); assert!(include(&marker_only, None, Some(RC_ON), None)); + // Neither is an env-shaped document AFTER the project one: pnpm + // writes the env document first. + let env_last = format!("---\n{}{env}", lock("")); + assert_eq!(super::pnpm_settings_major(&pinned_only, None), Some(11)); + assert_eq!(super::pnpm_settings_major(&env_last, None), None); + // Its bare entry is project evidence: the setting is off. + assert!(!include(&env_last, None, Some(RC_ON), None)); // A bare entry in the main document still proves it off. let bare = " b@2.0.0:\n resolution: {integrity: sha512-B==}\n"; assert!(!include( diff --git a/crates/socket-patch-core/src/utils/pnpm_workspace.rs b/crates/socket-patch-core/src/utils/pnpm_workspace.rs index c70385687..7573c6500 100644 --- a/crates/socket-patch-core/src/utils/pnpm_workspace.rs +++ b/crates/socket-patch-core/src/utils/pnpm_workspace.rs @@ -965,36 +965,54 @@ fn lockfile_off(setting: impl Fn(&str, &str) -> Option) -> Option) -> Option { + let texts = read_texts(view, &[MODULES_YAML, "package.json", PNPM_LOCK]).await; + pnpm_major_from(|rel| texts.get(rel).cloned()) +} + +/// The pnpm major the project at `view` pins ([`pinned_pnpm_major`]). +pub(crate) async fn pinned_pnpm_major_in(view: &ProjectView<'_>) -> Option { + let texts = read_texts(view, &[MODULES_YAML, "package.json"]).await; + pinned_pnpm_major(|rel| texts.get(rel).cloned()) +} + +/// The readable files of `rels` under `view`, by path. +async fn read_texts<'r>(view: &ProjectView<'_>, rels: &[&'r str]) -> BTreeMap<&'r str, String> { let mut texts = BTreeMap::new(); - for rel in [MODULES_YAML, "package.json", PNPM_LOCK] { + for &rel in rels { if let Ok(text) = view.read_text(rel).await { texts.insert(rel, text); } } - pnpm_major_from(|rel| texts.get(rel).cloned()) + texts } /// The installed pnpm's record of the project. pub(crate) const MODULES_YAML: &str = "node_modules/.modules.yaml"; -/// The pnpm major whose settings reading applies to a project whose files -/// `read` returns by project-relative path: the one that installed it -/// ([`MODULES_YAML`]), else the one package.json pins, else what the root -/// lock's grammar implies (`pnpm_settings_major`). -pub(crate) fn pnpm_major_from(read: impl Fn(&str) -> Option) -> Option { - use crate::patch::redirect::upstream::npm::{ - modules_yaml_pnpm_major, package_json_pnpm_major, pnpm_settings_major, - }; - let pinned = read(MODULES_YAML) +/// The pnpm major a project pins, from its files as `read` returns them by +/// project-relative path: the one that installed it ([`MODULES_YAML`]), +/// else the one package.json's `packageManager` names. +pub(crate) fn pinned_pnpm_major(read: impl Fn(&str) -> Option) -> Option { + use crate::patch::redirect::upstream::npm::{modules_yaml_pnpm_major, package_json_pnpm_major}; + let modules = read(MODULES_YAML); + modules .as_deref() .and_then(modules_yaml_pnpm_major) .or_else(|| { read("package.json") .as_deref() .and_then(package_json_pnpm_major) - }); + }) +} + +/// The pnpm major whose settings reading applies to a project whose files +/// `read` returns by project-relative path: the one it pins +/// ([`pinned_pnpm_major`]), else what the root lock's grammar implies +/// (`pnpm_settings_major`). +pub(crate) fn pnpm_major_from(read: impl Fn(&str) -> Option) -> Option { + let pinned = pinned_pnpm_major(&read); let lock = read(PNPM_LOCK).unwrap_or_default(); - pnpm_settings_major(&lock, pinned) + crate::patch::redirect::upstream::npm::pnpm_settings_major(&lock, pinned) } /// What pnpm does with its `lockfile` setting off, for the refusals and diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/golang.rs b/crates/socket-patch-core/src/vendor/lock_inventory/golang.rs index 25636f85d..f95d3a65f 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/golang.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/golang.rs @@ -43,6 +43,7 @@ pub(super) async fn inventory_go_sum_raw_in(view: &ProjectView<'_>) -> Option = Vec::new(); let mut unsupported: Vec = Vec::new(); - let npm = if every { - npm_family::inventory_npm_lock_raw_in(view).await - } else { - npm_family::inventory_npm_lock_in(view).await - }; - match npm { - Ok(Some((_, entries))) => out.extend(entries), - Ok(None) => {} + match npm_family::inventory_npm_trees_raw_in(view).await { + Ok(entries) if every => out.extend(entries), + Ok(entries) => out.extend(dedup_prefer_integrity(entries)), Err(diag) => unsupported.push(diag), } unsupported.extend(gem::unsupported_gem_layout_in(view).await); @@ -358,20 +359,23 @@ async fn union_views_in( (out, unsupported) } -/// Collapse duplicate (name, version) instances, preferring one that -/// carries a verifier. Each view applies it ONCE, at its public boundary, -/// over its `*_raw` body (collapsing is associative over concatenation — -/// the first verifier-bearing instance wins, else the first — so a union of -/// raw sub-views collapses to what collapsing each sub-view first would). +/// Collapse duplicate (name, version) instances, preferring a project +/// instance over a pnpm env-lockfile one ([`LockfileEntry::pnpm_env`]: a +/// config dependency never stands in for the project's own copy, whose +/// `resolved` url ledger liveness reads), then one that carries a +/// verifier. Each view applies it ONCE, at its public boundary, over its +/// `*_raw` body (collapsing is associative over concatenation — the first +/// most-preferred instance wins — so a union of raw sub-views collapses to +/// what collapsing each sub-view first would). fn dedup_prefer_integrity(raw: Vec) -> Vec { + let rank = |e: &LockfileEntry| (!e.pnpm_env, e.integrity != LockIntegrity::None); let mut seen: HashMap<(String, String), usize> = HashMap::new(); let mut out: Vec = Vec::new(); for entry in raw { let key = (entry.name.clone(), entry.version.clone()); match seen.get(&key) { Some(&i) => { - if out[i].integrity == LockIntegrity::None && entry.integrity != LockIntegrity::None - { + if rank(&entry) > rank(&out[i]) { out[i] = entry; } } diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs b/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs index 3b3edc459..e438d7499 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/npm_family.rs @@ -8,19 +8,23 @@ use std::path::Path; use crate::constants::npm_family::{BUN_LOCK, PNPM_SHRINKWRAP_LEGACY}; use crate::formats::governing_locks::{npm_governing_family, npm_lock_files, NpmLockFamily}; use crate::formats::yarn::{grammar as yarn_grammar, YarnLockGrammar}; +use crate::origin::hosted_patch_uuid; +use crate::utils::pnpm_workspace::lockfile_disabled; use crate::utils::purl::npm_purl; use crate::vendor::npm_flavor::NpmLockFlavor; use super::bun::{bun_text_lock_drives, inventory_bun_binary_in, inventory_bun_in}; +#[cfg(test)] +use super::dedup_prefer_integrity; use super::npm::inventory_package_lock_in; use super::pnpm::{ inventory_pnpm_lock_in, inventory_pnpm_lock_rel_in, inventory_pnpm_member_locks_in, - inventory_rush_pnpm_locks_in, + inventory_rush_pnpm_locks_in, is_rush, }; use super::view::ProjectView; use super::vlt::inventory_vlt_in; use super::yarn::{inventory_yarn_berry_in, inventory_yarn_classic_in}; -use super::{dedup_prefer_integrity, LockfileEntry, UnsupportedNpmLayout}; +use super::{LockIntegrity, LockfileEntry, UnsupportedNpmLayout}; use crate::vendor::npm_flavor::detect_npm_lock_flavor_in; // ── registry view ── @@ -40,28 +44,92 @@ use crate::vendor::npm_flavor::detect_npm_lock_flavor_in; /// the pnpm <=2-era `shrinkwrap.yaml` (same v5 grammar, older filename) or, /// failing that, the workspace members' own locks. A pnpm workspace with /// `sharedWorkspaceLockfile: false` reads those beside its root lock -/// ([`inventory_pnpm_member_locks_in`], #492). -/// Any remaining probe failure falls back to Rush's common lock when -/// `rush.json` is present. +/// ([`inventory_pnpm_member_locks_in`], #492). The flavor names that +/// routed lock; Rush's locks, which no flavor routes, are read beside it by +/// [`inventory_npm_trees_raw_in`]. #[cfg(test)] pub(crate) async fn inventory_npm_lock( project_root: &Path, ) -> Result)>, UnsupportedNpmLayout> { - inventory_npm_lock_in(&ProjectView::Disk(project_root)).await + let view = ProjectView::Disk(project_root); + let rush = is_rush(view).await; + inventory_npm_lock_raw_in(&view, rush) + .await + .map(|found| found.map(|(flavor, raw)| (flavor, dedup_prefer_integrity(raw)))) +} + +/// Every npm-family tree the inventory reads, collapsed: what +/// [`super::inventory_project`] lists for npm. +#[cfg(test)] +pub(crate) async fn inventory_npm_trees( + project_root: &Path, +) -> Result, UnsupportedNpmLayout> { + inventory_npm_trees_raw_in(&ProjectView::Disk(project_root)) + .await + .map(dedup_prefer_integrity) } -/// [`inventory_npm_lock`] over a [`ProjectView`]. -pub(super) async fn inventory_npm_lock_in( +/// Every guarded npm-family instance, uncollapsed: the routed lock +/// ([`inventory_npm_lock`]) and, whenever `rush.json` is present, Rush's +/// common and subspace locks, separate install trees beside it as hosted +/// mode and lockfile discovery read them (before: only when the root had +/// no lock; lock-model divergence N12). +pub(super) async fn inventory_npm_trees_raw_in( view: &ProjectView<'_>, +) -> Result, UnsupportedNpmLayout> { + let rush = is_rush(*view).await; + let mut out = inventory_npm_lock_raw_in(view, rush) + .await? + .map(|(_, entries)| entries) + .unwrap_or_default(); + if rush { + out.extend(guard_npm(inventory_rush_pnpm_locks_in(view).await)); + } + Ok(out) +} + +/// The routed lock's guarded instances ([`inventory_npm_lock`] before its +/// collapse). With pnpm's `lockfile` setting off ([`lockfile_disabled`]; +/// not under Rush, `rush` being [`is_rush`], the gate discovery shares, which runs +/// pnpm with settings of its own, as in lockfile discovery's +/// `extract_pnpm`), pnpm neither reads nor writes the root or member +/// locks, so a Socket-hosted pin in them is no lockfile basis. Discovery +/// keeps such a ref but withholds its lock pin; here the entry stays +/// listed (what it names is still the dependency set, and its hosted url +/// still shows ledger liveness the wiring) with its integrity dropped, so +/// nothing verifies or fetches against the pin (N14). Registry entries +/// keep theirs: a registry release's integrity is its content verifier +/// whether or not pnpm reads the lock. +async fn inventory_npm_lock_raw_in( + view: &ProjectView<'_>, + rush: bool, ) -> Result)>, UnsupportedNpmLayout> { - inventory_npm_lock_raw_in(view) - .await - .map(|found| found.map(|(flavor, raw)| (flavor, dedup_prefer_integrity(raw)))) + let mut found = inventory_npm_router_lock_raw_in(view).await?; + if let Some((NpmLockFlavor::Pnpm | NpmLockFlavor::PnpmLegacy, entries)) = &mut found { + // The settings are read only for a lock that pins something. + if !rush && entries.iter().any(is_hosted) && lockfile_disabled(view, false).await.is_some() + { + for entry in entries.iter_mut().filter(|e| is_hosted(e)) { + entry.integrity = LockIntegrity::None; + } + } + } + Ok(found) +} + +/// Whether `entry` resolves from Socket's patch server (see +/// [`inventory_npm_lock_raw_in`]). +fn is_hosted(entry: &LockfileEntry) -> bool { + entry + .resolved + .as_deref() + .and_then(|url| hosted_patch_uuid(url, &[])) + .is_some() } -/// [`inventory_npm_lock`] before its collapse: every -/// guarded instance ([`super::inventory_project_every_lock`]). -pub(super) async fn inventory_npm_lock_raw_in( +/// The lock the npm flavor router picks, with its fallbacks (see +/// [`inventory_npm_lock`]). +async fn inventory_npm_router_lock_raw_in( view: &ProjectView<'_>, ) -> Result)>, UnsupportedNpmLayout> { let (flavor, _warnings) = match detect_npm_lock_flavor_in(view).await { @@ -144,13 +212,8 @@ pub(super) async fn inventory_npm_lock_raw_in( return Ok(Some((NpmLockFlavor::Pnpm, guard_npm(members)))); } } - // Rush monorepos have no root package.json/lock pair; their - // single pnpm source-of-truth lives under common/config/rush/. - // The flavor probe (root-relative) can't see it, so every - // refusal that got this far falls back to Rush's locks, which - // read nothing outside a Rush monorepo (no rush.json). - let rush = inventory_rush_pnpm_locks_in(view).await; - return Ok((!rush.is_empty()).then(|| (NpmLockFlavor::Pnpm, guard_npm(rush)))); + // Rush's locks are read by the caller, whatever the probe says. + return Ok(None); } }; let raw = match flavor { diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs b/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs index 2479f21ab..6b5a06653 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/pnpm.rs @@ -121,21 +121,13 @@ fn pnpm_lock_text_inventory(text: &str) -> Option> { /// `common/config/subspaces//pnpm-lock.yaml`. `rush install` copies /// the source lock into common/temp and runs pnpm there. /// -/// The last fallback of [`super::npm_family::inventory_npm_lock_raw_in`]: -/// reached whenever the npm-family flavor probe refuses the project (any -/// refusal but the PnP ones, which propagate) and no earlier fallback (the -/// out-of-family root pnpm lock, a live sibling lock, pnpm <= 2's -/// `shrinkwrap.yaml`, the members' own locks) returned entries — typically -/// no root lock at all, where the probe refuses with -/// `vendor_rush_unsupported` or `vendor_lockfile_missing`. Outside a Rush -/// monorepo ([`is_rush`]) it reads nothing. The locks are the ones lockfile -/// discovery reads ([`rush_lock_rels`]), each read through the view. -/// Missing files/dirs are skipped fail-soft; the caller drops the whole -/// result when it comes back empty. +/// Called (via `npm_family::inventory_npm_trees_raw_in`) only when +/// `rush.json` is present ([`is_rush`]), which that caller checks once +/// through the view, so this reader does not stat it again; the locks are +/// read beside whatever lock the router picks. The locks are the ones +/// lockfile discovery reads ([`rush_lock_rels`]), each read through the +/// view. Missing files/dirs are skipped fail-soft. pub(super) async fn inventory_rush_pnpm_locks_in(view: &ProjectView<'_>) -> Vec { - if !is_rush(*view).await { - return Vec::new(); - } let mut out = Vec::new(); for rel in rush_lock_rels(*view).await { if let Some(entries) = inventory_pnpm_lock_rel_in(view, &rel).await { diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs b/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs index 2d9636f7f..be5994858 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs @@ -375,6 +375,7 @@ pub(super) fn python_lock_inventory(text: &str) -> Option> { out.push(LockfileEntry { ecosystem: "pypi", source_kind: SourceKind::Unspecified, + pnpm_env: false, purl, name, version: version.to_string(), @@ -416,6 +417,7 @@ async fn inventory_poetry_lock(view: &ProjectView<'_>) -> Option) -> Option) -> Option) -> Option .map(|(name, version, purl, _)| LockfileEntry { ecosystem: "pypi", source_kind: SourceKind::Unspecified, + pnpm_env: false, purl, name, version, @@ -722,6 +727,7 @@ async fn inventory_requirements_txt(view: &ProjectView<'_>) -> Option, integrity: LockIntegrity| LockfileEntry { ecosystem: "npm", source_kind: SourceKind::Unspecified, + pnpm_env: false, purl: format!("pkg:npm/{name}@{version}"), name: name.to_string(), version: version.to_string(), diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs index 5038478bb..f02cc40fb 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/tests.rs @@ -741,8 +741,9 @@ packages: ) .await; - let (flavor, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); - assert_eq!(flavor, NpmLockFlavor::Pnpm); + // No lock is routed (Rush's are no flavor's), and both are read. + assert!(inventory_npm_lock(tmp.path()).await.unwrap().is_none()); + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); // Union across the common lock and the subspace lock. assert_eq!(entry(&entries, "left-pad").version, "1.3.0"); assert_eq!(entry(&entries, "only-in-subspace").version, "9.9.9"); @@ -752,20 +753,22 @@ packages: async fn rush_json_without_any_lock_yields_none() { // rush.json but no common/subspace lock at all: nothing to inventory. let tmp = tempfile::tempdir().unwrap(); - write(tmp.path(), "rush.json", r#"{"rushVersion":"5.0.0"}"#).await; + write(tmp.path(), "rush.json", RUSH_JSON).await; assert!(inventory_npm_lock(tmp.path()).await.unwrap().is_none()); + assert!(inventory_npm_trees(tmp.path()).await.unwrap().is_empty()); } +/// Lock-model divergence N12: Rush's locks are their own install trees, +/// read whenever `rush.json` is present — beside a root lock too, as +/// hosted mode and lockfile discovery read them. #[tokio::test] -async fn root_pnpm_lock_wins_over_rush_fallback() { - // A plain pnpm project that also happens to carry a stray rush.json - // must route through the normal root-lock path, never the fallback. +async fn rush_locks_are_read_beside_the_root_lock() { let tmp = tempfile::tempdir().unwrap(); - write(tmp.path(), "rush.json", r#"{"rushVersion":"5.0.0"}"#).await; + write(tmp.path(), "rush.json", RUSH_JSON).await; write(tmp.path(), "pnpm-lock.yaml", PNPM_LOCK).await; write_nested( tmp.path(), - "common/config/rush/pnpm-lock.yaml", + RUSH_COMMON_LOCK, "lockfileVersion: '9.0' packages: @@ -776,15 +779,165 @@ packages: ) .await; - let (flavor, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + // The routed lock keeps its flavor and holds only its own entries + // (Rush's are no flavor's). + let (flavor, routed) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); assert_eq!(flavor, NpmLockFlavor::Pnpm); + assert!(!routed.iter().any(|e| e.name == "only-in-common")); + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); assert!(entries.iter().any(|e| e.name == "left-pad")); + assert!( + entries.iter().any(|e| e.name == "only-in-common"), + "the Rush common lock is read beside the root lock: {entries:?}" + ); + + // Beside another family's lock too. + tokio::fs::remove_file(tmp.path().join("pnpm-lock.yaml")) + .await + .unwrap(); + write(tmp.path(), "package-lock.json", PACKAGE_LOCK).await; + let (flavor, routed) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + assert_eq!(flavor, NpmLockFlavor::PackageLock); + assert!(!routed.iter().any(|e| e.name == "only-in-common")); + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); + assert!( + entries.iter().any(|e| e.name == "only-in-common"), + "{entries:?}" + ); + // The project union lists them too. + assert!(inventory_project(tmp.path()) + .await + .iter() + .any(|e| e.name == "only-in-common")); + + // Without rush.json they are not Rush's. + tokio::fs::remove_file(tmp.path().join("rush.json")) + .await + .unwrap(); + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); assert!( !entries.iter().any(|e| e.name == "only-in-common"), - "the root lock must win; the rush fallback must not run: {entries:?}" + "{entries:?}" ); } +/// The env document's registry instance of a +/// release comes first in the lock and may carry the only verifier (the +/// project's hosted instance loses its pin under `lockfile=false`), but a +/// config dependency never stands in for the project's own copy: the +/// collapsed view keeps the project's hosted instance, and the +/// every-instance view (ledger liveness) keeps both. +#[tokio::test] +async fn env_document_instances_never_displace_project_instances() { + let uuid = "1a2b3c4d-5e6f-4a1b-8c2d-0123456789ab"; + let url = + format!("https://patch.socket.dev/patch/npm/left-pad/1.3.0/tok/{uuid}/left-pad-1.3.0.tgz"); + let lock = format!( + "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n \ + left-pad:\n specifier: 1.3.0\n version: 1.3.0\n\npackages:\n\n \ + left-pad@1.3.0:\n resolution: {{integrity: sha512-UPSTREAM==}}\n\n---\n\ + lockfileVersion: '9.0'\n\npackages:\n\n left-pad@1.3.0:\n \ + resolution: {{integrity: sha512-PATCHED==, tarball: {url}}}\n" + ); + for lockfile_off in [false, true] { + let tmp = tempfile::tempdir().unwrap(); + write(tmp.path(), "pnpm-lock.yaml", &lock).await; + if lockfile_off { + write(tmp.path(), ".npmrc", "lockfile=false\n").await; + } + let (_, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + let left = entry(&entries, "left-pad"); + assert_eq!( + left.resolved.as_deref(), + Some(url.as_str()), + "{lockfile_off}" + ); + assert!(!left.pnpm_env, "{lockfile_off}"); + let every: Vec<_> = inventory_project_every_lock(tmp.path()) + .await + .into_iter() + .filter(|e| e.name == "left-pad") + .map(|e| (e.pnpm_env, e.resolved)) + .collect(); + assert_eq!( + every, + [(true, None), (false, Some(url.clone()))], + "{lockfile_off}" + ); + } +} + +/// Lock-model divergence N14: with pnpm's `lockfile` setting off, pnpm +/// never reads the lock, so a Socket-hosted pin in it is no lockfile +/// basis. Lockfile discovery keeps the ref and withholds its pin; the +/// inventory keeps the entry (and its hosted url) and drops its +/// integrity, so nothing fetches or verifies against it. Registry entries +/// keep theirs. Rush runs pnpm +/// with settings of its own, so its locks are untouched (as in discovery). +#[tokio::test] +async fn pnpm_lockfile_off_withholds_hosted_pins() { + let uuid = "1a2b3c4d-5e6f-4a1b-8c2d-0123456789ab"; + let url = + format!("https://patch.socket.dev/patch/npm/left-pad/1.3.0/tok/{uuid}/left-pad-1.3.0.tgz"); + let lock = format!( + "lockfileVersion: '9.0'\n\npackages:\n\n left-pad@1.3.0:\n \ + resolution: {{integrity: sha512-PATCHED==, tarball: {url}}}\n\n \ + right-pad@2.0.0:\n resolution: {{integrity: sha512-UPSTREAM==}}\n" + ); + let integrity = |entries: &[LockfileEntry], name: &str| entry(entries, name).integrity.clone(); + for (setting, withheld) in [ + (None, false), + (Some((".npmrc", "lockfile=false\n")), true), + (Some(("pnpm-workspace.yaml", "lockfile: false\n")), true), + (Some((".npmrc", "lockfile=true\n")), false), + ] { + let tmp = tempfile::tempdir().unwrap(); + write(tmp.path(), "pnpm-lock.yaml", &lock).await; + if let Some((file, text)) = setting { + write(tmp.path(), file, text).await; + } + let (_, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + let left = entry(&entries, "left-pad"); + assert_eq!(left.resolved.as_deref(), Some(url.as_str()), "{setting:?}"); + assert_eq!( + integrity(&entries, "left-pad"), + if withheld { + LockIntegrity::None + } else { + LockIntegrity::Sri("sha512-PATCHED==".into()) + }, + "{setting:?}" + ); + assert_eq!( + integrity(&entries, "right-pad"), + LockIntegrity::Sri("sha512-UPSTREAM==".into()), + "{setting:?}" + ); + } + + // Rush: the setting is not Rush's, so its lock keeps the pin. + let tmp = tempfile::tempdir().unwrap(); + write(tmp.path(), "rush.json", RUSH_JSON).await; + write(tmp.path(), ".npmrc", "lockfile=false\n").await; + write_nested(tmp.path(), RUSH_COMMON_LOCK, &lock).await; + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); + assert_eq!( + integrity(&entries, "left-pad"), + LockIntegrity::Sri("sha512-PATCHED==".into()) + ); + + // A directory named rush.json is not Rush (the gate discovery shares, + // `is_rush`), so the root lock still loses the pin. + let tmp = tempfile::tempdir().unwrap(); + write_nested(tmp.path(), "rush.json/placeholder", "").await; + write(tmp.path(), ".npmrc", "lockfile=false\n").await; + write(tmp.path(), "pnpm-lock.yaml", &lock).await; + let entries = inventory_npm_trees(tmp.path()).await.unwrap(); + assert_eq!(integrity(&entries, "left-pad"), LockIntegrity::None); + let (_, entries) = inventory_npm_lock(tmp.path()).await.unwrap().unwrap(); + assert_eq!(integrity(&entries, "left-pad"), LockIntegrity::None); +} + /// The inventory twin of discovery's /// `pnpm_rush_without_subspaces_and_unsafe_subspace_names`: a subspace dir /// whose name is not a safe path segment is never read, so the inventory @@ -841,11 +994,9 @@ packages: /// The sorted package names the npm-family inventory lists through `view` /// (empty when it lists none). async fn npm_inventory_names(view: &ProjectView<'_>) -> Vec { - let entries = super::npm_family::inventory_npm_lock_in(view) + let entries = super::npm_family::inventory_npm_trees_raw_in(view) .await - .unwrap() - .map(|(_, entries)| entries) - .unwrap_or_default(); + .unwrap(); let mut names: Vec = entries.into_iter().map(|e| e.name).collect(); names.sort(); names @@ -1524,6 +1675,7 @@ async fn lookup_matches_cargo_and_golang_purls() { LockfileEntry { ecosystem: "cargo", source_kind: SourceKind::Unspecified, + pnpm_env: false, name: "serde".into(), version: "1.0.200".into(), purl: "pkg:cargo/serde@1.0.200".into(), @@ -1533,6 +1685,7 @@ async fn lookup_matches_cargo_and_golang_purls() { LockfileEntry { ecosystem: "golang", source_kind: SourceKind::Unspecified, + pnpm_env: false, name: "github.com/x/y".into(), version: "v1.0.0".into(), purl: "pkg:golang/github.com/x/y@v1.0.0".into(), @@ -3134,7 +3287,8 @@ fn pnpm_resolution_tokens_cover_maps_the_grammar_refuses() { let lock = format!( "lockfileVersion: '9.0'\n\npackages:\n\n x@1.0.0:\n resolution:\n tarball: {url}\n nested:\n a: b\n\n y@1.0.0:\n resolution: {{integrity: sha512-a}}\n resolution: {{tarball: '{url}'}}\n\n z@1.0.0:\n resolution: {{integrity: sha512-z,\n tarball: \"{url}\"}}\n\n ok@1.0.0:\n resolution: {{integrity: sha512-ok}}\n" ); - let packages = crate::formats::pnpm::pnpm_packages(&lock); + let parsed = crate::formats::pnpm::PnpmLock::parse(&lock); + let packages = parsed.packages(); let by_key = |key: &str| packages.iter().find(|p| p.key == key).unwrap(); for key in ["x@1.0.0", "y@1.0.0", "z@1.0.0"] { let package = by_key(key); @@ -3938,6 +4092,7 @@ async fn lookup_matches_by_purl_identity() { let entry = |ecosystem: &'static str, name: &str, version: &str| LockfileEntry { ecosystem, source_kind: SourceKind::Unspecified, + pnpm_env: false, name: name.into(), version: version.into(), purl: format!("pkg:{ecosystem}/{name}@{version}"), diff --git a/crates/socket-patch-core/src/vendor/mod.rs b/crates/socket-patch-core/src/vendor/mod.rs index 3aaa2b11e..d6c9db46e 100644 --- a/crates/socket-patch-core/src/vendor/mod.rs +++ b/crates/socket-patch-core/src/vendor/mod.rs @@ -990,8 +990,8 @@ pub async fn lock_text_refusals( /// deletes; a gate that depended on it would make this under-predict, and /// the wet run still refuses. Yarn's restores rewrite key and checksum /// text, and legacy pnpm 7/8 locks (5.4 / 6.0) are not lock-text gated at -/// all, so neither is predicted here: a CRLF legacy lock, an override -/// conflict or an unsupported legacy entry previews `would_vendor` while +/// all, so neither is predicted here: a mixed line-ending legacy lock, an +/// override conflict or an unsupported legacy entry previews `would_vendor` while /// the wet takeover refuses it (keeping the hosted pin, #963) — a known /// preview gap the contract names. pub async fn pnpm_takeover_lock_text_refusals( @@ -2237,10 +2237,15 @@ mod pnpm_takeover_lock_text_refusal_tests { tmp } + /// Divergences N15: a CRLF hosted lock (an autocrlf checkout) takes over + /// like its LF twin; only a lock mixing CRLF and LF is refused. #[tokio::test] - async fn crlf_hosted_pnpm_lock_is_refused() { + async fn only_a_mixed_line_ending_hosted_pnpm_lock_is_refused() { let tmp = project("pnpm-lock.yaml", &HOSTED_V9.replace('\n', "\r\n")); let refused = pnpm_takeover_lock_text_refusals(tmp.path(), &[(PURL, UUID)]).await; + assert!(refused.is_empty(), "{refused:?}"); + let tmp = project("pnpm-lock.yaml", &HOSTED_V9.replacen('\n', "\r\n", 1)); + let refused = pnpm_takeover_lock_text_refusals(tmp.path(), &[(PURL, UUID)]).await; assert_eq!( refused.get(PURL).map(|(code, _)| *code), Some("vendor_lockfile_crlf_unsupported"), @@ -2267,8 +2272,8 @@ mod pnpm_takeover_lock_text_refusal_tests { } /// A known preview gap (CLI_CONTRACT, #853): a legacy pnpm 7/8 lock - /// (5.4 / 6.0) is not lock-text gated, so even a CRLF one the wet - /// takeover refuses (`vendor_lockfile_crlf_unsupported`) previews + /// (5.4 / 6.0) is not lock-text gated, so even one the wet takeover + /// refuses (a mixed line-ending lock, `vendor_lockfile_crlf_unsupported`) previews /// `would_vendor`. The wet run's refusal keeps the hosted pin (#963). #[tokio::test] async fn legacy_pnpm_project_is_out_of_scope() { diff --git a/crates/socket-patch-core/src/vendor/npm_common.rs b/crates/socket-patch-core/src/vendor/npm_common.rs index 48de7a329..0e477052c 100644 --- a/crates/socket-patch-core/src/vendor/npm_common.rs +++ b/crates/socket-patch-core/src/vendor/npm_common.rs @@ -659,7 +659,7 @@ pub(crate) fn is_safe_npm_name(name: &str) -> bool { /// The artifact path under the uuid dir: `[@scope/]-.tgz`, /// with the scope kept as a real subdirectory. -pub(super) fn tgz_rel_leaf(name: &str, version: &str) -> String { +pub(crate) fn tgz_rel_leaf(name: &str, version: &str) -> String { format!("{}.tgz", pkg_rel_leaf(name, version)) } diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock.rs b/crates/socket-patch-core/src/vendor/pnpm_lock.rs index a02a9bbca..b9bbe5654 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock.rs @@ -69,19 +69,23 @@ use super::npm_common::{ use super::parse_memo::ParseMemo; use super::path::parse_vendor_path; use super::pnpm_build_approvals as build_approvals; +use super::pnpm_lock_legacy::swap_block_sorted; use super::revert::{self, KeepPolicy}; use super::source::PackageSource; use super::state::{PnpmMeta, VendorEntry, WiringAction, WiringRecord}; use super::{RevertOpts, RevertOutcome, VendorOutcome, VendorWarning}; -use crate::constants::npm_family::PNPM_LOCK; +use crate::constants::npm_family::{PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY}; use crate::formats::pnpm::lines::{ indent_of, next_block, parse_key_line, section_bounds, split_lines, split_project_document, unquote_value, yaml_key, yaml_key_like, YamlBlock, }; +use crate::formats::pnpm::plan::{governing_pnpm_lock, match_key, KeyMatch, Target}; +use crate::formats::pnpm::sniff_lock_grammar; use crate::formats::pnpm::workspace; -use crate::formats::pnpm::{check_v9_lock_version as check_lock_version, vendored_npm_uuids}; +use crate::formats::pnpm::{vendored_npm_uuids, PnpmLockGrammar}; const PACKAGE_JSON: &str = "package.json"; +use crate::utils::line_endings::LineEndings; use crate::utils::pnpm_workspace::{governing_workspace_file, PNPM_WORKSPACE}; /// Refusal code for a workspace member whose pnpm settings live in an @@ -173,6 +177,22 @@ impl PnpmDialect { fn allows_revert_file(self, file: &str) -> bool { REVERT_ALLOWLIST.contains(&file) && (self == Self::V9 || file != PNPM_WORKSPACE) } + + /// The grammar of a lock this dialect rewrites: one the vendored target + /// rewrites ([`sniff_lock_grammar`]: 5.4, 6.0 and 9.0, the grammars its + /// rekey was captured against; N16) and this dialect splices. + pub(super) fn check_lock(self, text: &str) -> Result { + let grammar = sniff_lock_grammar(text)?; + match (self, grammar) { + (Self::V9, PnpmLockGrammar::V9) + | (Self::Legacy, PnpmLockGrammar::V54 | PnpmLockGrammar::V60) => Ok(grammar), + _ => Err(format!( + "{PNPM_LOCK} is a {} lock, which the {} planner does not write", + grammar.describe(), + self.flavor() + )), + } + } } pub(super) async fn vendor_pnpm_dialect( @@ -301,7 +321,9 @@ impl NpmLockBackend for PnpmBackend { mut pkg, mut lock, lock_prefix, + lock_eol, ws_text, + ws_eol, }, effective_key, } = plan; @@ -395,15 +417,18 @@ impl NpmLockBackend for PnpmBackend { .render(&pkg) .map_err(|e| format!("cannot serialize {PACKAGE_JSON}: {e}"))?; // An env document ahead of the project lock is carried over verbatim. + // Each file is written back in its own line endings (`read_pnpm_text`). let lock_out = format!("{lock_prefix}{}", lock.lines().join("\n")); + let ws_out = ws_edit.new_text.as_deref().map(|t| ws_eol.restore(t)); + let ws_was = ws_text.as_deref().map(|t| ws_eol.restore(t)); commit_surfaces( project_root, pkg_changed.then_some(new_pkg_bytes.as_slice()), &pkg_bytes, - ws_edit.new_text.as_deref().map(str::as_bytes), - ws_text.as_deref().map(str::as_bytes), + ws_out.as_deref().map(str::as_bytes), + ws_was.as_deref().map(str::as_bytes), ws_edit.created_file, - lock_changed.then_some(lock_out.as_bytes()), + lock_changed.then_some(lock_eol.restore(&lock_out).as_bytes()), ) .await?; if lock_changed { @@ -459,7 +484,11 @@ struct PnpmProject { lock: ProjectLock, /// Everything ahead of the project document, written back verbatim. lock_prefix: String, + /// The line endings the lock and the workspace file are written back + /// with; their texts here are LF ([`read_pnpm_text`]). + lock_eol: LineEndings, ws_text: Option, + ws_eol: LineEndings, } enum ProjectLock { @@ -521,7 +550,8 @@ pub(super) async fn git_branch_lock_refusal( } /// Read the pair, refusing (before any write) a file that is missing, -/// unreadable, not the shape the surgery has fixtures for, or CRLF. +/// unreadable, not the shape the surgery has fixtures for, or that mixes +/// CRLF and LF line endings ([`read_pnpm_text`]). async fn read_project( project_root: &Path, dialect: PnpmDialect, @@ -548,29 +578,32 @@ async fn read_project( ))); } }; - if let Some(refusal) = git_branch_lock_refusal( - &crate::vendor::lock_inventory::ProjectView::Disk(project_root), - ) - .await - { + let view = crate::vendor::lock_inventory::ProjectView::Disk(project_root); + if let Some(refusal) = git_branch_lock_refusal(&view).await { return Err(Box::new(refused(refusal.0, refusal.1))); } - let lock_text = match read_regular_to_string(&project_root.join(PNPM_LOCK)).await { - Ok(text) => text, - Err(e) => { - return Err(Box::new(refused( - "vendor_lockfile_missing", - format!("cannot read {PNPM_LOCK}: {e} — run `pnpm install` first"), - ))); - } - }; - let grammar = match dialect { - PnpmDialect::V9 => { - check_lock_version(&lock_text).map(|()| crate::formats::pnpm::PnpmLockGrammar::V9) - } - PnpmDialect::Legacy => super::pnpm_lock_legacy::check_lock_version(&lock_text), + // pnpm 1 / 2 read only `shrinkwrap.yaml`, whose grammar the rekey has no + // fixture for: there `pnpm-lock.yaml` is stray, and wiring it would + // patch nothing pnpm installs (N11, the choice hosted and discovery make). + // The pin matters only beside a shrinkwrap, so it is read only then. + let shrinkwrap = view.exists_no_follow(PNPM_SHRINKWRAP_LEGACY).await; + let pinned = match shrinkwrap { + true => crate::utils::pnpm_workspace::pinned_pnpm_major_in(&view).await, + false => None, }; - let grammar = match grammar { + if governing_pnpm_lock(true, shrinkwrap, pinned) == Some(PNPM_SHRINKWRAP_LEGACY) { + let why = "the project pins pnpm 1 / 2, which installs from shrinkwrap.yaml and ignores \ + pnpm-lock.yaml; vendored mode does not write that grammar — upgrade pnpm \ + and re-lock, or use `--mode hosted`"; + return Err(Box::new(refused( + "vendor_lockfile_version_unsupported", + why, + ))); + } + let (lock_text, lock_eol) = read_pnpm_text(project_root, PNPM_LOCK) + .await + .map_err(|e| read_refusal(PNPM_LOCK, e, "run `pnpm install` first"))?; + let grammar = match dialect.check_lock(&lock_text) { Ok(grammar) => grammar, Err(detail) => { return Err(Box::new(refused( @@ -579,23 +612,6 @@ async fn read_project( ))); } }; - // CRLF line endings (Windows autocrlf checkouts) break every structural - // probe below — `split_lines` keeps the trailing `\r`, so section headers - // like `packages:` never match — which would otherwise surface as a - // misleading "no packages entry … run `pnpm install`" refusal. pnpm - // itself tolerates CRLF, so name the real cause and fail closed before - // any probe runs. - if lock_text.contains('\r') { - return Err(Box::new(refused( - "vendor_lockfile_crlf_unsupported", - format!( - "{PNPM_LOCK} has CRLF line endings, which this rewriter cannot edit \ - byte-faithfully — normalize the file to LF (re-run `pnpm install`, \ - or add `pnpm-lock.yaml text eol=lf` to .gitattributes and re-checkout) \ - and retry" - ), - ))); - } if dialect == PnpmDialect::Legacy { return Ok(PnpmProject { pkg_bytes, @@ -606,7 +622,9 @@ async fn read_project( grammar, )?), lock_prefix: String::new(), + lock_eol, ws_text: None, + ws_eol: LineEndings::None, }); } let (lock_prefix, project_text) = match split_v9_lock(&lock_text) { @@ -636,21 +654,18 @@ async fn read_project( // encoding, permissions, a FIFO) must refuse — treating it as absent // would route into the create path, which OVERWRITES the user's // workspace definition with the root-only scaffold. - let ws_text: Option = - match read_regular_to_string(&project_root.join(PNPM_WORKSPACE)).await { - Ok(text) => Some(text), - Err(e) if e.kind() == std::io::ErrorKind::NotFound => None, - Err(e) => { - return Err(Box::new(refused( - "vendor_lockfile_missing", - format!( - "cannot read {PNPM_WORKSPACE}: {e} — vendoring mirrors the override \ - into it (pnpm >= 11 reads overrides from pnpm-workspace.yaml); fix \ - the file and retry" - ), - ))); - } - }; + let (ws_text, ws_eol) = match read_pnpm_text(project_root, PNPM_WORKSPACE).await { + Ok((text, eol)) => (Some(text), eol), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => (None, LineEndings::None), + Err(e) => { + return Err(read_refusal( + PNPM_WORKSPACE, + e, + "vendoring mirrors the override into it (pnpm >= 11 reads overrides from \ + pnpm-workspace.yaml); fix the file and retry", + )) + } + }; // A workspace member with its own lock (`sharedWorkspaceLockfile: // false`) and no workspace file of its own: pnpm reads `overrides:` // only from the workspace root's file, so the create path below would @@ -677,25 +692,72 @@ async fn read_project( ))); } } - // Same CRLF posture as the lock above: the workspace splices match exact - // LF lines, so a CRLF file dodges the conflict checks and the edit - // appends a DUPLICATE `overrides:` section — a duplicated mapping key - // pnpm refuses to parse. Fail closed naming the real cause. - if ws_text.as_deref().is_some_and(|t| t.contains('\r')) { - return Err(Box::new(refused( - "vendor_lockfile_crlf_unsupported", - format!( - "{PNPM_WORKSPACE} has CRLF line endings, which this rewriter cannot edit \ - byte-faithfully — normalize the file to LF and retry" - ), - ))); - } Ok(PnpmProject { pkg_bytes, pkg, lock: ProjectLock::V9(lines), lock_prefix, + lock_eol, ws_text, + ws_eol, + }) +} + +/// Read `file` under `root` as every vendored pnpm reader does (vendoring, +/// its pre-flight and the revert): its text with LF line endings, and the +/// endings it is written back with. A CRLF file (a Windows autocrlf +/// checkout) is edited like its LF twin and written back CRLF, as hosted +/// mode's byte splices keep it (divergences N15). A file that mixes endings +/// is a [`MixedLineEndings`] error ([`is_mixed_line_endings`]): no one +/// ending writes it back byte-faithfully, so vendoring refuses it and the +/// revert fails before any write. +async fn read_pnpm_text(root: &Path, file: &str) -> std::io::Result<(String, LineEndings)> { + let text = read_regular_to_string(&root.join(file)).await?; + match LineEndings::of(&text) { + LineEndings::Mixed => Err(std::io::Error::new( + std::io::ErrorKind::InvalidData, + MixedLineEndings(format!( + "{file} mixes CRLF and LF line endings, which this rewriter cannot write back \ + byte-faithfully — normalize the file to one line ending and retry" + )), + )), + LineEndings::Crlf => Ok(( + crate::utils::line_endings::to_lf(&text).into_owned(), + LineEndings::Crlf, + )), + eol => Ok((text, eol)), + } +} + +/// [`read_pnpm_text`]'s error for a file that mixes line endings, told +/// apart from an OS read error by its type, not its `io::ErrorKind`. +#[derive(Debug)] +struct MixedLineEndings(String); + +impl std::fmt::Display for MixedLineEndings { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.0) + } +} + +impl std::error::Error for MixedLineEndings {} + +/// Whether `e` is [`read_pnpm_text`]'s mixed-line-ending refusal. +fn is_mixed_line_endings(e: &std::io::Error) -> bool { + e.get_ref() + .is_some_and(|inner| inner.is::()) +} + +/// The vendoring refusal for a [`read_pnpm_text`] error other than a +/// missing file: `vendor_lockfile_crlf_unsupported` for mixed endings, else +/// `vendor_lockfile_missing` with `remedy`. +fn read_refusal(file: &str, e: std::io::Error, remedy: &str) -> Box { + Box::new(match is_mixed_line_endings(&e) { + true => refused("vendor_lockfile_crlf_unsupported", e.to_string()), + false => refused( + "vendor_lockfile_missing", + format!("cannot read {file}: {e} — {remedy}"), + ), }) } @@ -706,7 +768,7 @@ fn split_v9_lock(text: &str) -> Result<(&str, &str), String> { let (prefix, project) = split_project_document(text).map_err(|why| format!("{PNPM_LOCK} {why}"))?; if !prefix.is_empty() { - check_lock_version(project).map_err(|e| { + PnpmDialect::V9.check_lock(project).map_err(|e| { format!("the project document of {PNPM_LOCK} fails its version check: {e}") })?; } @@ -770,7 +832,7 @@ fn preflight_package( { return Err(Box::new(refused("vendor_override_conflict", detail))); } - if !lock_has_target_package_in(lines, name, version) { + if !lock_has_target_package(lines, name, version) { return Err(Box::new(refused( "vendor_lock_entry_not_found", format!( @@ -831,18 +893,29 @@ pub(super) async fn preflight_packages_dialect( /// `None`: cannot determine (missing/unreadable/unsupported lock) — /// callers must keep the entry, fail-safe. pub async fn pnpm_entry_in_use(entry: &VendorEntry, project_root: &Path) -> Option { - let text = read_regular_to_string(&project_root.join(PNPM_LOCK)) - .await - .ok()?; - if check_lock_version(&text).is_err() { - return None; - } - // Every `packages:`/`snapshots:` block key resolving into - // `.socket/vendor/npm//` — the format model's one walk - // ([`vendored_npm_uuids`], CRLF read like LF), collected once per lock - // bytes: a revert pass probes once per ledger entry. - let vendored = IN_USE_MEMO.parse_infallible(text.as_bytes(), || vendored_npm_uuids(&text)); - Some(vendored.contains(&entry.uuid)) + PnpmDialect::V9.entry_in_use(entry, project_root).await +} + +impl PnpmDialect { + /// Whether `entry`'s artifact is still consumed by a lock this dialect + /// reads: `None` when the lock is missing, unreadable or another + /// dialect's (callers keep the entry, fail-safe). + pub(super) async fn entry_in_use( + self, + entry: &VendorEntry, + project_root: &Path, + ) -> Option { + let text = read_regular_to_string(&project_root.join(PNPM_LOCK)) + .await + .ok()?; + self.check_lock(&text).ok()?; + // Every `packages:`/`snapshots:` block key resolving into + // `.socket/vendor/npm//` — the format model's one walk + // ([`vendored_npm_uuids`], CRLF read like LF), collected once per + // lock bytes: a revert pass probes once per ledger entry. + let vendored = IN_USE_MEMO.parse_infallible(text.as_bytes(), || vendored_npm_uuids(&text)); + Some(vendored.contains(&entry.uuid)) + } } /// FAIL-CLOSED revert guard for a ledger entry with NO wiring records, @@ -952,30 +1025,58 @@ pub(super) async fn revert_pnpm_dialect( return blocked; } } - // A v9 lock whose documents are not pnpm's layout cannot be reverted - // without guessing which document holds the wiring: fail before any - // surface is touched (the dry run too, so the preview never advertises - // a revert the wet run refuses) instead of reverting package.json and - // the workspace file around a lock left wired to the artifact. - let mut lock_text: Option> = None; - if dialect == PnpmDialect::V9 - && entry + // Read the lock and the workspace file the records touch up front, as + // vendoring reads them ([`read_pnpm_text`]), and fail before any surface + // is touched (the dry run too, so the preview never advertises a revert + // the wet run refuses) when one mixes CRLF and LF, or when a v9 lock's + // documents are not pnpm's layout: neither can be reverted without + // guessing, and reverting package.json and the workspace file around a + // lock left wired to the artifact would desync them. + let touches = |kind: fn(&str) -> bool| { + entry .wiring .iter() - .any(|r| r.file != PACKAGE_JSON && dialect.allows_revert_file(&r.file)) + .any(|r| dialect.allows_revert_file(&r.file) && kind(&r.file)) + }; + let read = |file| read_pnpm_text(project_root, file); + let lock_read = match touches(|f| f != PACKAGE_JSON) { + true => Some(read(PNPM_LOCK).await), + false => None, + }; + let ws_read = match touches(|f| f == PNPM_WORKSPACE) { + true => Some(read(PNPM_WORKSPACE).await), + false => None, + }; + for e in [&lock_read, &ws_read] + .into_iter() + .flatten() + .filter_map(|r| r.as_ref().err()) { - let read = read_regular_to_string(&project_root.join(PNPM_LOCK)).await; - if let Ok(text) = &read { - if let Err(why) = split_v9_lock(text) { - return RevertOutcome::failed(format!( - "{why}, so the vendored wiring of {} cannot be located; nothing was \ - reverted — restore the vendored {PNPM_LOCK} (or re-lock it with pnpm) \ - and re-run the revert", - entry.base_purl - )); - } + if is_mixed_line_endings(e) { + return RevertOutcome::failed(format!("{e}; nothing was reverted")); } - lock_text = Some(read); + } + let mut lock_prefix = String::new(); + let mut lock_lines: Option> = None; + if let Some(Ok((text, _))) = &lock_read { + // The wiring lives in the project document; an env document ahead + // of it is written back verbatim. + let (prefix, project) = match dialect { + PnpmDialect::V9 => match split_v9_lock(text) { + Ok(split) => split, + Err(why) => { + return RevertOutcome::failed(format!( + "{why}, so the vendored wiring of {} cannot be located; nothing was \ + reverted — restore the vendored {PNPM_LOCK} (or re-lock it with \ + pnpm) and re-run the revert", + entry.base_purl + )) + } + }, + PnpmDialect::Legacy => ("", text.as_str()), + }; + lock_prefix = prefix.to_string(); + lock_lines = Some(split_lines(project)); } if dry_run { return RevertOutcome::ok(); @@ -985,7 +1086,6 @@ pub(super) async fn revert_pnpm_dialect( // Partition by file through the allowlist (fail-closed skip+warning on // anything else — see REVERT_ALLOWLIST's security note). let mut touches_pkg = false; - let mut touches_lock = false; for rec in &entry.wiring { if !dialect.allows_revert_file(&rec.file) { outcome.warnings.push(VendorWarning::new( @@ -997,45 +1097,23 @@ pub(super) async fn revert_pnpm_dialect( )); continue; } - if rec.file == PACKAGE_JSON { - touches_pkg = true; - } else { - touches_lock = true; - } + touches_pkg |= rec.file == PACKAGE_JSON; } - // Load both surfaces up front (fail-closed on unparseable; a missing - // file degrades to a warning and the artifact removal still proceeds). - let mut lock_lines: Option> = None; - let mut lock_prefix = String::new(); - if touches_lock { - let read = match lock_text { - Some(read) => read, - None => read_regular_to_string(&project_root.join(PNPM_LOCK)).await, - }; - match read { - // The wiring lives in the project document; an env document - // ahead of it is written back verbatim. - Ok(text) => match dialect { - PnpmDialect::V9 => { - let (prefix, project) = match split_v9_lock(&text) { - Ok(split) => split, - Err(why) => return RevertOutcome::failed(why), - }; - lock_prefix = prefix.to_string(); - lock_lines = Some(split_lines(project)); - } - PnpmDialect::Legacy => lock_lines = Some(split_lines(&text)), - }, - Err(e) if e.kind() == std::io::ErrorKind::NotFound => { - outcome.warnings.push(VendorWarning::new( - "vendor_lockfile_missing", - format!("{PNPM_LOCK} is missing; lock fragments cannot be restored"), - )); - } - Err(e) => return RevertOutcome::failed(format!("cannot read {PNPM_LOCK}: {e}")), + // A missing lock degrades to a warning and the artifact removal still + // proceeds; an unreadable one fails. + let lock_eol = match lock_read { + Some(Ok((_, eol))) => eol, + Some(Err(e)) if e.kind() == std::io::ErrorKind::NotFound => { + outcome.warnings.push(VendorWarning::new( + "vendor_lockfile_missing", + format!("{PNPM_LOCK} is missing; lock fragments cannot be restored"), + )); + LineEndings::None } - } + Some(Err(e)) => return RevertOutcome::failed(format!("cannot read {PNPM_LOCK}: {e}")), + None => LineEndings::None, + }; let mut pkg_state: Option<(Value, JsonLayout)> = None; if touches_pkg { match read_regular_to_bytes(&project_root.join(PACKAGE_JSON)).await { @@ -1126,7 +1204,9 @@ pub(super) async fn revert_pnpm_dialect( if let Some(lines) = &lock_lines { if let Err(e) = atomic_write_bytes_preserving_mode( &project_root.join(PNPM_LOCK), - format!("{lock_prefix}{}", lines.join("\n")).as_bytes(), + lock_eol + .restore(&format!("{lock_prefix}{}", lines.join("\n"))) + .as_bytes(), ) .await { @@ -1164,9 +1244,10 @@ pub(super) async fn revert_pnpm_dialect( && build_approvals::is_ws_kind(&r.kind) }) .collect(); - if ws_rec.is_some() || !ws_approvals.is_empty() { + if let Some(read) = ws_read.filter(|_| ws_rec.is_some() || !ws_approvals.is_empty()) { if let Err(e) = revert_workspace( project_root, + read, ws_rec, &ws_approvals, entry, @@ -1195,9 +1276,11 @@ pub(super) async fn revert_pnpm_dialect( /// out of a file we edited (restoring the taken-over value, and dropping an /// `overrides:` section we created once it empties). Drift ⇒ warning, left /// alone. Build approvals vendoring added (`approvals`) are taken out too. -/// `Err` is a genuine write failure. +/// `read` is the file as [`read_pnpm_text`] read it (LF, with the endings +/// it is written back in). `Err` is a genuine read or write failure. async fn revert_workspace( project_root: &Path, + read: std::io::Result<(String, LineEndings)>, rec: Option<&WiringRecord>, approvals: &[&WiringRecord], entry: &VendorEntry, @@ -1213,8 +1296,8 @@ async fn revert_workspace( }; let entry_uuid = entry.uuid.as_str(); let path = project_root.join(PNPM_WORKSPACE); - let text = match read_regular_to_string(&path).await { - Ok(t) => t, + let (text, eol) = match read { + Ok(read) => read, Err(e) if e.kind() == std::io::ErrorKind::NotFound => { // ALREADY CONVERGED: for an Added override (no recorded // original) the reverted state is "no override" — a missing @@ -1231,18 +1314,6 @@ async fn revert_workspace( } Err(e) => return Err(format!("cannot read {PNPM_WORKSPACE}: {e}")), }; - // CRLF blindness fails CLOSED: the splices below match exact LF lines, - // so a CRLF-converted file would read an Added override as "already - // converged" and let the artifact it still points at be removed. Drift, - // not convergence — the keep gate holds the artifact until the user - // normalizes the file. - if text.contains('\r') { - warnings.push(drifted(format!( - "{PNPM_WORKSPACE} has CRLF line endings the revert splices cannot edit; \ - left alone — normalize the file to LF and re-run `vendor --revert`" - ))); - return Ok(()); - } // Fast path: a file we created that is still byte-identical to the // scaffold we wrote → delete it (byte-restore to "no file"). @@ -1278,7 +1349,7 @@ async fn revert_workspace( } dirty |= build_approvals::revert_ws(&mut lines, approvals); if dirty { - atomic_write_bytes_preserving_mode(&path, lines.join("\n").as_bytes()) + atomic_write_bytes_preserving_mode(&path, eol.restore(&lines.join("\n")).as_bytes()) .await .map_err(|e| format!("cannot write {PNPM_WORKSPACE}: {e}"))?; } @@ -1405,14 +1476,6 @@ impl EditCtx<'_> { vendor_value_is_for(value, self.name, self.version) } - /// Is `key` our rekeyed `name@file:` packages/snapshots key for - /// THIS name@version (any uuid)? - fn is_ours_key(&self, key: &str) -> bool { - key.strip_prefix(self.name) - .and_then(|rest| rest.strip_prefix("@file:")) - .is_some_and(|rest| self.is_ours(rest)) - } - /// The per-importer `specifier:` spelling: re-relativized for nested /// importers, root-relative for `.` (spike P7). fn spec_for_importer(&self, importer: &str) -> String { @@ -1678,56 +1741,26 @@ pub(super) fn check_lock_override( Ok(()) } -/// Pre-flight: does the lock have a packages entry vendoring can target — -/// the registry `name@version` key, or our own rekeyed `name@file:` key -/// (the in-sync / stale-uuid re-run)? The ours-probe binds to THIS -/// version's leaf — a sibling version's vendored entry is not a target -/// (see `vendor_value_is_for`). -fn lock_has_target_package(lines: &[String], name: &str, version: &str) -> bool { - let Some((start, end)) = section_bounds(lines, "packages") else { - return false; - }; - let reg_key = format!("{name}@{version}"); - let ours_prefix = format!("{name}@file:"); - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - if block.key == reg_key { - return true; - } - if let Some(rest) = block.key.strip_prefix(&ours_prefix) { - if vendor_value_is_for(rest, name, version) { - return true; - } - } - i = block.end; +/// Whether a packages/snapshots `key` is one the vendored edit targets for +/// `name@version` ([`match_key`]): `Some(false)` for the registry entry the +/// rekey repoints ([`Target::repoints`]: no peer suffix), `Some(true)` for a +/// rekey of this release already made (the in-sync / stale-uuid re-run), +/// `None` for every other key. The rekey probe binds to THIS version's leaf +/// and to this project's `.socket/vendor/`. +pub(super) fn target_key(key: &str, name: &str, version: &str) -> Option { + match match_key(key, name, version)? { + KeyMatch::Registry { suffix } => Target::Vendored.repoints(suffix).then_some(false), + KeyMatch::Rekeyed => Some(true), } - false } -/// [`lock_has_target_package`] over the backend's lock lines: answered from -/// the section index while the lines are the memoized split (the -/// candidate blocks are exactly those whose key is the registry key or -/// starts with `name@file:`, in lock order), else by the scan. -fn lock_has_target_package_in(lines: &LockLines, name: &str, version: &str) -> bool { - let Some(index) = lines.index() else { - return lock_has_target_package(lines, name, version); - }; - if index.packages.bounds.is_none() { - return false; - } - let reg_key = format!("{name}@{version}"); - let ours_prefix = format!("{name}@file:"); - index - .packages - .candidates(®_key, name) - .into_iter() - .any(|block| { - block.key == reg_key - || block - .key - .strip_prefix(&ours_prefix) - .is_some_and(|rest| vendor_value_is_for(rest, name, version)) - }) +/// Pre-flight: does the lock have a packages entry vendoring can target +/// ([`target_key`])? From the section index while the lines are the +/// memoized split, else by the scan ([`matching_blocks`]). +pub(super) fn lock_has_target_package(lines: &L, name: &str, version: &str) -> bool { + lines.bounds("packages").is_some_and(|(start, end)| { + !matching_blocks(lines, "packages", start, end, name, version).is_empty() + }) } /// Pre-flight fail-closed guard against reference forms the surgery does @@ -1821,60 +1854,39 @@ fn check_rewritable_refs_with( i = catalog.end; } } - if let Some(index) = index { - for candidate in index.snapshot_ref_candidates(®_key, name, version) { - match candidate { - SnapshotCandidate::Key(block) => { - let key = &index.snapshots.blocks[block].key; - if key.starts_with(&key_peer_prefix) { - return refuse("a peer-suffixed snapshot key", key); - } - } - SnapshotCandidate::Body(line) => { - let Some((dep, _repr, rest)) = parse_key_line(&lines[line], 6) else { - continue; - }; - let rest = unquote_value(rest); - if rest == reg_key || rest.starts_with(&key_peer_prefix) { - return refuse("an aliased snapshot reference", rest); - } - if dep == name && rest.starts_with(&val_peer_prefix) { - return refuse("a peer-suffixed snapshot reference", rest); - } - } - } - } - for entry in index.importer_ref_candidates(®_key, name, version) { - let dep = entry.dep.as_str(); - if let Some(v) = entry.ver.as_deref().map(unquote_value) { - if v == reg_key || v.starts_with(&key_peer_prefix) { - return refuse("an aliased importer version", v); - } - if dep == name && v.starts_with(&val_peer_prefix) { - return refuse("a peer-suffixed importer version", v); - } - if dep == name && v == version { - if let Some(s) = entry.spec.as_deref() { - if unquote_value(s).starts_with("catalog:") { - return refuse_catalog( - &format!("importer `{}` specifier", entry.importer), - s, - ); - } - } + // The positions the checks below read: from the index, the earliest + // position each refusal can hold at; by the scan, every one, read + // lazily so the scan stops at the first refusal. + type Refs<'a, T> = Box + 'a>; + let (snapshot_refs, importer_refs): (Refs, Refs) = match index { + Some(index) => ( + Box::new( + index + .snapshot_ref_candidates(®_key, name, version) + .into_iter(), + ), + Box::new( + index + .importer_ref_candidates(®_key, name, version) + .into_iter() + .map(ImporterDep::as_ref), + ), + ), + None => ( + Box::new(snapshot_positions(lines)), + Box::new(importer_refs(lines)), + ), + }; + for candidate in snapshot_refs { + match candidate { + SnapshotCandidate::Key(header) => { + let key = parse_key_line(&lines[header], 2).map_or("", |(key, ..)| key); + if key.starts_with(&key_peer_prefix) { + return refuse("a peer-suffixed snapshot key", key); } } - } - return Ok(()); - } - if let Some((start, end)) = section_bounds(lines, "snapshots") { - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - if block.key.starts_with(&key_peer_prefix) { - return refuse("a peer-suffixed snapshot key", &block.key); - } - for line in &lines[block.header + 1..block.end] { - let Some((dep, _repr, rest)) = parse_key_line(line, 6) else { + SnapshotCandidate::Body(line) => { + let Some((dep, _repr, rest)) = parse_key_line(&lines[line], 6) else { continue; }; let rest = unquote_value(rest); @@ -1885,52 +1897,74 @@ fn check_rewritable_refs_with( return refuse("a peer-suffixed snapshot reference", rest); } } - i = block.end; } } - if let Some((start, end)) = section_bounds(lines, "importers") { - let mut i = start + 1; - while let Some(importer) = next_block(lines, i, end) { - let mut k = importer.header + 1; - while k < importer.end { - let Some((dep, _repr, rest)) = parse_key_line(&lines[k], 6) else { - k += 1; - continue; - }; - if !rest.is_empty() { - k += 1; - continue; - } - let (spec, ver, f) = dep_field_lines(lines, k + 1, importer.end, 8); - if let Some((_, v)) = &ver { - let v = unquote_value(v); - if v == reg_key || v.starts_with(&key_peer_prefix) { - return refuse("an aliased importer version", v); - } - if dep == name && v.starts_with(&val_peer_prefix) { - return refuse("a peer-suffixed importer version", v); - } - if dep == name && v == version { - if let Some((_, s)) = spec { - // pnpm spells the default catalog `'catalog:'` - // (quoted) and named ones `catalog:` (bare). - if unquote_value(&s).starts_with("catalog:") { - return refuse_catalog( - &format!("importer `{}` specifier", importer.key), - &s, - ); - } - } - } - } - k = f; + for entry in importer_refs { + let Some(v) = entry.ver.map(unquote_value) else { + continue; + }; + if v == reg_key || v.starts_with(&key_peer_prefix) { + return refuse("an aliased importer version", v); + } + if entry.dep == name && v.starts_with(&val_peer_prefix) { + return refuse("a peer-suffixed importer version", v); + } + // pnpm spells the default catalog `'catalog:'` (quoted) and named + // ones `catalog:` (bare). + if let Some(s) = entry.spec.filter(|_| entry.dep == name && v == version) { + if unquote_value(s).starts_with("catalog:") { + return refuse_catalog(&format!("importer `{}` specifier", entry.importer), s); } - i = importer.end; } } Ok(()) } +/// Every snapshot block header and body line, in lock order: the scan's +/// [`SnapshotCandidate`]s. +fn snapshot_positions(lines: &[String]) -> impl Iterator + '_ { + let blocks = section_bounds(lines, "snapshots").into_iter(); + blocks + .flat_map(move |(start, end)| { + std::iter::successors(next_block(lines, start + 1, end), move |b| { + next_block(lines, b.end, end) + }) + }) + .flat_map(|b| { + std::iter::once(SnapshotCandidate::Key(b.header)) + .chain((b.header + 1..b.end).map(SnapshotCandidate::Body)) + }) +} + +/// One importer dep entry as the ref gate reads it ([`ImporterDep`], +/// borrowed). +struct ImporterRef<'a> { + importer: &'a str, + dep: &'a str, + spec: Option<&'a str>, + ver: Option<&'a str>, +} + +/// Every importer's [`DepEntry`]s, in lock order, read lazily. +fn importer_refs(lines: &[String]) -> impl Iterator> { + let blocks = section_bounds(lines, "importers").into_iter(); + blocks + .flat_map(move |(start, end)| { + std::iter::successors(next_block(lines, start + 1, end), move |b| { + next_block(lines, b.end, end) + }) + }) + .flat_map(move |importer| { + let key = parse_key_line(&lines[importer.header], 2).map_or("", |(key, ..)| key); + dep_entries(lines, importer.header + 1, importer.end, 6).map(move |e| ImporterRef { + importer: key, + dep: e.dep, + spec: e.spec.map(|(_, s)| s), + ver: e.ver.map(|(_, v)| v), + }) + }) +} + // ───────────────────────── package.json override ───────────────────────── /// Add/refresh `pnpm.overrides[@] = file:` on the @@ -2390,30 +2424,60 @@ pub(super) fn overrides_record( } } -/// Locate a dep entry's `specifier:`/`version:` field lines at `indent` -/// (4 for legacy root deps, 8 for modern importers). Returns their values plus -/// the index of the first non-field line. -#[allow(clippy::type_complexity)] -pub(super) fn dep_field_lines( +/// One dependency entry with nested `specifier:` / `version:` fields (each +/// with its line): a v9 importer dep (key at 6, fields at 8) or a legacy +/// 6.0 root dep (2 / 4). Borrowed from the lines. +pub(super) struct DepEntry<'l> { + pub(super) dep: &'l str, + pub(super) spec: Option<(usize, &'l str)>, + pub(super) ver: Option<(usize, &'l str)>, +} + +impl DepEntry<'_> { + /// `(specifier line, specifier, version line, version)`, owned so the + /// caller can rewrite the lines; `None` unless both fields are present. + pub(super) fn pair(&self) -> Option<(usize, String, usize, String)> { + let ((si, spec), (vi, ver)) = (self.spec?, self.ver?); + Some((si, spec.to_string(), vi, ver.to_string())) + } +} + +/// Every [`DepEntry`] keyed at `indent` in `[start, end)`, in order, read +/// lazily. A key with an inline value is not one; unknown fields are +/// skipped. +pub(super) fn dep_entries( lines: &[String], - mut f: usize, + start: usize, end: usize, indent: usize, -) -> (Option<(usize, String)>, Option<(usize, String)>, usize) { - let mut spec = None; - let mut ver = None; - while f < end { - let Some((field, _repr, fval)) = parse_key_line(&lines[f], indent) else { - break; - }; - match field { - "specifier" => spec = Some((f, fval.to_string())), - "version" => ver = Some((f, fval.to_string())), - _ => {} +) -> impl Iterator> { + let mut k = start; + std::iter::from_fn(move || { + while k < end { + k += 1; + let Some((dep, _repr, "")) = parse_key_line(&lines[k - 1], indent) else { + continue; + }; + let mut entry = DepEntry { + dep, + spec: None, + ver: None, + }; + while let Some((field, _repr, value)) = lines[..end] + .get(k) + .and_then(|l| parse_key_line(l, indent + 2)) + { + match field { + "specifier" => entry.spec = Some((k, value)), + "version" => entry.ver = Some((k, value)), + _ => {} + } + k += 1; + } + return Some(entry); } - f += 1; - } - (spec, ver, f) + None + }) } /// Edit 2: every importer's dep entry for the exact `name@version` — @@ -2430,92 +2494,41 @@ fn edit_importers( let mut changed = false; let mut i = start + 1; while let Some(importer) = next_block(lines.read(), i, end) { - let importer_key = importer.key.clone(); + let importer_key = importer.key; // Dep entries sit at 6-space indent under the 4-space dep-type // headers; their fields at 8. - let mut k = importer.header + 1; - while k < importer.end { - let Some((dep, _repr, rest)) = parse_key_line(&lines.read()[k], 6) else { - k += 1; - continue; - }; - if dep != ctx.name || !rest.is_empty() { - k += 1; + let pairs: Vec<_> = dep_entries(lines.read(), importer.header + 1, importer.end, 6) + .filter(|entry| entry.dep == ctx.name) + .filter_map(|entry| entry.pair()) + .collect(); + for (si, old_spec, vi, old_ver) in pairs { + let target = old_ver == ctx.version || (old_ver != ctx.spec && ctx.is_ours(&old_ver)); + if !target { continue; } - let (spec_idx, ver_idx, f) = dep_field_lines(lines.read(), k + 1, importer.end, 8); - if let (Some((si, old_spec)), Some((vi, old_ver))) = (spec_idx, ver_idx) { - let target = - old_ver == ctx.version || (old_ver != ctx.spec && ctx.is_ours(&old_ver)); - if target { - let was_ours = ctx.is_ours(&old_ver); - let importer_spec = ctx.spec_for_importer(&importer_key); - // Built before the splices below: `dep` borrows `lines[k]`. - let record = WiringRecord { - file: PNPM_LOCK.to_string(), - kind: KIND_LOCK_IMPORTER_DEP.to_string(), - action: WiringAction::Rewritten, - key: Some(format!("{importer_key}|{dep}")), - original: if was_ours { - None - } else { - Some(serde_json::json!({ - "specifier": old_spec, - "version": old_ver, - })) - }, - new: Some(serde_json::json!({ - "specifier": importer_spec, - "version": ctx.spec, - })), - }; - let lines = lines.write(); - lines[si] = format!(" specifier: {importer_spec}"); - lines[vi] = format!(" version: {}", ctx.spec); - wiring.push(record); - changed = true; - } - } - k = f; + let importer_spec = ctx.spec_for_importer(&importer_key); + wiring.push(WiringRecord { + file: PNPM_LOCK.to_string(), + kind: KIND_LOCK_IMPORTER_DEP.to_string(), + action: WiringAction::Rewritten, + key: Some(format!("{importer_key}|{}", ctx.name)), + original: (!ctx.is_ours(&old_ver)) + .then(|| serde_json::json!({ "specifier": old_spec, "version": old_ver })), + new: Some(serde_json::json!({ + "specifier": importer_spec, + "version": ctx.spec, + })), + }); + let lines = lines.write(); + lines[si] = format!(" specifier: {importer_spec}"); + lines[vi] = format!(" version: {}", ctx.spec); + changed = true; } i = importer.end; } Ok(changed) } -/// Fail closed on a half-drifted lock: when a `[start, end)` section -/// carries BOTH the registry-keyed entry and a socket file:-keyed entry -/// for this package, a rekey would splice a DUPLICATE mapping key (pnpm -/// refuses to parse those) and surgery cannot decide which block carries -/// the truth. -/// -/// `matches` is the section's [`matching_blocks`]: every other block is -/// neither, so it cannot change the answer. -fn check_no_split_entry( - matches: &[YamlBlock], - ctx: &EditCtx<'_>, - section: &str, -) -> Result<(), String> { - let reg_key = ctx.reg_key(); - let mut has_registry = false; - let mut has_ours = false; - for block in matches { - if block.key == reg_key { - has_registry = true; - } else if ctx.is_ours_key(&block.key) { - has_ours = true; - } - } - if has_registry && has_ours { - return Err(format!( - "{section} section carries BOTH `{reg_key}` and a `{}@file:…` entry (a \ - half-edited lock); run `pnpm install` to re-resolve it, then re-vendor", - ctx.name - )); - } - Ok(()) -} - /// Edit 3: rekey the `packages:` entry and rewrite its body — /// `resolution: {integrity: , tarball: }`, a `version:` line /// inserted after it, `deprecated:` dropped, everything else verbatim. @@ -2524,16 +2537,14 @@ fn edit_packages( ctx: &EditCtx<'_>, wiring: &mut Vec, ) -> Result { - let (start, end) = lines.bounds("packages").ok_or("no packages: section")?; + let bounds = lines.bounds("packages").ok_or("no packages: section")?; let reg_key = ctx.reg_key(); let new_key = ctx.new_key(); - let matches = matching_blocks(lines, "packages", start, end, ctx); - check_no_split_entry(&matches, ctx, "packages")?; - // The first registry-or-ours block is the one rewritten (every path // below returns). - if let Some(block) = matches.into_iter().next() { - let is_ours_key = ctx.is_ours_key(&block.key); + if let Some((block, is_ours_key)) = + target_block(lines, "packages", bounds, ctx.name, ctx.version)? + { let original_lines: Vec = lines.read()[block.header..block.end].to_vec(); let expected_resolution = format!( " resolution: {{integrity: {}, tarball: {}}}", @@ -2605,17 +2616,15 @@ fn edit_snapshot_rekey( ctx: &EditCtx<'_>, wiring: &mut Vec, ) -> Result { - let Some((start, end)) = lines.bounds("snapshots") else { + let Some(bounds) = lines.bounds("snapshots") else { return Ok(false); // a lock without snapshots has nothing to rekey }; let reg_key = ctx.reg_key(); let new_key = ctx.new_key(); - let matches = matching_blocks(lines, "snapshots", start, end, ctx); - check_no_split_entry(&matches, ctx, "snapshots")?; - // The first registry-or-ours block decides (every path below returns). - if let Some(block) = matches.into_iter().next() { - let is_ours_key = ctx.is_ours_key(&block.key); + if let Some((block, is_ours_key)) = + target_block(lines, "snapshots", bounds, ctx.name, ctx.version)? + { if block.key == new_key { return Ok(false); // in sync } @@ -2725,39 +2734,62 @@ fn edit_snapshot_refs( Ok(changed) } -/// The blocks of section `name` (`[start, end)`, its current bounds) whose -/// key is the registry key `name@version` or our `name@file:` key for this -/// version, in lock order — the only blocks the packages/snapshots edits +/// The blocks of `section` (`[start, end)`, its current bounds) whose key is +/// a [`target_key`] of `name@version`, each with that classification (`true`: +/// a rekey), in lock order — the only blocks the packages/snapshots edits /// act on. From the index while the lines are the memoized split (its /// candidates are a superset, filtered by the same predicate), else by the /// scan. fn matching_blocks( lines: &L, - name: &str, + section: &str, start: usize, end: usize, - ctx: &EditCtx<'_>, -) -> Vec { - let reg_key = ctx.reg_key(); - let keep = |key: &str| key == reg_key || ctx.is_ours_key(key); - if let Some(section) = lines.index().and_then(|index| index.section(name)) { + name: &str, + version: &str, +) -> Vec<(YamlBlock, bool)> { + let classify = |block: YamlBlock| Some((target_key(&block.key, name, version)?, block)); + if let Some(section) = lines.index().and_then(|index| index.section(section)) { return section - .candidates(®_key, ctx.name) + .candidates(name, version) .into_iter() - .filter(|block| keep(&block.key)) - .map(IndexedBlock::to_block) + .filter_map(|block| classify(block.to_block())) + .map(|(rekeyed, block)| (block, rekeyed)) .collect(); } let lines = lines.read(); - let mut found = Vec::new(); - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - i = block.end; - if keep(&block.key) { - found.push(block); - } + let blocks = std::iter::successors(next_block(lines, start + 1, end), |b| { + next_block(lines, b.end, end) + }); + blocks + .filter_map(classify) + .map(|(rekeyed, block)| (block, rekeyed)) + .collect() +} + +/// The block the packages/snapshots edit rewrites: the first +/// [`matching_blocks`] entry, with whether it is a rekey already made. +/// Fails closed on a half-drifted lock: when the section carries BOTH the +/// registry entry and a rekey of it, a rekey would splice a DUPLICATE +/// mapping key (pnpm refuses to parse those) and surgery cannot decide +/// which block carries the truth. +pub(super) fn target_block( + lines: &L, + section: &str, + (start, end): (usize, usize), + name: &str, + version: &str, +) -> Result, String> { + let matches = matching_blocks(lines, section, start, end, name, version); + let registry = matches.iter().find(|(_, rekeyed)| !rekeyed); + if let Some((registry, _)) = registry.filter(|_| matches.iter().any(|(_, rekeyed)| *rekeyed)) { + return Err(format!( + "{section} section carries BOTH `{}` and a vendored `file:` rekey of it (a \ + half-edited lock); run `pnpm install` to re-resolve it, then re-vendor", + registry.key + )); } - found + Ok(matches.into_iter().next()) } // ─────────────────────── memoized split + section index ──────────────────── @@ -2839,7 +2871,7 @@ impl std::ops::Deref for LockLines { /// Read / write access to lock lines for the edits, which run both on the /// backend's [`LockLines`] and on a plain `Vec` (unit tests). `index` is /// `Some` only while the lines are exactly the ones it was built from. -trait EditLines { +pub(super) trait EditLines { fn read(&self) -> &Vec; fn write(&mut self) -> &mut Vec; fn index(&self) -> Option<&LockIndex> { @@ -2922,8 +2954,9 @@ impl IndexedBlock { /// The blocks of one top-level section, in lock order, as the scans walk /// them ([`section_bounds`] + [`next_block`]), with two lookups: blocks by -/// exact key, and blocks by the key text before each `@file:` in it (so -/// `name` finds every key starting `name@file:`). +/// key (one leading `/` dropped, as [`match_key`] reads it), and blocks by +/// the key text before each `@file:` in it (so `name` finds every key +/// starting `name@file:`, and `""` every bare `file:` key). #[derive(Default)] struct SectionIndex { bounds: Option<(usize, usize)>, @@ -2945,13 +2978,14 @@ impl SectionIndex { while let Some(block) = next_block(lines, i, end) { i = block.end; let at = index.blocks.len(); - index.by_key.entry(block.key.clone()).or_default().push(at); - for (p, _) in block.key.match_indices("@file:") { - index - .by_file_prefix - .entry(block.key[..p].to_string()) - .or_default() - .push(at); + let key = block.key.strip_prefix('/').unwrap_or(&block.key); + index.by_key.entry(key.to_string()).or_default().push(at); + // A bare `file:` key files under `""`, a `name@file:` one under `name`. + let bare = block.key.starts_with("file:").then_some(0); + let named = block.key.match_indices("@file:").map(|(p, _)| p); + for p in bare.into_iter().chain(named) { + let prefix = block.key[..p].to_string(); + index.by_file_prefix.entry(prefix).or_default().push(at); } index.blocks.push(IndexedBlock { header: block.header, @@ -2964,14 +2998,18 @@ impl SectionIndex { index } - /// Every block whose key is `reg_key` or starts with `name@file:`, in - /// lock order. - fn candidates(&self, reg_key: &str, name: &str) -> Vec<&IndexedBlock> { - let mut at: Vec = self - .by_key - .get(reg_key) - .into_iter() - .chain(self.by_file_prefix.get(name)) + /// Every block whose key may name `name@version` without a suffix + /// ([`match_key`]): its registry key in either separator, or a + /// `name@file:` / bare `file:` key, in lock order. + fn candidates(&self, name: &str, version: &str) -> Vec<&IndexedBlock> { + let mut at: Vec = [format!("{name}@{version}"), format!("{name}/{version}")] + .iter() + .filter_map(|key| self.by_key.get(key)) + .chain( + [name, ""] + .iter() + .filter_map(|n| self.by_file_prefix.get(*n)), + ) .flatten() .copied() .collect(); @@ -2991,8 +3029,19 @@ struct ImporterDep { ver: Option, } +impl ImporterDep { + fn as_ref(&self) -> ImporterRef<'_> { + ImporterRef { + importer: &self.importer, + dep: &self.dep, + spec: self.spec.as_deref(), + ver: self.ver.as_deref(), + } + } +} + /// A position the snapshot scan of [`check_rewritable_refs`] may refuse -/// at: a block header (by block) or a body line. +/// at: a block header or a body line (by line). enum SnapshotCandidate { Key(usize), Body(usize), @@ -3004,7 +3053,7 @@ enum SnapshotCandidate { /// its package up instead of scanning; `*_paren` maps key on the text /// before each `(` (so `x` finds everything starting `x(`). #[derive(Default)] -struct LockIndex { +pub(super) struct LockIndex { /// First line index of every column-0 line, and all column-0 lines in /// order: [`section_bounds`] for any section. headers: HashMap, @@ -3086,53 +3135,40 @@ impl LockIndex { } } - if let Some((start, end)) = section_bounds(lines, "importers") { - let mut i = start + 1; - while let Some(importer) = next_block(lines, i, end) { - let mut k = importer.header + 1; - while k < importer.end { - let Some((dep, _repr, rest)) = parse_key_line(&lines[k], 6) else { - k += 1; - continue; - }; - if !rest.is_empty() { - k += 1; - continue; - } - let (spec, ver, f) = dep_field_lines(lines, k + 1, importer.end, 8); - if let Some((_, v)) = &ver { - let at = index.importer_deps.len(); - let v = unquote_value(v); - index.first_importer_ver.entry(v.to_string()).or_insert(at); - for prefix in paren_prefixes(v) { - index - .first_importer_ver_paren - .entry(prefix.to_string()) - .or_insert(at); - index - .first_importer_dep_ver_paren - .entry((dep.to_string(), prefix.to_string())) - .or_insert(at); - } - if spec - .as_ref() - .is_some_and(|(_, s)| unquote_value(s).starts_with("catalog:")) - { - index - .first_importer_catalog - .entry((dep.to_string(), v.to_string())) - .or_insert(at); - } - index.importer_deps.push(ImporterDep { - importer: importer.key.clone(), - dep: dep.to_string(), - spec: spec.map(|(_, s)| s), - ver: ver.map(|(_, v)| v), - }); - } - k = f; - } - i = importer.end; + index.importer_deps = importer_refs(lines) + .filter(|entry| entry.ver.is_some()) + .map(|e| ImporterDep { + importer: e.importer.to_string(), + dep: e.dep.to_string(), + spec: e.spec.map(str::to_string), + ver: e.ver.map(str::to_string), + }) + .collect(); + for (at, entry) in index.importer_deps.iter().enumerate() { + let (dep, v) = ( + &entry.dep, + unquote_value(entry.ver.as_deref().unwrap_or("")), + ); + index.first_importer_ver.entry(v.to_string()).or_insert(at); + for prefix in paren_prefixes(v) { + index + .first_importer_ver_paren + .entry(prefix.to_string()) + .or_insert(at); + index + .first_importer_dep_ver_paren + .entry((dep.clone(), prefix.to_string())) + .or_insert(at); + } + if entry + .spec + .as_deref() + .is_some_and(|s| unquote_value(s).starts_with("catalog:")) + { + index + .first_importer_catalog + .entry((dep.clone(), v.to_string())) + .or_insert(at); } } index @@ -3166,10 +3202,8 @@ impl LockIndex { ) -> Vec { let mut found: Vec<(usize, SnapshotCandidate)> = Vec::new(); if let Some(&block) = self.first_snapshot_key_paren.get(reg_key) { - found.push(( - self.snapshots.blocks[block].header, - SnapshotCandidate::Key(block), - )); + let header = self.snapshots.blocks[block].header; + found.push((header, SnapshotCandidate::Key(header))); } let body = [ self.first_snapshot_rest.get(reg_key), @@ -3307,10 +3341,26 @@ fn revert_lock_record( match rec.kind.as_str() { KIND_LOCK_OVERRIDES => revert_overrides_line(lines, rec, key, entry_uuid, dirty, warnings), KIND_LOCK_IMPORTER_DEP => revert_importer_dep(lines, rec, key, entry_uuid, dirty, warnings), - KIND_LOCK_PACKAGE => revert_block(lines, rec, key, "packages", entry_uuid, dirty, warnings), - KIND_LOCK_SNAPSHOT => { - revert_block(lines, rec, key, "snapshots", entry_uuid, dirty, warnings) - } + KIND_LOCK_PACKAGE => revert_block( + lines, + rec, + key, + "packages", + entry_uuid, + parse_vendor_path, + dirty, + warnings, + ), + KIND_LOCK_SNAPSHOT => revert_block( + lines, + rec, + key, + "snapshots", + entry_uuid, + parse_vendor_path, + dirty, + warnings, + ), KIND_LOCK_SNAPSHOT_REF => revert_snapshot_ref(lines, rec, key, entry_uuid, dirty, warnings), other => warnings.push(drifted(format!( "unknown wiring kind `{other}` for `{key}`; left alone" @@ -3413,89 +3463,115 @@ fn revert_importer_dep( )); return; }; + let pair = DepPair { + rec, + key, + dep, + entry_uuid, + label: "importer dep", + parse: parse_vendor_path, + }; let mut i = start + 1; while let Some(importer) = next_block(lines, i, end) { - if importer.key != importer_key { - i = importer.end; - continue; + if importer.key == importer_key { + return pair.restore(lines, importer.header + 1, importer.end, 6, dirty, warnings); } - let mut k = importer.header + 1; - while k < importer.end { - let Some((d, _repr, rest)) = parse_key_line(&lines[k], 6) else { - k += 1; - continue; - }; - if d != dep || !rest.is_empty() { - k += 1; - continue; - } - let (spec_idx, ver_idx, _) = dep_field_lines(lines, k + 1, importer.end, 8); - let (Some((si, live_spec)), Some((vi, live_ver))) = (spec_idx, ver_idx) else { - break; - }; - // ALREADY CONVERGED: both fields already equal the recorded - // pre-vendor original — an earlier partial revert (or the - // user, by hand) already restored this record. Not drift: - // stay silent so the drift-skip keep gate can converge. - if let Some(original) = rec.original.as_ref() { - if original.get("specifier").and_then(Value::as_str) == Some(live_spec.as_str()) - && original.get("version").and_then(Value::as_str) == Some(live_ver.as_str()) - { - return; - } - } - let new_ver = rec - .new - .as_ref() - .and_then(|n| n.get("version")) - .and_then(Value::as_str); - let ours = Some(live_ver.as_str()) == new_ver - || parse_vendor_path(&live_ver) - .is_some_and(|p| p.eco == "npm" && p.uuid == entry_uuid); - if !ours { - warnings.push(drifted(format!( - "importer dep `{key}` was re-resolved since vendoring ({live_ver}); left alone" - ))); - return; - } - let Some(original) = rec.original.as_ref() else { - warnings.push(drifted(format!( - "importer dep `{key}` has no recorded pre-vendor original; left as-is \ - (re-run `pnpm install` to re-resolve it)" - ))); - return; - }; - let (Some(orig_spec), Some(orig_ver)) = ( - original.get("specifier").and_then(Value::as_str), - original.get("version").and_then(Value::as_str), - ) else { - warnings.push(drifted(format!( - "importer dep `{key}` original is malformed" - ))); - return; - }; - lines[si] = format!(" specifier: {orig_spec}"); - lines[vi] = format!(" version: {orig_ver}"); - *dirty = true; + i = importer.end; + } + pair.removed(warnings); +} + +/// One recorded `specifier:` / `version:` pair to restore: a v9 importer +/// dep or a legacy 6.0 root dep. +pub(super) struct DepPair<'a> { + pub(super) rec: &'a WiringRecord, + pub(super) key: &'a str, + pub(super) dep: &'a str, + pub(super) entry_uuid: &'a str, + /// Warning prefix (`importer dep` / `root dep`). + pub(super) label: &'a str, + /// The ownership parse: [`parse_vendor_path`], or for pnpm <= 8 locks + /// [`super::path::parse_vendor_path_or_machine_absolute`]. + pub(super) parse: fn(&str) -> Option, +} + +impl DepPair<'_> { + fn removed(&self, warnings: &mut Vec) { + warnings.push(removed(format!( + "{} `{}` no longer exists; nothing to restore", + self.label, self.key + ))); + } + + /// Restore the pair of the `dep` entry at `indent` in `[start, end)` + /// ([`dep_entries`]); a missing entry is "removed". + pub(super) fn restore( + &self, + lines: &mut [String], + start: usize, + end: usize, + indent: usize, + dirty: &mut bool, + warnings: &mut Vec, + ) { + let (rec, key, label) = (self.rec, self.key, self.label); + let entry = dep_entries(lines, start, end, indent).find(|e| e.dep == self.dep); + let Some((Some((si, live_spec)), Some((vi, live_ver)))) = entry.map(|e| (e.spec, e.ver)) + else { + return self.removed(warnings); + }; + // ALREADY CONVERGED: both fields already equal the recorded + // pre-vendor original — an earlier partial revert (or the user, by + // hand) already restored this record. Not drift: stay silent so the + // drift-skip keep gate can converge. + fn field<'v>(v: Option<&'v Value>, f: &str) -> Option<&'v str> { + v.and_then(|v| v.get(f)).and_then(Value::as_str) + } + let original = rec.original.as_ref(); + if original.is_some() + && field(original, "specifier") == Some(live_spec) + && field(original, "version") == Some(live_ver) + { return; } - break; + let ours = field(rec.new.as_ref(), "version") == Some(live_ver) + || (self.parse)(live_ver).is_some_and(|p| p.eco == "npm" && p.uuid == self.entry_uuid); + if !ours { + return warnings.push(drifted(format!( + "{label} `{key}` was re-resolved since vendoring ({live_ver}); left alone" + ))); + } + if original.is_none() { + return warnings.push(drifted(format!( + "{label} `{key}` has no recorded pre-vendor original; left as-is \ + (re-run `pnpm install` to re-resolve it)" + ))); + } + let (Some(orig_spec), Some(orig_ver)) = + (field(original, "specifier"), field(original, "version")) + else { + return warnings.push(drifted(format!("{label} `{key}` original is malformed"))); + }; + let pad = " ".repeat(indent + 2); + lines[si] = format!("{pad}specifier: {orig_spec}"); + lines[vi] = format!("{pad}version: {orig_ver}"); + *dirty = true; } - warnings.push(removed(format!( - "importer dep `{key}` no longer exists; nothing to restore" - ))); } /// Restore a rekeyed packages/snapshots block: locate the block by the NEW /// key (from `rec.new`'s first line), verify ownership, splice the original /// lines back — merged with any dependency ref ANOTHER entry rewrote inside -/// the block since (#830, [`merge_live_dep_refs`]). -fn revert_block( +/// the block since (#830, [`merge_live_dep_refs`]). `parse` is the +/// ownership parse, as on [`DepPair::parse`]. +#[allow(clippy::too_many_arguments)] +pub(super) fn revert_block( lines: &mut Vec, rec: &WiringRecord, key: &str, section: &str, entry_uuid: &str, + parse: fn(&str) -> Option, dirty: &mut bool, warnings: &mut Vec, ) { @@ -3525,11 +3601,11 @@ fn revert_block( continue; } // Ours iff the live block is exactly what we wrote, or its key still - // points into OUR uuid dir (a re-serialized but unmoved entry). + // points into OUR uuid dir (a re-serialized but unmoved entry): v9's + // `name@file:` or legacy's bare `file:`. let live: Vec = lines[block.header..block.end].to_vec(); - let key_is_ours = new_key.rsplit_once("@file:").is_some_and(|(_, p)| { - parse_vendor_path(p).is_some_and(|v| v.eco == "npm" && v.uuid == entry_uuid) - }); + let path = new_key.rsplit_once("@file:").map_or(new_key, |(_, p)| p); + let key_is_ours = parse(path).is_some_and(|v| v.eco == "npm" && v.uuid == entry_uuid); if live != new_lines && !key_is_ours { warnings.push(drifted(format!( "{section} entry `{new_key}` was changed since vendoring; left alone" @@ -3544,7 +3620,25 @@ fn revert_block( return; }; let restored = merge_live_dep_refs(&original, &new_lines, &live); - lines.splice(block.header..block.end, restored); + // Legacy's bare `file:` rekey moved the block across the `/` vs + // `file:` sort boundary, so it goes back to its byte-sorted place + // (byte-identity with the pre-vendor lock); v9 rekeys in place. + if !new_key.starts_with("file:") { + lines.splice(block.header..block.end, restored); + } else if let Some(orig_key) = original.first().and_then(|l| parse_key_line(l, 2)) { + let orig_key = orig_key.0.to_string(); + if swap_block_sorted(lines, section, new_key, &orig_key, &restored).is_err() { + warnings.push(drifted(format!( + "{section} entry `{new_key}` vanished mid-restore; left alone" + ))); + return; + } + } else { + warnings.push(drifted(format!( + "{section} entry `{key}` original is malformed" + ))); + return; + } *dirty = true; return; } @@ -3767,11 +3861,7 @@ fn dep_map_refs(block: &[String]) -> Vec<(usize, &str, &str, &str)> { /// unwound on its own) — keeps that live value, so the lock stays /// resolvable whichever entry is unwound first. With `live == new` this is /// `original` verbatim. -pub(super) fn merge_live_dep_refs( - original: &[String], - new: &[String], - live: &[String], -) -> Vec { +fn merge_live_dep_refs(original: &[String], new: &[String], live: &[String]) -> Vec { let values = |block| -> HashMap<&str, &str> { dep_map_refs(block) .into_iter() @@ -3793,7 +3883,7 @@ pub(super) fn merge_live_dep_refs( /// Are two blocks equal up to their dependency-ref VALUES (what /// [`merge_live_dep_refs`] may change)? -pub(super) fn same_modulo_dep_refs(a: &[String], b: &[String]) -> bool { +fn same_modulo_dep_refs(a: &[String], b: &[String]) -> bool { let (refs_a, refs_b) = (dep_map_refs(a), dep_map_refs(b)); a.len() == b.len() && refs_a.len() == refs_b.len() @@ -3903,7 +3993,7 @@ pub(super) fn lines_value(lines: &[String]) -> Value { Value::Array(lines.iter().map(|l| Value::String(l.clone())).collect()) } -pub(super) fn value_lines(v: &Value) -> Option> { +fn value_lines(v: &Value) -> Option> { v.as_array().map(|a| { a.iter() .filter_map(Value::as_str) @@ -5706,28 +5796,44 @@ snapshots: assert!(matches!(fx.vendor(false).await, VendorOutcome::Done { .. })); } - /// A CRLF pnpm-lock.yaml (Windows autocrlf checkout) passes the version - /// sniff (`str::lines` strips `\r`) but breaks every structural probe — - /// the refusal must name the line endings as the cause, not the - /// misleading "no packages entry … run `pnpm install`". + /// Divergences N15: a CRLF pnpm-lock.yaml (a Windows autocrlf checkout) + /// is wired like its LF twin and keeps its CRLF endings, as hosted + /// mode's byte splices keep them, and the revert restores the CRLF + /// bytes. #[tokio::test] - async fn crlf_lock_refuses_naming_line_endings() { + async fn crlf_lock_is_wired_like_its_lf_twin_and_stays_crlf() { let crlf_lock = P1_BEFORE_LOCK.replace('\n', "\r\n"); let fx = fixture_with(P1_BEFORE_PKG, &crlf_lock).await; + let (result, entry, _) = expect_done(fx.vendor(false).await); + assert!(result.success, "{:?}", result.error); + let wired = P1_AFTER_LOCK.replace(SPIKE_INTEGRITY, &fx.actual_integrity().await); + assert_eq!(fx.read(PNPM_LOCK).await, wired.replace('\n', "\r\n")); + assert_eq!(fx.read(PACKAGE_JSON).await, P1_AFTER_PKG); + + let outcome = revert_pnpm(&entry.unwrap(), fx.root(), false).await; + assert!(outcome.success, "{:?}", outcome.error); + assert!(outcome.warnings.is_empty(), "{:?}", outcome.warnings); + assert_eq!(fx.read(PNPM_LOCK).await, crlf_lock, "CRLF bytes restored"); + assert_eq!(fx.read(PACKAGE_JSON).await, P1_BEFORE_PKG); + assert!(!fx.root().join(".socket/vendor/npm").join(UUID).exists()); + } + + /// N15's one remaining refusal: a lock mixing CRLF and LF has no single + /// ending to write back, so it is refused naming the line endings (not + /// the misleading "no packages entry … run `pnpm install`"). + #[tokio::test] + async fn mixed_line_ending_lock_refuses_naming_line_endings() { + let mixed = P1_BEFORE_LOCK.replacen('\n', "\r\n", 1); + let fx = fixture_with(P1_BEFORE_PKG, &mixed).await; let detail = expect_refused(fx.vendor(false).await, "vendor_lockfile_crlf_unsupported"); - assert!(detail.contains("CRLF"), "{detail}"); - assert!( - !detail.contains("make sure the package is installed"), - "no misleading install remedy: {detail}" - ); - assert_eq!(fx.read(PNPM_LOCK).await, crlf_lock, "lock untouched"); + assert!(detail.contains("mixes CRLF and LF"), "{detail}"); + assert_eq!(fx.read(PNPM_LOCK).await, mixed, "lock untouched"); assert_eq!(fx.read(PACKAGE_JSON).await, P1_BEFORE_PKG, "pkg untouched"); assert!( !fx.root().join(".socket/vendor").exists(), "refusal stages no artifact" ); } - #[tokio::test] async fn dry_run_writes_nothing() { let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; @@ -5992,6 +6098,87 @@ snapshots: format!("---\n{env}\n---\n{project}") } + /// N11: a project that pins pnpm 1 / 2 installs from `shrinkwrap.yaml`, + /// so its `pnpm-lock.yaml` is stray. Vendoring refuses it, with nothing + /// written, instead of wiring a lock pnpm never reads, which is the + /// choice hosted mode and discovery make (`plan::governing_pnpm_lock`). + /// With no pin, the `pnpm-lock.yaml` governs and is vendored as before. + #[tokio::test] + async fn a_pnpm_2_pin_beside_a_shrinkwrap_refuses() { + let shrinkwrap = "shrinkwrapVersion: 3\nshrinkwrapMinorVersion: 9\n"; + let pinned = P1_BEFORE_PKG.replacen('{', "{\n \"packageManager\": \"pnpm@2.25.7\",", 1); + let fx = fixture_with(&pinned, P1_BEFORE_LOCK).await; + tokio::fs::write(fx.root().join("shrinkwrap.yaml"), shrinkwrap) + .await + .unwrap(); + let detail = expect_refused( + fx.vendor(false).await, + "vendor_lockfile_version_unsupported", + ); + assert!( + detail.contains("pnpm 1 / 2") && detail.contains("shrinkwrap.yaml"), + "{detail}" + ); + assert_eq!(fx.read(PNPM_LOCK).await, P1_BEFORE_LOCK, "lock untouched"); + assert_eq!(fx.read(PACKAGE_JSON).await, pinned, "pkg untouched"); + assert!(!fx.root().join(".socket/vendor").exists()); + + let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; + tokio::fs::write(fx.root().join("shrinkwrap.yaml"), shrinkwrap) + .await + .unwrap(); + assert!(matches!(fx.vendor(false).await, VendorOutcome::Done { .. })); + } + + /// N15 on pnpm 11's two-document lock: vendored, then checked out with + /// autocrlf (`---\r` separators), the lock reverts to its CRLF bytes: + /// the document check reads the LF form, so `---\r` is a separator. + #[tokio::test] + async fn crlf_two_document_lock_reverts() { + let before = two_doc(ENV_DOC_CONFIG_DEP, P1_BEFORE_LOCK); + let fx = fixture_with(P1_BEFORE_PKG, &before).await; + let (_, entry, _) = expect_done(fx.vendor(false).await); + let crlf = fx.read(PNPM_LOCK).await.replace('\n', "\r\n"); + tokio::fs::write(fx.root().join(PNPM_LOCK), &crlf) + .await + .unwrap(); + let outcome = revert_pnpm(&entry.unwrap(), fx.root(), false).await; + assert!(outcome.success, "{:?}", outcome.error); + assert_eq!(fx.read(PNPM_LOCK).await, before.replace('\n', "\r\n")); + } + + /// A revert over a lock or workspace file that MIXES CRLF and LF fails + /// before any write, dry run included, as vendoring refuses such a + /// file: every surface and the artifact stay as they are. + #[tokio::test] + async fn mixed_line_ending_revert_fails_before_any_write() { + for file in [PNPM_LOCK, PNPM_WORKSPACE] { + let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; + let (_, entry, _) = expect_done(fx.vendor(false).await); + let entry = entry.unwrap(); + let mixed = fx.read(file).await.replacen('\n', "\r\n", 1); + tokio::fs::write(fx.root().join(file), &mixed) + .await + .unwrap(); + let pkg = fx.read(PACKAGE_JSON).await; + for dry_run in [true, false] { + let outcome = revert_pnpm(&entry, fx.root(), dry_run).await; + assert!(!outcome.success, "{file}"); + let error = outcome.error.unwrap_or_default(); + assert!( + error.contains(&format!("{file} mixes CRLF and LF")), + "{error}" + ); + } + assert_eq!(fx.read(file).await, mixed, "{file} left alone"); + assert_eq!(fx.read(PACKAGE_JSON).await, pkg, "{file}: nothing reverted"); + assert!( + fx.root().join(fx.rel_tgz()).exists(), + "{file}: artifact kept" + ); + } + } + /// #466: vendoring a two-document lock edits the PROJECT document only — /// byte-for-byte the single-document oracle — and leaves the env /// document untouched, even when a config dependency shares the @@ -6365,7 +6552,9 @@ snapshots: fn lock_version_remedy_is_version_aware() { use crate::formats::pnpm::{sniff_lock_grammar, PnpmLockGrammar}; - assert!(check_lock_version("lockfileVersion: '9.0'\n").is_ok()); + assert!(PnpmDialect::V9 + .check_lock("lockfileVersion: '9.0'\n") + .is_ok()); assert_eq!( sniff_lock_grammar("lockfileVersion: '9.0'\n"), Ok(PnpmLockGrammar::V9) @@ -6385,7 +6574,7 @@ snapshots: // …while the v9 BACKEND's own guard still holds them out (they can // only reach `vendor_pnpm` through a router bug). for head in ["lockfileVersion: 5.4\n", "lockfileVersion: '6.0'\n"] { - assert!(check_lock_version(head).is_err(), "{head}"); + assert!(PnpmDialect::V9.check_lock(head).is_err(), "{head}"); } // Too old for the allowlist (pnpm 6's 5.3): upgrade pnpm. @@ -6394,19 +6583,24 @@ snapshots: // Future versions: the fix is an allowlisted-emitting pnpm or a // newer build, never "pnpm >= 9" (the user's pnpm already is). - for head in ["lockfileVersion: '9.1'\n", "lockfileVersion: '10.0'\n"] { + // An unparseable major is not "old" either. + for head in [ + "lockfileVersion: '9.1'\n", + "lockfileVersion: '10.0'\n", + "lockfileVersion: 'v10'\n", + ] { let err = sniff_lock_grammar(head).unwrap_err(); assert!(err.contains("update socket-patch"), "{err}"); assert!(!err.contains("re-lock with pnpm >= 9"), "{err}"); // The v9 backend's guard agrees on the direction. - let err = check_lock_version(head).unwrap_err(); + let err = PnpmDialect::V9.check_lock(head).unwrap_err(); assert!(err.contains("update socket-patch"), "{err}"); } // No version in the head: unrecognizably old — upgrade advice stands. let err = sniff_lock_grammar("importers:\n").unwrap_err(); assert!(err.contains("re-lock with pnpm >= 9"), "{err}"); - assert!(check_lock_version("importers:\n").is_err()); + assert!(PnpmDialect::V9.check_lock("importers:\n").is_err()); } #[test] @@ -7260,69 +7454,66 @@ snapshots: assert_eq!(fx.read(PNPM_LOCK).await, P1_BEFORE_LOCK, "lock untouched"); } - /// A CRLF pnpm-workspace.yaml (Windows autocrlf checkout) dodges every - /// structural probe — the conflict checks miss the existing `overrides:` - /// section and the edit appends a DUPLICATE one, which pnpm rejects as a - /// duplicated mapping key (bricking every pnpm command). Same fail-closed - /// posture as the lock's CRLF refusal. + /// N15 for pnpm-workspace.yaml: a CRLF file (Windows autocrlf checkout) + /// is read like its LF twin, so the existing `overrides:` section is found + /// and extended (never a DUPLICATE section, which pnpm rejects), and the + /// file keeps its CRLF endings. #[tokio::test] - async fn crlf_workspace_file_refuses_instead_of_duplicating_overrides() { + async fn crlf_workspace_file_gains_the_override_in_its_own_section() { let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; let crlf_ws = "packages:\r\n - 'packages/*'\r\noverrides:\r\n other-pkg: 2.0.0\r\n"; write_ws(&fx, crlf_ws).await; - let detail = expect_refused(fx.vendor(false).await, "vendor_lockfile_crlf_unsupported"); - assert!(detail.contains(PNPM_WORKSPACE), "{detail}"); - assert!(detail.contains("CRLF"), "{detail}"); + let (result, entry, _) = expect_done(fx.vendor(false).await); + assert!(result.success, "{:?}", result.error); + let ws = fx.read(PNPM_WORKSPACE).await; + assert_eq!(ws.matches("overrides:").count(), 1, "{ws:?}"); + assert!(ws.contains(" other-pkg: 2.0.0\r\n"), "{ws:?}"); + assert!( + ws.contains(&format!("left-pad@1.3.0: file:{}\r\n", fx.rel_tgz())), + "{ws:?}" + ); + assert!(!ws.replace("\r\n", "").contains('\n'), "stays CRLF: {ws:?}"); + + let outcome = revert_pnpm(&entry.unwrap(), fx.root(), false).await; + assert!(outcome.success, "{:?}", outcome.error); assert_eq!( fx.read(PNPM_WORKSPACE).await, crlf_ws, - "workspace untouched" + "CRLF bytes restored" ); - assert_eq!(fx.read(PNPM_LOCK).await, P1_BEFORE_LOCK, "lock untouched"); - assert_eq!(fx.read(PACKAGE_JSON).await, P1_BEFORE_PKG, "pkg untouched"); } - /// The revert twin of the CRLF blindness: a workspace file converted to - /// CRLF since vendoring makes the Added override read as "already - /// converged" (the section probe misses it), so revert would remove the - /// artifact while the live override still points at it — pnpm >= 11 then - /// resolves through a deleted tarball. Must be a drift-keep instead. + /// The revert twin: a project vendored with LF files and then checked + /// out with autocrlf (every text file CRLF) reverts cleanly. The created + /// workspace scaffold reads as the scaffold and is deleted, the lock is + /// restored in CRLF, and the artifact goes. Before N15 the CRLF workspace + /// file was a drift-keep that held the artifact until the user + /// normalized it. #[tokio::test] - async fn crlf_workspace_revert_is_a_drift_keep_not_silent_convergence() { + async fn crlf_checkout_of_a_vendored_project_reverts_cleanly() { let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; let (_, entry, _) = expect_done(fx.vendor(false).await); let entry = entry.unwrap(); - let crlf_ws = fx.read(PNPM_WORKSPACE).await.replace('\n', "\r\n"); - tokio::fs::write(fx.root().join(PNPM_WORKSPACE), &crlf_ws) - .await - .unwrap(); + for file in [PNPM_WORKSPACE, PNPM_LOCK] { + let crlf = fx.read(file).await.replace('\n', "\r\n"); + tokio::fs::write(fx.root().join(file), crlf).await.unwrap(); + } let outcome = revert_pnpm(&entry, fx.root(), false).await; assert!(outcome.success, "{:?}", outcome.error); - assert!( - outcome - .warnings - .iter() - .any(|w| w.code == "vendor_lock_entry_drifted" && w.detail.contains(PNPM_WORKSPACE)), - "CRLF workspace must surface as drift: {:?}", - outcome.warnings - ); - assert!( - outcome.kept_artifact, - "the still-referenced artifact must be kept" - ); - assert!( - fx.root() - .join(format!(".socket/vendor/npm/{UUID}")) - .exists(), - "artifact dir survives the drift-keep" - ); + assert!(outcome.warnings.is_empty(), "{:?}", outcome.warnings); + assert!(!outcome.kept_artifact); + assert!(!fx.root().join(PNPM_WORKSPACE).exists(), "scaffold deleted"); assert_eq!( - fx.read(PNPM_WORKSPACE).await, - crlf_ws, - "CRLF file left alone" + fx.read(PNPM_LOCK).await, + P1_BEFORE_LOCK.replace('\n', "\r\n"), + "lock restored, still CRLF" ); + assert!(!fx + .root() + .join(format!(".socket/vendor/npm/{UUID}")) + .exists()); } /// A CRLF lock (a Windows autocrlf checkout) must never read as @@ -9778,19 +9969,53 @@ snapshots: assert_eq!(check_rewritable_refs(&lines, "left-pad", "1.3.0"), Ok(())); } + #[test] + fn a_sibling_checkouts_rekey_is_not_a_target() { + // Lock-model 2b (divergence N5, vendored planner): the planner reads + // its own rekeyed keys with the planning core's `match_key` + // (`origin::vendor_ref`, root-anchored), as hosted mode and discovery + // do, so a sibling checkout's `../.socket/vendor/…` rekey is not + // ours. + let art = format!(".socket/vendor/npm/{UUID}/left-pad-1.3.0.tgz"); + let target = |key: &str| target_key(key, "left-pad", "1.3.0"); + assert_eq!(target(&format!("left-pad@file:{art}")), Some(true)); + assert_eq!(target(&format!("file:{art}")), Some(true), "legacy rekey"); + assert_eq!(target("left-pad@1.3.0"), Some(false)); + assert_eq!( + target("/left-pad/1.3.0"), + Some(false), + "legacy registry key" + ); + assert_eq!(target("left-pad@1.3.0(react@18.2.0)"), None, "peer context"); + let sibling = format!("left-pad@file:../{art}"); + assert_eq!(target(&sibling), None); + let lock = split_lines(&format!( + "lockfileVersion: '9.0'\n\npackages:\n\n {sibling}:\n resolution: \ + {{integrity: sha512-x, tarball: file:../{art}}}\n version: 1.3.0\n" + )); + assert!(!lock_has_target_package(&lock, "left-pad", "1.3.0")); + } + /// A dep entry carrying a field that is neither `specifier` nor /// `version` (hand-edited or future-pnpm) is skipped, not misparsed: - /// both known fields are still found and the cursor lands past the - /// whole field run. + /// both known fields are still found and the entry ends at the first + /// non-field line. #[test] - fn dep_field_lines_ignores_unknown_fields() { + fn dep_entries_ignore_unknown_fields() { let lines = split_lines( - " specifier: ^1.0.0\n engines: whatever\n version: 1.0.0\n other:", + " dep:\n specifier: ^1.0.0\n engines: whatever\n version: 1.0.0\n other:\n inline: 1.0.0", + ); + let got: Vec<_> = dep_entries(&lines, 0, lines.len(), 6) + .map(|e| (e.dep, e.spec, e.ver)) + .collect(); + assert_eq!( + got, + vec![ + ("dep", Some((1, "^1.0.0")), Some((3, "1.0.0"))), + ("other", None, None), + ], + "fields end at the first non-field line; an inline value is no entry" ); - let (spec, ver, f) = dep_field_lines(&lines, 0, lines.len(), 8); - assert_eq!(spec, Some((0, "^1.0.0".to_string()))); - assert_eq!(ver, Some((2, "1.0.0".to_string()))); - assert_eq!(f, 3, "cursor stops at the first non-field line"); } /// The importer/snapshot edits are no-ops on a lock without their @@ -9937,6 +10162,7 @@ snapshots: "left-pad@1.3.0", "packages", UUID, + parse_vendor_path, &mut dirty, &mut warnings, ); @@ -9958,6 +10184,7 @@ snapshots: "left-pad@1.3.0", "packages", UUID, + parse_vendor_path, &mut dirty, &mut warnings, ); @@ -10126,7 +10353,7 @@ snapshots: let peer_suffixed = P1_BEFORE_LOCK.replace(" left-pad@1.3.0: {}", " left-pad@1.3.0(react@18.2.0): {}"); let absent = P1_BEFORE_LOCK.replace("left-pad@1.3.0", "left-pad@1.3.1"); - let crlf = P1_BEFORE_LOCK.replace('\n', "\r\n"); + let mixed = P1_BEFORE_LOCK.replacen('\n', "\r\n", 1); let legacy = P1_BEFORE_LOCK.replace("lockfileVersion: '9.0'", "lockfileVersion: '6.0'"); let cases: [(&str, &str, &str, &str); 6] = [ ( @@ -10154,9 +10381,9 @@ snapshots: "vendor_lock_entry_not_found", ), ( - "CRLF lock", + "mixed line-ending lock", P1_BEFORE_PKG, - &crlf, + &mixed, "vendor_lockfile_crlf_unsupported", ), ( @@ -10432,7 +10659,7 @@ snapshots: "seed {seed} refs {name}@{version}\n{text}" ); assert_eq!( - lock_has_target_package_in(&shared(&lines), name, version), + lock_has_target_package(&shared(&lines), name, version), lock_has_target_package(&lines, name, version), "seed {seed} target {name}@{version}\n{text}" ); @@ -10523,7 +10750,9 @@ snapshots: P1_BEFORE_LOCK, ))))), lock_prefix: String::new(), + lock_eol: LineEndings::Lf, ws_text: None, + ws_eol: LineEndings::None, }; let doc_of = |p: &PnpmProject| match &p.lock { ProjectLock::V9(LockLines::Shared(doc)) => Arc::clone(doc), @@ -10597,8 +10826,8 @@ snapshots: let ProjectLock::V9(lines) = &second.lock else { unreachable!() }; - assert!(!lock_has_target_package_in(lines, "left-pad", "1.3.0")); - assert!(lock_has_target_package_in(lines, "right-pad", "1.3.0")); + assert!(!lock_has_target_package(lines, "left-pad", "1.3.0")); + assert!(lock_has_target_package(lines, "right-pad", "1.3.0")); } /// pnpm-workspace.yaml spellings and document shapes the overrides surgery diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs b/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs index eac0fb8c4..acc5ee6e7 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs @@ -59,13 +59,12 @@ use serde_json::Value; use crate::manifest::schema::PatchRecord; use crate::patch::apply::PatchSources; -use crate::utils::fs::read_regular_to_string; use super::common::refused; use super::path::parse_vendor_path_or_machine_absolute; use super::pnpm_lock::{ - dep_field_lines, drifted, lines_value, merge_live_dep_refs, ref_record_blocks, removed, - revert_overrides_line, same_modulo_dep_refs, value_lines, DepRef, EditCtx, PnpmDialect, + dep_entries, drifted, lines_value, lock_has_target_package, ref_record_blocks, removed, + revert_block, revert_overrides_line, target_block, DepPair, DepRef, EditCtx, PnpmDialect, KIND_LOCK_OVERRIDES, KIND_LOCK_PACKAGE, }; use super::source::PackageSource; @@ -76,7 +75,7 @@ use crate::formats::pnpm::lines::{ next_block, parse_key_line, section_bounds, split_lines, unquote_value, yaml_key, yaml_key_like, yaml_value, }; -use crate::formats::pnpm::{sniff_lock_grammar, PnpmLock, PnpmLockGrammar}; +use crate::formats::pnpm::PnpmLockGrammar; /// The [`VendorEntry::flavor`] string legacy wirings are stamped with. /// Distinct from the v9 backend's `"pnpm"` so an older binary (which has no @@ -231,15 +230,6 @@ pub(super) struct LegacyLock { abs_root: String, } -pub(super) fn check_lock_version(text: &str) -> Result { - match sniff_lock_grammar(text) { - Ok(PnpmLockGrammar::V9) => Err(format!( - "{PNPM_LOCK} is a lockfileVersion 9.0 lock; not a legacy grammar" - )), - result => result, - } -} - pub(super) fn read_lock( lock_text: &str, project_root: &Path, @@ -311,7 +301,7 @@ pub(super) fn preflight_package( if let Err(detail) = check_rewritable_refs(&lock.lines, &ctx) { return Err(Box::new(refused("vendor_lock_entry_unsupported", detail))); } - if !lock_has_target_package(&lock.lines, &ctx) { + if !lock_has_target_package(&lock.lines, name, version) { return Err(Box::new(refused( "vendor_lock_entry_not_found", format!( @@ -339,14 +329,7 @@ pub(crate) async fn preflight_packages( /// (the `overrides:` declaration alone never counts); `None` when /// undeterminable — callers keep the entry, fail-safe. pub async fn pnpm_legacy_entry_in_use(entry: &VendorEntry, project_root: &Path) -> Option { - let text = read_regular_to_string(&project_root.join(PNPM_LOCK)) - .await - .ok()?; - match sniff_lock_grammar(&text) { - Ok(PnpmLockGrammar::V54 | PnpmLockGrammar::V60) => {} - _ => return None, - } - Some(PnpmLock::parse(&text).vendored_in_use(&entry.uuid)) + PnpmDialect::Legacy.entry_in_use(entry, project_root).await } impl LegacyLock { @@ -400,24 +383,6 @@ impl LegacyLock { // ─────────────────────────── pre-flight checks ─────────────────────────── -/// Does the lock have a packages entry vendoring can target — the grammar's -/// registry key, or our rekeyed `file:` key (the in-sync / stale-uuid -/// re-run)? -fn lock_has_target_package(lines: &[String], ctx: &Ctx<'_>) -> bool { - let Some((start, end)) = section_bounds(lines, "packages") else { - return false; - }; - let reg_key = ctx.reg_key(); - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - if block.key == reg_key || ctx.is_ours(&block.key) { - return true; - } - i = block.end; - } - false -} - /// Fail-closed guard against legacy reference forms the surgery does not /// rewrite: PEER-SUFFIXED dep paths (`/name/version_peer…` in v5.4, /// `/name@version(peer…)` in v6.0) and ALIASED references (a dep whose @@ -437,34 +402,24 @@ fn check_rewritable_refs(lines: &[String], ctx: &Ctx<'_>) -> Result<(), String> ctx.name, ctx.version )) }; - // Root dep maps (v5.4 bare values; v6.0 version: fields). + // Root dep maps: v5.4 bare values, v6.0 nested `version:` fields. for section in ROOT_DEP_SECTIONS { let Some((start, end)) = section_bounds(lines, section) else { continue; }; - let mut k = start + 1; - while k < end { - let Some((dep, _repr, rest)) = parse_key_line(&lines[k], 2) else { - k += 1; - continue; - }; - let value = if rest.is_empty() { - // v6.0 nested shape: read the version: field. - let (_, ver, f) = dep_field_lines(lines, k + 1, end, 4); - k = f; - match ver { - Some((_, v)) => v, - None => continue, - } - } else { - k += 1; - rest.to_string() - }; + let bare = lines[start + 1..end] + .iter() + .filter_map(|line| parse_key_line(line, 2)) + .filter(|(_, _, value)| !value.is_empty()) + .map(|(dep, _, value)| (dep, value)); + let nested = dep_entries(lines, start + 1, end, 2) + .filter_map(|entry| Some((entry.dep, entry.ver?.1))); + for (dep, value) in bare.chain(nested) { if value == reg_key || value.starts_with(&key_peer_prefix) { - return refuse("an aliased root dependency", &value); + return refuse("an aliased root dependency", value); } if dep == ctx.name && value.starts_with(&val_peer_prefix) { - return refuse("a peer-suffixed root dependency", &value); + return refuse("a peer-suffixed root dependency", value); } } } @@ -618,52 +573,33 @@ fn edit_root_deps_v60( let Some((start, end)) = section_bounds(lines, section) else { continue; }; - let mut k = start + 1; - while k < end { - let Some((dep, _repr, rest)) = parse_key_line(&lines[k], 2) else { - k += 1; - continue; - }; - if dep != ctx.name || !rest.is_empty() { - k += 1; + let pairs: Vec<_> = dep_entries(lines, start + 1, end, 2) + .filter(|entry| entry.dep == ctx.name) + .filter_map(|entry| entry.pair()) + .collect(); + for (si, old_spec, vi, old_ver) in pairs { + if !(old_ver == ctx.version || ctx.is_ours(&old_ver)) { continue; } - let (spec_f, ver_f, f) = dep_field_lines(lines, k + 1, end, 4); - if let (Some((si, old_spec)), Some((vi, old_ver))) = (spec_f, ver_f) { - let target = old_ver == ctx.version || ctx.is_ours(&old_ver); - if target { - hit = true; - if old_ver == ctx.spec && old_spec == abs_yaml { - k = f; - continue; // in sync - } - let was_ours = ctx.is_ours(&old_ver); - // Built before the splices below: `dep` borrows `lines[k]`. - let record = WiringRecord { - file: PNPM_LOCK.to_string(), - kind: KIND_LOCK_ROOT_DEP_PAIR.to_string(), - action: WiringAction::Rewritten, - key: Some(format!("{section}|{dep}")), - original: if was_ours { - None - } else { - Some(serde_json::json!({ - "specifier": old_spec, - "version": old_ver, - })) - }, - new: Some(serde_json::json!({ - "specifier": ctx.abs_spec, - "version": ctx.spec, - })), - }; - lines[si] = format!(" specifier: {abs_yaml}"); - lines[vi] = format!(" version: {}", ctx.spec); - wiring.push(record); - changed = true; - } + hit = true; + if old_ver == ctx.spec && old_spec == abs_yaml { + continue; // in sync } - k = f; + wiring.push(WiringRecord { + file: PNPM_LOCK.to_string(), + kind: KIND_LOCK_ROOT_DEP_PAIR.to_string(), + action: WiringAction::Rewritten, + key: Some(format!("{section}|{}", ctx.name)), + original: (!ctx.is_ours(&old_ver)) + .then(|| serde_json::json!({ "specifier": old_spec, "version": old_ver })), + new: Some(serde_json::json!({ + "specifier": ctx.abs_spec, + "version": ctx.spec, + })), + }); + lines[si] = format!(" specifier: {abs_yaml}"); + lines[vi] = format!(" version: {}", ctx.spec); + changed = true; } } Ok((changed, hit)) @@ -678,37 +614,14 @@ fn edit_packages( ctx: &Ctx<'_>, wiring: &mut Vec, ) -> Result { - let (start, end) = section_bounds(lines, "packages").ok_or("no packages: section")?; + let bounds = section_bounds(lines, "packages").ok_or("no packages: section")?; let reg_key = ctx.reg_key(); let new_key = ctx.new_key(); - - // Fail closed on a half-drifted lock carrying BOTH spellings. - let mut has_registry = false; - let mut has_ours = false; - let mut j = start + 1; - while let Some(block) = next_block(lines, j, end) { - if block.key == reg_key { - has_registry = true; - } else if ctx.is_ours(&block.key) { - has_ours = true; - } - j = block.end; - } - if has_registry && has_ours { - return Err(format!( - "packages section carries BOTH `{reg_key}` and a `file:…` entry (a \ - half-edited lock); run `pnpm install` to re-resolve it, then re-vendor" - )); - } - - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - let is_registry = block.key == reg_key; - let is_ours_key = ctx.is_ours(&block.key); - if !is_registry && !is_ours_key { - i = block.end; - continue; - } + // The first registry-or-ours block is the one rewritten (every path + // below returns). + if let Some((block, is_ours_key)) = + target_block(&*lines, "packages", bounds, ctx.name, ctx.version)? + { let original_lines: Vec = lines[block.header..block.end].to_vec(); let expected_resolution = format!( " resolution: {{integrity: {}, tarball: {}}}", @@ -845,7 +758,7 @@ fn edit_pkg_dep_refs( /// sort before `file:`-keys). pnpm sorts package keys with the `sort-keys` /// default compare (JS code-unit order), which plain Rust `str` ordering /// matches for these ASCII keys. -fn swap_block_sorted( +pub(super) fn swap_block_sorted( lines: &mut Vec, section: &str, old_key: &str, @@ -941,7 +854,16 @@ pub(super) fn revert_lock_record( KIND_LOCK_ROOT_DEP_PAIR => { revert_root_dep_pair(lines, rec, key, entry_uuid, dirty, warnings) } - KIND_LOCK_PACKAGE => revert_package_block(lines, rec, key, entry_uuid, dirty, warnings), + KIND_LOCK_PACKAGE => revert_block( + lines, + rec, + key, + "packages", + entry_uuid, + parse_vendor_path_or_machine_absolute, + dirty, + warnings, + ), KIND_LOCK_PKG_DEP_REF => revert_pkg_dep_ref(lines, rec, key, entry_uuid, dirty, warnings), other => warnings.push(drifted(format!( "unknown wiring kind `{other}` for `{key}`; left alone" @@ -1031,157 +953,15 @@ fn revert_root_dep_pair( ))); return; }; - let mut k = start + 1; - while k < end { - let Some((d, _repr, rest)) = parse_key_line(&lines[k], 2) else { - k += 1; - continue; - }; - if d != dep || !rest.is_empty() { - k += 1; - continue; - } - let (spec_f, ver_f, _) = dep_field_lines(lines, k + 1, end, 4); - let (Some((si, live_spec)), Some((vi, live_ver))) = (spec_f, ver_f) else { - break; - }; - // ALREADY CONVERGED: both fields already equal the recorded - // pre-vendor original — an earlier partial revert (or the user, by - // hand) already restored this record. Not drift: stay silent so - // the drift-skip keep gate can converge. - if let Some(original) = rec.original.as_ref() { - if original.get("specifier").and_then(Value::as_str) == Some(live_spec.as_str()) - && original.get("version").and_then(Value::as_str) == Some(live_ver.as_str()) - { - return; - } - } - let new_ver = rec - .new - .as_ref() - .and_then(|n| n.get("version")) - .and_then(Value::as_str); - let ours = Some(live_ver.as_str()) == new_ver - || parse_vendor_path_or_machine_absolute(&live_ver) - .is_some_and(|p| p.eco == "npm" && p.uuid == entry_uuid); - if !ours { - warnings.push(drifted(format!( - "root dep `{key}` was re-resolved since vendoring ({live_ver}); left alone" - ))); - return; - } - let Some(original) = rec.original.as_ref() else { - warnings.push(drifted(format!( - "root dep `{key}` has no recorded pre-vendor original; left as-is \ - (re-run `pnpm install` to re-resolve it)" - ))); - return; - }; - let (Some(orig_spec), Some(orig_ver)) = ( - original.get("specifier").and_then(Value::as_str), - original.get("version").and_then(Value::as_str), - ) else { - warnings.push(drifted(format!("root dep `{key}` original is malformed"))); - return; - }; - lines[si] = format!(" specifier: {orig_spec}"); - lines[vi] = format!(" version: {orig_ver}"); - *dirty = true; - return; - } - warnings.push(removed(format!( - "root dep `{key}` no longer exists; nothing to restore" - ))); -} - -/// Restore the rekeyed packages block: locate it by the NEW `file:` key, -/// verify ownership, then reinsert the ORIGINAL block at its byte-sorted -/// position (the rekey moved it across the `/` vs `file:` sort boundary, so -/// an in-place splice would restore it out of order and break byte-identity -/// with the pre-vendor lock) — merged with any dependency ref ANOTHER entry -/// rewrote inside the block since (#830, [`merge_live_dep_refs`]). -fn revert_package_block( - lines: &mut Vec, - rec: &WiringRecord, - key: &str, - entry_uuid: &str, - dirty: &mut bool, - warnings: &mut Vec, -) { - let Some(new_lines) = rec.new.as_ref().and_then(value_lines) else { - warnings.push(drifted(format!( - "record for `{key}` has no `new` fragment; left alone" - ))); - return; - }; - let Some((new_key, _repr, _rest)) = new_lines.first().and_then(|l| parse_key_line(l, 2)) else { - warnings.push(drifted(format!( - "record for `{key}` has a malformed fragment" - ))); - return; - }; - let Some((start, end)) = section_bounds(lines, "packages") else { - warnings.push(drifted(format!( - "packages section is gone; `{key}` not restored" - ))); - return; + let pair = DepPair { + rec, + key, + dep, + entry_uuid, + label: "root dep", + parse: parse_vendor_path_or_machine_absolute, }; - let mut i = start + 1; - while let Some(block) = next_block(lines, i, end) { - if block.key != new_key { - i = block.end; - continue; - } - let live: Vec = lines[block.header..block.end].to_vec(); - let key_is_ours = parse_vendor_path_or_machine_absolute(new_key) - .is_some_and(|p| p.eco == "npm" && p.uuid == entry_uuid); - if live != new_lines && !key_is_ours { - warnings.push(drifted(format!( - "packages entry `{new_key}` was changed since vendoring; left alone" - ))); - return; - } - let Some(original) = rec.original.as_ref().and_then(value_lines) else { - warnings.push(drifted(format!( - "packages entry `{key}` has no recorded pre-vendor original; left as-is \ - (re-run `pnpm install` to re-resolve it)" - ))); - return; - }; - let Some((orig_key, _r, _v)) = original.first().and_then(|l| parse_key_line(l, 2)) else { - warnings.push(drifted(format!( - "packages entry `{key}` original is malformed" - ))); - return; - }; - let restored = merge_live_dep_refs(&original, &new_lines, &live); - if swap_block_sorted(lines, "packages", new_key, orig_key, &restored).is_err() { - warnings.push(drifted(format!( - "packages entry `{new_key}` vanished mid-restore; left alone" - ))); - return; - } - *dirty = true; - return; - } - // ALREADY CONVERGED: an earlier partial revert restored this record — - // the restore rekeys the block back to its pre-vendor key, so the - // recorded `file:` key no longer matches while the original block is - // live (up to the dependency refs the merge kept). Not drift: stay - // silent so the drift-skip keep gate can converge instead of keeping - // the artifacts forever. - if let Some(orig) = rec.original.as_ref().and_then(value_lines) { - let mut j = start + 1; - while let Some(block) = next_block(lines, j, end) { - if same_modulo_dep_refs(&lines[block.header..block.end], &orig) { - return; - } - j = block.end; - } - } - warnings.push(removed(format!( - "packages entry `{new_key}` no longer exists; nothing to restore" - ))); + pair.restore(lines, start + 1, end, 2, dirty, warnings); } /// Restore one dependent packages block's dep ref to this entry. Keyed by @@ -2625,14 +2405,21 @@ packages: assert_eq!(fx.read(PNPM_LOCK).await, lock, "refusal writes nothing"); } - /// CRLF and non-allowlisted lock versions refuse with the same codes - /// and byte-untouched files as the v9 backend. + /// Divergences N15 on a legacy lock: a CRLF 5.4 lock is wired like its + /// LF twin (the captured pnpm 7 shape) and stays CRLF; vendoring used to + /// refuse it. Non-allowlisted lock versions still refuse, with + /// byte-untouched files, as the v9 backend does (N16, target-specific). #[tokio::test] - async fn crlf_and_foreign_versions_refuse() { + async fn crlf_is_wired_and_foreign_versions_refuse() { let crlf = T7_BEFORE_LOCK.replace('\n', "\r\n"); let fx = fixture_with(T_BEFORE_PKG, &crlf).await; - expect_refused(fx.vendor(false).await, "vendor_lockfile_crlf_unsupported"); - assert_eq!(fx.read(PNPM_LOCK).await, crlf, "refusal writes nothing"); + let (result, entry, _) = expect_done(fx.vendor(false).await); + assert!(result.success, "{:?}", result.error); + let wired = fx.expected_lock(T7_AFTER_LOCK).await; + assert_eq!(fx.read(PNPM_LOCK).await, wired.replace('\n', "\r\n")); + let outcome = revert_pnpm_legacy(&entry.unwrap(), fx.root(), false).await; + assert!(outcome.success, "{:?}", outcome.error); + assert_eq!(fx.read(PNPM_LOCK).await, crlf, "CRLF bytes restored"); // pnpm 6's 5.3 is NOT allowlisted. let old = T7_BEFORE_LOCK.replace("lockfileVersion: 5.4", "lockfileVersion: 5.3"); @@ -2861,8 +2648,8 @@ packages: ); } - /// Wiring records left alone during a WIRED revert (here: a - /// CRLF-converted lock whose LF-exact probes all miss) must KEEP the + /// Wiring records left alone during a WIRED revert (here: a root dep + /// re-resolved by hand since vendoring) must KEEP the /// artifact dir — the lock still resolves through the tarball, and /// deleting it bricks every subsequent install (the lossiness guard, /// present in the v9/npm/bun backends). @@ -2871,8 +2658,12 @@ packages: let fx = fixture_with(T_BEFORE_PKG, T7_BEFORE_LOCK).await; let (_, entry, _) = expect_done(fx.vendor(false).await); let entry = entry.unwrap(); - let crlf = fx.read(PNPM_LOCK).await.replace('\n', "\r\n"); - tokio::fs::write(fx.root().join(PNPM_LOCK), &crlf) + let drifted = fx.read(PNPM_LOCK).await.replacen( + &format!(" left-pad: file:{}\n", fx.rel_tgz()), + " left-pad: 1.3.1\n", + 1, + ); + tokio::fs::write(fx.root().join(PNPM_LOCK), &drifted) .await .unwrap(); @@ -2893,12 +2684,11 @@ packages: ); assert!( fx.root().join(fx.rel_tgz()).exists(), - "the CRLF lock still resolves through the tarball; deleting it bricks installs" + "the lock still resolves through the tarball; deleting it bricks installs" ); - assert_eq!( - fx.read(PNPM_LOCK).await, - crlf, - "the drifted lock is left alone" + assert!( + fx.read(PNPM_LOCK).await.contains(" left-pad: 1.3.1\n"), + "the drifted line is left alone" ); } diff --git a/crates/socket-patch-core/src/vendor/refusals.rs b/crates/socket-patch-core/src/vendor/refusals.rs index 52a4f1a40..508485d12 100644 --- a/crates/socket-patch-core/src/vendor/refusals.rs +++ b/crates/socket-patch-core/src/vendor/refusals.rs @@ -887,6 +887,8 @@ mod tests { /// A hosted pnpm pin's takeover meets the pnpm lock-text gates /// (#853); a ledger entry already at the selected uuid is in sync. + /// The lock mixes CRLF and LF endings, which the pnpm rewriter + /// cannot write back byte-faithfully (a pure CRLF lock is wired). #[tokio::test] async fn a_hosted_pin_meets_the_pnpm_gates_before_its_takeover() { let tmp = tempfile::tempdir().unwrap(); @@ -897,7 +899,7 @@ mod tests { .unwrap(); std::fs::write( tmp.path().join("pnpm-lock.yaml"), - HOSTED_V9.replace('\n', "\r\n"), + HOSTED_V9.replacen('\n', "\r\n", 1), ) .unwrap(); let pin = HostedPin { diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index f9d8f6d9c..e095a89d3 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -30,7 +30,7 @@ //! lockfile supplement, the vendored fetch inventory) read each format //! through ONE shared reader that yields EVERY entry, Socket-owned ones //! included: the npm-family entry models in `lock_inventory` -//! (`npm_lock_nodes`, `pnpm::pnpm_packages`, `yarn::classic_entries` / +//! (`npm_lock_nodes`, `PnpmLock::packages`, `yarn::classic_entries` / //! `berry_entries`, `BunLockb::parse_packages`, `vlt::vlt_lock_model`) and, //! for the other formats, the readers the writers own (`cargo_lock` / //! `cargo_config`, `go_mod_edit` / `go_sum_edit`, `formats::gem`, @@ -182,7 +182,7 @@ //! `vlt-lock.json`), so discovery and the inventory see one entry walk //! per format (an architecture test in this module enforces it). -use std::collections::{BTreeMap, BTreeSet, HashMap}; +use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; use std::path::{Path, PathBuf}; use std::sync::Mutex; @@ -250,6 +250,60 @@ pub const DIAG_REF_INVALID: &str = "patched_ref_invalid"; /// dependency version, a nuget source with no package mapping). pub const DIAG_REF_UNATTRIBUTABLE: &str = "patched_ref_unattributable"; +/// Which install tree of its file a record belongs to: the install-tree +/// key is the file (its [`Discovery::install_trees`] root) plus this part. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum TreePart { + /// The file's own tree (every format but the one below). + #[default] + File, + /// pnpm 11+'s env lockfile document (config dependencies, the pinned + /// package manager), which installs apart from the project's + /// `node_modules`: its own tree (owner decision, lock-model divergence + /// N4). Its wiring never exempts the project document in + /// [`Discovery::contest_across_locks`], and that contest skips it both + /// ways: no other lock decides a config dependency, and a config + /// dependency decides no other lock's ref. Within the document the + /// same-lock rules apply. Its entries never contest the project + /// document's refs, but the project document's copies DO withhold its + /// refs ([`TreePart::withheld_by`], owner decision on #2009). + PnpmEnv, +} + +impl TreePart { + /// Whether a copy of a package installed in tree `copy` of a ref's own + /// file withholds that ref (`self`): a copy in the same tree does, and + /// so does a pnpm lock's project document copy for a ref in the env + /// document of the same lock. A VEX statement covers the release, not + /// one install tree, so attesting a config dependency's wiring while + /// the project's `node_modules` installs the same `name@version` from + /// the registry, a `file:` copy or a bundled copy would claim the + /// release is fixed where it is not (owner decision on #2009). The + /// reverse does not hold: the env document's copies install into + /// pnpm's config store only and withhold no project ref. + pub(crate) fn withheld_by(self, copy: TreePart) -> bool { + self == copy || (self == TreePart::PnpmEnv && copy == TreePart::File) + } + + /// How diagnostics name this tree of root-relative `file`: + /// ` (env lockfile)` for the env document, else `file`. + pub(crate) fn label(self, file: &str) -> String { + match self { + TreePart::File => file.to_string(), + TreePart::PnpmEnv => format!("{file} (env lockfile)"), + } + } +} + +impl From for TreePart { + fn from(document: crate::formats::pnpm::PnpmDocument) -> Self { + match document { + crate::formats::pnpm::PnpmDocument::Project => TreePart::File, + crate::formats::pnpm::PnpmDocument::Env => TreePart::PnpmEnv, + } + } +} + /// One non-fatal discovery finding. The CLI surfaces each as a run warning /// (`code` is the stable routing tag, `file` the root-relative source). #[derive(Debug, Clone, PartialEq, Eq)] @@ -304,6 +358,8 @@ pub struct PatchedRef { /// package from, whose dependency sub-maps decide which dependencies /// install beside it (#1538). pub yarn_classic_block: Option, + /// The install tree of `source_file` the wiring is in ([`TreePart`]). + pub part: TreePart, } /// A hosted yarn classic pin's lock block, as lock-only VEX reads it @@ -362,9 +418,16 @@ impl PatchedRef { integrity_required, url: url.map(str::to_string), yarn_classic_block: None, + part: TreePart::File, } } + /// This ref, in install tree `part` of its file. + pub(crate) fn in_part(mut self, part: TreePart) -> Self { + self.part = part; + self + } + /// A vendored ref for the artifact `vref` names. pub fn vendored( purl: String, @@ -382,6 +445,7 @@ impl PatchedRef { integrity_required: false, url: None, yarn_classic_block: None, + part: TreePart::File, } } @@ -483,6 +547,8 @@ pub struct ResolvedElsewhere { /// which pip compares as a string /// ([`Discovery::resolved_elsewhere_as_spelled`]). pub as_spelled: bool, + /// The install tree of `file` the entry is in ([`TreePart`]). + pub part: TreePart, } impl ResolvedElsewhere { @@ -554,6 +620,8 @@ pub struct UnpatchedCopy { /// What makes the build patch that copy, when it is not the usual /// "depend on the registry release instead" (a bundled copy, #758). pub remedy: Option, + /// The install tree of `file` the entry is in ([`TreePart`]). + pub part: TreePart, } /// A lock entry that installs SOME package (at `version`, when known) under @@ -727,8 +795,9 @@ pub enum CopyTarget { /// extractor has run ([`Discovery::unattest_unwired_copies`]). #[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)] pub struct UnwiredCopy { - /// Only refs of this root-relative file; `None` = refs of any file. - pub(crate) scope: Option, + /// Only refs of this root-relative file, in a tree this tree of it + /// withholds ([`TreePart::withheld_by`]); `None` = refs of any file. + pub(crate) scope: Option<(PathBuf, TreePart)>, pub(crate) target: CopyTarget, /// Root-relative file that shows the copy. pub(crate) file: PathBuf, @@ -739,7 +808,11 @@ pub struct UnwiredCopy { impl UnwiredCopy { fn covers(&self, r: &PatchedRef) -> bool { - if self.scope.as_ref().is_some_and(|f| *f != r.source_file) { + if self + .scope + .as_ref() + .is_some_and(|(f, part)| *f != r.source_file || !r.part.withheld_by(*part)) + { return false; } match &self.target { @@ -803,8 +876,10 @@ pub struct Discovery { /// another entry of the same lock (an `npm:` alias added after the /// pin), the npm twin lock, another manager's lock /// ([`Discovery::contest_across_locks`]), npm 6's legacy mirror, a Bun - /// registry entry. Each is diagnosed [`DIAG_REF_UNATTRIBUTABLE`] and is - /// neither attested nor unwound by the management commands, but it is + /// registry entry, the project document's registry entry beside a pnpm + /// env-lockfile ref ([`Discovery::contest_within_locks`]). Each is + /// diagnosed [`DIAG_REF_UNATTRIBUTABLE`] and is neither attested nor + /// unwound by the management commands, but it is /// still a pin a live lock records: the rollout's recorded view counts /// it ([`HostedPin::recorded`]), so the re-scan that rewires the other /// copy is not capped as NEW (#1195). Validated and sorted like `refs`. @@ -1051,11 +1126,22 @@ impl Discovery { /// lock's size (#993), and every reader before then only asks whether /// some entry matches. pub(crate) fn resolved_elsewhere(&mut self, file: &str, purl: Option) { + self.resolved_elsewhere_in(file, TreePart::File, purl); + } + + /// [`Discovery::resolved_elsewhere`] in install tree `part` of `file`. + pub(crate) fn resolved_elsewhere_in( + &mut self, + file: &str, + part: TreePart, + purl: Option, + ) { let Some(purl) = purl else { return }; self.elsewhere.push(ResolvedElsewhere { purl: canonical_base_purl(&purl), file: PathBuf::from(file), as_spelled: false, + part, }); } @@ -1068,6 +1154,7 @@ impl Discovery { purl: canonical_base_purl(&purl), file: PathBuf::from(file), as_spelled: true, + part: TreePart::File, }); } @@ -1087,6 +1174,7 @@ impl Discovery { purl, file: PathBuf::from(file), as_spelled: false, + part: TreePart::File, }); } @@ -1104,11 +1192,23 @@ impl Discovery { purl: Option, key: &str, how: &str, + ) { + self.unpatched_copy_in(file, TreePart::File, purl, key, how); + } + + /// [`Discovery::unpatched_copy`] in install tree `part` of `file`. + pub(crate) fn unpatched_copy_in( + &mut self, + file: &str, + part: TreePart, + purl: Option, + key: &str, + how: &str, ) { let Some(purl) = purl else { return; }; - self.resolved_elsewhere(file, Some(purl.clone())); + self.resolved_elsewhere_in(file, part, Some(purl.clone())); // Deduplicated once, in `finalize`: a per-push scan is quadratic // over a lock with thousands of registry blocks. self.unpatched_copies.push(UnpatchedCopy { @@ -1117,6 +1217,7 @@ impl Discovery { key: key.to_string(), how: how.to_string(), remedy: None, + part, }); } @@ -1168,7 +1269,10 @@ impl Discovery { let purls: BTreeSet<&str> = self .refs .iter() - .filter(|r| r.source_file == u.file) + // A file-level copy, in the project tree: it withholds a + // pnpm env-lockfile ref of the same file too + // ([`TreePart::withheld_by`]). + .filter(|r| r.source_file == u.file && r.part.withheld_by(TreePart::File)) .filter(|r| match &u.version { Some(v) => r.purl.rsplit_once('@').is_some_and(|(_, rv)| rv == v), None => true, @@ -1187,6 +1291,7 @@ impl Discovery { key: u.key.clone(), how: u.how.clone(), remedy: None, + part: TreePart::File, }); } } @@ -1199,9 +1304,17 @@ impl Discovery { /// that copy whatever the wiring does, so the ref is diagnosed /// ([`DIAG_REF_UNATTRIBUTABLE`], naming the entry) and not emitted, but /// [`Discovery::shadowed`]. Its uuid stays recognized (rule 11). + /// + /// A pnpm env-lockfile ref ([`TreePart::PnpmEnv`]) is withheld by the + /// project document's copies too ([`TreePart::withheld_by`]): a `file:` + /// copy shadows it as above, and a registry entry of the same release + /// moves it to [`Discovery::rewirable`] (a re-run rewires that entry, + /// and the project ref then stands in its place, see + /// [`Discovery::drop_env_twins`]). fn contest_within_locks(&mut self) { + self.drop_env_twins(); self.expand_unidentified_copies(); - if self.unpatched_copies.is_empty() { + if self.unpatched_copies.is_empty() && self.refs.iter().all(|r| r.part == TreePart::File) { return; } let refs = std::mem::take(&mut self.refs); @@ -1209,26 +1322,53 @@ impl Discovery { let copy = self .unpatched_copies .iter() - .find(|c| c.purl == r.purl && c.file == r.source_file) + .find(|c| c.purl == r.purl && c.file == r.source_file && r.part.withheld_by(c.part)) .cloned(); - match copy { - Some(c) => { - let file = r.source_file.to_string_lossy().into_owned(); - self.diag( - DIAG_REF_UNATTRIBUTABLE, - &file, - format!( - "{file}: {} is wired to Socket patch {} but lock entry `{}` {}; \ - that copy stays UNPATCHED and nothing is attested", - r.purl, r.uuid, c.key, c.how - ), - ); - // Withheld, not lost: rollback / remove / the takeover - // still unwind the wiring (#828). - self.shadow(r); - } - None => self.refs.push(r), + if let Some(c) = copy { + let file = r.source_file.to_string_lossy().into_owned(); + let label = r.part.label(&file); + let entry = if c.part == r.part { + "" + } else { + "the project document's " + }; + self.diag( + DIAG_REF_UNATTRIBUTABLE, + &file, + format!( + "{label}: {} is wired to Socket patch {} but {entry}lock entry `{}` {}; \ + that copy stays UNPATCHED and nothing is attested", + r.purl, r.uuid, c.key, c.how + ), + ); + // Withheld, not lost: rollback / remove / the takeover + // still unwind the wiring (#828). + self.shadow(r); + continue; + } + let registry = r.part == TreePart::PnpmEnv + && self.elsewhere.iter().any(|e| { + e.file == r.source_file && e.part == TreePart::File && e.covers(&r.purl) + }); + if registry { + let file = r.source_file.to_string_lossy().into_owned(); + let label = r.part.label(&file); + self.diag( + DIAG_REF_UNATTRIBUTABLE, + &file, + format!( + "{label}: {} is wired to Socket patch {}, but the project document of \ + the same lock resolves that version from elsewhere (not a Socket \ + patch); a VEX statement covers the release, not one install tree, so \ + the patch is not attested while `node_modules` installs the unpatched \ + copy — re-run `socket-patch scan` / `vendor` to rewire it too", + r.purl, r.uuid + ), + ); + self.withhold_rewirable(r); + continue; } + self.refs.push(r); } } @@ -1245,14 +1385,17 @@ impl Discovery { /// 11). This generalizes the npm extractor's shrinkwrap/package-lock /// rule to every pair of locks. PEP 723 script locks (`*.py.lock`) /// neither contest nor are contested: each is scoped to its own script's - /// install. Files of one install tree ([`Discovery::install_trees`]) are - /// one lock here: pip resolves a direct reference and a compatible `==` - /// pin of the same version in `requirements.txt` and its `-r` includes - /// to the reference (#1086). + /// install. Neither is a pnpm env lockfile document + /// ([`TreePart::PnpmEnv`]): pnpm installs its config dependencies into + /// its config store whatever else runs, and its wiring exempts no + /// other tree. Files of one install tree ([`Discovery::install_trees`]) + /// are one lock here: pip resolves a direct reference and a compatible + /// `==` pin of the same version in `requirements.txt` and its `-r` + /// includes to the reference (#1086). fn contest_across_locks(&mut self) { // The purls each install tree wires. let mut wiring: BTreeMap> = BTreeMap::new(); - for r in &self.refs { + for r in self.refs.iter().filter(|r| r.part == TreePart::File) { wiring .entry(self.tree_of(&r.source_file).to_path_buf()) .or_default() @@ -1270,7 +1413,7 @@ impl Discovery { let mut contested = Vec::new(); let refs = std::mem::take(&mut self.refs); for r in refs { - let other = (!is_script_lock(&r.source_file)) + let other = (!is_script_lock(&r.source_file) && r.part == TreePart::File) .then(|| { let tree = self.tree_of(&r.source_file); // Only a PyPI ref can match another spelling of its @@ -1289,9 +1432,13 @@ impl Discovery { .find(|e| { let other = self.tree_of(&e.file); // A tree that wires the release itself, in any - // spelling, is no contester. + // spelling, is no contester. A pnpm env + // lockfile installs config dependencies apart + // from every `node_modules`: it contests no + // other tree. e.covers(&r.purl) && other != tree + && e.part == TreePart::File && !wiring.get(other).is_some_and(|purls| { purls.contains(&r.purl) || (pypi @@ -1347,6 +1494,52 @@ impl Discovery { } } + /// Drop a pnpm env-lockfile ref ([`TreePart::PnpmEnv`]) that the same + /// file's project document also wires (the same `(purl, uuid, mode)`): + /// one wiring of the package in that file, whose project instance is + /// the one an installed `node_modules` copy is verified against, even + /// when the two pins differ. Runs before any contest + /// ([`Discovery::contest_within_locks`]), so the twin leaves no + /// diagnostic or [`Unattested`] mark (keyed by `(purl, uuid)`, which + /// would withhold the project ref) behind, and a withheld project ref + /// never leaves it attested in its place. The project twin counts + /// wherever an extractor put it (`refs`, `shadowed` or `rewirable`). + fn drop_env_twins(&mut self) { + let lists = [&self.refs, &self.shadowed, &self.rewirable]; + if !lists + .iter() + .copied() + .flatten() + .any(|r| r.part == TreePart::PnpmEnv) + { + return; + } + let project: HashSet<(PathBuf, String, String, WiringMode)> = lists + .into_iter() + .flatten() + .filter(|r| r.part == TreePart::File) + .map(|r| { + ( + r.source_file.clone(), + r.purl.clone(), + r.uuid.clone(), + r.mode, + ) + }) + .collect(); + for refs in [&mut self.refs, &mut self.shadowed, &mut self.rewirable] { + refs.retain(|r| { + r.part == TreePart::File + || !project.contains(&( + r.source_file.clone(), + r.purl.clone(), + r.uuid.clone(), + r.mode, + )) + }); + } + } + /// The install tree `file` belongs to: its tree's root file, or itself. fn tree_of<'a>(&'a self, file: &'a Path) -> &'a Path { self.install_trees @@ -1514,11 +1707,12 @@ impl Discovery { self.contested.dedup(); for refs in [&mut self.refs, &mut self.shadowed, &mut self.rewirable] { refs.sort_by(|a, b| { - (&a.source_file, &a.purl, &a.uuid, a.mode).cmp(&( + (&a.source_file, &a.purl, &a.uuid, a.mode, a.part).cmp(&( &b.source_file, &b.purl, &b.uuid, b.mode, + b.part, )) }); refs.dedup(); @@ -4603,7 +4797,7 @@ mod tests { /// ARCHITECTURE GUARD (rule 12): the npm-family extractors read their /// locks through the entry models the lock inventory shares - /// (`classic_entries` / `berry_entries`, `pnpm_packages`, + /// (`classic_entries` / `berry_entries`, `PnpmLock::packages`, /// `bun_text_entries`, `BunLockb::parse_packages`), never through the /// grammar primitives those models wrap, nor a second parser of the /// same lock (a JSON read of `bun.lock`) — so discovery and the diff --git a/crates/socket-patch-core/src/vex/discover/npm.rs b/crates/socket-patch-core/src/vex/discover/npm.rs index 7ea719b9b..5a5c58a0e 100644 --- a/crates/socket-patch-core/src/vex/discover/npm.rs +++ b/crates/socket-patch-core/src/vex/discover/npm.rs @@ -60,13 +60,14 @@ use serde_json::Value; use super::{ npm_purl, npm_vendored_tarball_names, parse_json, vendor_ref, CopyTarget, DiscoverCtx, - Discovery, LocateOpts, Located, PatchedRef, UnattestedKind, UnwiredCopy, VendorRef, + Discovery, LocateOpts, Located, PatchedRef, TreePart, UnattestedKind, UnwiredCopy, VendorRef, DIAG_LOCKFILE_UNPARSEABLE, DIAG_REF_INVALID, DIAG_REF_UNATTRIBUTABLE, }; use crate::constants::npm_family::{NPM_LOCKS, PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY}; +use crate::formats::pnpm::plan::governing_pnpm_lock; use crate::formats::pnpm::{ - classify_pnpm_key, entry_bundled, entry_field, pnpm_registry_key, Bundled, PnpmKey, PnpmLock, - PnpmPackage, + classify_pnpm_key, entry_bundled, entry_field, pnpm_registry_key, Bundled, PnpmDocument, + PnpmKey, PnpmLock, PnpmPackage, }; use crate::formats::text::strip_bom; use crate::patch::redirect::NpmUserPatches; @@ -354,7 +355,7 @@ pub(super) fn mark_user_patched( how }; out.unwired_copy(UnwiredCopy { - scope: Some(r.source_file.clone()), + scope: Some((r.source_file.clone(), r.part)), target: CopyTarget::Ref { purl: r.purl.clone(), uuid: r.uuid.clone(), @@ -735,14 +736,13 @@ fn entry_ref( /// ## Files /// /// * `pnpm-lock.yaml` at the root. -/// * `shrinkwrap.yaml` (pnpm 1 / 2) ONLY when there is no `pnpm-lock.yaml`: -/// pnpm >= 3 never reads it (it migrates it to `pnpm-lock.yaml` once), and -/// pnpm <= 2 never writes `pnpm-lock.yaml`, so a pair means one of them is -/// migration debris — the same call `lock_inventory` makes (it reads -/// shrinkwrap.yaml only on `vendor_lockfile_missing`). The hosted rewriter -/// edits both when both exist, so the live one is covered either way; a -/// stale shrinkwrap left carrying a Socket url must not attest from the -/// lockfile basis after `pnpm-lock.yaml` was reverted. +/// * `shrinkwrap.yaml` (pnpm 1 / 2) ONLY when there is no `pnpm-lock.yaml`, +/// or the project pins pnpm 1 / 2: pnpm >= 3 never reads it (it migrates +/// it to `pnpm-lock.yaml` once), and pnpm <= 2 reads nothing else, so a +/// pair means one of them is migration debris. The choice is the hosted +/// planner's (`formats::pnpm::plan::governing_pnpm_lock`, N11): the +/// other file is never rewritten, and a stale Socket url left in it must +/// not attest from the lockfile basis. /// * Rush (`rush.json` present): `common/config/rush/pnpm-lock.yaml` and every /// `common/config/subspaces//pnpm-lock.yaml` — the only non-root /// locks in scope, exactly the set `scan --mode hosted` collects and @@ -750,6 +750,16 @@ fn entry_ref( /// exists, like the rewriter (no precedence between files — rule 10). /// Vendoring refuses Rush (`npm_flavor`), so these only carry hosted refs. /// +/// Every document of a lock is read. pnpm 11+'s env lockfile document +/// (config dependencies, the pinned package manager; identified by its +/// importers, [`PnpmLock::packages`]) is an install tree of its own +/// ([`TreePart::PnpmEnv`]): its refs are emitted, and they never contest +/// or are contested by another lock, nor contest the project document's. +/// The project document's registry entry, `file:` copy or bundled copy of +/// the same `name@version` withholds an env ref, though (owner decision on +/// #2009, [`TreePart::withheld_by`]): a VEX statement covers the release, +/// not one install tree. +/// /// With pnpm's `gitBranchLockfile` on and a `pnpm-lock..yaml` /// present (#556), every ref of the root and member locks is marked /// [`UnattestedKind::PnpmGitBranchLock`] ([`unattest_git_branch_lock`]). @@ -810,17 +820,25 @@ async fn extract_pnpm(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { // The root and member locks read (Rush's are not: its pnpm runs with a // workspace of its own). let mut read: Vec = Vec::new(); - let root_lock = ctx.exists(PNPM_LOCK).await; - if root_lock { - extract_pnpm_lock(ctx, PNPM_LOCK, disabled, out).await; - // pnpm never reads the debris shrinkwrap: the patches it still names - // are recognized as NOT wired (module docs, rule 11), so a ledger - // record its stale Socket url alone keeps "mentioned" is dead. - ctx.recognize_ignored(PNPM_SHRINKWRAP_LEGACY).await; - read.push(PNPM_LOCK.to_string()); - } else { - extract_pnpm_lock(ctx, PNPM_SHRINKWRAP_LEGACY, disabled, out).await; - read.push(PNPM_SHRINKWRAP_LEGACY.to_string()); + // The lock pnpm reads, as the hosted planner picks it + // (`formats::pnpm::plan::governing_pnpm_lock`, N11). pnpm never reads + // the other one: the patches it still names are recognized as NOT + // wired (module docs, rule 11), so a ledger record its stale Socket + // url alone keeps "mentioned" is dead. + let (lock, shrinkwrap) = (PNPM_LOCK, PNPM_SHRINKWRAP_LEGACY); + let both = (ctx.exists(lock).await, ctx.exists(shrinkwrap).await); + // The pin decides only between the two, so it is read only then. + let pinned = match both { + (true, true) => crate::utils::pnpm_workspace::pinned_pnpm_major_in(&ctx.view).await, + _ => None, + }; + if let Some(governing) = governing_pnpm_lock(both.0, both.1, pinned) { + extract_pnpm_lock(ctx, governing, disabled, out).await; + read.push(governing.to_string()); + if both == (true, true) { + let shadowed = if governing == lock { shrinkwrap } else { lock }; + ctx.recognize_ignored(shadowed).await; + } } if rush { // The common lock, then each subspace's, sorted for deterministic @@ -882,7 +900,7 @@ async fn extract_pnpm(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { /// ruled out. See [`unattest_git_branch_lock`]. fn unattest_unlisted_branch_locks(file: &str, why: &str, out: &mut Discovery) { out.unwired_copy(UnwiredCopy { - scope: Some(PathBuf::from(file)), + scope: Some((PathBuf::from(file), TreePart::File)), target: CopyTarget::Any, file: PathBuf::from(file), detail: format!( @@ -904,7 +922,9 @@ fn unattest_unlisted_branch_locks(file: &str, why: &str, out: &mut Discovery) { /// the wiring in `file` is intact (rollback, remove and list still manage /// it, and the ledgers keep it live), and no `scan` / `vendor` re-run can /// rewire the branch lock. The mark names the branch lock in `file`'s own -/// directory when there is one. +/// directory when there is one. A branch lock replaces the whole file, so +/// the mark is the project tree's, which covers the env lockfile +/// document's refs too ([`TreePart::withheld_by`]). fn unattest_git_branch_lock(file: &str, branch_locks: &[String], out: &mut Discovery) { let dir = Path::new(file).parent().unwrap_or(Path::new("")); let shown = branch_locks @@ -913,7 +933,7 @@ fn unattest_git_branch_lock(file: &str, branch_locks: &[String], out: &mut Disco .or(branch_locks.first()) .map_or(file, String::as_str); out.unwired_copy(UnwiredCopy { - scope: Some(PathBuf::from(file)), + scope: Some((PathBuf::from(file), TreePart::File)), target: CopyTarget::Any, file: PathBuf::from(shown), detail: format!( @@ -950,22 +970,63 @@ async fn extract_pnpm_lock( ); return; } + // Each document is its own install tree: pnpm 11+'s env lockfile is + // read as [`TreePart::PnpmEnv`], so its entries never contest the + // project document's refs (the project document's copies do withhold + // its refs, [`TreePart::withheld_by`]). + for document in lock.documents() { + let tree = PnpmTree::of(file, document); + extract_pnpm_document(ctx, &tree, &lock, document, disabled, out).await; + } +} + +/// One install tree of a pnpm lock: the lock's root-relative `file`, the +/// [`TreePart`] its records carry, and the `label` its diagnostics name it +/// by (` (env lockfile)` for the env document). +struct PnpmTree<'a> { + file: &'a str, + part: TreePart, + label: String, +} + +impl<'a> PnpmTree<'a> { + fn of(file: &'a str, document: PnpmDocument) -> Self { + let part = TreePart::from(document); + PnpmTree { + file, + part, + label: part.label(file), + } + } +} + +/// The refs and copies of one document of a pnpm lock, recorded in its +/// install tree (see [`extract_pnpm_lock`]). +async fn extract_pnpm_document( + ctx: &DiscoverCtx<'_>, + tree: &PnpmTree<'_>, + lock: &PnpmLock<'_>, + document: PnpmDocument, + disabled: Option<&str>, + out: &mut Discovery, +) { let mut copies: Vec = Vec::new(); let mut bundles: Vec<(&str, Bundled<'_>)> = Vec::new(); let mut withheld: BTreeSet = BTreeSet::new(); - for package in lock.packages() { + for package in lock.packages_of(document) { let withhold = disabled.map(|_| &mut withheld); - pnpm_entry_ref(ctx, file, package, &mut copies, withhold, out); + pnpm_entry_ref(ctx, tree, package, &mut copies, withhold, out); bundles.extend(entry_bundled(&package.entry).map(|b| (package.key, b))); } - record_pnpm_file_copies(ctx, file, copies, out).await; - record_pnpm_bundled_copies(file, bundles, out); + record_pnpm_file_copies(ctx, tree, copies, out).await; + record_pnpm_bundled_copies(tree, bundles, out); + let label = &tree.label; if let (Some(source), false) = (disabled, withheld.is_empty()) { out.diag( DIAG_REF_UNATTRIBUTABLE, - file, + tree.file, format!( - "{file}: {}, so its hosted pins of {} are not attested from the lock alone, \ + "{label}: {}, so its hosted pins of {} are not attested from the lock alone, \ only from an installed tree whose every copy verifies; \ `socket-patch scan --mode vendored` wires them without a lock", crate::utils::pnpm_workspace::describe_lockfile_disabled(source), @@ -991,7 +1052,12 @@ async fn extract_pnpm_lock( /// stay refs: the wiring is intact and no rewire could clear the bundled /// copy, so the ledgers' liveness gates (`vendor --check`, `scan`) keep /// seeing them live, and the copy is no cross-lock evidence either. -fn record_pnpm_bundled_copies(file: &str, bundles: Vec<(&str, Bundled<'_>)>, out: &mut Discovery) { +fn record_pnpm_bundled_copies( + tree: &PnpmTree<'_>, + bundles: Vec<(&str, Bundled<'_>)>, + out: &mut Discovery, +) { + let (file, label) = (tree.file, &tree.label); for (parent, bundled) in bundles { let (targets, what) = match bundled { Bundled::All => ( @@ -1008,11 +1074,11 @@ fn record_pnpm_bundled_copies(file: &str, bundles: Vec<(&str, Bundled<'_>)>, out }; for target in targets { out.unwired_copy(UnwiredCopy { - scope: Some(PathBuf::from(file)), + scope: Some((PathBuf::from(file), tree.part)), target, file: PathBuf::from(file), detail: format!( - "{file} entry `{parent}` {what}, which pnpm unpacks from that package's \ + "{label} entry `{parent}` {what}, which pnpm unpacks from that package's \ own tarball without locking its version, so no Socket wiring reaches it" ), kind: UnattestedKind::BundledCopy, @@ -1042,11 +1108,11 @@ struct PnpmFileCopy { /// read is left alone. async fn record_pnpm_file_copies( ctx: &DiscoverCtx<'_>, - file: &str, + tree: &PnpmTree<'_>, copies: Vec, out: &mut Discovery, ) { - let lock_dir = std::path::Path::new(file) + let lock_dir = std::path::Path::new(tree.file) .parent() .map(|d| d.to_string_lossy().replace('\\', "/")) .unwrap_or_default(); @@ -1096,8 +1162,9 @@ async fn record_pnpm_file_copies( } else { "tarball" }; - out.unpatched_copy( - file, + out.unpatched_copy_in( + tree.file, + tree.part, npm_purl(&name, &version), ©.key, &format!( @@ -1127,7 +1194,8 @@ pub(crate) async fn pnpm_lock_file_copies( let ctx = DiscoverCtx::with_origins(root, &[]); let lock = PnpmLock::parse(text); let mut copies: Vec = Vec::new(); - for package in lock.packages() { + // The vendored backend wires the project document alone. + for package in lock.packages_of(PnpmDocument::Project) { // As `pnpm_entry_ref` classifies them: an entry without a tarball // is a directory copy; one with a tarball is a copy unless it is a // Socket wiring (or one discovery rejects as such). @@ -1153,7 +1221,8 @@ pub(crate) async fn pnpm_lock_file_copies( })); } let mut scratch = Discovery::default(); - record_pnpm_file_copies(&ctx, file, copies, &mut scratch).await; + let tree = PnpmTree::of(file, PnpmDocument::Project); + record_pnpm_file_copies(&ctx, &tree, copies, &mut scratch).await; let purl = crate::utils::purl_key::canonical_base_purl(&purl); scratch .unpatched_copies @@ -1185,12 +1254,13 @@ fn pnpm_file_copy(package: &PnpmPackage<'_>, directory: bool) -> Option, - file: &str, + tree: &PnpmTree<'_>, package: &PnpmPackage<'_>, copies: &mut Vec, withheld: Option<&mut BTreeSet>, out: &mut Discovery, ) { + let (file, label) = (tree.file, &tree.label); let key = package.key; let Some(resolution) = &package.resolution else { if resolution_is_socket_shaped(ctx, package) { @@ -1198,7 +1268,7 @@ fn pnpm_entry_ref( DIAG_REF_INVALID, file, format!( - "{file}: packages entry {key:?} references a Socket patch through a \ + "{label}: packages entry {key:?} references a Socket patch through a \ resolution that is not a flat pnpm resolution map; it is ignored" ), ); @@ -1207,7 +1277,7 @@ fn pnpm_entry_ref( }; let Some(tarball) = resolution.tarball() else { // A plain registry entry (integrity only) or a directory/git dep. - out.resolved_elsewhere(file, pnpm_registry_key_purl(key)); + out.resolved_elsewhere_in(file, tree.part, pnpm_registry_key_purl(key)); copies.extend(pnpm_file_copy(package, true)); return; }; @@ -1217,14 +1287,14 @@ fn pnpm_entry_ref( .map(|sri| LockIntegrity::Sri(sri.to_string())); let located = ctx.locate(tarball, LocateOpts::LITERAL_CHECKED); if let Some(vref) = located.vendored { - pnpm_vendored_ref(file, package, tarball, &vref, integrity, out); + pnpm_vendored_ref(tree, package, tarball, &vref, integrity, out); } else if located.decorated_leaf { // pnpm fetches the literal file (`#` / `::` included). out.diag( DIAG_REF_INVALID, file, format!( - "{file}: packages entry {key:?} (tarball {tarball:?}) is not a literal \ + "{label}: packages entry {key:?} (tarball {tarball:?}) is not a literal \ .socket/vendor/npm// path; it is ignored" ), ); @@ -1234,7 +1304,7 @@ fn pnpm_entry_ref( DIAG_REF_INVALID, file, format!( - "{file}: packages entry {key:?} resolves from the Socket patch server but \ + "{label}: packages entry {key:?} resolves from the Socket patch server but \ its key is not a registry name@version" ), ); @@ -1248,25 +1318,20 @@ fn pnpm_entry_ref( } None => integrity, }; - out.push(PatchedRef::hosted( - purl, - uuid, - file, - Some(tarball), - integrity, - true, - )); + out.push( + PatchedRef::hosted(purl, uuid, file, Some(tarball), integrity, true).in_part(tree.part), + ); } else { // A registry-keyed entry fetching some other tarball, or a user's // `file:` tarball. - out.resolved_elsewhere(file, pnpm_registry_key_purl(key)); + out.resolved_elsewhere_in(file, tree.part, pnpm_registry_key_purl(key)); copies.extend(pnpm_file_copy(package, false)); } } /// A vendored packages entry (see [`extract_pnpm`] for the key shapes). fn pnpm_vendored_ref( - file: &str, + tree: &PnpmTree<'_>, package: &PnpmPackage<'_>, tarball: &str, vref: &VendorRef, @@ -1277,8 +1342,11 @@ fn pnpm_vendored_ref( let invalid = |out: &mut Discovery, why: &str| { out.diag( DIAG_REF_INVALID, - file, - format!("{file}: packages entry {key:?} (tarball {tarball:?}) {why}; it is ignored"), + tree.file, + format!( + "{}: packages entry {key:?} (tarball {tarball:?}) {why}; it is ignored", + tree.label + ), ); }; // (name, the key's own `file:` path if the key was rekeyed). @@ -1289,7 +1357,7 @@ fn pnpm_vendored_ref( PnpmKey::V9File { name, path } => (Some(name), Some(path)), // Registry-form key with a hand-pointed `file:` tarball. PnpmKey::Registry { name, version } => { - return finish_vendored(file, name, version, vref, integrity, invalid, out) + return finish_vendored(tree, name, version, vref, integrity, invalid, out) } PnpmKey::Other => return invalid(out, "has no registry name@version key"), }; @@ -1304,11 +1372,11 @@ fn pnpm_vendored_ref( let Some(version) = entry_field(&package.entry, "version") else { return invalid(out, "has no (single) `version:` line"); }; - finish_vendored(file, name, version, vref, integrity, invalid, out) + finish_vendored(tree, name, version, vref, integrity, invalid, out) } fn finish_vendored( - file: &str, + tree: &PnpmTree<'_>, name: &str, version: &str, vref: &VendorRef, @@ -1325,7 +1393,7 @@ fn finish_vendored( &format!("names {purl} but its tarball is not that package's vendored npm tarball"), ); } - out.push(PatchedRef::vendored(purl, vref, file, integrity)); + out.push(PatchedRef::vendored(purl, vref, tree.file, integrity).in_part(tree.part)); } /// [`pnpm_registry_key`] (the lock inventory's key grammar) as a validated @@ -3359,6 +3427,290 @@ mod tests { assert_eq!(out.unattested.len(), 1, "vendored: {:#?}", out.unattested); } + /// pnpm 11+'s env lockfile document is read as an install tree of its + /// own (divergence N4): a Socket wiring of a config dependency is a + /// ref, but one the project document of the same lock withholds when it + /// resolves the same release from the registry (rewirable), a `file:` + /// copy (shadowed) or a bundled copy (unattested): a VEX statement + /// covers the release, not one tree (owner decision). The env document's + /// entries never contest the project document's refs; its wiring does + /// not exempt the project document from contesting another lock; and + /// it contests no other lock's ref. + #[tokio::test] + async fn pnpm_env_document_is_its_own_install_tree() { + let url = hosted_url("npm", "left-pad", "1.3.0", UUID_A, "left-pad-1.3.0.tgz"); + let wired = + format!(" left-pad@1.3.0:\n resolution: {{integrity: {SRI}, tarball: {url}}}\n\n"); + let registry = " left-pad@1.3.0:\n resolution: {integrity: sha512-UPSTREAM==}\n\n"; + let env = |packages: &str| { + format!( + "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n configDependencies:\n \ + left-pad:\n specifier: 1.3.0\n version: 1.3.0\n\npackages:\n\n{packages}" + ) + }; + let project = |packages: &str| { + format!( + "---\nlockfileVersion: '9.0'\n\nimporters:\n\n .:\n dependencies:\n \ + left-pad:\n specifier: 1.3.0\n version: 1.3.0\n\npackages:\n\n{packages}" + ) + }; + let hosted = [("pkg:npm/left-pad@1.3.0", UUID_A, WiringMode::Hosted)]; + let file_copy = " left-pad@file:forks/left-pad:\n \ + resolution: {directory: forks/left-pad, type: directory}\n\n"; + let bundle = " host-pkg@1.0.0:\n resolution: {integrity: sha512-HOST==}\n \ + bundledDependencies: true\n\n"; + + // Wired in the env document, the project document resolving the + // same release elsewhere: never attested. A registry entry a + // re-run can rewire withholds it as rewirable, a `file:` copy + // shadows it, and a bundled copy (whose version the lock does not + // record) leaves it a ref marked unattested. + #[derive(Debug, PartialEq)] + enum Withheld { + Rewirable, + Shadowed, + Unattested, + } + for (case, project_packages, how) in [ + ("registry", registry.to_string(), Withheld::Rewirable), + ("file: copy", file_copy.to_string(), Withheld::Shadowed), + ( + "registry + file: copy", + format!("{registry}{file_copy}"), + Withheld::Shadowed, + ), + ("bundled copy", bundle.to_string(), Withheld::Unattested), + ( + "registry + bundled copy", + format!("{registry}{bundle}"), + Withheld::Rewirable, + ), + ] { + for crlf in [false, true] { + let p = Project::new(); + let text = format!("{}{}", env(&wired), project(&project_packages)); + let text = if crlf { + text.replace('\n', "\r\n") + } else { + text + }; + p.write("pnpm-lock.yaml", text); + p.write( + "forks/left-pad/package.json", + r#"{"name":"left-pad","version":"1.3.0"}"#, + ); + let out = p.discover().await; + let case = format!("{case} (crlf: {crlf})"); + let withheld = match how { + Withheld::Rewirable => &out.rewirable, + Withheld::Shadowed => &out.shadowed, + Withheld::Unattested => &out.refs, + }; + assert_eq!(withheld.len(), 1, "{case}: {out:#?}"); + // The real lock path, with the tree carried beside it. + assert_eq!( + ( + withheld[0].purl.as_str(), + withheld[0].source_file.as_path(), + withheld[0].part + ), + ( + "pkg:npm/left-pad@1.3.0", + std::path::Path::new("pnpm-lock.yaml"), + TreePart::PnpmEnv + ), + "{case}" + ); + assert!(out.contested.is_empty(), "{case}: {:#?}", out.contested); + if how == Withheld::Unattested { + // The ref stays (no re-run clears a bundled copy), but + // the VEX plan omits it. + assert_eq!(out.unattested.len(), 1, "{case}: {:#?}", out.unattested); + let u = &out.unattested[0]; + assert_eq!( + (u.purl.as_str(), u.uuid.as_str(), u.kind), + ( + "pkg:npm/left-pad@1.3.0", + UUID_A, + UnattestedKind::BundledCopy + ), + "{case}" + ); + } else { + assert!(out.refs.is_empty(), "{case}: {:#?}", out.refs); + assert_eq!( + out.hosted_claim("pkg:npm/left-pad@1.3.0", UUID_A), + Some(false), + "{case}" + ); + let diag = out + .diagnostics + .iter() + .find(|d| d.code == DIAG_REF_UNATTRIBUTABLE) + .unwrap_or_else(|| panic!("{case}: {:#?}", out.diagnostics)); + assert!( + diag.detail.starts_with("pnpm-lock.yaml (env lockfile): ") + && diag.detail.contains("project document"), + "{case}: {}", + diag.detail + ); + } + } + } + + // Wired in the env document, the project document resolving + // another version from the registry: a ref, attested. + let p = Project::new(); + p.write( + "pnpm-lock.yaml", + format!( + "{}{}", + env(&wired), + project(®istry.replace("1.3.0", "1.2.0")) + ), + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert_eq!(out.refs[0].part, TreePart::PnpmEnv); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + assert_eq!( + out.hosted_claim("pkg:npm/left-pad@1.3.0", UUID_A), + Some(true) + ); + + // Wired in the project document, the env document resolving the + // same release from the registry: a ref. + let p = Project::new(); + p.write( + "pnpm-lock.yaml", + format!("{}{}", env(registry), project(&wired)), + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert_eq!(out.refs[0].part, TreePart::File); + assert!(out.contested.is_empty(), "{:#?}", out.contested); + + // Wired in both documents, the pins differing: ONE ref, the project + // document's. + let env_wired = wired.replace(SRI, "sha512-ENVPIN=="); + let p = Project::new(); + p.write( + "pnpm-lock.yaml", + format!("{}{}", env(&env_wired), project(&wired)), + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert_eq!(out.refs.len(), 1, "{:#?}", out.refs); + assert_eq!(out.refs[0].part, TreePart::File); + assert_eq!( + out.refs[0].locked_integrity, + Some(LockIntegrity::Sri(SRI.into())) + ); + + // Wired in both documents, the project's ref withheld (a `file:` + // copy of the release installs beside it): its env twin is dropped + // too, never attested in the project ref's place. + let p = Project::new(); + p.write( + "pnpm-lock.yaml", + format!( + "{}{}", + env(&env_wired), + project(&format!("{wired}{file_copy}")) + ), + ); + p.write( + "forks/left-pad/package.json", + r#"{"name":"left-pad","version":"1.3.0"}"#, + ); + let out = p.discover().await; + assert!(out.refs.is_empty(), "{:#?}", out.refs); + assert!( + out.shadowed + .iter() + .chain(&out.rewirable) + .all(|r| r.part == TreePart::File), + "{:#?} {:#?}", + out.shadowed, + out.rewirable + ); + assert_ne!( + out.hosted_claim("pkg:npm/left-pad@1.3.0", UUID_A), + Some(true) + ); + // The dropped twin leaves no diagnostic of its own behind. + assert!( + out.diagnostics + .iter() + .all(|d| !d.detail.contains("(env lockfile)")), + "{:#?}", + out.diagnostics + ); + + // Wired in both documents, the env document bundling a copy: its + // twin is dropped before any mark, so the project ref (whose tree + // the env document's copies never reach) is attested. + let p = Project::new(); + p.write( + "pnpm-lock.yaml", + format!( + "{}{}", + env(&format!("{env_wired}{bundle}")), + project(&wired) + ), + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert_eq!(out.refs[0].part, TreePart::File); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + + // The env document's registry entry contests no other lock's ref. + let npm_lock = format!( + r#"{{"name":"m","lockfileVersion":3,"packages":{{"":{{"name":"m"}}, + "node_modules/left-pad":{{"version":"1.3.0","resolved":"{url}","integrity":"{SRI}"}}}}}}"# + ); + let p = Project::new(); + p.write("package-lock.json", &npm_lock); + p.write( + "pnpm-lock.yaml", + format!("{}{}", env(registry), project("")), + ); + let out = p.discover().await; + assert_refs(&out, &hosted); + assert!(out.contested.is_empty(), "{:#?}", out.contested); + + // The env document's wiring does not exempt the project document, + // whose registry entry still contests package-lock.json's ref (and + // withholds the env document's own ref). + let p = Project::new(); + p.write("package-lock.json", &npm_lock); + p.write( + "pnpm-lock.yaml", + format!("{}{}", env(&wired), project(registry)), + ); + let out = p.discover().await; + assert!(out.refs.is_empty(), "{:#?}", out.refs); + assert_eq!( + out.rewirable + .iter() + .map(|r| (r.source_file.as_path(), r.part)) + .collect::>(), + [ + (std::path::Path::new("package-lock.json"), TreePart::File), + (std::path::Path::new("pnpm-lock.yaml"), TreePart::PnpmEnv), + ] + ); + assert_eq!(out.contested.len(), 1, "{:#?}", out.contested); + let c = &out.contested[0]; + assert_eq!( + (c.file.as_path(), c.other.as_path()), + ( + std::path::Path::new("package-lock.json"), + std::path::Path::new("pnpm-lock.yaml") + ) + ); + } + /// The pnpm bundled mark stays in its own lock: a same-version ref in /// another lock (a `package-lock.json` twin) is neither marked nor /// contested by it — the pnpm lock does not record the bundled version, @@ -3984,6 +4336,15 @@ mod tests { out.recognized_files(UUID_A, WiringMode::Hosted), vec![std::path::Path::new("shrinkwrap.yaml")] ); + // N11: a project that pins pnpm 1 / 2 installs from + // `shrinkwrap.yaml`, so there it is the lock that attests and the + // stray `pnpm-lock.yaml` is the shadowed one, as the hosted planner + // picks them (`plan::governing_pnpm_lock`). + with_lock.write("package.json", r#"{"packageManager": "pnpm@2.25.7"}"#); + assert_refs( + &run(&with_lock).await, + &[("pkg:npm/left-pad@1.3.0", UUID_A, WiringMode::Hosted)], + ); } /// `--patch-server-url` deployments count; the same URL without the diff --git a/crates/socket-patch-core/src/vex/discover/testing/golden.rs b/crates/socket-patch-core/src/vex/discover/testing/golden.rs index 54509be0e..589d75633 100644 --- a/crates/socket-patch-core/src/vex/discover/testing/golden.rs +++ b/crates/socket-patch-core/src/vex/discover/testing/golden.rs @@ -37,7 +37,7 @@ use std::path::{Path, PathBuf}; use serde_json::{json, Map, Value}; use crate::vex::discover::{ - ContestedRef, Diag, Discovery, PatchedRef, Recognized, ResolvedElsewhere, Unattested, + ContestedRef, Diag, Discovery, PatchedRef, Recognized, ResolvedElsewhere, TreePart, Unattested, UnlockedPin, UnpatchedCopy, WiringMode, }; @@ -150,8 +150,9 @@ fn render(out: &Discovery, root: &Path) -> Value { url, // The block's lines; the lock itself is the golden's input. yarn_classic_block: _, + part, } = r; - json!({ + let mut rendered = json!({ "purl": purl, "uuid": uuid, "mode": mode(*m), @@ -161,7 +162,13 @@ fn render(out: &Discovery, root: &Path) -> Value { "integrity_required": integrity_required, "url": url, "lockfile_basis_ok": r.lockfile_basis_ok(), - }) + }); + // Only off the default, so the goldens of single-tree files stay + // byte-identical. + if *part != TreePart::File { + rendered["part"] = json!(format!("{part:?}")); + } + rendered }; let refs: Vec = refs.iter().map(render_ref).collect(); let diagnostics: Vec = diagnostics @@ -255,6 +262,8 @@ fn render(out: &Discovery, root: &Path) -> Value { key, how, remedy, + // A pnpm env-lockfile copy contests only its own tree. + part: _, } = c; let mut copy = json!({ "purl": purl, @@ -301,6 +310,7 @@ fn render(out: &Discovery, root: &Path) -> Value { key, how, remedy, + part: _, } = c; let mut copy = json!({ "purl": purl, diff --git a/crates/socket-patch-core/src/vex/discover/user_patched.rs b/crates/socket-patch-core/src/vex/discover/user_patched.rs index f418476db..7fd7fcede 100644 --- a/crates/socket-patch-core/src/vex/discover/user_patched.rs +++ b/crates/socket-patch-core/src/vex/discover/user_patched.rs @@ -75,10 +75,12 @@ pub(crate) async fn extract(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { if index.is_empty() { return; } - // Each mark is an `UnwiredCopy` for exactly that ref, as the npm and - // Bun extractors' (`npm::mark_user_patched`): it lands only on a ref - // that survives every contest, and a ref one of those already covers - // keeps that first reason. + // Each mark is an `UnwiredCopy` for exactly that ref, in its own tree + // of its file, as the npm and Bun extractors' (`npm::mark_user_patched`): + // it lands only on a ref that survives every contest, and a ref one of + // those already covers keeps that first reason. A pnpm env-lockfile ref + // (`TreePart::PnpmEnv`) of a user-patched package is marked too: the + // index is per package, and a missed attestation is the safe side. let marks: Vec = out .refs .iter() @@ -97,7 +99,7 @@ pub(crate) async fn extract(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { sources.join(", ") ); Some(UnwiredCopy { - scope: Some(r.source_file.clone()), + scope: Some((r.source_file.clone(), r.part)), target: CopyTarget::Ref { purl: r.purl.clone(), uuid: r.uuid.clone(), diff --git a/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden b/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden index 8b8c62706..5b9361677 100644 --- a/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden +++ b/crates/socket-patch-core/tests/equivalence/pnpm_depscan_sized.golden @@ -1,3 +1,3 @@ # A depscan-sized seeded pnpm-lock.yaml set + shuffled overrides. # -0 782731128c8cd746 f3e10497dfc8fec6 +0 b120df27c737dd90 1ab3b36d03d3b3f8 diff --git a/crates/socket-patch-core/tests/pnpm_hosted.rs b/crates/socket-patch-core/tests/pnpm_hosted.rs index 33a07f3a2..0b237e870 100644 --- a/crates/socket-patch-core/tests/pnpm_hosted.rs +++ b/crates/socket-patch-core/tests/pnpm_hosted.rs @@ -134,19 +134,35 @@ fn mixed_newlines_preserve_unrelated_bytes() { roundtrip("pnpm-lock.yaml", lock, &dep("left-pad"), 1); } +/// pnpm 1's early shrinkwrap (`shrinkwrapVersion: 3`) is no durable +/// lock: a lone one refuses the dependency. Beside a `pnpm-lock.yaml` it +/// is shadowed (N11: pnpm >= 3 never reads it), so it refuses nothing and +/// only the governing lock is rewritten. #[test] -fn early_pnpm1_refuses_non_durable_shrinkwrap_across_all_locks() { +fn early_pnpm1_shrinkwrap_refuses_only_when_it_governs() { let old = "shrinkwrapVersion: 3\npackages:\n /left-pad/1.3.0:\n resolution:\n integrity: sha1-UPSTREAM=\n"; let newer = "lockfileVersion: '9.0'\npackages:\n left-pad@1.3.0:\n resolution: {integrity: sha512-UPSTREAM==}\n"; - let files = BTreeMap::from([ - ("shrinkwrap.yaml".into(), old.into()), - ("pnpm-lock.yaml".into(), newer.into()), - ]); - let result = rewrite_registry_redirect(&files, &[dep("left-pad")]); + let alone = BTreeMap::from([("shrinkwrap.yaml".into(), old.into())]); + let result = rewrite_registry_redirect(&alone, &[dep("left-pad")]); assert!(result.files.is_empty() && result.edits.is_empty()); assert!(result.refused_pnpm_uuids.contains("patch-id")); assert_eq!( result.warnings[0].code, "redirect_pnpm_legacy_lockfile_unsupported" ); + + let both = BTreeMap::from([ + ("shrinkwrap.yaml".into(), old.into()), + ("pnpm-lock.yaml".into(), newer.into()), + ]); + let result = rewrite_registry_redirect(&both, &[dep("left-pad")]); + assert!( + result.refused_pnpm_uuids.is_empty(), + "{:?}", + result.warnings + ); + assert_eq!( + result.files.keys().map(String::as_str).collect::>(), + ["pnpm-lock.yaml"] + ); } diff --git a/docs/design/lock-model-divergences.md b/docs/design/lock-model-divergences.md index 2a2003e2b..e79113a10 100644 --- a/docs/design/lock-model-divergences.md +++ b/docs/design/lock-model-divergences.md @@ -33,7 +33,7 @@ follow from the install-tree decision. N11 was decided on 2026-11-05. | J12 | scala-cli | Guidance only (`patch/redirect/scala_guidance.rs`) | Back-port: hosted wires scala-cli through the merged planner (a same-GAV Socket repository instead of a suffixed version). | | P3 | Is a shadowed `pdm.lock` rewritten? | Only when it governs (`pdm_drives`, ~564) | Rewrite it like every other tree, per the install-tree decision. | | P4 | Are `-r` includes of `requirements.txt` rewritten? | Root file only (`patch/redirect/requirements.rs` ~208) | Follow includes, per the install-tree decision. An include outside the repository root stays bounded in workspace runs: never read or rewritten (owner, 2026-11-05; #2025 (draft)). | -| N11 | Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` rewritten? | Yes; its all-or-nothing gate covers both files | **Owner decision (2026-11-05): shadowed.** Swept for stale identities, never written, and out of the all-or-nothing gate. pnpm never reads it, and an unparseable debris key in it can refuse the dependency in every lock. When the project's pnpm evidence pins pnpm 1 / 2, `shrinkwrap.yaml` governs instead and vendored mode refuses. Fixed in #2029 (draft). | +| N11 | Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` rewritten? | Yes; its all-or-nothing gate covers both files | **Resolved (owner: shadowed) in #2009**; see N11 under npm. | ## JVM @@ -84,19 +84,19 @@ follow from the install-tree decision. N11 was decided on 2026-11-05. | N1 | pnpm `gitBranchLockfile` with a branch lock present: is the root lock the install tree? | Hosted withholds the root locks and refuses (`hosted/engine.rs` ~930). Vendored refuses. Discovery and the inventory never check the setting. | Hosted. Discovery withholds pnpm root and member-lock attestations whenever any `pnpm-lock..yaml` exists, whatever the settings or environment say (owner, 2026-11-05; #2010 (draft)). | false attestation | | N2 | npm user patches (#711) and `bun patch` (#367) | Hosted skips that one dependency and records it as refused. Vendored refuses the whole npm v4 lock, or the one bun dependency. Discovery never reads `patchedDependencies` / `"patched"` and attests a pre-#711 hosted pin. | Hosted's per-dependency skip, in discovery too, for every package manager's user patches (owner decision; #2010 (draft) for npm and bun, #2018 (draft) for pnpm, yarn and vlt). The override is the existing `--allow-patch-conflicts`; `vex` never attests under a user patch, even with it. | false attestation | | N3 | A workspace member's stray npm/bun/vlt lock | Discovery treats it as absent. Hosted and vendored refuse. The inventory reads it as the project's lock. | The inventory ignores it | | -| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree; documents identified by content, not position. Fixed in #2009 (draft). | | -| N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere. The vendored writers, reverters and GC claim only this project's vendor paths in #2025 (draft); the pnpm readers and planner follow in #2009 / #2029 (draft). | revert could rewrite another project's wiring | +| N4 | pnpm 11's env-lockfile document | See [Decisions](lock-model-unification.md#decisions-under-the-hosted-rule). `main_document` takes the text after the last `---`, so a crafted multi-document lock chooses its own "project". | Its own tree. **Resolved in #2009**: both documents are read through `PnpmLock`; the env document (the first, when its importers carry only `configDependencies` / `packageManagerDependencies`) is its own install tree (`TreePart::PnpmEnv`, see its docs), and the project document's copies of a release withhold an env ref of it (owner decision: a VEX statement covers the release, not one tree). | | +| N5 | What counts as Socket wiring | Discovery and the unwind use the allowlist. The inventory records hosted URLs on any host. `parse_vendor_path` accepts `.socket/vendor/` anywhere, so a `../.socket/vendor/…` entry from a sibling checkout is "ours" to the vendored writer and reverter. The hosted pnpm "vendored?" check is a substring heuristic. | The `origin` classifier everywhere. The vendored writers, reverters and GC claim only this project's vendor paths in #2025 (draft). pnpm part **resolved in #2009**: both modes read pnpm keys with one rule, `formats::pnpm::plan::match_key` (`origin::vendor_ref` plus the artifact leaf), so neither takes a sibling checkout's rekey for its own. The vendored reverter still finds its wiring with `parse_vendor_path` (values such as pnpm 7/8's absolute specifier need it). | revert could rewrite another project's wiring | | N6 | `npm-shrinkwrap.json` and `package-lock.json` | Hosted, vendored and discovery read both, with a contest. The inventory reads the shrinkwrap only. | Both, everywhere | | | N7 | npm lockfileVersion | Hosted rewrites v1 and v4. Vendored accepts v2/v3 only. The inventory ignores v1. Discovery reads v1 and v2+. | Hosted, once vendored `file:` wiring under npm 6 and npm 12.1 v4 has fixtures | | | N8 | npm `file:` directory link identity (#1481) | Hosted and discovery give a versionless link its source directory's `name@version`. Vendored matches the entry's own version, wiring the registry copies with no warning. | Hosted | silent vendored success | | N9 | yarn classic blocks reached only through an alias | Hosted skips them. Vendored rewrites them. Discovery counts an alias block as an install of the real package. | Back-port: hosted rewrites alias blocks (vendored supports what hosted lacks) | | | N10 | Locks of several npm-family package managers in one project | Hosted rewrites every family; discovery reads every family with the contest. The inventory and vendored pick one family (vlt > bun > pnpm > yarn > npm). | Union (separate trees) | | -| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | Shadowed (owner, 2026-11-05; see above). Fixed in #2029 (draft). | | -| N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. Fixed in #2009 (draft). | target-specific (vendored) | +| N11 | `shrinkwrap.yaml` beside `pnpm-lock.yaml` | Hosted reads and splices it, and its all-or-nothing gate covers both. Discovery sweeps it as ignored. The inventory reads it only as a fallback. Vendored never reads it. | **Resolved (owner: shadowed) in #2009**: hosted never rewrites it and leaves it out of the gate; discovery keeps sweeping it; vendored never reads it. A pnpm 1 / 2 pin flips the pair: `shrinkwrap.yaml` governs, `pnpm-lock.yaml` is shadowed, and vendored mode refuses the project (`vendor_lockfile_version_unsupported`). A lone `shrinkwrap.yaml` keeps today's behavior. | | +| N12 | Rush common and subspace locks | Hosted and discovery always read them. The inventory reads them only when there is no root lock (#2003 aligns the listing itself). Vendored refuses Rush. | Always read. Vendored's refusal stays. **Resolved in #2009** (the inventory reads them whenever `rush.json` exists). | target-specific (vendored) | | N13 | Plug'n'Play | Vendored refuses. The hosted berry preflight has no PnP gate. The inventory returns an error, so `scan` sees nothing in a project hosted pinned. | The inventory reads PnP locks. Vendored keeps its refusal. | target-specific (vendored) | -| N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery. Fixed in #2009 (draft). | target-specific | -| N15 | CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml` | Vendored refuses; the rest handle it | Handle it everywhere | | -| N16 | Lock versions | pnpm: vendored accepts only 5.4/6.0/9.0; hosted any but early shrinkwrap. vlt: vendored refuses a lock with no version; hosted rewrites and warns. | Hosted | | +| N14 | pnpm `lockfile=false` | Hosted refuses. Discovery withholds hosted refs, keeps vendored ones. Vendored proceeds. The inventory ignores the setting. | The modes stay as they are; the inventory follows discovery. **Resolved in #2009**: under the setting (not under Rush), the inventory keeps a Socket-hosted pnpm entry but drops its integrity, as discovery withholds the lock pin. | target-specific | +| N15 | CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml` | Vendored refuses; the rest handle it | Handle it everywhere. **Resolved in #2009**: the vendored planner and reverter edit a CRLF lock or workspace file like its LF twin and write it back CRLF. Only a file mixing CRLF and LF is still refused (`vendor_lockfile_crlf_unsupported`), since no single ending writes it back byte-faithfully. | | +| N16 | Lock versions | pnpm: vendored accepts only 5.4/6.0/9.0; hosted any but early shrinkwrap. vlt: vendored refuses a lock with no version; hosted rewrites and warns. | Hosted. **pnpm, #2009**: two target-specific checks: hosted refuses only pnpm 1's early shrinkwrap (`plan::hosted_refuses_lock`), and the vendored dialects route by `formats::pnpm::sniff_lock_grammar` (`PnpmDialect::check_lock`). Vendored keeps its 5.4/6.0/9.0 range: no other version has a captured pnpm `file:` override shape that proves its rekey, so the other versions stay refused, target-specifically. vlt is unchanged. | target-specific (pnpm vendored) | | N17 | Bun workspaces, yarn classic offline mirror | Refused differently by the two modes | Stays | target-specific | | N18 | An unparseable npm lock | Hosted warns and skips. Vendored refuses the primary lock. Discovery diagnoses. The inventory returns nothing silently. | Discovery's diagnostic, surfaced by every consumer | | | N19 | yarn.lock with a stray top-level line | Discovery rejects the file; hosted and the inventory scan its blocks | Discovery's rule | | diff --git a/docs/design/lock-model-unification.md b/docs/design/lock-model-unification.md index f6d8196f2..448294e08 100644 --- a/docs/design/lock-model-unification.md +++ b/docs/design/lock-model-unification.md @@ -81,7 +81,7 @@ Non-goal: adding new hosted or vendored backends. | May a reader or writer read project files outside the git repository root? | Several do: vendored paths spelled `../.socket/vendor/…`, an ancestor `go.work`, out-of-root `-r` includes, parent NuGet configs. | **Owner decision:** no. Every project-file read is bounded to the git repository root (the project root when there is no repository). Machine-level tool configuration (for example `GOENV`, the user `NuGet.Config`) is listed separately for review. **2026-11-05:** the bound applies to workspace runs only. Global runs (`--global` / `-g` / `--global-prefix`) read anywhere. Without a git checkout, upward lookups (an ancestor `go.work`, parent NuGet configs, workspace and settings files) behave as before (#2025). | | In a workspace run, what if a file the tool honors sits above the repository root (an ancestor `pnpm-workspace.yaml`, `.npmrc`, `go.work`, `.cargo/config.toml`)? | Each walk reads it, or stops at a different place. | **Owner decision, 2026-11-05:** such a marker file above the repository root and below the home directory makes hosted and vendored mode refuse with a message naming the file, and readers withhold. Nothing at or above home counts (#2025). | | Do out-of-repository `-r` includes of `requirements.txt` count? | Hosted rewrites the root file only; the inventory and discovery follow includes wherever they point. | **Owner decision, 2026-11-05:** includes are followed (P4), but an include outside the repository root stays bounded in workspace runs: never read or rewritten (#2025). | -| Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` read (N11)? | Hosted rewrites both under one all-or-nothing gate. Discovery sweeps it as ignored. The inventory reads it as a fallback. Vendored never reads it. | **Owner decision, 2026-11-05:** it is shadowed: swept for stale identities, never rewritten, and outside the gate. If the project's pnpm evidence (install record or `package.json` `packageManager`) pins pnpm 1 or 2, which read only `shrinkwrap.yaml`, then `shrinkwrap.yaml` governs and vendored mode refuses the project (#2029). | +| Is a `shrinkwrap.yaml` beside `pnpm-lock.yaml` read (N11)? | Hosted rewrites both under one all-or-nothing gate. Discovery sweeps it as ignored. The inventory reads it as a fallback. Vendored never reads it. | **Owner decision, 2026-11-05:** it is shadowed: swept for stale identities, never rewritten, and outside the gate. If the project's pnpm evidence (install record or `package.json` `packageManager`) pins pnpm 1 or 2, which read only `shrinkwrap.yaml`, then `shrinkwrap.yaml` governs and vendored mode refuses the project (#2009). | | Does a pnpm branch lock (`pnpm-lock..yaml`) make the root lock unreliable (N1)? | Hosted withholds the root locks and refuses when `gitBranchLockfile` is set. Vendored refuses. Discovery and the inventory never check. | **Owner decision, 2026-11-05:** discovery withholds pnpm root and member-lock attestations whenever any `pnpm-lock..yaml` exists, whatever the settings or environment say (#2010). | ## The model @@ -291,7 +291,7 @@ A format whose planner merge is large splits into two PRs, readers first. | 0 | `lock-model/0-design` (#2001) | This document and the size budget | | 1 | `lock-model/1-origin` (#2002) | The `origin` module: discovery's classifier and the redirect's hosted-URL rule, moved verbatim. Callers import it from `crate::origin`. | | 2a | `lock-model/2a-pnpm-readers` (#2009) | Pilot, readers. One pnpm entry walk for the inventory, discovery, the store crawler, the hosted planner and the vendored in-use check. The env-lockfile document becomes its own tree (see Decisions). | -| 2b | `lock-model/2b-pnpm-planner` (#2029) | Pilot, planner. One pnpm rewire planner for both modes. `shrinkwrap.yaml` beside `pnpm-lock.yaml` is shadowed (see Decisions). | +| 2b | `lock-model/2b-pnpm-planner` (#2029, folded into #2009) | Pilot, planner. One pnpm rewire planner for both modes. `shrinkwrap.yaml` beside `pnpm-lock.yaml` is shadowed (see Decisions). | | 3 | `lock-model/3-npm` | package-lock and shrinkwrap. Folds `vendor/npm_origin.rs`'s second walk into the records. | | 4 | `lock-model/4-yarn` | Classic and Berry. One block model for every reader and writer, `BerryStanzas` included. | | 5 | `lock-model/5-bun-vlt` | `bun.lock`, `bun.lockb` and vlt. The vlt writers move off their line grammar. | diff --git a/docs/ecosystems.md b/docs/ecosystems.md index c3e2610a1..826a086ff 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -171,7 +171,9 @@ Each Legacy format has an upgrade path and an undo path. Both work in v5: from its legacy mirror, alias nodes included) and rewires both locks in npm 12's dual-lock state. Majors 6–12 are measured in [npm compatibility](testing/npm-compatibility.md). -- **pnpm** — hosted rewriting supports legacy `shrinkwrap.yaml` (pnpm 1/2), +- **pnpm** — hosted rewriting supports legacy `shrinkwrap.yaml` (pnpm 1/2; never + rewritten beside a `pnpm-lock.yaml`, which pnpm >= 3 reads instead; a + project pinning pnpm 1/2 keeps `shrinkwrap.yaml` and leaves the stray lock), lockfileVersion 5.x (pnpm 3–7), 6.0 (pnpm 8), and 9.0 (pnpm 9–12). Early pnpm 1 locks with shrinkwrapVersion 3 and no positive minor version are refused because those installers discard hosted URLs; the tested