diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index c9f242c3a..ca8f1a1ee 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -544,7 +544,7 @@ Every VEX statement's impact string records which patch-application mode persist `vendored` and `redirected` are disjoint in practice (the modes conflict); if a PURL somehow appears in both sets, `vendored` wins. -**Patch hosts (manifest-less VEX).** A hosted lockfile reference counts only when it points at Socket's patch server or the operator's `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin. A hosted URL on any OTHER host — a staging patch server used without `--patch-server-url`, or a look-alike host — is not a hosted pin at all (v5.0: there is no redirect ledger to vouch for it), so `vex` does not attest it, and `list`, `rollback`, `remove`, `vendor` and `repair` do not see it either. Pass `--patch-server-url` for a non-production patch server. (A pre-v5 redirect ledger still on disk can keep such a record in play under the pre-v5 rule: judged by the ledger's own recorded wiring, attesting only an installed tree that hashes to the record unless `--no-verify` / `--vex-no-verify` trusts it; do not combine `--no-verify` with lockfiles you do not trust.) +**Patch hosts (manifest-less VEX).** A hosted lockfile reference counts only when it points at Socket's patch server or the operator's `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin. A hosted URL on any OTHER host — a staging patch server used without `--patch-server-url`, or a look-alike host — is not a hosted pin at all (v5.0: there is no redirect ledger to vouch for it), so `vex` does not attest it, and `list`, `rollback`, `remove`, `vendor` and `repair` do not see it either. Pass `--patch-server-url` for a non-production patch server. A pre-v5 redirect ledger still on disk does not change this: a record whose lockfile or recorded wiring resolves the package from a hosted-shaped URL on an origin this run was not given (or one carrying userinfo) is never attested from the ledger, whatever is installed and even under `--no-verify` / `--vex-no-verify`: `vex` reports it as `redirect_unwired`, with a `vex_claim_unwired` note naming the cause. (A patch that `apply`'s manifest also records is still verified as an agent-mode patch, on that record.) `scan` does not treat such wiring as gone: its takeover and hosted-wiring advisories still see it. ### Manifest-less VEX (lockfile discovery) @@ -581,9 +581,9 @@ Recognition rules that hold for every ecosystem: * **Patch hosts.** A hosted reference counts only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin, with no userinfo. The uuid is the URL's LAST canonical-uuid path segment, because grant tokens may themselves be uuid-shaped. The Go module prefix is fixed. `socket-patch-` registry / repository / source names count only through a pin. For a URL on any other host, see **Patch hosts** above. * **Pins, not definitions.** A registry, index or source *definition* alone (cargo `[registries]`, nuget ``, pom ``, uv index tables, `.npmrc`) never makes a reference, because it survives a reverted pin. Sections the package manager ignores are not read: npm's v2 `dependencies` mirror, a `.cargo/config.toml` shadowed by `.cargo/config`. A Socket pin inside a maven `` is diagnosed, never a reference. * **Contested locks.** When one lock wires a package to a patch and another lock resolves the same `name@version` from a non-Socket source, the build's bytes depend on which package manager runs. The reference is then dropped with a `patched_ref_unattributable` diagnostic naming both files. This applies across npm / pnpm / yarn / bun and across uv / pylock / poetry / pdm / Pipfile.lock / requirements. PEP 723 script locks neither contest nor are contested. The root `requirements.txt` and its in-root `-r` includes count as one lock here: pip reads them as one requirement set, where a direct reference wins over a compatible `==` pin of the same version in another file of the set (#1086). A PyPI version matches under PEP 440 equality, so a requirements line pinned in any spelling that can only install the wired release (`==1.2.3.0`, `==v1.2.3`, `===1.2.3`) contests it like `==1.2.3` does; a range or wildcard (`~=1.2.3`, `==1.2.*`) does not (#1602). A **bundled** npm copy (`inBundle: true`, or v1 `bundled: true`, beneath a dependency that bundles it; the root project's own bundles install from the lock, #1618) of the same `name@version` contests the reference too, in the same lock, in the other npm lock of a shrinkwrap/package-lock pair, or in any other lock. npm unpacks it from the parent package's tarball, so no rewire reaches it and it stays unpatched. Bun and vlt unpack bundled copies the same way (#469, #471). pnpm unpacks bundled copies too, but its lock cannot tie one to a reference (see **Unattested references** below). For Bun, that is a `bun.lock` entry whose meta is `{ "bundled": true }`, or a `bun.lockb` record that a dependency edge with the `bundled` behavior bit reaches, including one Bun shares with a regular install. Such an entry is never a reference, and it contests the reference the same way. For vlt, the lock records no node for a bundled copy, so the copy is found in the installed store: a real package directory inside a store package's own `node_modules`. yarn classic's `yarn.lock` records nothing about bundled copies either (#758), so for a lock with a Socket wiring they are found in the installed `node_modules`: each package a parent's `bundleDependencies` / `bundledDependencies` names inside the parent's own `node_modules`, the dependencies of those beside them, and everything nested under them. Such a copy contests the reference in `yarn.lock` (the diagnostic names the parent's lock entry and the copy's directory); a lock-only `vex` with no `node_modules` can't see it. Hosted and vendored scans skip these copies with `redirect_bun_bundled_instance_skipped` / `redirect_vlt_bundled_instance_skipped` / `vendor_bundled_instance_skipped`, and a yarn classic hosted scan names one with `redirect_copy_unpatched`. When a bundled copy is the only instance, vendoring refuses with `vendor_lock_entry_not_rewritable`. A reference withheld only because such an unreachable copy installs beside it (a bundled npm / Bun / vlt copy, an npm copy beneath a `hasShrinkwrap` package (#753) or one npm installs from a git / url / `file:` spec, or a yarn classic git or `file:` directory block of the same `name@version`) is still the rewriter's own wiring of exactly one package version: it is never attested, but `rollback`, `remove`, `list` and the hosted → vendored takeover treat it as an ordinary hosted pin and restore its upstream registry entry (#828), rather than refusing it with `hosted_wiring_contested`. Another entry of the **same** npm, Bun or yarn lock that resolves the wired `name@version` from a non-Socket source (for example a workspace member added after the rewire, then `npm install` / `bun install` / `yarn install`; for yarn berry, a registry locator such as `left-pad@npm:1.3.0` beside the hosted `…::__archiveUrl=` one) contests the reference too (#588). The package manager installs both entries, and that copy stays unpatched. Re-running `scan` / `vendor` rewires every copy. -* **Unattested references.** Some evidence shows a build may run a copy no wiring reaches, but cannot be tied to the reference's exact `name@version` or cannot say the build runs it. The reference then stays a reference: `list`, `rollback` and `remove` find it, and the ledgers' liveness gates (`vendor --check`, `scan`) keep treating the wiring as live, because re-running `scan` / `vendor` could never clear the evidence. Only `vex` omits it, as a run warning and as the `failed[].reason`. Eight cases besides Gradle's (`vex_gradle_lock_above_base`) and NuGet's (`vex_nuget_lock_other_version`, see the table above); the eighth, **hosted sbt pins a build bypasses** (`vex_sbt_pin_bypassed`), is under **Hosted sbt** above: **pnpm bundled copies** (`vex_pnpm_bundled_copy`): a `packages:` entry's `bundledDependencies:` names the copies pnpm unpacks from that package's own tarball, but pnpm never locks them, so the bundled version is not in the lock. A pnpm reference whose package name a `bundledDependencies` list in the same lock names is omitted whatever its version (a missed attestation when the bundled copy is another version, never a false one), and `bundledDependencies: true` (or a value that cannot be read) omits every reference of that lock. It reaches no other lock. **deno.lock** (`vex_deno_lock_copy`): `deno install` installs a `package.json` project's npm dependencies from `deno.lock` and never reads `package-lock.json` / `pnpm-lock.yaml` / `yarn.lock`, so an npm-family reference whose `name@version` the `deno.lock` npm section also locks is omitted (#406). Whether Deno or another package manager populates `node_modules` (`nodeModulesDir: "manual"` allows either) is not in the files, so this holds whatever `nodeModulesDir` says. **Maven reactor roots** (`vex_maven_reactor_root`): a hosted pin in a root `pom.xml` that declares `` / ``, or whose subprojects Maven 4 discovers (see the hosted Maven rewriter), is omitted, because a module's own literal `` overrides a root pin and module poms are not read (#261); hosted scan refuses such roots (`redirect_maven_multimodule_unsupported`), so this only covers pins an older release wrote. **Maven classifier copies** (`vex_maven_classifier_unpatched`): a `` dependency of a hosted-pinned GA is its own coordinate; the classifier-less pin does not retarget it and the grant serves only the main jar, so Maven fetches it from the next repository. When a live, non-profile classifier copy other than `sources` / `javadoc` (`tests`, a native or JDK-specific build) resolves any version other than the suffixed pin — its own literal ``, a `` entry with the same classifier, or no version in the root pom — the pin is omitted. `sources` / `javadoc` copies never run on a classpath and do not withhold the statement; hosted scan only warns about them (`redirect_maven_classifier_unsupported`) and skips the dep when an executable copy names the patched release. **pnpm branch locks** (`vex_pnpm_git_branch_lockfile`): with `gitBranchLockfile` on, pnpm installs a git branch from its own `pnpm-lock..yaml` (the **pnpm per-branch locks** state above), which no wiring reaches. `vex` does not decide whether the setting is on: the presence of any branch lock (at the root, or in a workspace member that installs from its own lock under `sharedWorkspaceLockfile: false`, read from the project's files and never from the environment) omits every reference of the root pnpm lock (`pnpm-lock.yaml`, or `shrinkwrap.yaml` without it) and of each member's own lock, hosted and vendored alike, whatever `gitBranchLockfile` or pnpm's `lockfile` setting says (#556; deterministic and fail-closed). This is checked only when a root or member pnpm lock wires a patch. Hosted mode refuses every pnpm pin when the setting is on and vendored mode refuses the project, so this also covers wiring written before the branch lock appeared. The run warning names the branch lock in the marked lock's own directory. Rush locks are not affected. **Self-patched npm and Bun packages** (`vex_npm_patched_dependency`): a reference in `package-lock.json` / `npm-shrinkwrap.json` / `bun.lock` / `bun.lockb` to a package the project patches itself is omitted, by the hosted rewriters' own per-dependency rule (`redirect_npm_patched_dependency_skipped`, #711; `redirect_bun_patched_dependency_skipped`, #367): a root `package.json` `patchedDependencies` key for its `name@version`, its bare name, or `name@` of any tarball (npm and Bun alike); for npm also a non-null `patched` record on any present npm lock's entry for that `name@version`; for `bun.lock` also the keys it mirrors (the run warning then names `bun.lock` as the key's source). npm applies the user's diff on top of whatever tarball the lock names and Bun applies it only to the resolution it is keyed on, so both modes leave such a package unwired; this covers pins an earlier release wrote, and an npm one fails `npm ci` with `EPATCHFAILED` until `socket-patch rollback ` restores the registry entry (the remedy says so). A key naming the pinned tarball itself (a patch made on top of the pin) is no conflict for the hosted rewriters, which keep the pin, but the reference is omitted all the same: the user's diff runs over the Socket bytes, never the Socket patch alone; a reference with neither a hosted url nor a vendored tarball is matched by its `name@version` and counts every tarball key. Only a reference that survives the lock contests carries this reason. **Packages patched with pnpm, Yarn or patch-package** (`vex_npm_patched_dependency` too): a reference of any npm-family lock (pnpm, Rush, yarn classic and berry, vlt, npm, Bun; hosted or vendored) to a `name@version` the project patches itself is omitted, read from the project, its declared workspace members and (on disk) the ancestor workspace settings up to the git repository root, by the consent gate's own walk: pnpm's `patchedDependencies` (`package.json`'s `pnpm.patchedDependencies`, `pnpm-workspace.yaml`'s `patchedDependencies`, and the `patchedDependencies` map or a `(patch_hash=…)` key suffix in the reference's own pnpm lock or the root one), a yarn berry `patch:` descriptor outside Yarn's builtin compatibility patches (`package.json`'s dependency maps and `resolutions`, or a `patch:` locator `yarn.lock` resolves, including one wrapping the hosted tarball), or a patch-package file (`patches/+.patch`, or under a `--patch-dir` a `package.json` script names; it counts for every version of ``, since patch-package applies it to whichever version is installed). A key's version may be exact, a range or absent (every version), and metadata that cannot be read (an unparseable `package.json`, a `--patch-dir` the script computes) omits every npm reference. This is the `user_patch_conflict` consent gate's evidence (see [Existing project patches](#existing-project-patches)) minus native npm / Bun and the other ecosystems, and the evidence the in-memory hosted engine withholds a package by. The package manager (or the postinstall) applies the user's diff on top of whatever tarball the lock names, so the install never runs the Socket patch alone: the reference is omitted even when an `--allow-patch-conflicts` run wired it, for as long as the patch is declared. Yarn classic and vlt have no patch mechanism of their own (vlt's `vlt.json` `modifiers` rewrite specs, not contents), so patch-package is what is read for them. npm's and Bun's own top-level `patchedDependencies` are judged by the rule above, not here. **Go workspaces** (`go_workspace_override`): a hosted or vendored Go `replace` that the workspace go resolves (an ancestor's `go.work`, one `GOWORK` names, members outside the project included) overrides, ignores or rejects, by the same check hosted and vendored modes run before writing one (`redirect_golang_workspace_conflict`, `vendor_go_workspace_conflict`), is omitted under the reason `vex`'s own go.mod gate reports. So is a replace in the root `go.work` while go builds another one. +* **Unattested references.** Some evidence shows a build may run a copy no wiring reaches, but cannot be tied to the reference's exact `name@version` or cannot say the build runs it. The reference then stays a reference: `list`, `rollback` and `remove` find it, and the ledgers' liveness gates (`vendor --check`, `scan`) keep treating the wiring as live, because re-running `scan` / `vendor` could never clear the evidence. Only `vex` omits it, as a run warning and as the `failed[].reason`. Ten cases besides Gradle's (`vex_gradle_lock_above_base`) and NuGet's (`vex_nuget_lock_other_version`, see the table above); one of them, **hosted sbt pins a build bypasses** (`vex_sbt_pin_bypassed`), is under **Hosted sbt** above: **pnpm bundled copies** (`vex_pnpm_bundled_copy`): a `packages:` entry's `bundledDependencies:` names the copies pnpm unpacks from that package's own tarball, but pnpm never locks them, so the bundled version is not in the lock. A pnpm reference whose package name a `bundledDependencies` list in the same lock names is omitted whatever its version (a missed attestation when the bundled copy is another version, never a false one), and `bundledDependencies: true` (or a value that cannot be read) omits every reference of that lock. It reaches no other lock. **deno.lock** (`vex_deno_lock_copy`): `deno install` installs a `package.json` project's npm dependencies from `deno.lock` and never reads `package-lock.json` / `pnpm-lock.yaml` / `yarn.lock`, so an npm-family reference whose `name@version` the `deno.lock` npm section also locks is omitted (#406). Whether Deno or another package manager populates `node_modules` (`nodeModulesDir: "manual"` allows either) is not in the files, so this holds whatever `nodeModulesDir` says. **Maven reactor roots** (`vex_maven_reactor_root`): a hosted pin in a root `pom.xml` that declares `` / ``, or whose subprojects Maven 4 discovers (see the hosted Maven rewriter), is omitted, because a module's own literal `` overrides a root pin and module poms are not read (#261); hosted scan refuses such roots (`redirect_maven_multimodule_unsupported`), so this only covers pins an older release wrote. **Maven classifier copies** (`vex_maven_classifier_unpatched`): a `` dependency of a hosted-pinned GA is its own coordinate; the classifier-less pin does not retarget it and the grant serves only the main jar, so Maven fetches it from the next repository. When a live, non-profile classifier copy other than `sources` / `javadoc` (`tests`, a native or JDK-specific build) resolves any version other than the suffixed pin — its own literal ``, a `` entry with the same classifier, or no version in the root pom — the pin is omitted. `sources` / `javadoc` copies never run on a classpath and do not withhold the statement; hosted scan only warns about them (`redirect_maven_classifier_unsupported`) and skips the dep when an executable copy names the patched release. **pnpm branch locks** (`vex_pnpm_git_branch_lockfile`): with `gitBranchLockfile` on, pnpm installs a git branch from its own `pnpm-lock..yaml` (the **pnpm per-branch locks** state above), which no wiring reaches. `vex` does not decide whether the setting is on: the presence of any branch lock (at the root, or in a workspace member that installs from its own lock under `sharedWorkspaceLockfile: false`, read from the project's files and never from the environment) omits every reference of the root pnpm lock (`pnpm-lock.yaml`, or `shrinkwrap.yaml` without it) and of each member's own lock, hosted and vendored alike, whatever `gitBranchLockfile` or pnpm's `lockfile` setting says (#556; deterministic and fail-closed). This is checked only when a root or member pnpm lock wires a patch. Hosted mode refuses every pnpm pin when the setting is on and vendored mode refuses the project, so this also covers wiring written before the branch lock appeared. The run warning names the branch lock in the marked lock's own directory. Rush locks are not affected. **Self-patched npm and Bun packages** (`vex_npm_patched_dependency`): a reference in `package-lock.json` / `npm-shrinkwrap.json` / `bun.lock` / `bun.lockb` to a package the project patches itself is omitted, by the hosted rewriters' own per-dependency rule (`redirect_npm_patched_dependency_skipped`, #711; `redirect_bun_patched_dependency_skipped`, #367): a root `package.json` `patchedDependencies` key for its `name@version`, its bare name, or `name@` of any tarball (npm and Bun alike); for npm also a non-null `patched` record on any present npm lock's entry for that `name@version`; for `bun.lock` also the keys it mirrors (the run warning then names `bun.lock` as the key's source). npm applies the user's diff on top of whatever tarball the lock names and Bun applies it only to the resolution it is keyed on, so both modes leave such a package unwired; this covers pins an earlier release wrote, and an npm one fails `npm ci` with `EPATCHFAILED` until `socket-patch rollback ` restores the registry entry (the remedy says so). A key naming the pinned tarball itself (a patch made on top of the pin) is no conflict for the hosted rewriters, which keep the pin, but the reference is omitted all the same: the user's diff runs over the Socket bytes, never the Socket patch alone; a reference with neither a hosted url nor a vendored tarball is matched by its `name@version` and counts every tarball key. Only a reference that survives the lock contests carries this reason. **Packages patched with pnpm, Yarn or patch-package** (`vex_npm_patched_dependency` too): a reference of any npm-family lock (pnpm, Rush, yarn classic and berry, vlt, npm, Bun; hosted or vendored) to a `name@version` the project patches itself is omitted, read from the project, its declared workspace members and (on disk) the ancestor workspace settings up to the git repository root, by the consent gate's own walk: pnpm's `patchedDependencies` (`package.json`'s `pnpm.patchedDependencies`, `pnpm-workspace.yaml`'s `patchedDependencies`, and the `patchedDependencies` map or a `(patch_hash=…)` key suffix in the reference's own pnpm lock or the root one), a yarn berry `patch:` descriptor outside Yarn's builtin compatibility patches (`package.json`'s dependency maps and `resolutions`, or a `patch:` locator `yarn.lock` resolves, including one wrapping the hosted tarball), or a patch-package file (`patches/+.patch`, or under a `--patch-dir` a `package.json` script names; it counts for every version of ``, since patch-package applies it to whichever version is installed). A key's version may be exact, a range or absent (every version), and metadata that cannot be read (an unparseable `package.json`, a `--patch-dir` the script computes) omits every npm reference. This is the `user_patch_conflict` consent gate's evidence (see [Existing project patches](#existing-project-patches)) minus native npm / Bun and the other ecosystems, and the evidence the in-memory hosted engine withholds a package by. The package manager (or the postinstall) applies the user's diff on top of whatever tarball the lock names, so the install never runs the Socket patch alone: the reference is omitted even when an `--allow-patch-conflicts` run wired it, for as long as the patch is declared. Yarn classic and vlt have no patch mechanism of their own (vlt's `vlt.json` `modifiers` rewrite specs, not contents), so patch-package is what is read for them. npm's and Bun's own top-level `patchedDependencies` are judged by the rule above, not here. **Go workspaces** (`go_workspace_override`): a hosted or vendored Go `replace` that the workspace go resolves (an ancestor's `go.work`, one `GOWORK` names, members outside the project included) overrides, ignores or rejects, by the same check hosted and vendored modes run before writing one (`redirect_golang_workspace_conflict`, `vendor_go_workspace_conflict`), is omitted under the reason `vex`'s own go.mod gate reports. So is a replace in the root `go.work` while go builds another one. **Unpinned Cargo declarations** (`vex_cargo_unpinned_declaration`): cargo resolves each source on its own, so when one declaration of a crate pins it to `socket-patch-` but another — the root `Cargo.toml`'s or a workspace member's, read as hosted scan reads them — is not pinned, with a version requirement that may select the locked patched version, that declaration builds a separate, unpatched copy, possibly of the very version the pin covers. Not pinned means crates.io or another registry that no Socket `[patch.]` registry entry redirects, a `registry-index`, a path or git — the last three even beside `registry = "socket-patch-"`, which they override (hosted scan's own pinned check). The `Cargo.lock` reference is omitted, naming the unpinned declaration's manifest. Hosted scan pins every declaration, so this is a partial revert or a hand edit; the warning says whether re-running `scan` pins it again (a crates.io declaration whose requirement matches no other locked version of the crate) or cannot (`redirect_cargo_toml_dep_unrewritable`: roll the patch back or change the declaration). A declaration whose requirement excludes the patched version (cargo resolves it to another version), and a `[workspace.dependencies]` entry no manifest inherits, do not withhold the statement. When the lock already holds a same-version copy from another source, the reference is withheld as an unpatched lock copy instead. A declaration pinned to ANOTHER Socket patch, which the lock does not resolve the crate from, adds that patch's `Cargo.toml` reference, so the package is gated as `wiring_conflict`. **Vendored Cargo copies a `[patch]` does not reach** (`vex_cargo_patch_bypassed`): a `[patch.]` entry replaces the crate only in declarations from ``, and `vendor` writes `[patch.crates-io]`. So a vendored reference is omitted when a live declaration of the crate (the root `Cargo.toml`'s or a workspace member's) names another registry, a `registry-index` or a git repository, with a version requirement that may select the patched version, unless a `[patch]` table for its source points at the same copy (`[patch.]` for a registry dependency, `[patch.""]` for a `registry-index` one, `[patch.""]` for a git one; URLs compare as cargo canonicalizes them: the host's case, a lock's `registry+` prefix, one trailing `/` and a `.git` suffix ignored, a GitHub URL case-insensitive, a `sparse+` index another source than the git one, and the crates.io git index is crates.io); cargo builds that declaration's own, unpatched copy. Workspace members are read for this whenever the root manifest or the project cargo config wires a Socket copy through `[patch]`, whatever the lock says. It is also omitted when `Cargo.lock` locks the crate at exactly the patched version from any source (a registry, crates.io included, or git) beside the vendored copy; the warning names that lock entry. A path declaration (the user's own fork) does not withhold the statement. Only `[patch]` entries cargo honors count: a manifest entry the project config or a crates.io URL table replaces covers nothing. The warning ends with the remedy for its evidence: for a declaration, re-running `vendor` does not help (declare the crate from crates.io there, point a `[patch]` table for its source at the copy, or roll the patch back); for a stale lock entry a `[patch]` table at the copy redirects (crates.io under `[patch.crates-io]`, or a git repository under its own table), re-lock (`cargo update -p `, or re-run `vendor` for crates.io); for another lock entry, `cargo tree -i @` names the dependency that declares it. * **Shrinkwrap without a twin (#899).** npm >= 12 never reads `npm-shrinkwrap.json`: on a project whose only npm lock is the shrinkwrap it resolves the tree from the registry and writes a fresh `package-lock.json`, so a Socket reference there reaches npm <= 11 only. The reference is still discovered, so `list`, `rollback` and `remove` manage it, but `vex` omits the patch (`vex_npm_shrinkwrap_only`, as a run warning and as the `failed[].reason`). A `package-lock.json` beside it (wired the same way) makes it attestable again. Hosted and vendored runs still rewire the shrinkwrap and warn `redirect_npm_shrinkwrap_only` / `vendor_npm_shrinkwrap_only`. -* **Lockless pins.** With no lock to name a version, a `Cargo.toml` pin (every declaration on `socket-patch-`, that registry defined on the patch host for the same uuid) or an exclusive nuget exact-id mapping is never a reference on its own, so v5.0 does not attest it (nor does `list` show it, or `rollback` / `remove` restore it — restore those files from version control). Only a pre-v5 redirect-ledger record naming a version the pin admits keeps it live. The same holds for a gem wired only in the `Gemfile` (the pre-bundler-2.6 mixed state, lock not converged). +* **Lockless pins.** With no lock to name a version, a `Cargo.toml` pin (every declaration on `socket-patch-`, that registry defined on the patch host for the same uuid) or an exclusive nuget exact-id mapping is never a reference on its own, so v5.0 does not attest it (nor does `list` show it, or `rollback` / `remove` restore it — restore those files from version control). Only a pre-v5 redirect-ledger record naming a version the pin admits keeps it live. For Cargo, declarations are read as for a locked pin (**Unpinned Cargo declarations**): one that is no pin admits nothing, and its requirement (any version, without one) is excluded from the pin; a `[workspace.dependencies]` template no manifest inherits is no declaration at all. The same holds for a gem wired only in the `Gemfile` (the pre-bundler-2.6 mixed state, lock not converged). **Record resolution.** A candidate's record must carry the patch uuid the lockfile actually **wires**. It is taken from the first source that has one: the manifest (matched qualifier-insensitively), the hosted records above (this run's, then a pre-v5 ledger's), then the vendor ledger's embedded records. If none has it and the run is online, `vex` fetches the patch view by uuid from the patch API — for a v5 hosted checkout this is the normal path. The fetch uses `get`'s API client: the public proxy when no token is configured, and a one-shot 401/403 fallback to the proxy (free patches only). At most 10 fetches run concurrently. Fetched records stay in memory: `vex` never writes the manifest. A candidate still has no record under `--offline`, after a transport error or a 404, or when the patch is refused (paid without an entitled token); it is then omitted as `record_unavailable`, and the run is not aborted. A record whose uuid or package disagrees with the wiring is omitted as `record_mismatch`. The informational `socket-patch.vendor.json` marker is never a record source. When the lockfile wires a package to patch U, a manifest or ledger record for that package under another uuid is superseded, and a human-mode `Note:` says so. @@ -2738,6 +2738,8 @@ match it, withholds the statement: | `vex_maven_reactor_root` | run warning and `failed[].reason` | A hosted Maven pin sits in a root `pom.xml` that declares `` / `` (a profile's included), where a module's own literal `` overrides it; only the root pom is read, so no statement (#261). `list`, `rollback` and `remove` still find the pin. | | `go_workspace_override` | run warning and `failed[].reason` | A hosted or vendored Go `replace` is wired, but the workspace go resolves (an ancestor's `go.work`, one `GOWORK` names, members outside the project included) overrides, ignores or rejects it, or go builds another `go.work` than the root's that holds it; no statement while it does. `list`, `rollback` and `remove` still find the replace. | | `vex_maven_classifier_unpatched` | run warning and `failed[].reason` | A hosted Maven pin's GA also has a live executable `` copy (not `sources` / `javadoc`) at a version other than the pin, which the classifier-less pin does not retarget and the grant does not serve; no statement while it does. `list`, `rollback` and `remove` still find the pin. | +| `vex_cargo_unpinned_declaration` | run warning and `failed[].reason` | A hosted Cargo pin is wired, but another declaration of the crate (the root's or a workspace member's) is not pinned (crates.io or another registry no Socket `[patch]` entry redirects, a `registry-index`, a path or git) and may select the patched version, so cargo builds a separate unpatched copy for it; no statement while it does. The warning says whether re-running `scan` can pin it. `list`, `rollback` and `remove` still find the pin. | +| `vex_cargo_patch_bypassed` | run warning and `failed[].reason` | A vendored Cargo copy is wired, but a declaration of the crate from another registry or git accepts the patched version, or `Cargo.lock` locks that version from a registry or git, and the copy's `[patch]` entry does not redirect it; no statement while it does. `list`, `rollback` and `remove` still find the wiring. | | `vex_pnpm_git_branch_lockfile` | run warning and `failed[].reason` | An npm pin is wired in a pnpm lock, but a `pnpm-lock..yaml` exists, which pnpm installs the branch from instead when `gitBranchLockfile` is on (#556); the branch lock's presence withholds the statement whatever the setting says, until the branch locks are folded in and deleted. `list`, `rollback` and `remove` still find the pin, and `vendor --check` and `scan` still treat the wiring as live. | | `vex_npm_patched_dependency` | run warning and `failed[].reason` | An npm-family pin is wired, but the project patches that package itself: npm's or Bun's `patchedDependencies` or npm's lock-entry `patched` record (#711, #367), which hosted and vendored mode never wire, or pnpm's `patchedDependencies`, a Yarn `patch:` descriptor or a patch-package file, which they wire only with `--allow-patch-conflicts`, under the user's patch; no statement while the patch is declared. `list`, `rollback` and `remove` still find the pin. | | `vex_sbt_resolution_unverified` | run warning and `failed[].reason` | A vendored sbt pin (`socket-patch-vendor.sbt`) whose resolution the build's local evidence does not verify, by the same check and build-wide blockers as a hosted sbt pin (no evidence or evidence older than a build source, a declared project with none, a `dependencyOverrides` / `resolvers` reassignment, a `build.sbt.lock`, project definitions that cannot be read statically, another version or other bytes resolved); no statement while it holds. The note names the cause and its remedy: `sbt update` clears stale or missing evidence, keeping the setting's existing entries (appending with `+=` / `++=` in place of a `:=`, `~=` or `--=`) clears a reassignment, removing what forces another version clears a version conflict, and a build whose projects cannot be read statically (a computed `Project(...)`, `projectMatrix`, `crossProject`) is never attested. `vendor --check` and `scan` still treat the wiring as live. | diff --git a/crates/socket-patch-cli/src/commands/scan/mod.rs b/crates/socket-patch-cli/src/commands/scan/mod.rs index 7e4c6dee9..e63db545c 100644 --- a/crates/socket-patch-cli/src/commands/scan/mod.rs +++ b/crates/socket-patch-cli/src/commands/scan/mod.rs @@ -981,8 +981,10 @@ fn overlap_from_states( /// them to right now — the truth source for takeover direction. /// /// Both directions are proved by lockfile discovery with the same liveness -/// rules `vex` gates attestations on (core `Discovery::redirect_record_live` -/// / `Discovery::vendor_entry_live`). `redirect` holds the overlap PURLs the +/// rules `vex` gates attestations on (core `Discovery::redirect_record_verdict` +/// / `Discovery::vendor_entry_live`); a hosted record on a patch server this +/// run was not given (`RecordLiveness::Undecided`) counts as wired, since the +/// wiring is still there. `redirect` holds the overlap PURLs the /// lock routes to the hosted patch server (the vendored ledger entry is /// stale); `vendored` holds those it routes to a committed /// `.socket/vendor//` artifact. @@ -1067,7 +1069,7 @@ pub(super) async fn classify_overlap_takeover_with( let mut liveness = LedgerLiveness::new(cwd, discovery, None); for purl in overlap { let hosted_live = match redirect_uuid_by_purl.get(&PurlKey::new(&purl)) { - Some(uuid) => liveness.redirect_record(&purl, uuid).await, + Some(uuid) => liveness.redirect_record_retained(&purl, uuid).await, None => discovery.wires_package(&purl, WiringMode::Hosted), }; // `None`: undecided (a JVM pom that cannot be read, or a tree still @@ -1394,8 +1396,9 @@ async fn gradle_scan( /// pin names the purl (`redirect_state`, the lockfiles' hosted state — see /// [`crate::commands::hosted_state_from_pins`]) AND lockfile discovery /// proves the current lockfile still routes it to that hosted patch — core -/// `Discovery::redirect_record_live`, the same liveness rule `vex` gates -/// hosted attestations on. +/// `Discovery::redirect_record_verdict`, the same liveness rule `vex` gates +/// hosted attestations on, keeping a record on a patch server this run was +/// not given (`RecordLiveness::Undecided`): its wiring is still retained. /// /// Deliberately NOT routed through [`classify_overlap_takeover`]: that /// classifier keys on purls present in BOTH ledgers, so hosted-only wiring @@ -1438,7 +1441,7 @@ pub(super) async fn hosted_wiring_retained_purls( let mut liveness = LedgerLiveness::new(cwd, &discovery, None); let mut out = Vec::new(); for (purl, uuid) in candidates { - if liveness.redirect_record(&purl, uuid).await { + if liveness.redirect_record_retained(&purl, uuid).await { out.push(purl); } } diff --git a/crates/socket-patch-cli/src/commands/vex_sources.rs b/crates/socket-patch-cli/src/commands/vex_sources.rs index bf15aadc0..739df5cf0 100644 --- a/crates/socket-patch-cli/src/commands/vex_sources.rs +++ b/crates/socket-patch-cli/src/commands/vex_sources.rs @@ -44,7 +44,10 @@ //! whatever raw text survives. Only for a uuid no read file mentions //! (formats no extractor reads, patch servers outside the host allowlist) //! does the ledger's own recorded wiring decide — only files that PIN the -//! resolution, never a leftover registry definition; a vendor entry +//! resolution, never a leftover registry definition — and a hosted +//! record whose lock entry or recorded wiring routes the uuid through a +//! hosted-shaped url on an origin this run was not given is never +//! attested (core `RecordLiveness::Undecided`, fail closed); a vendor entry //! recording no wiring (repair's reconstruction) probes its ecosystem's //! root locks; a lock inventory entry resolving the purl elsewhere is //! dead. A reverted lockfile plus a leftover ledger or artifact must not @@ -82,7 +85,7 @@ use socket_patch_core::utils::purl::strip_purl_qualifiers; use socket_patch_core::utils::purl_key::{canonical_base_purl, PurlKey}; use socket_patch_core::vendor::state::{VendorArtifact, VendorEntry, VendorState}; use socket_patch_core::vex::discover::{ - vendor_ref, Discovery, LedgerLiveness, PatchedRef, WiringMode, + vendor_ref, Discovery, LedgerLiveness, PatchedRef, RecordLiveness, WiringMode, }; use socket_patch_core::vex::{FailedPatch, Unattested, UnattestedKind}; @@ -232,6 +235,14 @@ const NOTE_MAVEN_REACTOR_ROOT: &str = "vex_maven_reactor_root"; /// classifier copy at another version, which the pin does not reach /// (`vex::Unattested`). const NOTE_MAVEN_CLASSIFIER_UNPATCHED: &str = "vex_maven_classifier_unpatched"; +/// Omission tag and note: a hosted Cargo pin beside another declaration of +/// the crate that names no Socket registry, which cargo resolves as a +/// separate, unpatched package (`vex::Unattested`). +const NOTE_CARGO_UNPINNED_DECLARATION: &str = "vex_cargo_unpinned_declaration"; +/// Omission tag and note: a vendored Cargo copy beside a copy of the crate +/// from a source its `[patch]` entry does not redirect (another registry, +/// git), which cargo builds unpatched (`vex::Unattested`). +const NOTE_CARGO_PATCH_BYPASSED: &str = "vex_cargo_patch_bypassed"; /// Omission tag and note: a pnpm lock pins the patch, but a /// `pnpm-lock..yaml` exists, which pnpm installs from instead when /// `gitBranchLockfile` is on; its presence withholds (`vex::Unattested`). @@ -300,6 +311,15 @@ fn unattested_note(kind: UnattestedKind) -> (&'static str, &'static str) { "not attested while that classifier copy resolves an unpatched version; the \ wiring itself is intact, so re-running `scan` does not change this", ), + UnattestedKind::CargoUnpinnedDeclaration => ( + NOTE_CARGO_UNPINNED_DECLARATION, + "not attested while that declaration stands", + ), + UnattestedKind::CargoPatchBypassed => ( + NOTE_CARGO_PATCH_BYPASSED, + "not attested while that copy can build beside the vendored one; the vendored \ + wiring itself is intact (the warning above says how to clear it)", + ), UnattestedKind::PnpmGitBranchLock => ( NOTE_PNPM_GIT_BRANCH_LOCKFILE, "not attested while any branch lock exists (or the members that could hold one \ @@ -660,9 +680,14 @@ pub(crate) async fn plan( continue; } } else if cand.redirected { - let live = assumed.contains(strip_purl_qualifiers(&cand.key)) - || liveness.redirect_record(&cand.key, &cand.uuid).await; - if live { + let verdict = if assumed.contains(strip_purl_qualifiers(&cand.key)) { + RecordLiveness::Live + } else { + liveness + .redirect_record_verdict(&cand.key, &cand.uuid) + .await + }; + if verdict.attests() { Basis::Hosted { lockfile_basis: false, } @@ -671,6 +696,9 @@ pub(crate) async fn plan( // purl: verify it as the agent-mode patch it now is. Basis::Installed } else { + if verdict == RecordLiveness::Undecided { + notes.push(foreign_origin_note(&cand.key, &cand.uuid)); + } notes.extend(dead_claim_notes( &discovery, &cand.key, @@ -1096,6 +1124,20 @@ fn dead_claim_notes( notes } +/// Why a hosted-ledger record whose wiring resolves from a patch server +/// this run was not given ([`RecordLiveness::Undecided`]) is not attested: +/// fail closed, and say how a staging server is admitted. +fn foreign_origin_note(key: &str, uuid: &str) -> PlanNote { + PlanNote { + code: NOTE_CLAIM_UNWIRED, + detail: format!( + "{key}: the hosted ledger records patch {uuid}, but its wiring resolves it from a \ + patch server this run was not given; pass --patch-server-url for a non-production \ + patch server; not attested" + ), + } +} + /// A local record for `uuid` (manifest, then redirect ledger, then vendor /// entries) with the key it is filed under. fn local_record_by_uuid( @@ -2090,6 +2132,106 @@ mod tests { ); } + /// A hosted Cargo pin beside an unpinned declaration of the crate is + /// omitted as `vex_cargo_unpinned_declaration`, the note naming the + /// declaration and the remedy. + #[tokio::test] + async fn a_cargo_pin_beside_an_unpinned_declaration_is_gated() { + const PURL: &str = "pkg:cargo/serde@1.0.1"; + let tmp = tempfile::tempdir().unwrap(); + let mut redirect = RedirectState::new(); + redirect.records.insert(PURL.into(), record(U1)); + let mut found = discovery(vec![hosted_ref(PURL, U1, true)]); + found.unattested.push(socket_patch_core::vex::Unattested { + purl: PURL.into(), + uuid: U1.into(), + file: "crates/a/Cargo.toml".into(), + detail: "crates/a/Cargo.toml declares serde \"1\" from crates.io, not registry \ + \"socket-patch-…\"; re-run `socket-patch scan` to pin it too" + .into(), + kind: UnattestedKind::CargoUnpinnedDeclaration, + }); + let gated = plan( + &common(tmp.path()), + Sources { + manifest: PatchManifest::new(), + vendor: VendorState::new(), + redirect: Some(redirect), + discovery: found, + }, + &[], + &RunApiClient::new(), + ) + .await; + assert!(gated.view.patches.is_empty() && gated.hosted.is_empty()); + assert_eq!( + gated.gated, + vec![failed(PURL, NOTE_CARGO_UNPINNED_DECLARATION)] + ); + assert!( + gated + .notes + .iter() + .any(|n| n.code == NOTE_CARGO_UNPINNED_DECLARATION + && n.detail.contains("crates/a/Cargo.toml declares serde") + && n.detail.ends_with( + "re-run `socket-patch scan` to pin it too; not attested while that \ + declaration stands" + )), + "{:?}", + gated.notes + ); + } + + /// A vendored Cargo copy beside a copy of the crate its `[patch]` + /// entry does not redirect is omitted as `vex_cargo_patch_bypassed`, + /// the note naming that copy and the remedy. + #[tokio::test] + async fn a_vendored_cargo_copy_beside_a_bypassing_source_is_gated() { + const PURL: &str = "pkg:cargo/serde@1.0.1"; + let tmp = tempfile::tempdir().unwrap(); + let mut manifest = PatchManifest::new(); + manifest.patches.insert(PURL.into(), record(U1)); + let vref = vendor_ref(&format!(".socket/vendor/cargo/{U1}/serde-1.0.1")).unwrap(); + let mut found = discovery(vec![PatchedRef::vendored( + PURL.into(), + &vref, + "Cargo.toml", + None, + )]); + found.unattested.push(socket_patch_core::vex::Unattested { + purl: PURL.into(), + uuid: U1.into(), + file: "crates/a/Cargo.toml".into(), + detail: "crates/a/Cargo.toml declares serde \"1\" from registry `my-corp`".into(), + kind: UnattestedKind::CargoPatchBypassed, + }); + let gated = plan( + &common(tmp.path()), + Sources { + manifest, + vendor: VendorState::new(), + redirect: None, + discovery: found, + }, + &[], + &RunApiClient::new(), + ) + .await; + assert!(gated.view.patches.is_empty() && gated.hosted.is_empty()); + assert_eq!(gated.gated, vec![failed(PURL, NOTE_CARGO_PATCH_BYPASSED)]); + assert!( + gated + .notes + .iter() + .any(|n| n.code == NOTE_CARGO_PATCH_BYPASSED + && n.detail.contains("from registry `my-corp`") + && n.detail.contains("says how to clear it")), + "{:?}", + gated.notes + ); + } + /// Ledger-only records with no wiring anywhere are gated, except the /// manifest-owned one, which falls back to agent-mode verification; an /// in-run confirmed purl is live by construction. diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/cargo.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/cargo.rs index dbcef78ef..09436e756 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/cargo.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/cargo.rs @@ -913,48 +913,31 @@ fn cargo_hosted_f_spoofed_references_never_attest() { let run = fx.vex(&["--proxy-url", &api.uri()]); assert_nothing_discovered(&run, "foreign host"); assert_eq!(api.hits(), 0, "a foreign-host uuid is never even fetched"); - // DELIBERATE (the staging fallback): a uuid on a host outside the - // allowlist is not a lockfile reference, but it is not "recognized" - // either, so a redirect-ledger record for it falls back to the ledger's - // own recorded wiring files — which still name it — so that a staging - // `scan --mode hosted` run without `--patch-server-url` keeps - // attesting. Hash verification still judges the copy the build - // consumes (the foreign registry's): nothing is installed here, so the - // default run omits it; only `--no-verify`, which trusts the ledger's - // record by definition, attests. + // Fail closed: a ledger record wired through a hosted-shaped url on an + // unconfigured origin is `redirect_unwired` whatever is installed, even + // under `--no-verify` (CLI_CONTRACT.md, "Patch hosts (manifest-less VEX)"). write_redirect_ledger(&fx, CARGO_PURL, cargo_record(U), CARGO_HOSTED_EDITS); - let run = fx.vex(&["--offline"]); - assert_omitted( - &run, - CARGO_PURL, - "package_not_found", - "foreign host + ledger", - ); - fx.install_crate(SOCKET_CARGO_SRC, PATCHED_RS); - let run = fx.vex(&["--offline"]); - assert_omitted( - &run, - CARGO_PURL, - "package_not_found", - "foreign host + ledger: Socket's copy is not what this lock builds", - ); - fx.install_crate("evil.example-0123456789abcdef", PRISTINE_RS); - let run = fx.vex(&["--offline"]); - assert_eq!( - run.code, - Some(1), - "foreign registry's pristine copy: {}", - run.env - ); - let run = fx.vex(&["--offline", "--no-verify"]); - assert_attested( - &fx, - &run, - CARGO_PURL, - U, - "redirected", - "staging fallback, --no-verify", - ); + for (state, install) in [ + ("not installed", None), + ("Socket's patched copy", Some(SOCKET_CARGO_SRC)), + ( + "the foreign registry's patched copy", + Some("evil.example-0123456789abcdef"), + ), + ] { + if let Some(src) = install { + fx.install_crate(src, PATCHED_RS); + } + for extra in [&["--offline"][..], &["--offline", "--no-verify"][..]] { + let run = fx.vex(extra); + assert_omitted( + &run, + CARGO_PURL, + "redirect_unwired", + &format!("foreign host + ledger, {state} {extra:?}"), + ); + } + } // The same foreign origin configured as the operator's patch server. let fx = Fx::new(); diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/maven.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/maven.rs index 64c46fcf9..c7128d6ae 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/maven.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/maven.rs @@ -37,9 +37,9 @@ //! manifest and ledgers are deleted and the standalone `vex` re-attests. //! //! Documented limitations are asserted explicitly (never skipped): the -//! legacy same-GAV hosted fallback, a staging (non-allowlisted) host kept -//! alive by a ledger under `--no-verify`, and agent-mode (`apply`) patches, -//! which have no lockfile wiring to discover. +//! legacy same-GAV hosted fallback and agent-mode (`apply`) patches, which +//! have no lockfile wiring to discover. A staging (non-allowlisted) host is +//! never attested, even by a ledger under `--no-verify` (fail closed). #[path = "../prebuilt_common/mod.rs"] mod prebuilt_common; @@ -736,7 +736,8 @@ fn maven_hosted_pin_without_trusted_checksums_still_attests() { /// Socket url whose uuid segment differs from the id) is not a patch /// reference — with no manifest and no ledger there is nothing to attest, /// and the API is never asked about the uuid. A ledger record for the -/// same uuid is kept dead by the rejected text (rule 11). +/// same uuid is never attested (`redirect_unwired`): dead by the rejected +/// text (rule 11), or on a foreign origin. #[test] fn maven_hosted_spoofed_repositories_never_attest() { let golden = std::fs::read_to_string(fixture_dir("maven/pom/basic/expected/pom.xml")).unwrap(); @@ -775,37 +776,29 @@ fn maven_hosted_spoofed_repositories_never_attest() { } continue; } - // DOCUMENTED LIMITATION (vex_sources.rs module docs, "only for a - // uuid no read file mentions ... patch servers outside the host - // allowlist ... does the ledger's own recorded wiring decide"): a - // NON-allowlisted host is how a staging patch server run without - // `--patch-server-url` looks, so a ledger whose pom.xml still pins - // `-socket.` under a `socket-patch-` repository keeps - // the claim alive. With verification on, the installed tree is - // still the evidence — nothing installed is `package_not_found` - // (the not-installed lockfile basis is only for DISCOVERED refs) - // and a pristine served jar is `not_applied` — so nothing attests. - // Under `--no-verify` the ledger record attests: `--no-verify` - // trusts the records, and the wiring gate cannot tell a staging - // host from a hostile one — the limitation CLI_CONTRACT.md records - // under "Patch hosts (manifest-less VEX)". - let (code, env) = fx.vex(&["--offline"]); - assert_omitted(code, &env, MVN_PURL, "package_not_found", label); + // Fail closed: a ledger record wired through a hosted-shaped url on an + // unconfigured origin is `redirect_unwired` whatever is installed, even + // under `--no-verify` (CLI_CONTRACT.md, "Patch hosts (manifest-less VEX)"). let sfx_dir = fx.m2().join(format!("org/slf4j/slf4j-api/{MVN_SUFFIXED}")); - put( - &sfx_dir, - &format!("slf4j-api-{MVN_SUFFIXED}.pom"), - b"", - ); - put( - &sfx_dir, - &format!("slf4j-api-{MVN_SUFFIXED}.jar"), - MVN_PRISTINE, - ); - let (code, env) = fx.vex(&["--offline"]); - assert_omitted(code, &env, MVN_PURL, "not_applied", label); - let (code, env) = fx.vex(&["--offline", "--no-verify"]); - assert_eq!(code, Some(0), "{label}: documented staging fallback: {env}"); + for installed in [false, true] { + if installed { + put( + &sfx_dir, + &format!("slf4j-api-{MVN_SUFFIXED}.pom"), + b"", + ); + put( + &sfx_dir, + &format!("slf4j-api-{MVN_SUFFIXED}.jar"), + MVN_PRISTINE, + ); + } + for extra in [&["--offline"][..], &["--offline", "--no-verify"][..]] { + let (code, env) = fx.vex(extra); + let cell = format!("{label} (installed: {installed}) {extra:?}"); + assert_omitted(code, &env, MVN_PURL, "redirect_unwired", &cell); + } + } } } diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/yarn.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/yarn.rs index 520900e4f..9e6ce4db7 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/yarn.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/yarn.rs @@ -1045,43 +1045,43 @@ fn uuid_on_a_foreign_host_is_not_a_hosted_reference() { assert_eq!(env["error"]["code"], "manifest_not_found", "{cell}: {env}"); assert_eq!(request_count(&rt, &server), 0, "{cell}: network call"); - // DOCUMENTED FALLBACK (vex_sources `redirect_record_live`): a - // uuid the discovery allowlist does not recognize is judged by - // the ledger's raw-text fallback, which accepts ANY host so a - // staging patch server used without `--patch-server-url` keeps - // working. The fallback never trusts the look-alike lock's pin: - // with verification on it attests only an installed tree that - // hashes to the record. (`--no-verify` trusts the ledger by - // definition — CLI_CONTRACT.md, "Patch hosts (manifest-less - // VEX)".) + // Fail closed: a ledger record wired through a hosted-shaped url on an + // unconfigured origin is `redirect_unwired` whatever is installed, even + // under `--no-verify` (CLI_CONTRACT.md, "Patch hosts (manifest-less VEX)"). write_redirect_ledger(cwd, flavor, record(UUID)); - let (code, env) = vex_json(cwd, &["--offline"]); - assert_omitted( - cwd, - code, - &env, - "package_not_found", - &format!("{cell} + ledger, not installed"), - ); - install(cwd, b"tampered bytes\n"); - let (code, env) = vex_json(cwd, &["--offline"]); - assert_omitted( - cwd, - code, - &env, - "hash_mismatch", - &format!("{cell} + ledger, tampered"), - ); - install(cwd, PATCHED); - let (code, env) = vex_json(cwd, &["--offline"]); - assert_attested( - cwd, - code, - &env, - UUID, - "redirected", - &format!("{cell} + ledger, verified"), - ); + for (state, bytes) in [ + ("not installed", None), + ("tampered", Some(&b"tampered bytes\n"[..])), + ("hashes to the record", Some(PATCHED)), + ] { + if let Some(bytes) = bytes { + install(cwd, bytes); + } + for extra in [&["--offline"][..], &["--offline", "--no-verify"][..]] { + let (code, env) = vex_json(cwd, extra); + assert_omitted( + cwd, + code, + &env, + "redirect_unwired", + &format!("{cell} + ledger, {state} {extra:?}"), + ); + if !origin.contains('@') { + // The note names the cause and the remedy. + assert!( + env["warnings"] + .as_array() + .is_some_and(|w| w.iter().any(|w| { + w["code"] == "vex_claim_unwired" + && w["detail"] + .as_str() + .is_some_and(|d| d.contains("--patch-server-url")) + })), + "{cell} + ledger, {state}: {env}" + ); + } + } + } } } } diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index d682993d0..dbdc75785 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -973,7 +973,7 @@ pub async fn read_candidate_files( // unsatisfiable. Keyed `/Cargo.toml` for the cargo rewriter. if out.files.contains_key("Cargo.toml") && candidates.iter().any(|c| c.dep.ecosystem == "cargo") { - for rel in crate::utils::cargo_workspace::member_manifests_in(view) { + for rel in crate::utils::cargo_workspace::member_manifests_in(view).await { out.read(view, unreadable, &rel).await; } } diff --git a/crates/socket-patch-core/src/hosted/memory/mod.rs b/crates/socket-patch-core/src/hosted/memory/mod.rs index 4c40d8845..8553adf4d 100644 --- a/crates/socket-patch-core/src/hosted/memory/mod.rs +++ b/crates/socket-patch-core/src/hosted/memory/mod.rs @@ -53,7 +53,7 @@ use std::time::Instant; use crate::api::client::PatchApi; use crate::api::types::{PatchResponse, PatchSearchResult}; use crate::crawlers::Ecosystem; -use crate::utils::cargo_workspace::member_manifests_in; +use crate::utils::cargo_workspace::member_manifests_in_memory; use crate::vendor::lock_inventory::{ inventory_project_diagnosed_in, MemoryEntry, MemoryProject, ProjectView, }; @@ -279,7 +279,7 @@ fn demote_cargo_members(states: &mut [RootState], warnings: &mut Vec Option hosted_patch_url_uuids(url, extra_origins)?.pop() } -/// EVERY canonical-uuid path segment of a Socket-HOSTED url, in path order -/// (the grant token first when it is uuid-shaped, the patch uuid last), or -/// `None` when `url` is not on an accepted origin — [`hosted_patch_uuid`]'s -/// exact acceptance rules, without electing one segment. `vex` discovery -/// uses it to RECOGNIZE every Socket identity a lockfile mentions, including -/// the malformed or rejected shapes whose "last uuid" is not a patch. -pub fn hosted_patch_url_uuids(url: &str, extra_origins: &[String]) -> Option> { +/// `url` normalized the way [`hosted_patch_url_uuids`] reads it: `\/` +/// unescaped, a percent-encoded `http(s)://` url decoded, and a Cargo +/// `sparse+` / `registry+` source prefix stripped. `None` without a +/// `://`. +fn normalized_hosted_url(url: &str) -> Option { use crate::utils::purl::percent_decode_purl_component; let unescaped = url.trim().replace("\\/", "/"); @@ -92,13 +90,49 @@ pub fn hosted_patch_url_uuids(url: &str, extra_origins: &[String]) -> Option { - &decoded[kind.len() + 1..] + decoded[kind.len() + 1..].to_string() } - _ => decoded.as_str(), - }; - let parsed = reqwest::Url::parse(text).ok()?; + _ => decoded, + }) +} + +/// The patch uuid `url` names when read as a hosted url of its OWN origin +/// ([`hosted_patch_uuid`]'s grammar with the url's origin taken as +/// configured): what a lock entry on an origin a run was not given claims +/// to resolve from. Only a url in a Socket patch server's path shape (a +/// `patch`, `patch-registry` or `gopatch` segment) counts: any other url +/// merely ending in the uuid (an artifact store's blob path) resolves from +/// elsewhere. Userinfo is dropped first: Socket never writes one, so +/// `https://patch.socket.dev@evil.example/patch/…` is a look-alike on the +/// host after the `@`, never an accepted origin. +pub(crate) fn hosted_shape_uuid(url: &str) -> Option { + let normalized = normalized_hosted_url(url)?; + let mut parsed = reqwest::Url::parse(&normalized).ok()?; + if !parsed + .path_segments()? + .any(|s| matches!(s, "patch" | "patch-registry" | "gopatch")) + { + return None; + } + let _ = parsed.set_username(""); + let _ = parsed.set_password(None); + let origin = parsed.origin().ascii_serialization(); + hosted_patch_uuid(parsed.as_str(), &[origin]) +} + +/// EVERY canonical-uuid path segment of a Socket-HOSTED url, in path order +/// (the grant token first when it is uuid-shaped, the patch uuid last), or +/// `None` when `url` is not on an accepted origin — [`hosted_patch_uuid`]'s +/// exact acceptance rules, without electing one segment. `vex` discovery +/// uses it to RECOGNIZE every Socket identity a lockfile mentions, including +/// the malformed or rejected shapes whose "last uuid" is not a patch. +pub fn hosted_patch_url_uuids(url: &str, extra_origins: &[String]) -> Option> { + use crate::utils::purl::percent_decode_purl_component; + + let text = normalized_hosted_url(url)?; + let parsed = reqwest::Url::parse(&text).ok()?; if !matches!(parsed.scheme(), "https" | "http") || !parsed.username().is_empty() || parsed.password().is_some() diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 91f3a625c..609e14fca 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -2142,60 +2142,21 @@ fn rewrite_cargo_with( } let other_versions = cargo_lock_other_versions(cargo_lock.as_deref(), &dep.name, &dep.version); - // The root manifest's `[workspace.dependencies]` verdicts feed its - // members' `workspace = true` inheritors. - let mut root_workspace: BTreeMap = BTreeMap::new(); - let mut toml_plans: Vec<(usize, CargoTomlPlan)> = Vec::new(); - let mut excluded: Vec<(String, String)> = Vec::new(); - let mut refused: Option<(String, String)> = None; - for (i, (path, text)) in manifests.iter().enumerate() { - match plan_cargo_toml( - text, - path, - &dep.name, - &dep.version, - &other_versions, - ®, - &root_workspace, - ) { - Ok(plan) => { - if let Err(reason) = validate_cargo_toml_pins( - &plan.content, - &dep.name, - &dep.version, - &other_versions, - ®, - &plan.workspace, - &root_workspace, - ) { - refused = Some((path.clone(), reason)); - break; - } - if path == "Cargo.toml" { - root_workspace = plan.workspace.clone(); - } - excluded.extend(plan.excluded.iter().map(|req| (path.clone(), req.clone()))); - if plan.found { - toml_plans.push((i, plan)); - } - } - Err(reason) => { - refused = Some((path.clone(), reason)); - break; + let (toml_plans, excluded) = + match plan_cargo_tomls(&manifests, &dep.name, &dep.version, &other_versions, ®) { + Ok(planned) => planned, + Err((path, reason)) => { + result.warnings.push(RewriteWarning { + code: "redirect_cargo_toml_dep_unrewritable".into(), + detail: format!( + "{} in {path} cannot be pinned ({reason}); dependency skipped \ + (nothing rewritten)", + dep.name + ), + }); + continue; } - } - } - if let Some((path, reason)) = refused { - result.warnings.push(RewriteWarning { - code: "redirect_cargo_toml_dep_unrewritable".into(), - detail: format!( - "{} in {path} cannot be pinned ({reason}); dependency skipped \ - (nothing rewritten)", - dep.name - ), - }); - continue; - } + }; // Declared, but no declaration's requirement accepts the patched // version: cargo resolves each to another version, so a pin cannot // reach the locked one (the TS twin's requirement-coverage refusal). @@ -2460,7 +2421,7 @@ fn crlf_to_lf(text: &str) -> Option { /// A workspace-member manifest key the caller supplied: `/Cargo.toml`, /// a plain repo-relative path (never absolute, never `..`, never under the /// ledger's `.socket/` or a build `target/`). -fn is_cargo_member_manifest_key(key: &str) -> bool { +pub(crate) fn is_cargo_member_manifest_key(key: &str) -> bool { let Some(dir) = key.strip_suffix("/Cargo.toml") else { return false; }; @@ -2577,7 +2538,7 @@ fn cargo_manifest_package_id( /// in either the git-index or the sparse spelling. Compared the way cargo's /// `CanonicalUrl` does: host casing, a trailing `/` and a `.git` suffix do /// not make a different source (and github paths are case-insensitive). -fn is_crates_io_source(source: &str) -> bool { +pub(crate) fn is_crates_io_source(source: &str) -> bool { let url = source.trim(); let url = url .strip_prefix("registry+") @@ -3634,7 +3595,7 @@ fn validate_cargo_toml_pins( Ok(()) } -struct CargoTomlPlan { +pub(crate) struct CargoTomlPlan { content: String, edits: Vec, /// `false` when every occurrence already carried our registry (idempotent @@ -3778,6 +3739,58 @@ fn cargo_lock_other_versions(lock: Option<&str>, crate_name: &str, version: &str versions } +/// The Cargo.toml half of a hosted pin of `crate_name`@`version` to +/// registry `reg` across `manifests` (root `Cargo.toml` first, then the +/// members, as `(path, text)`): each manifest's plan (by index) that found +/// a declaration, and every requirement that excludes the version, by +/// path — or the first refusal, `(path, reason)`, which skips the whole +/// dependency. `vex` discovery runs it dry to say whether a re-scan can +/// pin a declaration it found unpinned. +#[allow(clippy::type_complexity)] +pub(crate) fn plan_cargo_tomls( + manifests: &[(String, String)], + crate_name: &str, + version: &str, + other_versions: &[String], + reg: &str, +) -> Result<(Vec<(usize, CargoTomlPlan)>, Vec<(String, String)>), (String, String)> { + // The root manifest's `[workspace.dependencies]` verdicts feed its + // members' `workspace = true` inheritors. + let mut root_workspace: BTreeMap = BTreeMap::new(); + let mut toml_plans: Vec<(usize, CargoTomlPlan)> = Vec::new(); + let mut excluded: Vec<(String, String)> = Vec::new(); + for (i, (path, text)) in manifests.iter().enumerate() { + let plan = plan_cargo_toml( + text, + path, + crate_name, + version, + other_versions, + reg, + &root_workspace, + ) + .map_err(|reason| (path.clone(), reason))?; + validate_cargo_toml_pins( + &plan.content, + crate_name, + version, + other_versions, + reg, + &plan.workspace, + &root_workspace, + ) + .map_err(|reason| (path.clone(), reason))?; + if path == "Cargo.toml" { + root_workspace = plan.workspace.clone(); + } + excluded.extend(plan.excluded.iter().map(|req| (path.clone(), req.clone()))); + if plan.found { + toml_plans.push((i, plan)); + } + } + Ok((toml_plans, excluded)) +} + /// `Err` carries the refusal reason: an occurrence exists that cannot be /// pinned to the managed registry, so the whole dep must be skipped. /// `inherited` is the workspace root's verdicts when planning a member. diff --git a/crates/socket-patch-core/src/utils/cargo_workspace.rs b/crates/socket-patch-core/src/utils/cargo_workspace.rs index db99948a0..6aa8e0388 100644 --- a/crates/socket-patch-core/src/utils/cargo_workspace.rs +++ b/crates/socket-patch-core/src/utils/cargo_workspace.rs @@ -11,10 +11,11 @@ //! would follow it). Cargo still reads those manifests; the rewriter's //! Cargo.lock dependents check refuses a crate one of them depends on. -use std::collections::BTreeSet; +use std::collections::{BTreeMap, BTreeSet}; use std::path::Path; use std::sync::Arc; +use futures_util::StreamExt; use toml_edit::{DocumentMut, Item, Table}; use crate::utils::workspace_globs::segment_glob_matches; @@ -32,18 +33,179 @@ pub fn member_manifests(root: &Path) -> Vec { member_manifests_with(&DiskTree(root)) } -/// [`member_manifests`] over a [`ProjectView`]: the in-memory variant -/// reads the manifests the host supplied and expands member globs against -/// the supplied paths (a directory exists when some supplied path lives -/// under it; symbolic links are never directories). -pub fn member_manifests_in(view: &ProjectView<'_>) -> Vec { - match view { - ProjectView::Disk(_) | ProjectView::Snapshot(_) => { - let root = view.disk_root().expect("a disk view has a root"); - member_manifests(root) +/// [`member_manifests`] of an in-memory project: the manifests the host +/// supplied, member globs expanded against the supplied paths (a directory +/// exists when some supplied path lives under it; symbolic links are never +/// directories). +pub fn member_manifests_in_memory(project: &MemoryProject) -> Vec { + member_manifests_with(&MemoryTree(project)) +} + +/// [`member_manifests`] over a [`ProjectView`], every read, probe and +/// listing going through the view: a snapshot's overlays and content cache +/// apply and its recording notes the walk; in memory it reads the supplied +/// files. The manifests are read concurrently (bounded). +pub async fn member_manifests_in(view: &ProjectView<'_>) -> Vec { + member_manifest_texts_in(view) + .await + .into_iter() + .map(|(rel, _)| rel) + .collect() +} + +/// In-flight manifest reads of one [`member_manifest_texts_in`] level. +const MEMBER_READ_CONCURRENCY: usize = 16; + +/// [`member_manifests_in`], each key with the text the walk read through +/// the view (so a caller need not read it again). +pub(crate) async fn member_manifest_texts_in(view: &ProjectView<'_>) -> Vec<(String, String)> { + let Some((facts, _)) = view_manifest(view, "Cargo.toml").await else { + return Vec::new(); + }; + let mut frontier: Vec = Vec::new(); + if let Some((members, exclude)) = &facts.workspace { + let mut excluded: BTreeSet = BTreeSet::new(); + for pattern in exclude { + excluded.extend(expand_glob_in(view, pattern, CARGO_SKIP).await); + } + for pattern in members { + for dir in expand_glob_in(view, pattern, CARGO_SKIP).await { + if !excluded.contains(&dir) { + frontier.push(dir); + } + } + } + } + frontier.extend( + facts + .path_deps + .iter() + .filter_map(|dep| normalize_rel("", dep)), + ); + let mut found: BTreeMap = BTreeMap::new(); + let mut tried: BTreeSet = BTreeSet::new(); + while !frontier.is_empty() { + let mut level: Vec = Vec::new(); + for dir in std::mem::take(&mut frontier) { + if found.len() + level.len() >= MAX_MANIFESTS { + break; + } + if !walkable(&dir) || tried.contains(&dir) { + continue; + } + tried.insert(dir.clone()); + if every_segment(&dir, |at| view_is_real_dir(view, at)) { + level.push(dir); + } + } + let read: Vec<(String, Option)> = + crate::utils::concurrent::ordered_concurrent( + level, + MEMBER_READ_CONCURRENCY, + |dir| async move { + let manifest = view_manifest(view, &format!("{dir}/Cargo.toml")).await; + (dir, manifest) + }, + ) + .collect() + .await; + for (dir, manifest) in read { + let Some((facts, text)) = manifest else { + continue; + }; + frontier.extend( + facts + .path_deps + .iter() + .filter_map(|dep| normalize_rel(&dir, dep)), + ); + found.insert(dir, text); + } + } + found + .into_iter() + .map(|(dir, text)| (format!("{dir}/Cargo.toml"), text)) + .collect() +} + +/// A manifest the view walk read: its walk facts and its text. +type ReadManifest = (Arc, String); + +/// A regular (non-symlink) manifest read through the view: its walk facts +/// and its text. +async fn view_manifest(view: &ProjectView<'_>, rel: &str) -> Option { + if view.is_symlink(rel) { + return None; + } + let text = view.read_text(rel).await.ok()?; + let facts = FACTS_MEMO + .parse(text.as_bytes(), || { + text.parse::() + .map(|doc| ManifestFacts::of(&doc)) + }) + .ok()?; + Some((facts, text)) +} + +/// [`DirTree::is_real_dir`] through the view's own mediated probes (a +/// snapshot notes each and sees its overlays). +fn view_is_real_dir(view: &ProjectView<'_>, rel: &str) -> bool { + view.is_dir(rel) && !view.is_symlink(rel) +} + +/// [`expand_glob`] over a [`ProjectView`]: the same walk, listing through +/// the view's `list_dir` (a real directory's real sub-directories, as +/// [`DirTree::child_dirs`] answers). +async fn expand_glob_in(view: &ProjectView<'_>, pattern: &str, skip: &[&str]) -> Vec { + let Some(normalized) = normalize_rel("", pattern.trim_end_matches('/')) else { + return Vec::new(); + }; + let segments: Vec<&str> = normalized.split('/').filter(|s| !s.is_empty()).collect(); + let mut out = Vec::new(); + // [`expand_from`]'s recursion as an explicit stack, children pushed in + // reverse so they are visited in the same order. + let mut stack: Vec<(String, usize)> = vec![(String::new(), 0)]; + while let Some((at, i)) = stack.pop() { + if out.len() >= MAX_MANIFESTS { + break; + } + let Some(seg) = segments.get(i) else { + out.push(at); + continue; + }; + if !segment_is_pattern(seg) { + let next = join_rel(&at, seg); + if view_is_real_dir(view, &next) { + stack.push((next, i + 1)); + } + continue; + } + let Ok(entries) = view.list_dir(&at).await else { + continue; + }; + let children = glob_children( + entries.into_iter().filter(|e| e.is_dir).map(|e| e.name), + seg, + skip, + ); + let mut next: Vec<(String, usize)> = Vec::new(); + if *seg == "**" { + next.push((at.clone(), i + 1)); + next.extend(children.iter().map(|child| (join_rel(&at, child), i))); + } else { + let pattern = cargo_segment(seg); + for child in &children { + let name: Vec = child.chars().collect(); + if segment_glob_matches(&pattern, &name) { + next.push((join_rel(&at, child), i + 1)); + } + } } - ProjectView::Memory(project) => member_manifests_with(&MemoryTree(project)), + stack.extend(next.into_iter().rev()); } + out.sort(); + out.dedup(); + out } /// The two directory questions a glob walk asks, keyed by `/`-separated @@ -258,28 +420,36 @@ fn is_real_dir(path: &Path) -> bool { /// Every component of repo-relative `dir` is a real directory under /// the root — none is a symbolic link (which may lead outside the root). pub(crate) fn is_real_dir_path(tree: &dyn DirTree, dir: &str) -> bool { + every_segment(dir, |at| tree.is_real_dir(at)) +} + +/// Whether `real` holds for every ancestor-or-self prefix of `dir`. +fn every_segment(dir: &str, real: impl Fn(&str) -> bool) -> bool { let mut at = String::new(); dir.split('/').all(|seg| { at = join_rel(&at, seg); - tree.is_real_dir(&at) + real(&at) }) } +/// Whether the walk may enter `dir` at all. `target` is cargo's build +/// directory at every level: the glob walk already skips it, and a literal +/// `members = ["target/gen"]` or a path dependency into it names a manifest +/// the next `cargo clean` deletes — pinning it would record a hosted edit +/// whose file can vanish, blocking the rollback of every other cargo edit. +fn walkable(dir: &str) -> bool { + !dir.is_empty() && !dir.split('/').any(|seg| seg == "target") +} + fn enqueue( tree: &dyn Tree, dir: String, dirs: &mut BTreeSet, queue: &mut Vec<(String, Arc)>, ) { - // `target` is cargo's build directory at every level: the glob walk - // already skips it, and a literal `members = ["target/gen"]` or a path - // dependency into it names a manifest the next `cargo clean` deletes — - // pinning it would record a hosted edit whose file can vanish, blocking - // the rollback of every other cargo edit. - if dir.is_empty() + if !walkable(&dir) || dirs.len() >= MAX_MANIFESTS || dirs.contains(&dir) - || dir.split('/').any(|seg| seg == "target") || !is_real_dir_path(tree, &dir) { return; @@ -404,6 +574,23 @@ fn cargo_segment(seg: &str) -> Vec { seg.replace("[^", "[\0^").chars().collect() } +/// The sub-directories a wildcard segment `seg` may match, sorted: never a +/// dot-directory (unless `seg` itself starts with `.`) or a name in `skip`. +fn glob_children( + entries: impl IntoIterator, + seg: &str, + skip: &[&str], +) -> Vec { + let mut children: Vec = entries + .into_iter() + .filter(|name| { + (!name.starts_with('.') || seg.starts_with('.')) && !skip.contains(&name.as_str()) + }) + .collect(); + children.sort(); + children +} + fn expand_from( tree: &dyn DirTree, at: String, @@ -428,13 +615,7 @@ fn expand_from( let Some(entries) = tree.child_dirs(&at) else { return; }; - let mut children: Vec = entries - .into_iter() - .filter(|name| { - (!name.starts_with('.') || seg.starts_with('.')) && !skip.contains(&name.as_str()) - }) - .collect(); - children.sort(); + let children = glob_children(entries, seg, skip); if *seg == "**" { expand_from(tree, at.clone(), tail, skip, out); for child in children { @@ -478,6 +659,124 @@ mod tests { format!("[package]\nname = \"{name}\"\nversion = \"0.1.0\"\n") } + /// Over a tracked snapshot the walk reads through the view: its reads + /// fill the content cache, so files written just now (racy by stat + /// alone) — including a `.socket/` copy's manifest the walk follows + /// through `[patch]` — still leave a read set that holds; a member + /// added under a glob afterwards breaks it (the listing was recorded). + #[tokio::test] + async fn snapshot_walk_records_what_it_reads() { + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + write( + root, + "Cargo.toml", + "[workspace]\nmembers = [\"crates/*\"]\n\n\ + [patch.crates-io]\nserde = { path = \".socket/vendor/cargo/u1/serde-1.0.0\" }\n", + ); + write(root, "crates/a/Cargo.toml", &pkg("a")); + write( + root, + ".socket/vendor/cargo/u1/serde-1.0.0/Cargo.toml", + &pkg("serde"), + ); + let snap = crate::vendor::lock_inventory::DiskSnapshot::tracked(root); + snap.begin_recording(); + let walked = member_manifests_in(&ProjectView::Snapshot(&snap)).await; + assert_eq!(walked, member_manifests(root)); + assert!(walked.contains(&".socket/vendor/cargo/u1/serde-1.0.0/Cargo.toml".to_string())); + let reads = snap.end_recording().expect("the walk keeps the recording"); + assert!(reads.unchanged(), "fresh files read through the view hold"); + write(root, "crates/b/Cargo.toml", &pkg("b")); + assert!(!reads.unchanged(), "a new member dir changes the listing"); + } + + /// A snapshot's overlays are part of the walk: an overlaid member + /// manifest (a pending write) is a member before it reaches the disk. + #[tokio::test] + async fn snapshot_walk_sees_overlays() { + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + write( + root, + "Cargo.toml", + "[workspace]\nmembers = [\"crates/*\"]\n", + ); + write(root, "crates/a/Cargo.toml", &pkg("a")); + let snap = crate::vendor::lock_inventory::DiskSnapshot::new(root); + snap.overlay("crates/c/Cargo.toml", pkg("c").as_bytes()); + assert_eq!( + member_manifests_in(&ProjectView::Snapshot(&snap)).await, + vec!["crates/a/Cargo.toml", "crates/c/Cargo.toml"] + ); + assert_eq!(member_manifests(root), vec!["crates/a/Cargo.toml"]); + } + + /// The view walk answers exactly as the disk walk over every shape the + /// memo test covers (globs, `**`, excludes, nested path dependencies, + /// `[patch]` copies, unparseable manifests) plus a symlinked member + /// directory, and returns the text it read for each. + #[tokio::test] + async fn view_walk_matches_the_disk_walk() { + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + write( + root, + "Cargo.toml", + "[workspace]\nmembers = [\"crates/*\", \"tools/**\", \"lit\"]\n\ + exclude = [\"crates/x\"]\n\ + [patch.crates-io]\nserde = { path = \".socket/vendor/cargo/u1/serde-1.0.0\" }\n\ + [workspace.dependencies]\nshared = { path = \"libs/shared\" }\n", + ); + write( + root, + "crates/a/Cargo.toml", + &format!("{}[dependencies.b]\npath = \"../b\"\n", pkg("a")), + ); + write(root, "crates/b/Cargo.toml", &pkg("b")); + write(root, "crates/x/Cargo.toml", &pkg("x")); + write(root, "crates/.hidden/Cargo.toml", &pkg("hidden")); + write(root, "crates/target/Cargo.toml", &pkg("target")); + write(root, "crates/bad/Cargo.toml", "not = [toml"); + write(root, "tools/deep/t/Cargo.toml", &pkg("t")); + write(root, "lit/Cargo.toml", &pkg("lit")); + write(root, "libs/shared/Cargo.toml", &pkg("shared")); + write( + root, + ".socket/vendor/cargo/u1/serde-1.0.0/Cargo.toml", + &format!( + "{}[dependencies]\nderive = {{ path = \"../derive\" }}\n", + pkg("serde") + ), + ); + write( + root, + ".socket/vendor/cargo/u1/derive/Cargo.toml", + &pkg("derive"), + ); + #[cfg(unix)] + { + write(root, "elsewhere/l/Cargo.toml", &pkg("l")); + std::os::unix::fs::symlink(root.join("elsewhere"), root.join("crates/link")).unwrap(); + } + let disk = member_manifests(root); + assert!(disk.contains(&"tools/deep/t/Cargo.toml".to_string())); + assert!(!disk.iter().any(|k| k.starts_with("crates/link"))); + assert_eq!(member_manifests_in(&ProjectView::Disk(root)).await, disk); + let snap = crate::vendor::lock_inventory::DiskSnapshot::new(root); + assert_eq!( + member_manifests_in(&ProjectView::Snapshot(&snap)).await, + disk + ); + for (rel, text) in member_manifest_texts_in(&ProjectView::Disk(root)).await { + assert_eq!( + text, + std::fs::read_to_string(root.join(&rel)).unwrap(), + "{rel}" + ); + } + } + #[test] fn members_globs_excludes_and_path_dependencies() { let tmp = tempfile::tempdir().unwrap(); diff --git a/crates/socket-patch-core/src/vex/discover/cargo.rs b/crates/socket-patch-core/src/vex/discover/cargo.rs index 40e6ab008..be574e805 100644 --- a/crates/socket-patch-core/src/vex/discover/cargo.rs +++ b/crates/socket-patch-core/src/vex/discover/cargo.rs @@ -29,13 +29,24 @@ //! (cargo < 1.41) files it in `[metadata]` as `"checksum //! ()"` instead — read as the same pin. //! -//! `Cargo.toml` decides whether cargo still USES that entry (rule 10): cargo +//! The manifests decide whether cargo still USES that entry (rule 10): cargo //! re-resolves a locked package whose source no longer matches the -//! manifest's declaration. So for the lock entry's crate: +//! manifest's declaration. The manifests are the root `Cargo.toml` and every +//! workspace member's (`/Cargo.toml`, enumerated as the hosted rewriter +//! enumerates the manifests it pins: +//! [`crate::utils::cargo_workspace::member_manifests_in`], minus `.socket/` +//! and `target/`), whose declarations count alike. So for the lock entry's +//! crate: //! -//! * not declared in the root `Cargo.toml` at all (a workspace member's dep -//! the root lock describes; no / unparseable manifest) → ref; -//! * some declaration pins `socket-patch-` (the same uuid) → ref; +//! * not declared in any of them (a transitive-only crate; no / unparseable +//! root manifest) → ref; +//! * some declaration names `socket-patch-` (the same uuid) → ref; +//! the other declarations that may select the locked version then decide +//! whether it is attested ([`beside_the_pin`]: one pinned to ANOTHER +//! patch adds its conflict ref, one cargo builds its own copy for marks +//! the ref [`super::UnattestedKind::CargoUnpinnedDeclaration`]). A +//! `[workspace.dependencies]` template counts only when a manifest +//! inherits it; //! * declarations pin only OTHER `socket-patch-` registries → the lock //! ref AND a `Cargo.toml` ref for `` at the locked version (no pin): //! the files disagree and the CLI gates the package as `wiring_conflict`; @@ -105,28 +116,36 @@ //! config), and a `[patch.""]` table in the manifest //! replaces `[patch.crates-io]` wholesale — both leave the lock looking //! exactly like live Socket wiring. +//! A ref whose build may also compile a copy of the version the wiring does +//! not reach (a `[patch.]` entry replaces the crate only in +//! declarations from ``) stays a ref but is marked +//! [`super::UnattestedKind::CargoPatchBypassed`] ([`patch_bypassed`]). //! A `.socket/cargo-patches/` path (the retired redirect backend) carries no //! uuid and is not a ref. Socket-shaped entries in a `.cargo/config.toml` //! that cargo ignores (because `.cargo/config` exists) are diagnosed, never //! read. -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet}; use toml_edit::{DocumentMut, Item, TableLike}; +use crate::patch::redirect::is_crates_io_source as is_crates_io_lock_source; + use super::{ names_vendor_dir, simple_purl, socket_patch_name_uuid, toml_or_diag, vendor_ref, DiscoverCtx, - Discovery, PatchedRef, TomlDiag, UnlockedPin, VendorRef, DIAG_REF_INVALID, + Discovery, PatchedRef, TomlDiag, UnattestedKind, UnlockedPin, VendorRef, DIAG_REF_INVALID, DIAG_REF_UNATTRIBUTABLE, }; use crate::formats::cargo::{is_registry_source, CargoLock, CopyClaim, LockedPackage}; use crate::patch::redirect::generation::{hosted_pin_name, PIN_NAME_PREFIX}; +use crate::patch::redirect::{cargo_req_selects, CargoReqMatch}; use crate::utils::digest::is_hex64_lower; use crate::vendor::cargo_config::{ - effective_config_rel_in, patch_entries, registry_definitions, CargoPatchEntry, CONFIG_LEGACY, - CONFIG_TOML, + effective_config_rel_in, patch_entries, registry_definitions, CONFIG_LEGACY, CONFIG_TOML, +}; +use crate::vendor::cargo_manifest::{ + crates_io_url_alias_tables, is_crates_io_source, is_crates_io_url_alias, }; -use crate::vendor::cargo_manifest::{crates_io_url_alias_tables, is_crates_io_source}; use crate::vendor::cargo_tag; use crate::vendor::lock_inventory::LockIntegrity; @@ -145,29 +164,67 @@ const DEP_TABLES: [&str; 5] = [ pub(crate) async fn extract(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { let lock = load_lock(ctx, out).await; - let manifest = read_toml(ctx, CARGO_TOML, out).await; + let root = read_toml(ctx, CARGO_TOML, out).await; let config = read_config(ctx, out).await; + // Every `[patch]` entry pointing at a Socket copy, the root manifest's + // (with why cargo ignores it, if it does) and then the config's. + let mut wired_all: Vec<(WiredCopy, Option)> = Vec::new(); + let mut shadow: Option = None; + if let Some(doc) = &root { + let s = Shadow::new(doc, config.as_ref()); + wired_all.extend(socket_copies(doc).into_iter().map(|w| { + let why = s.why(&w.table, &w.key, Some(&w.path), None); + (w, why) + })); + shadow = Some(s); + } + if let Some((_, doc)) = &config { + wired_all.extend(socket_copies(doc).into_iter().map(|w| (w, None))); + } + // The root manifest, then every workspace member's — the manifests the + // hosted rewriter pins a crate in (each that declares it). + let mut manifests: Vec<(String, DocumentMut)> = Vec::new(); + if let Some(doc) = root { + // Any Socket pin the root or the config carries, a `[patch]` + // entry included: members may hold the declarations it governs. + let root_pins = doc.to_string().contains(PIN_NAME_PREFIX) + || config + .as_ref() + .is_some_and(|c| c.1.to_string().contains(PIN_NAME_PREFIX)); + // A vendored copy wired in the root or the config is weighed against + // every member's declarations ([`patch_bypassed`]). + let vendored = !wired_all.is_empty(); + manifests.push((CARGO_TOML.to_string(), doc)); + manifests.extend(read_members(ctx, &lock, root_pins, vendored, out).await); + } + let manifest: Option<&DocumentMut> = manifests.first().map(|(_, doc)| doc); - // Crate name → the `registry` value of each root-manifest declaration - // (`None` = declared with no registry key). Absent map = no usable - // manifest. - let decls = manifest.as_ref().map(|doc| { - let mut decls = manifest_declarations(doc); + // Crate name → each declaration of it in the root or a member manifest + // (its file, `registry` value — `None` = declared with no registry key — + // and version requirement). Absent map = no usable root manifest. + let decls = manifest.map(|doc| { + let mut decls = manifest_declarations(&manifests); + // `[patch]` is read from the root manifest only: cargo ignores (and + // warns about) a member's. for patch in patch_entries(doc) { if let Some(reg) = patch.registry { let name = patch.name.to_string(); - decls.entry(name).or_default().push(Some(reg.to_string())); + // An entry cargo ignores (a crates.io URL table replaces + // `[patch.crates-io]`; the config redefines the key) is + // Socket's wiring still (the ref stays) but pins nothing. + let mut decl = Decl::patch(CARGO_TOML, patch.source, reg); + decl.live = shadow + .as_ref() + .is_none_or(|s| s.why(patch.source, patch.key, None, Some(reg)).is_none()); + decls.entry(name).or_default().push(decl); } } - decls - }); - // The config's `[patch]` registry pins are also declarations cargo - // honors (only matters for crates the manifest also declares). - let decls = decls.map(|mut decls| { - if let Some((_, doc)) = &config { + // The config's `[patch]` registry pins are also declarations cargo + // honors (only matters for crates the manifests also declare). + if let Some((file, doc)) = &config { for patch in patch_entries(doc) { if let (Some(reg), Some(existing)) = (patch.registry, decls.get_mut(patch.name)) { - existing.push(Some(reg.to_string())); + existing.push(Decl::patch(file, patch.source, reg)); } } } @@ -178,65 +235,32 @@ pub(crate) async fn extract(ctx: &DiscoverCtx<'_>, out: &mut Discovery) { .map(|(_, doc)| registry_definitions(doc).into_iter().collect()) .unwrap_or_default(); - hosted_from_lock(ctx, &lock, decls.as_ref(), &definitions, out); + hosted_from_lock(ctx, &lock, &manifests, decls.as_ref(), &definitions, out); unpatched_twins(ctx, &lock, out); - if let (Some(decls), Some(doc)) = (&decls, &manifest) { - unresolved_manifest_pins(ctx, &lock, decls, doc, &definitions, out); - } - if let Some(doc) = &manifest { - // Cargo drops a manifest crates.io `[patch]` item that the project - // config redefines under the same key (any version), and a - // `[patch.""]` table replaces `[patch.crates-io]` - // wholesale — a Socket entry there is dead wiring, never attested. - let config_keys: Vec<(&str, String, Option)> = config - .iter() - .flat_map(|(file, doc)| { - patch_entries(doc) - .into_iter() - .filter(|e| is_crates_io_source(e.source)) - .map(move |e| (*file, e.key.to_string(), e.path.map(same_path_key))) - }) - .collect(); - let aliases = crates_io_url_alias_tables(doc); - let shadowed = |entry: &CargoPatchEntry<'_>| -> Option { - if !is_crates_io_source(entry.source) { - return None; - } - if let (true, Some(alias)) = (entry.source == "crates-io", aliases.first()) { - return Some(format!( - "the `[patch.\"{alias}\"]` table replaces `[patch.crates-io]` in cargo" - )); - } - // The same key wiring the same copy (a half-migrated project) - // is not a shadow: the config's own ref carries it. - let own = entry.path.map(same_path_key); - config_keys - .iter() - .find(|(_, key, path)| key == entry.key && *path != own) - .map(|(file, key, _)| { - format!("{file} redefines `[patch]` key `{key}`, which replaces it in cargo") - }) - }; - vendored_from_patches(ctx, CARGO_TOML, doc, &lock, &shadowed, out).await; - } - if let Some((file, doc)) = &config { - vendored_from_patches(ctx, file, doc, &lock, &|_| None, out).await; + if let Some(decls) = &decls { + unresolved_manifest_pins(ctx, &lock, decls, &definitions, out); } - let wired: Vec = manifest + // Only the entries cargo honors redirect declarations to a copy: a + // shadowed manifest entry covers nothing (cargo builds whatever replaced + // it). + let wired: Vec = wired_all .iter() - .chain(config.iter().map(|(_, doc)| doc)) - .flat_map(socket_copies) + .filter(|(_, why)| why.is_none()) + .map(|(w, _)| w.clone()) .collect(); - unattributed_tags(&lock, &wired, out); -} - -/// Every Socket copy a file's `[patch]` tables point at. -fn socket_copies(doc: &DocumentMut) -> Vec { - patch_entries(doc) - .into_iter() - .filter_map(|e| e.path.and_then(vendor_ref)) - .filter(|v| v.eco == "cargo") - .collect() + let bypass = Bypass { + lock: &lock, + decls: decls.as_ref(), + wired: &wired, + }; + if let Some(doc) = manifest { + vendored_from_patches(ctx, CARGO_TOML, doc, shadow.as_ref(), &bypass, out).await; + } + if let Some((file, doc)) = &config { + vendored_from_patches(ctx, file, doc, None, &bypass, out).await; + } + let copies: Vec = wired_all.into_iter().map(|(w, _)| w.vref).collect(); + unattributed_tags(&lock, &copies, out); } /// Diagnose Socket-tagged sourceless lock entries that no visible @@ -311,6 +335,60 @@ async fn read_toml(ctx: &DiscoverCtx<'_>, file: &str, out: &mut Discovery) -> Op parse_toml(file, &text, out) } +/// Every workspace member manifest (`/Cargo.toml`) of the root one, +/// parsed — the set the hosted rewriter pins in +/// ([`crate::utils::cargo_workspace::member_manifests_in`], keeping the +/// keys it accepts: never under `.socket/` — a vendored copy is Socket's, +/// not a declaration — or a build `target/`). Unparseable ones are skipped +/// (diagnosed). +/// +/// Members only ever decide about Socket wiring, so they are parsed only +/// when there is some they could decide about: a lock with a +/// Socket-sourced package, or a `socket-patch-` name in a member's own +/// text, the root's or the config's (a pin to diagnose or recognize, a +/// `[patch]` entry members declare, a lockless pin, [`UnlockedPin`]), +/// whatever the lock says (stale, unparseable or absent). `vendored` (the +/// root manifest or the config wires a Socket copy through `[patch]`, +/// whose lock entries are sourceless) always reads them: a member's +/// declaration from a source the `[patch]` does not redirect bypasses the +/// copy ([`patch_bypassed`]). +async fn read_members( + ctx: &DiscoverCtx<'_>, + lock: &Lock, + root_pins: bool, + vendored: bool, + out: &mut Discovery, +) -> Vec<(String, DocumentMut)> { + let socket_sourced = + vendored || (matches!(lock, Lock::Parsed(_)) && has_socket_sourced(ctx, lock)); + let texts: Vec<(String, String)> = + crate::utils::cargo_workspace::member_manifest_texts_in(&ctx.view) + .await + .into_iter() + .filter(|(rel, _)| crate::patch::redirect::is_cargo_member_manifest_key(rel)) + .collect(); + if !socket_sourced && !root_pins && !texts.iter().any(|(_, t)| t.contains(PIN_NAME_PREFIX)) { + return Vec::new(); + } + let mut members = Vec::new(); + for (rel, text) in texts { + ctx.adopt_read(&rel, &text); + if let Some(doc) = parse_toml(&rel, &text, out) { + members.push((rel, doc)); + } + } + members +} + +/// Whether the lock resolves some package from a Socket patch server. +fn has_socket_sourced(ctx: &DiscoverCtx<'_>, lock: &Lock) -> bool { + lock.packages().iter().any(|p| { + p.source + .as_deref() + .is_some_and(|s| source_uuid(ctx, s).is_some()) + }) +} + fn parse_toml(file: &str, text: &str, out: &mut Discovery) -> Option { toml_or_diag(file, text, TomlDiag::TrimEnd, out).map(toml_edit::Document::into_mut) } @@ -355,36 +433,312 @@ async fn read_config( // ── manifest / config shapes ───────────────────────────────────────────── -/// Every dependency declaration in a manifest, keyed by the crate it names -/// (`package = "…"` when renamed, else the key): the `registry` value of -/// each (`None` = no registry key). `workspace = true` inheritors are -/// skipped — the `[workspace.dependencies]` entry they inherit is itself -/// collected. -fn manifest_declarations(doc: &DocumentMut) -> BTreeMap>> { - let mut decls: BTreeMap>> = BTreeMap::new(); - for dep in dependency_entries(doc) { - decls.entry(dep.name).or_default().push(dep.registry); +/// Where cargo takes one declaration of a crate from. +#[derive(Debug, Clone, PartialEq, Eq)] +enum DeclSource { + /// A `[patch.]` entry's `registry` pin: it declares no + /// dependency, but redirects the declarations cargo resolves from + /// `
` ([`Decl::patched_by`]). + Patch(String), + /// A registry dependency: its `registry`, else crates.io. + Registry, + /// `registry-index = ""`. + RegistryIndex(String), + /// `path = "…"` (cargo builds it locally, whatever `registry` says). + Path, + /// `git = ""` (empty when the value is not a string). + Git(String), +} + +impl DeclSource { + /// [`Decl::patched_by`] for a declaration from this source with `registry` + /// (`None` = no registry key). + fn patched_by(&self, registry: Option<&str>, table: &str) -> bool { + match self { + DeclSource::Registry => match registry.map(str::trim) { + None | Some("crates-io") => is_crates_io_source(table), + Some(reg) => reg == table, + }, + DeclSource::RegistryIndex(url) => { + (is_crates_io_url_alias(url) && is_crates_io_source(table)) + || patch_source_key(url) == patch_source_key(table) + } + DeclSource::Git(url) => git_key(url).is_some_and(|k| git_key(table) == Some(k)), + DeclSource::Path | DeclSource::Patch(_) => false, + } + } + + /// The source a `Cargo.lock` `source` names, as far as a `[patch]` table + /// can redirect it: crates.io, or a git repository (its `?query` / `#rev` + /// cut). `None` for anything else. + fn of_lock_source(source: &str) -> Option { + if is_crates_io_lock_source(source) { + return Some(DeclSource::Registry); + } + let repo = source.trim().strip_prefix("git+")?; + Some(DeclSource::Git( + repo.split(['?', '#']).next().unwrap_or(repo).to_string(), + )) + } +} + +/// One declaration of a crate: the root-relative file holding it, its +/// `registry` value (`None` = no registry key), its version requirement +/// and where cargo takes it from. +#[derive(Debug, Clone)] +struct Decl { + file: String, + registry: Option, + /// The `version` requirement (`None` for a `[patch]` registry pin, or + /// a declaration without one). + version: Option, + source: DeclSource, + /// `false` for a `[workspace.dependencies]` template no manifest + /// inherits (`workspace = true`): cargo resolves it for nobody. + live: bool, +} + +impl Decl { + /// A `[patch.
]` entry's registry pin. + fn patch(file: &str, table: &str, registry: &str) -> Self { + Decl { + file: file.to_string(), + registry: Some(registry.to_string()), + version: None, + source: DeclSource::Patch(table.to_string()), + live: true, + } + } + + /// The Socket patch uuid a well-formed `socket-patch-` registry + /// names. + fn socket_uuid(&self) -> Option { + socket_patch_name_uuid(self.registry.as_deref()?, false) + } + + /// Whether the registry is `socket-patch-…`-shaped (well-formed or not). + fn socket_shaped(&self) -> bool { + self.registry + .as_deref() + .is_some_and(|r| r.trim().starts_with(PIN_NAME_PREFIX)) + } + + /// The Socket patch this declaration itself pins the crate to: a + /// `socket-patch-` registry on a registry dependency or a + /// `[patch]` entry. Beside `path`, `git` or `registry-index` the + /// registry key does not decide where cargo takes the crate from (the + /// hosted rewriter's own pinned check), so that is no pin. + fn pin(&self) -> Option { + match self.source { + DeclSource::Registry | DeclSource::Patch(_) => self.socket_uuid(), + _ => None, + } + } + + /// Whether the `[patch.
]` table redirects this declaration: + /// + /// * a registry dependency of that registry (`crates-io`, or a + /// crates.io URL alias, for one with no `registry` or + /// `registry = "crates-io"`, which cargo reads as crates.io too); + /// * a `registry-index` dependency of that index URL + /// ([`patch_source_key`], the spelling `[patch]` tables compare in); + /// one naming the crates.io git index is crates.io itself (the same + /// cargo source), so `[patch.crates-io]` redirects it too; + /// * a git dependency of that repository ([`git_key`], cargo's + /// canonical spelling). + /// + /// Any other difference reads as another source: at worst a missed + /// redirect, so a withheld statement, never a false one. + fn patched_by(&self, table: &str) -> bool { + self.source.patched_by(self.registry.as_deref(), table) + } + + /// "`` declares `` `` from ``", for + /// messages. A `path` / `git` / `registry-index` overrides a Socket + /// registry named beside it: that is said too, or the message reads as if + /// the registry were missing. + fn describe(&self, name: &str) -> String { + let req = self + .version + .as_deref() + .map_or_else(|| "with no version".to_string(), |v| format!("{v:?}")); + let beside = match self.registry.as_deref() { + Some(reg) + if reg.starts_with(PIN_NAME_PREFIX) && self.source != DeclSource::Registry => + { + format!(", which overrides its registry = \"{reg}\"") + } + _ => String::new(), + }; + format!( + "{} declares {name} {req} from {}{beside}", + self.file, + self.origin() + ) + } + + /// Where cargo takes this dependency from, for messages. + fn origin(&self) -> String { + match (&self.source, self.registry.as_deref()) { + (DeclSource::Git(_), _) => "a git repository".to_string(), + (DeclSource::Path, _) => "a local path".to_string(), + (DeclSource::RegistryIndex(url), _) => format!("the registry index {url}"), + (_, Some(reg)) => format!("registry `{reg}`"), + (_, None) => "crates.io".to_string(), + } + } +} + +/// `url` with its scheme and host lowercased (both are case-insensitive), +/// trimmed, or `None` when empty. +fn lower_host(url: &str) -> Option { + let url = url.trim(); + if url.is_empty() { + return None; + } + let host_end = url.find("://").map(|s| s + 3).map_or(0, |start| { + start + url[start..].find('/').unwrap_or(url.len() - start) + }); + Some(format!( + "{}{}", + url[..host_end].to_ascii_lowercase(), + &url[host_end..] + )) +} + +/// A git repository URL as cargo's `CanonicalUrl` spells it: one trailing +/// `/` dropped, a GitHub URL lowercased whole (GitHub paths are +/// case-insensitive), a trailing `.git` dropped. `None` when empty. +fn git_key(url: &str) -> Option { + let mut url = lower_host(url)?; + if url.ends_with('/') { + url.pop(); + } + let github = url + .split_once("://") + .is_some_and(|(_, rest)| rest.split('/').next() == Some("github.com")); + if github { + url = url.to_ascii_lowercase(); + } + if let Some(stripped) = url.strip_suffix(".git") { + url = stripped.to_string(); + } + Some(url) +} + +/// The source of a lock entry of `name` at exactly `version` that is not a +/// Socket patch server's: a copy cargo builds from crates.io, another +/// registry or git beside the patched one. +fn same_version_foreign_entry<'l>( + ctx: &DiscoverCtx<'_>, + lock: &'l Lock, + name: &str, + version: &str, +) -> Option<&'l str> { + lock.packages() + .iter() + .filter(|p| p.name == name && p.version == version) + .find_map(|p| { + p.source + .as_deref() + .filter(|s| source_uuid(ctx, s).is_none()) + }) +} + +/// Crate name → its declarations ([`Decl`]). +type Decls = BTreeMap>; + +/// Every dependency declaration in `manifests` (root first, then members), +/// keyed by the crate it names (`package = "…"` when renamed, else the +/// key). `workspace = true` inheritors are not declarations of their own — +/// the root's `[workspace.dependencies]` template they inherit is, and it +/// is live only when some manifest inherits its key. +fn manifest_declarations(manifests: &[(String, DocumentMut)]) -> Decls { + let entries: Vec<(&str, DepEntry)> = manifests + .iter() + .flat_map(|(file, doc)| { + dependency_entries(doc) + .into_iter() + .map(move |d| (file.as_str(), d)) + }) + .collect(); + let inherited: BTreeSet<&str> = entries + .iter() + .filter(|(_, d)| d.inherits) + .map(|(_, d)| d.key.as_str()) + .collect(); + let mut decls = Decls::new(); + for (file, dep) in entries.iter().filter(|(_, d)| !d.inherits) { + let decl = Decl { + file: file.to_string(), + registry: dep.registry.clone(), + version: dep.version.clone(), + source: dep.source.clone(), + live: !dep.template || inherited.contains(&dep.key.as_str()), + }; + decls.entry(dep.name.clone()).or_default().push(decl); + } + decls +} + +/// The distinct values, in first-seen order. +fn distinct(values: impl IntoIterator) -> Vec { + let mut out: Vec = Vec::new(); + for value in values { + if !out.contains(&value) { + out.push(value); + } } + out +} + +/// The distinct files among `decls`, in order, joined for a message. +fn decl_files<'a>(decls: impl IntoIterator) -> String { + distinct(decls.into_iter().map(|d| d.file.as_str())).join(", ") +} + +/// Each of `decls` described ([`Decl::describe`]), joined for a message. +fn describe_all<'a>(decls: impl IntoIterator, name: &str) -> String { decls + .into_iter() + .map(|d| d.describe(name)) + .collect::>() + .join(", and ") } -/// One manifest dependency declaration (see [`manifest_declarations`]). +/// One manifest dependency entry (see [`manifest_declarations`]). struct DepEntry { + /// The table key. + key: String, + /// The crate: `package = "…"` when renamed, else the key. name: String, registry: Option, /// The `version` requirement (a bare string entry is one). version: Option, + source: DeclSource, + /// A `[workspace.dependencies]` template. + template: bool, + /// A `workspace = true` inheritor (of the template keyed `key`). + inherits: bool, } fn dependency_entries(doc: &DocumentMut) -> Vec { let mut deps = Vec::new(); - let mut tables: Vec<&dyn TableLike> = Vec::new(); + let mut tables: Vec<(&dyn TableLike, bool)> = Vec::new(); let root = doc.as_table(); for kind in DEP_TABLES { - tables.extend(root.get(kind).and_then(Item::as_table_like)); + tables.extend( + root.get(kind) + .and_then(Item::as_table_like) + .map(|t| (t, false)), + ); } if let Some(ws) = root.get("workspace").and_then(Item::as_table_like) { - tables.extend(ws.get("dependencies").and_then(Item::as_table_like)); + tables.extend( + ws.get("dependencies") + .and_then(Item::as_table_like) + .map(|t| (t, true)), + ); } if let Some(targets) = root.get("target").and_then(Item::as_table_like) { for (_, target) in targets.iter() { @@ -392,27 +746,47 @@ fn dependency_entries(doc: &DocumentMut) -> Vec { continue; }; for kind in DEP_TABLES { - tables.extend(target.get(kind).and_then(Item::as_table_like)); + tables.extend( + target + .get(kind) + .and_then(Item::as_table_like) + .map(|t| (t, false)), + ); } } } - for table in tables { + for (table, template) in tables { for (key, item) in table.iter() { let dep = if let Some(version) = item.as_str() { DepEntry { + key: key.to_string(), name: key.to_string(), registry: None, version: Some(version.to_string()), + source: DeclSource::Registry, + template, + inherits: false, } } else if let Some(entry) = item.as_table_like() { - if entry.get("workspace").and_then(Item::as_bool) == Some(true) { - continue; - } let field = |k: &str| entry.get(k).and_then(Item::as_str).map(str::to_string); + let source = if entry.contains_key("git") { + DeclSource::Git(field("git").unwrap_or_default()) + } else if entry.contains_key("path") { + DeclSource::Path + } else if let Some(url) = field("registry-index") { + DeclSource::RegistryIndex(url) + } else { + DeclSource::Registry + }; DepEntry { + key: key.to_string(), name: field("package").unwrap_or_else(|| key.to_string()), registry: field("registry"), version: field("version"), + source, + template, + inherits: !template + && entry.get("workspace").and_then(Item::as_bool) == Some(true), } } else { continue; @@ -437,10 +811,15 @@ fn source_uuid(ctx: &DiscoverCtx<'_>, source: &str) -> Option { fn hosted_from_lock( ctx: &DiscoverCtx<'_>, lock: &Lock, - decls: Option<&BTreeMap>>>, + manifests: &[(String, DocumentMut)], + decls: Option<&Decls>, definitions: &BTreeMap, out: &mut Discovery, ) { + let manifests = ManifestTexts { + docs: manifests, + texts: std::cell::OnceCell::new(), + }; for pkg in lock.packages() { let Some(source) = pkg.source.as_deref() else { continue; @@ -488,43 +867,64 @@ fn hosted_from_lock( true, ); let Some(regs) = decls.and_then(|d| d.get(&pkg.name)) else { - // Not declared by the root manifest (a member's dependency the - // root lock describes), or no usable manifest: the lock is the - // only word. + // Declared by no manifest (a transitive-only crate), or no + // usable root manifest: the lock is the only word. out.push(lock_ref); continue; }; - if regs.iter().any(|r| r.as_deref() == Some(reg.as_str())) { + // Some declaration names this patch's registry (even one a `path`, + // `git` or `registry-index` overrides): Socket's wiring is present, + // so the lock ref stands, and the other declarations decide whether + // it is attested ([`beside_the_pin`]). + if regs + .iter() + .any(|d| d.registry.as_deref() == Some(reg.as_str())) + { out.push(lock_ref); + beside_the_pin(ctx, lock, &manifests, pkg, &purl, &uuid, regs, out); continue; } - let others: Vec = regs + // Pins to another Socket patch that may select this version; one + // whose requirement excludes it (or a template nobody inherits) is + // about another version, so it neither conflicts nor keeps the + // stale entry. + let others: Vec<(&str, String)> = regs .iter() - .flatten() - .filter_map(|r| socket_patch_name_uuid(r, false)) + .filter(|d| { + d.live + && cargo_req_selects(d.version.as_deref(), &pkg.version, &[]) + != CargoReqMatch::NotOurs + }) + .filter_map(|d| Some((d.file.as_str(), d.pin()?))) .collect(); if others.is_empty() { + let files = decl_files(regs); + let verb = if files.contains(", ") { + "declare" + } else { + "declares" + }; out.diag( DIAG_REF_INVALID, CARGO_LOCK, format!( - "{CARGO_LOCK}: {purl} resolves from Socket patch {uuid}, but {CARGO_TOML} \ - declares {} without registry = \"{reg}\" — cargo re-resolves it from the \ + "{CARGO_LOCK}: {purl} resolves from Socket patch {uuid}, but {files} \ + {verb} {} without registry = \"{reg}\" — cargo re-resolves it from the \ declared source; not counted", pkg.name ), ); continue; } - // The manifest routes the crate to a DIFFERENT Socket patch than the + // A manifest routes the crate to a DIFFERENT Socket patch than the // lock pins: emit both wirings and let the CLI's conflict gate refuse // to pick one. out.push(lock_ref); - for other in others { + for (file, other) in others { out.push(PatchedRef::hosted( purl.clone(), other, - CARGO_TOML, + file, None, None, true, @@ -533,6 +933,224 @@ fn hosted_from_lock( } } +/// The root and member manifests, and their texts as the hosted rewriter's +/// planner reads them, serialized once on first need. +struct ManifestTexts<'a> { + docs: &'a [(String, DocumentMut)], + texts: std::cell::OnceCell>, +} + +impl ManifestTexts<'_> { + fn texts(&self) -> &[(String, String)] { + self.texts.get_or_init(|| { + self.docs + .iter() + .map(|(file, doc)| (file.clone(), doc.to_string())) + .collect() + }) + } +} + +/// What one declaration of a crate means for its Socket-locked +/// `name@version` ([`route`]). +enum Route { + /// Routes it to this Socket patch: its own `socket-patch-` + /// registry, or a `[patch]` entry's that redirects it. + Pin(String), + /// Cargo builds its own, unpatched copy for it, which may be this + /// version: [`CargoReqMatch::Ours`] when only this locked version + /// matches its requirement, [`CargoReqMatch::Ambiguous`] when another + /// locked one does too. + Unpinned(CargoReqMatch), + /// Says nothing about this version: its requirement excludes it, it is + /// a `[patch]` entry (a redirection, counted through the declarations + /// it redirects), a template nobody inherits, or a malformed + /// `socket-patch-*` name (diagnosed by [`unresolved_manifest_pins`]). + Elsewhere, +} + +/// Classify `d` for the locked `version` of its crate, `others` being the +/// other versions the lock holds of it and `patch_pins` the `[patch.
]` +/// Socket registry pins (table, uuid) cargo honors. +fn route(d: &Decl, patch_pins: &[(&str, String)], version: &str, others: &[String]) -> Route { + if matches!(d.source, DeclSource::Patch(_)) || !d.live { + return Route::Elsewhere; + } + let selects = cargo_req_selects(d.version.as_deref(), version, others); + if selects == CargoReqMatch::NotOurs { + return Route::Elsewhere; + } + if let Some(uuid) = d.pin() { + return Route::Pin(uuid); + } + if d.source == DeclSource::Registry && d.socket_shaped() { + return Route::Elsewhere; + } + match patch_pins.iter().find(|(table, _)| d.patched_by(table)) { + Some((_, uuid)) => Route::Pin(uuid.clone()), + None => Route::Unpinned(selects), + } +} + +/// The other declarations beside a lock entry pinned to Socket patch +/// `uuid` ([`route`] for each): +/// +/// * one routed to ANOTHER Socket patch that may select this version is a +/// conflict: its wiring is emitted beside the lock ref, so the CLI's +/// conflict gate refuses to pick one; +/// * one cargo builds its own copy for (crates.io, another registry or +/// registry index, a path, git — including a `socket-patch-` +/// registry a `path` / `git` / `registry-index` overrides) marks the ref +/// [`UnattestedKind::CargoUnpinnedDeclaration`]: cargo resolves each +/// source on its own, so that copy is unpatched and may be this very +/// version. The hosted rewriter pins every declaration, so this is a +/// partial revert or a hand edit. The ref stays (rollback, remove and +/// the ledgers still see the pin); the detail names the declarations and +/// whether a re-scan can pin them. +/// +/// When the lock already holds a same-version copy from another source, +/// [`unpatched_twins`] withholds the ref and names that entry (#679), so +/// nothing is added here. +#[allow(clippy::too_many_arguments)] +fn beside_the_pin( + ctx: &DiscoverCtx<'_>, + lock: &Lock, + manifests: &ManifestTexts<'_>, + pkg: &LockedPackage, + purl: &str, + uuid: &str, + regs: &[Decl], + out: &mut Discovery, +) { + let same_name = || lock.packages().iter().filter(|p| p.name == pkg.name); + if same_version_foreign_entry(ctx, lock, &pkg.name, &pkg.version).is_some() { + return; + } + let others = distinct( + same_name() + .filter(|p| p.version != pkg.version) + .map(|p| p.version.clone()), + ); + // The patches the lock resolves the crate from. + let locked_from: Vec = same_name() + .filter_map(|p| p.source.as_deref().and_then(|s| source_uuid(ctx, s))) + .collect(); + let patch_pins: Vec<(&str, String)> = regs + .iter() + .filter_map(|d| match &d.source { + DeclSource::Patch(table) if d.live => Some((table.as_str(), d.socket_uuid()?)), + _ => None, + }) + .collect(); + let mut unpinned: Vec<(&Decl, CargoReqMatch)> = Vec::new(); + for d in regs { + match route(d, &patch_pins, &pkg.version, &others) { + // A patch the lock resolves the crate from at all is its own + // ref (that registry serves only its own version, a separate, + // patched package): no conflict. + Route::Pin(other) if other != uuid && !locked_from.contains(&other) => { + out.push(PatchedRef::hosted( + purl.to_string(), + other, + &d.file, + None, + None, + true, + )); + } + Route::Unpinned(selects) => unpinned.push((d, selects)), + Route::Pin(_) | Route::Elsewhere => {} + } + } + let Some((first, _)) = unpinned.first() else { + return; + }; + let declared = describe_all(unpinned.iter().map(|(d, _)| *d), &pkg.name); + let (that, each) = if unpinned.len() == 1 { + ("that declaration", "the crate declaring it") + } else { + ("those declarations", "each crate declaring one") + }; + // What a re-scan can do: the hosted rewriter pins a crates.io + // declaration whose requirement selects only the patched version among + // the locked ones, and refuses every other kind. + let refused: Vec = unpinned + .iter() + .filter_map(|(d, selects)| { + // `registry = "crates-io"` is crates.io itself, which the + // rewriter re-pins like a declaration with no `registry` (#386). + let registry = d.registry.as_deref().filter(|r| r.trim() != "crates-io"); + let why = match (&d.source, registry, selects) { + (DeclSource::Registry, None, CargoReqMatch::Ours) => return None, + (DeclSource::Registry, None, _) => { + let also: Vec<&str> = others + .iter() + .filter(|v| { + cargo_req_selects(d.version.as_deref(), v, &[]) + != CargoReqMatch::NotOurs + }) + .map(String::as_str) + .collect(); + format!( + "its requirement also matches {} {} locked beside it", + pkg.name, + also.join(", ") + ) + } + (DeclSource::Registry | DeclSource::RegistryIndex(_), ..) => { + "it names another registry".to_string() + } + _ => "it builds a path or git source".to_string(), + }; + Some(format!("{}'s declaration ({why})", d.file)) + }) + .collect(); + // Every declaration may pass on its own and the rewriter still refuse the + // dependency over another one (the pin's own requirement matching another + // locked version, a `path` template nobody inherits): it skips the whole + // dependency on the first refusal, so ask its planner, dry, over the same + // manifests. + let planner_refusal = || { + crate::patch::redirect::plan_cargo_tomls( + manifests.texts(), + &pkg.name, + &pkg.version, + &others, + &hosted_pin_name(uuid), + ) + .err() + }; + let remedy = if !refused.is_empty() { + format!( + "`socket-patch scan` cannot pin {}, so change that declaration or roll the \ + patch back", + refused.join(", or ") + ) + } else if let Some((file, why)) = planner_refusal() { + format!( + "`socket-patch scan` cannot pin {} ({file}: {why}), so change that declaration \ + or roll the patch back", + pkg.name + ) + } else { + "re-run `socket-patch scan` to pin it too".to_string() + }; + out.unattested( + purl, + uuid, + &first.file, + format!( + "{declared}, not registry \"{}\": cargo resolves {that} as a separate \ + package, so {each} builds an unpatched {} that the requirement allows to be \ + {}; {remedy}", + hosted_pin_name(uuid), + pkg.name, + pkg.version + ), + UnattestedKind::CargoUnpinnedDeclaration, + ); +} + /// Record every lock block that resolves a Socket-hosted crate's /// `name@version` from another source ([`Discovery::unpatched_copy`]). /// Cargo never unifies packages of different sources, so a dependency that @@ -584,88 +1202,178 @@ fn unpatched_twins(ctx: &DiscoverCtx<'_>, lock: &Lock, out: &mut Discovery) { } /// Diagnose `Cargo.toml` Socket-registry pins no lock entry turns into a ref -/// (no exact version to attest), and malformed `socket-patch-*` names. +/// (no exact version to attest), and malformed `socket-patch-*` names. One +/// diagnostic per crate and patch, naming every manifest that pins it. /// /// With NO `Cargo.lock` at all (a library that gitignores it; `rewrite_cargo` /// supports that state — `LockCommit::Absent` — and writes only the pin and -/// the registry definition) a pin is still live wiring for the crate: every -/// declaration of it routes to `socket-patch-`, the project config defines -/// that registry on the Socket host for the same `U`, and that registry -/// serves only the patched version. It is recorded as an [`UnlockedPin`] -/// carrying the manifest's version requirements, so a redirect-ledger record -/// whose exact version satisfies them stays live -/// ([`Discovery::hosted_claim`]); it is still not a ref (no version to -/// attest on its own). +/// the registry definition) a pin is still live wiring for the crate: no +/// declaration routes it to another Socket registry, the project config +/// defines that registry on the Socket host for the same `U`, and that +/// registry serves only the patched version. It is recorded as ONE +/// [`UnlockedPin`] per crate and patch, carrying the pinning declarations' +/// version requirements (from the root and every member) and, as +/// `excluded_reqs`, those of the declarations without a pin: the rewriter +/// leaves a declaration alone only when its requirement excludes the +/// patched version, so a redirect-ledger record stays live +/// ([`Discovery::hosted_claim`]) when its exact version satisfies every +/// pinned requirement and no unpinned one. An unpinned declaration with no +/// version requirement (git, path) may resolve any version: no pin. It is +/// still not a ref (no version to attest on its own). fn unresolved_manifest_pins( ctx: &DiscoverCtx<'_>, lock: &Lock, - decls: &BTreeMap>>, - manifest: &DocumentMut, + decls: &Decls, definitions: &BTreeMap, out: &mut Discovery, ) { - let deps = dependency_entries(manifest); for (name, regs) in decls { - for reg in regs.iter().flatten() { - if !reg.trim().starts_with(PIN_NAME_PREFIX) { - continue; - } - let Some(uuid) = socket_patch_name_uuid(reg, false) else { - out.diag( - DIAG_REF_INVALID, - CARGO_TOML, - format!( - "{CARGO_TOML}: {name} is pinned to registry {reg:?}, which is not a \ - socket-patch- name" - ), - ); - continue; - }; - let socket_sourced = lock.packages().iter().any(|p| { - p.name == *name - && p.source - .as_deref() - .and_then(|s| source_uuid(ctx, s)) - .is_some() - }); - if socket_sourced { - continue; // a ref (or the conflict pair) came from the lock - } + let malformed = distinct( + regs.iter() + .filter(|d| d.socket_shaped() && d.socket_uuid().is_none()) + .map(|d| (d.file.as_str(), d.registry.as_deref().unwrap_or_default())), + ); + // Classified like a locked pin ([`route`]): only a live [`Decl::pin`] + // pins. + let live: Vec<&Decl> = regs.iter().filter(|d| d.live).collect(); + let uuids = distinct(live.iter().filter_map(|d| d.pin())); + for (file, reg) in malformed { + out.diag( + DIAG_REF_INVALID, + file, + format!( + "{file}: {name} is pinned to registry {reg:?}, which is not a \ + socket-patch- name" + ), + ); + } + // A well-formed Socket registry that pins nothing: say so, since the + // file reads like wiring. + let ineffective = distinct( + regs.iter() + .filter(|d| d.socket_uuid().is_some() && (d.pin().is_none() || !d.live)) + .map(|d| { + let why = if d.live { + format!( + "cargo builds it from {}, whatever `registry` says", + d.origin() + ) + } else if matches!(d.source, DeclSource::Patch(_)) { + "cargo ignores that `[patch]` entry (a crates.io URL table or the project \ + config replaces it)" + .to_string() + } else { + "no manifest inherits that `[workspace.dependencies]` entry".to_string() + }; + (d.file.as_str(), why) + }), + ); + for (file, why) in ineffective { + out.diag( + DIAG_REF_INVALID, + file, + format!("{file}: {name}'s Socket patch registry pins nothing there: {why}"), + ); + } + if uuids.is_empty() { + continue; + } + // Patches the lock resolves the crate from, and those a ref + // already carries (the lock's, or a conflict pair beside it). + let locked_from: Vec = lock + .packages() + .iter() + .filter(|p| p.name == *name) + .filter_map(|p| p.source.as_deref().and_then(|s| source_uuid(ctx, s))) + .collect(); + // Whether the lock also resolves the crate from a non-Socket source + // (or a path): then "only from other Socket patches" is not what it + // says. + let locked_elsewhere = lock.packages().iter().any(|p| { + p.name == *name + && p.source + .as_deref() + .is_none_or(|s| source_uuid(ctx, s).is_none()) + }); + let carried = |uuid: &str| { + locked_from.iter().any(|u| u == uuid) + || out.refs.iter().any(|r| { + r.uuid == uuid + && crate::utils::purl::purl_parts(&r.purl) + .is_some_and(|(ty, n, _)| ty == "cargo" && n == *name) + }) + }; + let uuids: Vec = uuids.into_iter().filter(|u| !carried(u)).collect(); + for uuid in &uuids { + let pinning: Vec<&Decl> = live + .iter() + .copied() + .filter(|d| d.pin().as_deref() == Some(uuid.as_str())) + .collect(); + let reg = pinning[0].registry.as_deref().unwrap_or_default(); + let file = pinning[0].file.as_str(); let why = match lock { Lock::Unparseable => continue, // already diagnosed Lock::Absent => { - let only_this_registry = regs.iter().all(|r| r.as_deref() == Some(reg)); + // Another Socket registry (or a malformed one) for the + // same crate is a conflict, never a pin. + let only_this_patch = live + .iter() + .filter(|d| { + d.socket_shaped() + && matches!(d.source, DeclSource::Registry | DeclSource::Patch(_)) + }) + .all(|d| d.pin().as_deref() == Some(uuid.as_str())); let defined_here = definitions - .get(reg.as_str()) + .get(reg) .and_then(|index| source_uuid(ctx, index)) - .is_some_and(|u| u == uuid); - let version_reqs: Vec = deps + .is_some_and(|u| u == *uuid); + let version_reqs = distinct(pinning.iter().filter_map(|d| d.version.clone())); + // Every live dependency declaration this patch does + // not pin (a `path` / `git` / `registry-index` one + // beside a `socket-patch-` registry included); `None` + // when one carries no requirement: it may resolve any + // version. + let excluded_reqs: Option> = live .iter() - .filter(|d| d.name == *name && d.registry.as_deref() == Some(reg)) - .filter_map(|d| d.version.clone()) + .filter(|d| !matches!(d.source, DeclSource::Patch(_)) && d.pin().is_none()) + .map(|d| d.version.clone()) .collect(); - if only_this_registry && defined_here && !version_reqs.is_empty() { + if let (true, true, false, Some(excluded_reqs)) = ( + only_this_patch, + defined_here, + version_reqs.is_empty(), + excluded_reqs, + ) { + let excluded_reqs = distinct(excluded_reqs); out.unlocked_pin(UnlockedPin { ecosystem: "cargo".to_string(), name: name.clone(), uuid: uuid.clone(), - file: CARGO_TOML.into(), + file: file.into(), version_reqs, - index_url: definitions.get(reg.as_str()).cloned(), + excluded_reqs, + index_url: definitions.get(reg).cloned(), }); } format!("there is no {CARGO_LOCK} to fix its version") } + Lock::Parsed { .. } if !locked_from.is_empty() && !locked_elsewhere => format!( + "{CARGO_LOCK} resolves it only from other Socket patches, at versions \ + this pin's requirement excludes (a stale lock: the next build \ + re-resolves this declaration)" + ), Lock::Parsed { .. } => format!( "{CARGO_LOCK} does not resolve it from a Socket patch server (a stale \ lock, or a patch server outside the allowlist — see \ --patch-server-url)" ), }; + let files = decl_files(pinning.iter().copied()); out.diag( DIAG_REF_UNATTRIBUTABLE, - CARGO_TOML, - format!("{CARGO_TOML}: {name} is pinned to Socket patch {uuid}, but {why}"), + file, + format!("{files}: {name} is pinned to Socket patch {uuid}, but {why}"), ); } } @@ -712,46 +1420,396 @@ async fn copy_tag(ctx: &DiscoverCtx<'_>, artifact_rel: &str, out: &mut Discovery } } -/// Vendored refs from one file's `[patch]` tables. `shadowed` names why -/// cargo ignores an entry (a higher-precedence item replaces it), if so. -async fn vendored_from_patches( - ctx: &DiscoverCtx<'_>, - file: &str, - doc: &DocumentMut, - lock: &Lock, - shadowed: &(dyn Fn(&CargoPatchEntry<'_>) -> Option + Sync), - out: &mut Discovery, -) { - for entry in patch_entries(doc) { - let Some(path) = entry.path else { - continue; // a git / registry patch - }; - if !names_vendor_dir(path) { - continue; // a user path, or the uuid-less `.socket/cargo-patches/` - } - let name = entry.name; - if let Some(why) = shadowed(&entry) { - out.diag( - DIAG_REF_INVALID, - file, - format!( - "{file}: [patch] entry `{}` for {name} points at {path:?}, but {why}; \ - not counted", - entry.key - ), - ); - continue; - } - let Some(vref) = vendor_ref(path).filter(|v| v.eco == "cargo") else { - out.diag( - DIAG_REF_INVALID, - file, - format!( - "{file}: [patch] path {path:?} for {name} is not this project's \ - .socket/vendor/cargo//- copy" - ), - ); - continue; +/// One `[patch.
]` entry that points at a Socket copy. +#[derive(Debug, Clone)] +struct WiredCopy { + table: String, + key: String, + path: String, + vref: VendorRef, +} + +/// Every Socket copy a file's `[patch]` tables point at. +fn socket_copies(doc: &DocumentMut) -> Vec { + patch_entries(doc) + .into_iter() + .filter_map(|e| { + let path = e.path?; + let vref = vendor_ref(path).filter(|v| v.eco == "cargo")?; + Some(WiredCopy { + table: e.source.to_string(), + key: e.key.to_string(), + path: path.to_string(), + vref, + }) + }) + .collect() +} + +/// Why cargo ignores a root-manifest `[patch]` entry, if it does: a +/// `[patch.""]` table replaces `[patch.crates-io]` +/// wholesale, and a project-config `[patch]` item for the same source and +/// key replaces the manifest's (unless both wire the same path, the +/// half-migrated shape attested through the config). Config entries are +/// never shadowed. +struct Shadow { + /// The manifest's `[patch.""]` table keys. + aliases: Vec, + /// The config's `[patch]` items. + config_keys: Vec, + /// The config's `[registries.] index` values: a `[patch.]` + /// table patches the same source as `[patch.""]`. + registries: BTreeMap, +} + +/// One config `[patch]` item as [`Shadow`] compares it: (file, source +/// table, key, path, registry). +type ConfigPatchItem = (&'static str, String, String, Option, Option); + +/// A `[patch.]` table name, a `registry-index` or a lock `source` +/// as cargo's `CanonicalUrl` spells it, for comparison: a URL's host case, +/// a trailing `/` or `.git` and a GitHub path's case ignored, a lock's +/// `registry+` prefix dropped, `sparse+` kept (another source); a registry +/// name verbatim. +fn patch_source_key(source: &str) -> String { + let t = source.trim(); + // A sparse index keeps its `sparse+` (another cargo source than the git + // index at the same URL); only the host case and a trailing `/` differ. + let key = if let Some(rest) = t.strip_prefix("sparse+") { + lower_host(rest).map(|url| format!("sparse+{}", url.trim_end_matches('/'))) + } else if t.contains("://") { + git_key(t.strip_prefix("registry+").unwrap_or(t)) + } else { + None + }; + key.unwrap_or_else(|| t.to_string()) +} + +/// Whether two `[patch.]` table names patch the same source: +/// crates.io under any spelling, else the same name or URL +/// ([`patch_source_key`]). +fn same_patch_source(a: &str, b: &str) -> bool { + if is_crates_io_source(a) || is_crates_io_source(b) { + return is_crates_io_source(a) && is_crates_io_source(b); + } + patch_source_key(a) == patch_source_key(b) +} + +impl Shadow { + fn new(manifest: &DocumentMut, config: Option<&(&'static str, DocumentMut)>) -> Self { + let config_keys = config + .into_iter() + .flat_map(|(file, doc)| { + patch_entries(doc).into_iter().map(move |e| { + ( + *file, + e.source.to_string(), + e.key.to_string(), + e.path.map(same_path_key), + e.registry.map(|r| r.trim().to_string()), + ) + }) + }) + .collect(); + let registries = config + .and_then(|(_, doc)| doc.get("registries")) + .and_then(toml_edit::Item::as_table_like) + .into_iter() + .flat_map(|t| t.iter()) + .filter_map(|(name, item)| { + let index = item.as_table_like()?.get("index")?.as_str()?; + Some((name.to_string(), index.to_string())) + }) + .collect(); + Shadow { + aliases: crates_io_url_alias_tables(manifest), + config_keys, + registries, + } + } + + /// `table`, a registry name resolved to its configured index URL. + fn resolved<'s>(&'s self, table: &'s str) -> &'s str { + self.registries + .get(table.trim()) + .map_or(table, String::as_str) + } + + /// Why cargo ignores the manifest's `[patch.
]` item `key` + /// (wiring `path`, or Socket registry `registry`), if it does. + fn why( + &self, + table: &str, + key: &str, + path: Option<&str>, + registry: Option<&str>, + ) -> Option { + if let (true, Some(alias)) = (table == "crates-io", self.aliases.first()) { + return Some(format!( + "the `[patch.\"{alias}\"]` table replaces `[patch.crates-io]` in cargo" + )); + } + // Only identical wiring is the same: the same path, or the same + // registry for path-less items. Two path-less items otherwise (a + // registry pin and a git fork, say) are not. + let own = path.map(same_path_key); + let own_reg = registry.map(str::trim); + let same_wiring = |p: &Option, r: &Option| match (&own, own_reg) { + (Some(_), _) => *p == own, + (None, Some(reg)) => p.is_none() && r.as_deref() == Some(reg), + (None, None) => false, + }; + self.config_keys + .iter() + .find(|(_, src, k, p, r)| { + k == key + && same_patch_source(self.resolved(src), self.resolved(table)) + && !same_wiring(p, r) + }) + .map(|(file, _, k, _, _)| { + format!("{file} redefines `[patch]` key `{k}`, which replaces it in cargo") + }) + } +} + +/// What a vendored copy is weighed against for copies its `[patch]` wiring +/// does not redirect ([`patch_bypassed`]). +struct Bypass<'a> { + lock: &'a Lock, + /// Every declaration in the root and member manifests (`None`: no + /// usable root manifest). + decls: Option<&'a Decls>, + /// The `[patch]` entries pointing at a Socket copy that cargo honors + /// (a shadowed manifest entry redirects nothing). + wired: &'a [WiredCopy], +} + +/// Mark the vendored ref `purl` (the copy `vref` of `name`@`version`) +/// [`UnattestedKind::CargoPatchBypassed`] when the build may also compile +/// a copy of that version its wiring does not reach. A `[patch.
]` +/// entry replaces the crate only in declarations from `
` (v5 +/// `vendor` writes `[patch.crates-io]`), so either of these builds +/// unpatched bytes: +/// +/// * a declaration (the root's or a workspace member's) that [`route`] +/// finds [`Route::Unpinned`] against the honored entries pointing at +/// this copy — the classifier hosted pins use, with this copy's tables +/// as the pins: a live dependency that may select `version` from a +/// source no such table redirects (another registry, a +/// `registry-index`, git, or crates.io when the copy is wired for +/// another source only). A path declaration (the user's own fork) is +/// left alone, as the lock's copy claim leaves its sourceless entry +/// alone; a `socket-patch-*` registry dependency is hosted wiring; +/// * a `Cargo.lock` entry of `name` at exactly `version` with a non-Socket +/// source: that copy is already locked beside the vendored one. +/// +/// The detail ends with the remedy for its evidence. The ref stays +/// (rollback, remove and the ledgers still see the wiring), but nothing +/// attests it. +fn patch_bypassed( + ctx: &DiscoverCtx<'_>, + bypass: &Bypass<'_>, + vref: &VendorRef, + name: &str, + version: &str, + purl: &str, + out: &mut Discovery, +) { + let pins: Vec<(&str, String)> = bypass + .wired + .iter() + .filter(|w| w.vref.uuid == vref.uuid && w.vref.leaf == vref.leaf) + .map(|w| (w.table.as_str(), vref.uuid.clone())) + .collect(); + let others = distinct( + bypass + .lock + .packages() + .iter() + .filter(|p| p.name == name && cargo_tag::strip_tag(&p.version) != version) + .map(|p| cargo_tag::strip_tag(&p.version).to_string()), + ); + let mut declared: Vec<&Decl> = Vec::new(); + let mut pinned_elsewhere: Vec<(&Decl, String)> = Vec::new(); + for d in bypass.decls.and_then(|d| d.get(name)).into_iter().flatten() { + match route(d, &pins, version, &others) { + Route::Unpinned(_) if d.source != DeclSource::Path => declared.push(d), + Route::Pin(uuid) if uuid != vref.uuid => pinned_elsewhere.push((d, uuid)), + _ => {} + } + } + // A declaration pinned to ANOTHER Socket patch builds that patch's copy, + // not this one. Only a ref for that patch at this very package, emitted by + // the hosted pass (from the lock, or a conflict pair beside a lock pin) + // and not itself withheld, puts the pair before the CLI's conflict gate; + // otherwise (a stale lock, a lock line whose config index disagrees, + // another locked version) nothing else says so. "Withheld" here is exactly + // the CLI's `bypassed_ref`: a file-scoped mark on the ref's own file; any + // other mark leaves the ref in the conflict gate. + let gated = |uuid: &str| { + out.refs.iter().any(|r| { + r.uuid == uuid + && r.purl == purl + && !out.unattested.iter().any(|u| { + u.kind.file_scoped() + && u.uuid == uuid + && u.purl == purl + && u.file == r.source_file + }) + }) + }; + let other_patch: Vec<(&Decl, String)> = pinned_elsewhere + .into_iter() + .filter(|(_, uuid)| !gated(uuid)) + .collect(); + let other_declared = other_patch + .iter() + .map(|(d, uuid)| format!("{} (Socket patch {uuid})", d.describe(name))) + .collect::>() + .join(", and "); + // Named in every detail, so one `vex` run shows each cause. + let also_other = if other_patch.is_empty() { + String::new() + } else { + format!(". Also: {other_declared}, which no wiring of that patch pairs with this copy") + }; + let locked = same_version_foreign_entry(ctx, bypass.lock, name, version); + let redirects = format!( + "which no [patch] entry pointing at {} redirects", + vref.artifact_rel + ); + let (file, detail) = match (declared.first(), locked) { + (Some(first), _) => { + let evidence = locked.map_or_else(String::new, |source| { + format!("; {CARGO_LOCK} also locks {name} {version} from {source}") + }); + let declared = describe_all(declared.iter().copied(), name); + ( + first.file.as_str(), + format!( + "{declared}, {redirects}: cargo resolves that declaration as a separate \ + package, so the crate declaring it builds an unpatched {name} that the \ + requirement allows to be {version}{evidence}. Re-running `vendor` does \ + not change this: declare {name} from crates.io there (or point a \ + [patch] table for its source at the copy), or roll the patch \ + back{also_other}" + ), + ) + } + (None, Some(source)) => { + // An entry a `[patch]` table pointing at the copy redirects + // (crates.io under `[patch.crates-io]`, a git repository under its + // own table) is stale: cargo would redirect it. Any other is + // declared by some dependency, or left over from one that was + // removed. + let stale = DeclSource::of_lock_source(source).and_then(|src| { + pins.iter() + .find(|(table, _)| src.patched_by(None, table)) + .map(|(table, _)| *table) + }); + let remedy = if let Some(table) = stale { + // `vendor` writes `[patch.crates-io]` only, so it re-locks + // only a crates.io entry. + let vendor = if is_crates_io_source(table) { + "re-run `socket-patch vendor`, or " + } else { + "" + }; + let table = if table.contains("://") { + format!("\"{table}\"") + } else { + table.to_string() + }; + format!( + "the entry is stale (`[patch.{table}]` redirects its source), so \ + re-lock: {vendor}`cargo update -p {name}`" + ) + } else { + format!( + "`cargo tree -i {name}@{version}` names the dependency that declares it \ + (change that declaration or roll the patch back); if none does any \ + more, re-lock with `cargo update -p {name}`" + ) + }; + ( + CARGO_LOCK, + format!( + "{CARGO_LOCK} also locks {name} {version} from {source}, {redirects}: \ + cargo builds that copy unpatched beside the vendored one; \ + {remedy}{also_other}" + ), + ) + } + (None, None) => { + let Some((first, _)) = other_patch.first() else { + return; + }; + ( + first.file.as_str(), + format!( + "{other_declared}, which no wiring of that patch for {name} {version} pairs \ + with this copy (a stale {CARGO_LOCK}, or a registry definition that \ + disagrees): cargo resolves that declaration from that patch's registry, \ + so the crate declaring it builds {name} with another patch than the copy \ + at {}, and the requirement allows it to be {version}. Keep one patch for \ + {name}: pin that declaration to this patch (or drop it), or roll this \ + patch back", + vref.artifact_rel + ), + ) + } + }; + out.unattested( + purl, + &vref.uuid, + file, + detail, + UnattestedKind::CargoPatchBypassed, + ); +} + +/// Vendored refs from one file's `[patch]` tables. `shadow` (the root +/// manifest's) names why cargo ignores an entry (a higher-precedence item +/// replaces it), if so. +async fn vendored_from_patches( + ctx: &DiscoverCtx<'_>, + file: &str, + doc: &DocumentMut, + shadow: Option<&Shadow>, + bypass: &Bypass<'_>, + out: &mut Discovery, +) { + let lock = bypass.lock; + for entry in patch_entries(doc) { + let Some(path) = entry.path else { + continue; // a git / registry patch + }; + if !names_vendor_dir(path) { + continue; // a user path, or the uuid-less `.socket/cargo-patches/` + } + let name = entry.name; + if let Some(why) = shadow.and_then(|s| s.why(entry.source, entry.key, entry.path, None)) { + out.diag( + DIAG_REF_INVALID, + file, + format!( + "{file}: [patch] entry `{}` for {name} points at {path:?}, but {why}; \ + not counted", + entry.key + ), + ); + continue; + } + let Some(vref) = vendor_ref(path).filter(|v| v.eco == "cargo") else { + out.diag( + DIAG_REF_INVALID, + file, + format!( + "{file}: [patch] path {path:?} for {name} is not this project's \ + .socket/vendor/cargo//- copy" + ), + ); + continue; }; // One `-` directory, named for THIS entry's crate. let version = if vref.leaf.contains('/') { @@ -832,6 +1890,7 @@ async fn vendored_from_patches( continue; } } + patch_bypassed(ctx, bypass, &vref, name, version, &purl, out); out.push(PatchedRef::vendored(purl, &vref, file, None)); } } @@ -880,6 +1939,15 @@ mod tests { format!("[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n[dependencies]\n{deps}\n") } + /// A workspace root over `crates/*` that is also a package declaring + /// `deps`. + fn workspace_root(deps: &str) -> String { + format!( + "[workspace]\nmembers = [\"crates/*\"]\n\n{}", + manifest(deps) + ) + } + fn pinned(name: &str, version: &str, uuid: &str) -> String { format!("{name} = {{ version = \"{version}\", registry = \"socket-patch-{uuid}\" }}") } @@ -1337,6 +2405,79 @@ mod tests { assert!(!toml_ref.lockfile_basis_ok()); } + /// No declaration pins the lock's patch A, and the one pinning B + /// cannot select the locked version: B's pin is about another version, + /// so there is no conflict pair, and nothing routes the crate to A, so + /// the stale lock entry is no ref. (B's pin that may select it is the + /// conflict pair, see `manifest_and_lock_disagreeing_on_the_patch_emit_both`.) + #[tokio::test] + async fn another_patch_pin_excluding_the_locked_version_is_no_conflict() { + let p = Project::new(); + p.write("Cargo.toml", manifest(&pinned("serde", "=1.0.2", UUID_B))); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), + ); + let out = run(&p).await; + assert_refs(&out, &[]); + assert!( + diag_codes(&out).contains(&DIAG_REF_INVALID), + "{:#?}", + out.diagnostics + ); + // B's pin, which the lock does not reflect, is still diagnosed. + assert!( + out.diagnostics + .iter() + .any(|d| d.code == DIAG_REF_UNATTRIBUTABLE + && d.file == std::path::Path::new("Cargo.toml") + && d.detail.contains(UUID_B) + && d.detail.contains("only from other Socket patches")), + "{:#?}", + out.diagnostics + ); + } + + /// A well-formed Socket registry that pins nothing — beside `path`, + /// `git` or `registry-index`, or on a `[workspace.dependencies]` entry no + /// manifest inherits — is diagnosed, since the file reads like wiring. + #[tokio::test] + async fn an_ineffective_socket_registry_is_diagnosed() { + let reg = format!("registry = \"socket-patch-{UUID_A}\""); + let p = mixed_workspace( + &pinned("serde", "1", UUID_A), + &format!("serde = {{ path = \"../serde\", version = \"1\", {reg} }}"), + &[], + ); + let out = run(&p).await; + assert!( + out.diagnostics.iter().any(|d| d.code == DIAG_REF_INVALID + && d.file == std::path::Path::new("crates/a/Cargo.toml") + && d.detail.contains("pins nothing there") + && d.detail.contains("a local path")), + "{:#?}", + out.diagnostics + ); + + let p = Project::new(); + p.write( + "Cargo.toml", + format!( + "[workspace]\nmembers = [\"crates/*\"]\n\n[workspace.dependencies]\n{}\n", + pinned("serde", "1", UUID_A) + ), + ); + member(&p, "a", "other = \"1\""); + let out = run(&p).await; + assert!( + out.diagnostics.iter().any(|d| d.code == DIAG_REF_INVALID + && d.file == std::path::Path::new("Cargo.toml") + && d.detail.contains("no manifest inherits")), + "{:#?}", + out.diagnostics + ); + } + /// A project-config definition of the pinned registry that indexes a /// different patch (or a non-Socket host) is what cargo resolves /// against: the lock entry is stale. @@ -1362,67 +2503,876 @@ mod tests { } } - /// A rotated grant token in the definition is the same patch. + /// A workspace root that declares nothing itself, with one member. + fn workspace_with_member(p: &Project, member_deps: &str) { + p.write("Cargo.toml", "[workspace]\nmembers = [\"crates/*\"]\n"); + member(p, "a", member_deps); + } + + /// The hosted rewriter pins the crate in every member manifest that + /// declares it, so a member's declaration decides like the root's. A + /// member reverted to crates.io (the root declares nothing) makes cargo + /// re-resolve from crates.io: no ref, diagnosed against the member, and + /// the stale lock entry's patch is recognized (a ledger record for it is + /// dead). #[tokio::test] - async fn rotated_token_in_the_definition_still_agrees() { + async fn reverted_member_manifest_pin_drops_the_stale_lock_entry() { + for decl in [ + "serde = \"1.0.1\"".to_string(), + "serde = { version = \"1\", registry = \"my-corp\" }".to_string(), + ] { + let p = Project::new(); + workspace_with_member(&p, &decl); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), + ); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert_refs(&out, &[]); + assert_eq!(diag_codes(&out), vec![DIAG_REF_INVALID], "{decl}"); + assert!( + out.diagnostics[0].detail.contains("crates/a/Cargo.toml"), + "{:#?}", + out.diagnostics + ); + assert_eq!( + out.hosted_claim("pkg:cargo/serde@1.0.1", UUID_A), + Some(false), + "{decl}" + ); + } + } + + /// The member's own pin keeps the lock entry a ref; a member pin for + /// ANOTHER patch emits the conflict pair, attributed to the member. + #[tokio::test] + async fn member_manifest_pins_count_like_the_roots() { let p = Project::new(); - p.write("Cargo.toml", manifest(&pinned("serde", "1", UUID_A))); + workspace_with_member(&p, &pinned("serde", "1", UUID_A)); p.write( "Cargo.lock", lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), ); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + assert!(out.diagnostics.is_empty(), "{:#?}", out.diagnostics); + + let p = Project::new(); + workspace_with_member(&p, &pinned("serde", "1", UUID_B)); p.write( - ".cargo/config.toml", - format!( - "[registries.socket-patch-{UUID_A}]\nindex = \"sparse+https://patch.socket.dev/patch-registry/cargo/other-token/{UUID_A}/index/\"\n" - ), + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), ); let out = run(&p).await; assert_refs( &out, - &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + &[ + ("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted), + ("pkg:cargo/serde@1.0.1", UUID_B, WiringMode::Hosted), + ], + ); + let toml_ref = out.refs.iter().find(|r| r.uuid == UUID_B).unwrap(); + assert_eq!( + toml_ref.source_file, + std::path::PathBuf::from("crates/a/Cargo.toml") ); } - /// Definitions alone never make a ref; a pin with no Socket-sourced lock - /// entry is diagnosed, not guessed at. + /// The root's pin still decides when a member declares the crate under + /// another version requirement the pin does not reach (the rewriter + /// leaves such a declaration alone): some declaration pins the patch. #[tokio::test] - async fn definitions_and_unresolved_pins_are_not_refs() { - // Definition only, crates.io lock. + async fn a_root_pin_with_an_unpinned_member_declaration_is_a_ref() { let p = Project::new(); - p.write(".cargo/config.toml", registry_block(UUID_A)); - p.write("Cargo.toml", manifest("serde = \"1\"")); + pinned_root_workspace(&p); + member(&p, "a", "serde = \"0.9\""); p.write( "Cargo.lock", - lock(&[("serde", "1.0.1", Some(CRATES_IO), Some(CKSUM))]), + lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), ); let out = run(&p).await; - assert_refs(&out, &[]); - assert!(out.diagnostics.is_empty(), "{:#?}", out.diagnostics); - // The leftover definition names the patch on the Socket host: it is - // recognized, so a ledger claim for it is dead (rule 11). - assert_eq!( - out.hosted_claim("pkg:cargo/serde@1.0.1", UUID_A), - Some(false) + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], ); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + } - // Pin, but no lock at all / a lock resolving crates.io. - for with_lock in [false, true] { - let p = Project::new(); - p.write(".cargo/config.toml", registry_block(UUID_A)); - p.write("Cargo.toml", manifest(&pinned("serde", "1", UUID_A))); - if with_lock { - p.write( - "Cargo.lock", - lock(&[("serde", "1.0.1", Some(CRATES_IO), Some(CKSUM))]), - ); - } - let out = run(&p).await; - assert_refs(&out, &[]); - assert_eq!( - diag_codes(&out), - vec![DIAG_REF_UNATTRIBUTABLE], - "{with_lock}" + /// A workspace whose root and member `crates/a` declare serde with the + /// given dependency lines, locked to Socket patch A at 1.0.1 (plus + /// `extra` lock entries). + fn mixed_workspace(root_dep: &str, member_dep: &str, extra: &[(&str, &str)]) -> Project { + let p = Project::new(); + p.write("Cargo.toml", workspace_root(root_dep)); + member(&p, "a", member_dep); + let socket = index(UUID_A); + let mut pkgs = vec![("serde", "1.0.1", Some(socket.as_str()), Some(CKSUM))]; + pkgs.extend( + extra + .iter() + .map(|(v, s)| ("serde", *v, Some(*s), Some(CKSUM))), + ); + p.write("Cargo.lock", lock(&pkgs)); + p.write(".cargo/config.toml", registry_block(UUID_A)); + p + } + + /// A `[patch.crates-io]` Socket pin that cargo ignores (a crates.io URL + /// table replaces it) pins no declaration: the member's crates.io + /// declaration is unpinned, so the ref is not attested. + #[tokio::test] + async fn a_shadowed_patch_pin_pins_no_declaration() { + let patch = format!( + "serde = \"1\"\n\n[patch.crates-io]\nserde = {{ registry = \"socket-patch-{UUID_A}\" }}\n" + ); + let p = mixed_workspace(&patch, "serde = \"1\"", &[]); + let out = run(&p).await; + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + let shadowed = format!( + "{patch}\n[patch.\"https://github.com/rust-lang/crates.io-index\"]\nother = {{ path = \"other\" }}\n" + ); + let p = mixed_workspace(&shadowed, "serde = \"1\"", &[]); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + assert!( + out.unattested + .iter() + .any(|u| u.kind == UnattestedKind::CargoUnpinnedDeclaration), + "{:#?}", + out.unattested + ); + } + + /// The one unattested mark of `out`, asserted to be patch A's `kind` + /// mark on `purl` naming `file`. + fn one_mark<'a>( + out: &'a Discovery, + kind: UnattestedKind, + purl: &str, + file: &str, + ) -> &'a Unattested { + assert_eq!(out.unattested.len(), 1, "{:#?}", out.unattested); + let u = &out.unattested[0]; + assert_eq!(u.kind, kind); + assert_eq!((u.purl.as_str(), u.uuid.as_str()), (purl, UUID_A)); + assert_eq!(u.file, std::path::PathBuf::from(file)); + u + } + + /// [`one_mark`] for the hosted serde ref's + /// [`UnattestedKind::CargoUnpinnedDeclaration`]. + fn unpinned_mark<'a>(out: &'a Discovery, file: &str) -> &'a Unattested { + one_mark( + out, + UnattestedKind::CargoUnpinnedDeclaration, + "pkg:cargo/serde@1.0.1", + file, + ) + } + + /// The root pins serde to Socket, but a member declares it from crates.io + /// at a requirement the patched 1.0.1 satisfies. Cargo resolves that + /// declaration as a separate crates.io package, so the member builds + /// unpatched bytes: the ref stays (rollback / remove find it, the ledger + /// claim lives) but is not attested, naming the member. + #[tokio::test] + async fn a_root_pin_beside_an_unpinned_member_is_unattested() { + for member in [ + "serde = \"1\"", + "serde = \"=1.0.1\"", + "serde = { version = \"1\", registry = \"my-corp\" }", + "serde = { git = \"https://github.com/serde-rs/serde\" }", + "legacy = { package = \"serde\", version = \"1.0\" }", + ] { + let p = mixed_workspace(&pinned("serde", "1", UUID_A), member, &[]); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + assert!( + out.diagnostics.is_empty(), + "{member}: {:#?}", + out.diagnostics + ); + assert_eq!( + out.hosted_claim("pkg:cargo/serde@1.0.1", UUID_A), + Some(true), + "{member}" + ); + let u = unpinned_mark(&out, "crates/a/Cargo.toml"); + assert!( + u.detail.contains("crates/a/Cargo.toml declares serde") + && u.detail + .contains(&format!("not registry \"socket-patch-{UUID_A}\"")) + && !u.detail.contains("already locks"), + "{member}: {}", + u.detail + ); + } + let p = mixed_workspace(&pinned("serde", "1", UUID_A), "serde = \"1\"", &[]); + let u = &run(&p).await.unattested[0]; + assert!( + u.detail.contains("serde \"1\" from crates.io"), + "{}", + u.detail + ); + } + + /// The mirror image: a member pins serde, the root declares it from + /// crates.io. The root's declaration builds unpatched bytes. + #[tokio::test] + async fn a_member_pin_beside_an_unpinned_root_is_unattested() { + let p = mixed_workspace("serde = \"1.0\"", &pinned("serde", "1", UUID_A), &[]); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + let u = unpinned_mark(&out, "Cargo.toml"); + assert!( + u.detail + .contains("Cargo.toml declares serde \"1.0\" from crates.io"), + "{}", + u.detail + ); + } + + /// What the hosted rewriter writes — every declaration pinned — is + /// attested; so is a pin beside an unpinned declaration that the + /// patched version cannot satisfy (cargo resolves it to another + /// version, never this one). + #[tokio::test] + async fn every_declaration_pinned_is_attested() { + for (root, member) in [ + (pinned("serde", "1", UUID_A), pinned("serde", "1.0", UUID_A)), + (pinned("serde", "1", UUID_A), "serde = \"0.9\"".to_string()), + ( + pinned("serde", "1", UUID_A), + "serde = \">=1.0.2\"".to_string(), + ), + // A `[patch]` registry entry redirects declarations; it declares + // no dependency of its own. + ( + format!( + "{}\n\n[patch.crates-io]\nserde = {{ version = \"1\", registry = \"my-corp\" }}", + pinned("serde", "1", UUID_A) + ), + pinned("serde", "1", UUID_A), + ), + ] { + let p = mixed_workspace(&root, &member, &[]); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + assert!(out.unattested.is_empty(), "{member}: {:#?}", out.unattested); + assert!( + out.diagnostics.is_empty(), + "{member}: {:#?}", + out.diagnostics + ); + } + } + + /// The lock already holds the member's crates.io copy at the patched + /// version beside the Socket entry: the crates.io-twin rule (#679) + /// withholds the ref (shadowed, its diagnostic naming that lock entry) + /// and no unattested mark is added for a ref that no longer stands. + #[tokio::test] + async fn a_locked_same_version_twin_is_left_to_the_twin_rule() { + let p = mixed_workspace( + &pinned("serde", "1", UUID_A), + "serde = \"1\"", + &[("1.0.1", CRATES_IO)], + ); + let out = run(&p).await; + assert_refs(&out, &[]); + assert_eq!(out.shadowed.len(), 1, "{:#?}", out.shadowed); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + assert!( + out.diagnostics + .iter() + .any(|d| d.code == DIAG_REF_UNATTRIBUTABLE + && d.detail.contains(&format!("serde 1.0.1 ({CRATES_IO})"))), + "{:#?}", + out.diagnostics + ); + } + + /// The remedy follows what a re-scan can do: the hosted rewriter pins a + /// crates.io declaration whose requirement selects only the patched + /// version, but refuses one that also matches another locked version + /// (`redirect_cargo_toml_dep_unrewritable`), one from another registry + /// or `registry-index`, and a path / git one. + #[tokio::test] + async fn the_remedy_says_whether_a_rescan_can_pin_it() { + let p = mixed_workspace(&pinned("serde", "=1.0.1", UUID_A), "serde = \"1\"", &[]); + let u = run(&p).await.unattested.remove(0); + assert!( + u.detail.contains("re-run `socket-patch scan`"), + "{}", + u.detail + ); + // `registry = "crates-io"` is crates.io, which a re-scan re-pins + // (#386). + let p = mixed_workspace( + &pinned("serde", "=1.0.1", UUID_A), + "serde = { version = \"1\", registry = \"crates-io\" }", + &[], + ); + let u = run(&p).await.unattested.remove(0); + assert!( + u.detail.contains("re-run `socket-patch scan`"), + "{}", + u.detail + ); + + // `"1"` also matches the crates.io 1.0.200 locked beside it. + let p = mixed_workspace( + &pinned("serde", "=1.0.1", UUID_A), + "serde = \"1\"", + &[("1.0.200", CRATES_IO)], + ); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + let u = unpinned_mark(&out, "crates/a/Cargo.toml"); + assert!( + u.detail.contains("also matches serde 1.0.200") + && u.detail.contains("roll the patch back") + && !u.detail.contains("re-run `socket-patch scan`"), + "{}", + u.detail + ); + + for (member, from) in [ + ( + "serde = { version = \"1\", registry = \"my-corp\" }", + "from registry `my-corp`", + ), + ( + "serde = { version = \"1\", registry-index = \"https://corp.example/index\" }", + "from the registry index https://corp.example/index", + ), + ("serde = { path = \"../serde\" }", "from a local path"), + ] { + let p = mixed_workspace(&pinned("serde", "1", UUID_A), member, &[]); + let out = run(&p).await; + let u = unpinned_mark(&out, "crates/a/Cargo.toml"); + assert!( + u.detail.contains(from) + && u.detail.contains("roll the patch back") + && !u.detail.contains("re-run `socket-patch scan`"), + "{member}: {}", + u.detail + ); + } + // Every unpinned declaration passes on its own, but the rewriter + // refuses the dependency over the PIN's own requirement (`"1"` also + // matches the crates.io 1.0.200), so a re-scan pins nothing. + let p = mixed_workspace( + &pinned("serde", "1", UUID_A), + "serde = \"=1.0.1\"", + &[("1.0.200", CRATES_IO)], + ); + let out = run(&p).await; + let u = unpinned_mark(&out, "crates/a/Cargo.toml"); + assert!( + u.detail.contains("cannot pin serde (Cargo.toml:") + && u.detail.contains("roll the patch back") + && !u.detail.contains("re-run `socket-patch scan`"), + "{}", + u.detail + ); + } + + /// A declaration that names the Socket registry but also a `path`, + /// `git` or `registry-index` source is not a pin (the hosted rewriter's + /// own check): cargo builds that source. Beside a real pin it withholds + /// the statement; alone it still keeps the ref (Socket's wiring is + /// present) but never attests it. + #[tokio::test] + async fn a_socket_registry_beside_another_source_is_not_a_pin() { + let reg = format!("registry = \"socket-patch-{UUID_A}\""); + for extra in [ + "path = \"../serde\"", + "git = \"https://github.com/serde-rs/serde\"", + "registry-index = \"https://corp.example/index\"", + ] { + let member = format!("serde = {{ version = \"1\", {reg}, {extra} }}"); + let p = mixed_workspace(&pinned("serde", "1", UUID_A), &member, &[]); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + let u = unpinned_mark(&out, "crates/a/Cargo.toml"); + assert!( + u.detail.contains(&format!( + "which overrides its registry = \"socket-patch-{UUID_A}\"" + )), + "{extra}: {}", + u.detail + ); + + let p = Project::new(); + p.write("Cargo.toml", manifest(&member)); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), + ); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + unpinned_mark(&out, "Cargo.toml"); + } + } + + /// A `[workspace.dependencies]` entry is a template: cargo resolves it + /// only for a manifest that inherits it (`workspace = true`). An + /// unpinned template nobody inherits builds nothing; an inherited one + /// does. + #[tokio::test] + async fn an_unpinned_workspace_template_counts_only_when_inherited() { + for (member_dep, inherited) in [ + ("other = \"1\"", false), + ("serde = { workspace = true }", true), + ] { + let p = Project::new(); + p.write( + "Cargo.toml", + format!( + "[workspace]\nmembers = [\"crates/*\"]\n\n[workspace.dependencies]\n\ + serde = \"1\"\n\n[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n\ + [dependencies]\n{}\n", + pinned("serde", "1", UUID_A) + ), + ); + member(&p, "a", member_dep); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), + ); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + if inherited { + unpinned_mark(&out, "Cargo.toml"); + } else { + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + } + } + } + + /// The root pins the lock's patch A, a member pins ANOTHER patch B for + /// the same version: the files disagree, so both wirings are emitted + /// and the CLI's conflict gate refuses to pick one. A member pin whose + /// requirement excludes the locked version is about another version. + #[tokio::test] + async fn a_member_pinned_to_another_patch_is_a_conflict_pair() { + let p = mixed_workspace( + &pinned("serde", "1", UUID_A), + &pinned("serde", "1", UUID_B), + &[], + ); + let out = run(&p).await; + assert_refs( + &out, + &[ + ("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted), + ("pkg:cargo/serde@1.0.1", UUID_B, WiringMode::Hosted), + ], + ); + let b = out.refs.iter().find(|r| r.uuid == UUID_B).unwrap(); + assert_eq!( + b.source_file, + std::path::PathBuf::from("crates/a/Cargo.toml") + ); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + + let p = mixed_workspace( + &pinned("serde", "1", UUID_A), + &pinned("serde", "0.9", UUID_B), + &[], + ); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + + // The lock resolves the member's B pin too, at another version: B's + // registry serves only that version, a separate patched package + // with its own ref, not a conflict over 1.0.1. + let b_index = index(UUID_B); + let p = mixed_workspace( + &pinned("serde", "1", UUID_A), + &pinned("serde", "1", UUID_B), + &[("1.0.5", b_index.as_str())], + ); + let out = run(&p).await; + assert_refs( + &out, + &[ + ("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted), + ("pkg:cargo/serde@1.0.5", UUID_B, WiringMode::Hosted), + ], + ); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + } + + /// A manifest under `.socket/` (a path dependency into Socket's own + /// tree, as a vendored copy is) is not a declaration the hosted + /// rewriter pins, so it never decides: the lock stays the only word. + #[tokio::test] + async fn manifests_under_dot_socket_are_not_declarations() { + let p = Project::new(); + p.write( + "Cargo.toml", + manifest("local = { path = \".socket/local\" }"), + ); + p.write( + ".socket/local/Cargo.toml", + "[package]\nname = \"local\"\nversion = \"0.1.0\"\n\n[dependencies]\nserde = \"1\"\n", + ); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), + ); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + } + + /// With no lock, a member's pin is an unlocked pin carrying the member's + /// requirement, attributed to the member manifest. + #[tokio::test] + async fn lockless_member_pin_is_an_unlocked_pin() { + let p = Project::new(); + workspace_with_member(&p, &pinned("serde", "1", UUID_A)); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert_refs(&out, &[]); + assert_eq!(out.unlocked_pins.len(), 1, "{:#?}", out.unlocked_pins); + let pin = &out.unlocked_pins[0]; + assert_eq!(pin.file, std::path::PathBuf::from("crates/a/Cargo.toml")); + assert_eq!(pin.version_reqs, vec!["1".to_string()]); + assert_eq!(diag_codes(&out), vec![DIAG_REF_UNATTRIBUTABLE]); + assert_eq!( + out.diagnostics[0].file, + std::path::PathBuf::from("crates/a/Cargo.toml") + ); + } + + /// A workspace whose root pins `serde` and declares `crates/*` members. + fn pinned_root_workspace(p: &Project) { + p.write("Cargo.toml", workspace_root(&pinned("serde", "1", UUID_A))); + } + + fn member(p: &Project, dir: &str, deps: &str) { + p.write( + &format!("crates/{dir}/Cargo.toml"), + format!("[package]\nname = \"{dir}\"\nversion = \"0.1.0\"\n\n[dependencies]\n{deps}\n"), + ); + } + + /// A member's unpinned declaration at a version the pin does not reach + /// (`serde = "0.9"` beside the root's pinned `serde = "1"`, which the + /// rewriter leaves alone) keeps the lockless pin; it only excludes the + /// versions it accepts. One that accepts the record's version, or that has + /// no requirement at all (a git dependency), leaves the record unrouted. + #[tokio::test] + async fn lockless_pin_survives_a_member_declaring_another_version() { + let record = "pkg:cargo/serde@1.0.1"; + for (member_decl, live) in [ + ("serde = \"0.9\"", true), + ("serde = \"1.0\"", false), + ("serde = { git = \"https://example.com/serde\" }", false), + ] { + let p = Project::new(); + pinned_root_workspace(&p); + member(&p, "a", member_decl); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert_refs(&out, &[]); + assert_eq!( + out.hosted_claim(record, UUID_A), + Some(live), + "{member_decl}: {:#?}", + out.unlocked_pins + ); + } + let p = Project::new(); + pinned_root_workspace(&p); + member(&p, "a", "serde = \"0.9\""); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert_eq!(out.unlocked_pins.len(), 1); + assert_eq!(out.unlocked_pins[0].excluded_reqs, vec!["0.9".to_string()]); + assert_eq!( + out.hosted_claim("pkg:cargo/serde@0.9.5", UUID_A), + Some(false), + "the member resolves 0.9 from crates.io" + ); + } + + /// A `socket-patch-` registry beside `path`, `git` or + /// `registry-index` is no pin (cargo builds the path / git / index + /// source; the rewriter's own pinned check agrees): lockless, such a + /// member declaration is an unpinned one, so it excludes the versions it + /// accepts (all of them with no requirement), and alone it makes no + /// lockless pin at all. + #[tokio::test] + async fn lockless_path_or_git_beside_a_socket_registry_is_no_pin() { + let record = "pkg:cargo/serde@1.0.1"; + let reg = format!("registry = \"socket-patch-{UUID_A}\""); + for member_decl in [ + format!("serde = {{ path = \"../serde\", version = \"1\", {reg} }}"), + format!("serde = {{ git = \"https://example.com/serde\", {reg} }}"), + format!( + "serde = {{ version = \"1\", registry-index = \"sparse+https://x.example/\", {reg} }}" + ), + ] { + let p = Project::new(); + pinned_root_workspace(&p); + member(&p, "a", &member_decl); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert_refs(&out, &[]); + assert_eq!( + out.hosted_claim(record, UUID_A), + Some(false), + "{member_decl}: {:#?}", + out.unlocked_pins + ); + } + // The member's overridden registry alone: no pin anywhere. + let p = Project::new(); + workspace_with_member( + &p, + &format!("serde = {{ path = \"../serde\", version = \"1\", {reg} }}"), + ); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert!(out.unlocked_pins.is_empty(), "{:#?}", out.unlocked_pins); + } + + /// A `[workspace.dependencies]` template no manifest inherits declares + /// nothing cargo resolves: lockless, a Socket-pinned one makes no pin, + /// and an unpinned one excludes no version from the root's pin. Once a + /// member inherits the pinned template, it is the pin. + #[tokio::test] + async fn lockless_uninherited_template_is_not_a_declaration() { + let template = |dep: &str| { + format!("[workspace]\nmembers = [\"crates/*\"]\n\n[workspace.dependencies]\n{dep}\n") + }; + let p = Project::new(); + p.write("Cargo.toml", template(&pinned("serde", "1", UUID_A))); + member(&p, "a", "other = \"1\""); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert!(out.unlocked_pins.is_empty(), "{:#?}", out.unlocked_pins); + + let p = Project::new(); + p.write("Cargo.toml", template(&pinned("serde", "1", UUID_A))); + member(&p, "a", "serde = { workspace = true }"); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert_eq!(out.unlocked_pins.len(), 1, "{:#?}", out.unlocked_pins); + assert_eq!( + out.hosted_claim("pkg:cargo/serde@1.0.1", UUID_A), + Some(true) + ); + + let p = Project::new(); + p.write( + "Cargo.toml", + format!( + "[workspace]\nmembers = [\"crates/*\"]\n\n[workspace.dependencies]\n\ + serde = \"1\"\n\n[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n\ + [dependencies]\n{}\n", + pinned("serde", "1", UUID_A) + ), + ); + member(&p, "a", "other = \"1\""); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert_eq!(out.unlocked_pins.len(), 1, "{:#?}", out.unlocked_pins); + assert!(out.unlocked_pins[0].excluded_reqs.is_empty()); + assert_eq!( + out.hosted_claim("pkg:cargo/serde@1.0.1", UUID_A), + Some(true) + ); + } + + /// The root and two members pin the same patch — one lockless pin carrying + /// every requirement, one diagnostic listing each manifest once. + #[tokio::test] + async fn one_lockless_pin_and_diagnostic_per_crate_and_patch() { + let p = Project::new(); + pinned_root_workspace(&p); + member(&p, "a", &pinned("serde", "1.0", UUID_A)); + member( + &p, + "b", + &format!( + "{}\n\n[dev-dependencies]\n{}", + pinned("serde", "1", UUID_A), + pinned("serde", "1", UUID_A) + ), + ); + p.write(".cargo/config.toml", registry_block(UUID_A)); + let out = run(&p).await; + assert_eq!(out.unlocked_pins.len(), 1, "{:#?}", out.unlocked_pins); + let pin = &out.unlocked_pins[0]; + assert_eq!(pin.file, std::path::PathBuf::from("Cargo.toml")); + assert_eq!(pin.version_reqs, vec!["1".to_string(), "1.0".to_string()]); + assert!(pin.excluded_reqs.is_empty()); + assert_eq!(diag_codes(&out), vec![DIAG_REF_UNATTRIBUTABLE]); + assert!( + out.diagnostics[0] + .detail + .starts_with("Cargo.toml, crates/a/Cargo.toml, crates/b/Cargo.toml: serde"), + "{}", + out.diagnostics[0].detail + ); + } + + /// Members only matter to Socket wiring, so a project with a lock that + /// resolves nothing from a Socket patch server — or no lock and no + /// `socket-patch-` pin anywhere — reads no member manifest as discovery's + /// own (none in [`Discovery::read`], which the full run collects). + #[tokio::test] + async fn members_are_read_only_when_socket_wiring_could_need_them() { + let member_read = |out: &Discovery| { + out.read + .iter() + .any(|r| r.file == std::path::Path::new("crates/a/Cargo.toml")) + }; + // A plain lock: no walk. + let p = Project::new(); + workspace_with_member(&p, "serde = \"1\""); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(CRATES_IO), Some(CKSUM))]), + ); + assert!(!member_read(&p.discover().await)); + // No lock, no pin anywhere: walked, not adopted. + let p = Project::new(); + workspace_with_member(&p, "serde = \"1\""); + assert!(!member_read(&p.discover().await)); + // A Socket-sourced lock entry: read. + let p = Project::new(); + workspace_with_member(&p, "serde = \"1\""); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), + ); + assert!(member_read(&p.discover().await)); + // No lock, a member pin: read. + let p = Project::new(); + workspace_with_member(&p, &pinned("serde", "1", UUID_A)); + assert!(member_read(&p.discover().await)); + // A stale crates.io lock beside a member pin still reads the member, + // so the pin is diagnosed (a stale lock or a server outside the + // allowlist) and its uuid recognized. + let p = Project::new(); + workspace_with_member(&p, &pinned("serde", "1", UUID_A)); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(CRATES_IO), Some(CKSUM))]), + ); + let out = p.discover().await; + assert!(member_read(&out)); + assert!( + out.diagnostics + .iter() + .any(|d| d.code == DIAG_REF_UNATTRIBUTABLE), + "{:?}", + out.diagnostics + ); + } + + /// A rotated grant token in the definition is the same patch. + #[tokio::test] + async fn rotated_token_in_the_definition_still_agrees() { + let p = Project::new(); + p.write("Cargo.toml", manifest(&pinned("serde", "1", UUID_A))); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), + ); + p.write( + ".cargo/config.toml", + format!( + "[registries.socket-patch-{UUID_A}]\nindex = \"sparse+https://patch.socket.dev/patch-registry/cargo/other-token/{UUID_A}/index/\"\n" + ), + ); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + } + + /// Definitions alone never make a ref; a pin with no Socket-sourced lock + /// entry is diagnosed, not guessed at. + #[tokio::test] + async fn definitions_and_unresolved_pins_are_not_refs() { + // Definition only, crates.io lock. + let p = Project::new(); + p.write(".cargo/config.toml", registry_block(UUID_A)); + p.write("Cargo.toml", manifest("serde = \"1\"")); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(CRATES_IO), Some(CKSUM))]), + ); + let out = run(&p).await; + assert_refs(&out, &[]); + assert!(out.diagnostics.is_empty(), "{:#?}", out.diagnostics); + // The leftover definition names the patch on the Socket host: it is + // recognized, so a ledger claim for it is dead (rule 11). + assert_eq!( + out.hosted_claim("pkg:cargo/serde@1.0.1", UUID_A), + Some(false) + ); + + // Pin, but no lock at all / a lock resolving crates.io. + for with_lock in [false, true] { + let p = Project::new(); + p.write(".cargo/config.toml", registry_block(UUID_A)); + p.write("Cargo.toml", manifest(&pinned("serde", "1", UUID_A))); + if with_lock { + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(CRATES_IO), Some(CKSUM))]), + ); + } + let out = run(&p).await; + assert_refs(&out, &[]); + assert_eq!( + diag_codes(&out), + vec![DIAG_REF_UNATTRIBUTABLE], + "{with_lock}" ); // With NO lock the pin is `rewrite_cargo`'s own `LockCommit::Absent` // output (a library that gitignores Cargo.lock): live wiring for a @@ -1479,23 +3429,69 @@ mod tests { // A malformed socket-patch-* registry name. let p = Project::new(); p.write( - "Cargo.toml", - manifest("serde = { version = \"1\", registry = \"socket-patch-UUID\" }"), + "Cargo.toml", + manifest("serde = { version = \"1\", registry = \"socket-patch-UUID\" }"), + ); + let out = run(&p).await; + assert_eq!(diag_codes(&out), vec![DIAG_REF_INVALID]); + } + + /// `[patch.crates-io] x = { registry = "socket-patch-" }` (hand-wired) + /// is a declaration cargo honors. + #[tokio::test] + async fn patch_table_registry_pin_counts_as_the_declaration() { + let p = Project::new(); + p.write( + "Cargo.toml", + format!( + "{}\n[patch.crates-io]\n{}\n", + manifest("serde = \"1\""), + pinned("serde", "1", UUID_A) + ), + ); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), + ); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + assert!(out.diagnostics.is_empty(), "{:#?}", out.diagnostics); + // The crates.io declaration is what the entry redirects: not unpinned. + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + + // An explicit `registry = "crates-io"` is crates.io, so a + // `[patch.""]` pin redirects it too. + let p = Project::new(); + p.write( + "Cargo.toml", + format!( + "{}\n[patch.\"https://github.com/rust-lang/crates.io-index\"]\n{}\n", + manifest("serde = { version = \"1\", registry = \"crates-io\" }"), + pinned("serde", "1", UUID_A) + ), + ); + p.write( + "Cargo.lock", + lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), ); let out = run(&p).await; - assert_eq!(diag_codes(&out), vec![DIAG_REF_INVALID]); - } + assert_refs( + &out, + &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + ); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); - /// `[patch.crates-io] x = { registry = "socket-patch-" }` (hand-wired) - /// is a declaration cargo honors. - #[tokio::test] - async fn patch_table_registry_pin_counts_as_the_declaration() { + // A declaration from ANOTHER registry is not redirected by + // `[patch.crates-io]`: cargo builds my-corp's serde for it. let p = Project::new(); p.write( "Cargo.toml", format!( "{}\n[patch.crates-io]\n{}\n", - manifest("serde = \"1\""), + manifest("serde = \"1\"\nalt = { package = \"serde\", version = \"1\", registry = \"my-corp\" }"), pinned("serde", "1", UUID_A) ), ); @@ -1504,11 +3500,12 @@ mod tests { lock(&[("serde", "1.0.1", Some(&index(UUID_A)), Some(CKSUM))]), ); let out = run(&p).await; - assert_refs( - &out, - &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], + assert_eq!(out.unattested.len(), 1, "{:#?}", out.unattested); + assert!( + out.unattested[0].detail.contains("from registry `my-corp`"), + "{:#?}", + out.unattested ); - assert!(out.diagnostics.is_empty(), "{:#?}", out.diagnostics); } /// A registry pin in the effective config's `[patch]` counts as a @@ -1531,6 +3528,7 @@ mod tests { &[("pkg:cargo/serde@1.0.1", UUID_A, WiringMode::Hosted)], ); assert!(out.diagnostics.is_empty(), "{:#?}", out.diagnostics); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); } #[tokio::test] @@ -2213,4 +4211,634 @@ mod tests { } } } + + // ── vendored: copies the [patch] does not redirect ─────────────── + + const MY_CORP: &str = "registry+https://my-corp.example/index"; + + /// A v5-vendored workspace: the root wires cfg-if 1.0.4 to the tagged + /// copy for patch A through `[patch.crates-io]` (plus `root_extra` + /// appended to the root manifest) and declares it from crates.io; the + /// member `crates/a` declares it with `member_dep`. The lock holds the + /// tagged copy plus `extra` entries `(version, source)`. + fn vendored_workspace(member_dep: &str, root_extra: &str, extra: &[(&str, &str)]) -> Project { + let rel = vendor_path(UUID_A, "cfg-if-1.0.4"); + let p = Project::new(); + p.write( + "Cargo.toml", + format!( + "{}\n[patch.crates-io]\n\ + cfg-if-socket-aaaaaaaa = {{ package = \"cfg-if\", path = \"{rel}\" }}\n\ + {root_extra}\n", + workspace_root("cfg-if = \"1\"") + ), + ); + member(&p, "a", member_dep); + let tagged = crate::vendor::cargo_tag::tag_version("1.0.4", UUID_A); + let mut pkgs = vec![("cfg-if", tagged.as_str(), None, None)]; + pkgs.extend( + extra + .iter() + .map(|(v, s)| ("cfg-if", *v, Some(*s), Some(CKSUM))), + ); + p.write("Cargo.lock", lock(&pkgs)); + p + } + + /// [`one_mark`] for the vendored cfg-if ref's + /// [`UnattestedKind::CargoPatchBypassed`]. + fn bypassed_mark<'a>(out: &'a Discovery, file: &str) -> &'a Unattested { + one_mark( + out, + UnattestedKind::CargoPatchBypassed, + "pkg:cargo/cfg-if@1.0.4", + file, + ) + } + + /// `[patch.crates-io]` redirects only crates.io declarations. A member + /// that declares cfg-if from another registry or from git, at a + /// requirement the patched 1.0.4 satisfies, resolves its own unpatched + /// copy, which may be that very version. The vendored ref stays (rollback + /// / remove find it, the ledger claim lives) but is not attested, naming + /// the member. + #[tokio::test] + async fn a_member_from_another_source_bypasses_the_vendored_copy() { + for (member, origin) in [ + ( + "cfg-if = { version = \"1\", registry = \"my-corp\" }", + "from registry `my-corp`", + ), + ( + "fork = { package = \"cfg-if\", version = \"=1.0.4\", registry = \"my-corp\" }", + "from registry `my-corp`", + ), + ( + "cfg-if = { git = \"https://example.com/cfg-if\" }", + "from a git repository", + ), + ( + "cfg-if = { version = \"1\", registry-index = \"sparse+https://my-corp.example/index/\" }", + "from the registry index sparse+https://my-corp.example/index/", + ), + ] { + let p = vendored_workspace(member, "", &[]); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/cfg-if@1.0.4", UUID_A, WiringMode::Vendored)], + ); + assert!( + out.diagnostics.is_empty(), + "{member}: {:#?}", + out.diagnostics + ); + assert_eq!( + out.vendored_claim( + "pkg:cargo/cfg-if@1.0.4", + UUID_A, + &vendor_path(UUID_A, "cfg-if-1.0.4") + ), + Some(true), + "{member}" + ); + let u = bypassed_mark(&out, "crates/a/Cargo.toml"); + assert!( + u.detail.contains("crates/a/Cargo.toml declares cfg-if") + && u.detail.contains(origin) + && !u.detail.contains("also locks"), + "{member}: {}", + u.detail + ); + } + } + + /// The lock already holds a same-version cfg-if from another source + /// (another registry, git, even crates.io) beside the vendored copy: + /// cargo builds that copy unpatched whoever declares it, so the mark + /// names the lock entry. A copy at another version is no such evidence. + #[tokio::test] + async fn a_same_version_lock_entry_from_any_source_bypasses_the_vendored_copy() { + for source in [MY_CORP, "git+https://example.com/cfg-if#abc123", CRATES_IO] { + let p = vendored_workspace("cfg-if = \"1\"", "", &[("1.0.4", source)]); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/cfg-if@1.0.4", UUID_A, WiringMode::Vendored)], + ); + let u = bypassed_mark(&out, "Cargo.lock"); + assert!( + u.detail + .contains(&format!("Cargo.lock also locks cfg-if 1.0.4 from {source}")), + "{source}: {}", + u.detail + ); + } + let p = vendored_workspace("cfg-if = \"1\"", "", &[("0.1.10", MY_CORP)]); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/cfg-if@1.0.4", UUID_A, WiringMode::Vendored)], + ); + assert!(out.unattested.is_empty(), "{:#?}", out.unattested); + } + + /// What the `[patch]` wiring redirects stays attested: a member + /// declaring cfg-if from crates.io, under any spelling, and a + /// declaration from elsewhere whose requirement excludes the patched + /// version, or a registry dependency whose `[patch.]` table + /// also points at the same copy; and a `[workspace.dependencies]` + /// template from another registry that no manifest inherits. A user's + /// own path fork is the documented exception (see + /// `a_same_version_fork_beside_the_tagged_copy_is_ignored`). + #[tokio::test] + async fn declarations_the_patch_redirects_keep_the_vendored_copy_attested() { + let rel = vendor_path(UUID_A, "cfg-if-1.0.4"); + let covered = format!("\n[patch.my-corp]\ncfg-if = {{ path = \"{rel}\" }}\n"); + let template = "\n[workspace.dependencies]\n\ + cfg-if = { version = \"1\", registry = \"my-corp\" }\n"; + for (member, root_extra) in [ + ("cfg-if = \"1\"", ""), + ("cfg-if = \"=1.0.4\"", ""), + ("cfg-if = { version = \"1\", registry = \"crates-io\" }", ""), + ("alias = { package = \"cfg-if\", version = \"1.0\" }", ""), + ("cfg-if = { version = \"0.1\", registry = \"my-corp\" }", ""), + ( + "cfg-if = { version = \"1\", registry = \"my-corp\" }", + covered.as_str(), + ), + ("cfg-if = { path = \"../fork\" }", ""), + ("cfg-if = \"1\"", template), + ] { + let p = vendored_workspace(member, root_extra, &[]); + let out = run(&p).await; + assert!( + out.refs.iter().any(|r| r.purl == "pkg:cargo/cfg-if@1.0.4" + && r.uuid == UUID_A + && r.mode == WiringMode::Vendored), + "{member}: {:#?}", + out.refs + ); + assert!(out.unattested.is_empty(), "{member}: {:#?}", out.unattested); + } + } + + /// A `[patch.""]` table pointing at the same copy redirects a + /// declaration from that repository (a trailing `/` or `.git` aside), + /// so the copy stays attested; one from another repository still + /// bypasses it. + #[tokio::test] + async fn a_git_patch_table_for_the_copy_covers_that_repository_only() { + let rel = vendor_path(UUID_A, "cfg-if-1.0.4"); + let table = + format!("\n[patch.\"https://example.com/cfg-if\"]\ncfg-if = {{ path = \"{rel}\" }}\n"); + for member in [ + "cfg-if = { git = \"https://example.com/cfg-if\" }", + "cfg-if = { git = \"https://example.com/cfg-if.git/\", branch = \"main\" }", + ] { + let out = run(&vendored_workspace(member, &table, &[])).await; + assert_refs( + &out, + &[("pkg:cargo/cfg-if@1.0.4", UUID_A, WiringMode::Vendored)], + ); + assert!(out.unattested.is_empty(), "{member}: {:#?}", out.unattested); + } + let out = run(&vendored_workspace( + "cfg-if = { git = \"https://example.com/fork/cfg-if\" }", + &table, + &[], + )) + .await; + bypassed_mark(&out, "crates/a/Cargo.toml"); + } + + /// The pre-v5 wiring in the project config redirects only crates.io + /// too: a member's git declaration bypasses it the same way. + #[tokio::test] + async fn a_member_from_git_bypasses_the_config_wired_copy() { + let p = Project::new(); + p.write( + ".cargo/config.toml", + patch_config(&[("cfg-if", &vendor_path(UUID_A, "cfg-if-1.0.4"))]), + ); + workspace_with_member(&p, "cfg-if = { git = \"https://example.com/cfg-if\" }"); + p.write("Cargo.lock", lock(&[("cfg-if", "1.0.4", None, None)])); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/cfg-if@1.0.4", UUID_A, WiringMode::Vendored)], + ); + bypassed_mark(&out, "crates/a/Cargo.toml"); + } + + /// A vendored project's lock has no Socket-sourced package (the copy's + /// entry is sourceless), and a lockless one has no `socket-patch-` pin, so + /// the member walk must not hinge on either: the members are read whenever + /// a `[patch]` wires a Socket copy, lock or no lock. + #[tokio::test] + async fn members_are_read_for_a_vendored_copy_without_a_socket_lock_entry() { + for with_lock in [true, false] { + let p = vendored_workspace( + "cfg-if = { version = \"1\", registry = \"my-corp\" }", + "", + &[], + ); + if !with_lock { + std::fs::remove_file(p.root().join("Cargo.lock")).unwrap(); + } + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/cfg-if@1.0.4", UUID_A, WiringMode::Vendored)], + ); + bypassed_mark(&out, "crates/a/Cargo.toml"); + } + } + + /// The one vendored cfg-if ref of `out`: attested (no mark) or not. + fn assert_vendored_attested(out: &Discovery, attested: bool, what: &str) { + assert_refs( + out, + &[("pkg:cargo/cfg-if@1.0.4", UUID_A, WiringMode::Vendored)], + ); + assert_eq!( + out.unattested.is_empty(), + attested, + "{what}: {:#?}", + out.unattested + ); + } + + /// The project config redefines the manifest's crates.io `[patch]` key + /// with a git fork, so cargo ignores the manifest's entry for the copy and + /// builds the fork for crates.io declarations: that shadowed entry + /// redirects nothing, so the copy's other (honored) `[patch.my-corp]` + /// entry is not attested beside the fork. + #[tokio::test] + async fn a_shadowed_manifest_entry_covers_no_declaration() { + let rel = vendor_path(UUID_A, "cfg-if-1.0.4"); + let p = vendored_workspace( + "cfg-if = \"1\"", + &format!("\n[patch.my-corp]\ncfg-if = {{ path = \"{rel}\" }}\n"), + &[], + ); + p.write( + ".cargo/config.toml", + "[patch.crates-io]\ncfg-if-socket-aaaaaaaa = { package = \"cfg-if\", git = \ + \"https://example.com/fork/cfg-if\" }\n", + ); + let out = run(&p).await; + assert_vendored_attested(&out, false, "shadowed"); + let u = bypassed_mark(&out, "Cargo.toml"); + assert!( + u.detail + .contains("Cargo.toml declares cfg-if \"1\" from crates.io"), + "{}", + u.detail + ); + assert!( + diag_codes(&out).contains(&DIAG_REF_INVALID), + "{:#?}", + out.diagnostics + ); + } + + /// A pin to another patch beside a lock that also resolves the crate from + /// crates.io is not described as "only from other Socket patches". + #[tokio::test] + async fn another_patch_pin_beside_a_foreign_twin_is_not_called_socket_only() { + let p = mixed_workspace( + &pinned("serde", "1", UUID_A), + &pinned("serde", "1", UUID_B), + &[("1.0.1", CRATES_IO)], + ); + let out = run(&p).await; + let diag = out + .diagnostics + .iter() + .find(|d| d.code == DIAG_REF_UNATTRIBUTABLE && d.detail.contains(UUID_B)) + .unwrap_or_else(|| panic!("{:#?}", out.diagnostics)); + assert!( + !diag.detail.contains("only from other Socket patches"), + "{}", + diag.detail + ); + } + + /// The [`Shadow`] of a bare manifest under project config `config`. + fn shadow(config: &str) -> super::Shadow { + let manifest: toml_edit::DocumentMut = "[package]\nname = \"x\"\n".parse().unwrap(); + let config: toml_edit::DocumentMut = config.parse().unwrap(); + super::Shadow::new(&manifest, Some(&(".cargo/config.toml", config))) + } + + /// A config item for the same key replaces the manifest's `[patch]` item + /// in cargo whatever the source table and whether or not either wires a + /// path: a path-less config git fork shadows a path-less manifest registry + /// pin, and a `[patch.my-corp]` config item shadows the manifest's + /// `[patch.my-corp]` copy. + #[test] + fn config_items_shadow_pathless_and_named_source_entries() { + let s = shadow( + "[patch.crates-io]\n\ + cfg-if-socket-aaaaaaaa = { package = \"cfg-if\", git = \"https://example.com/f\" }\n\ + [patch.my-corp]\n\ + cfg-if = { git = \"https://example.com/f\" }\n\ + [patch.\"https://My-Corp.example/index/\"]\n\ + cfg-if = { path = \"../fork\" }\n", + ); + // A path-less registry pin under the same key: shadowed. + assert!(s + .why( + "crates-io", + "cfg-if-socket-aaaaaaaa", + None, + Some("socket-patch-aaaaaaaa") + ) + .is_some()); + // The same key under another source: not. + assert!(s + .why( + "other", + "cfg-if-socket-aaaaaaaa", + None, + Some("socket-patch-aaaaaaaa") + ) + .is_none()); + // A named non-crates.io table: shadowed. + assert!(s.why("my-corp", "cfg-if", Some("vendor/x"), None).is_some()); + // A URL table, compared as an index. + assert!(s + .why( + "https://my-corp.example/index", + "cfg-if", + Some("vendor/x"), + None + ) + .is_some()); + // A canonical-equivalent git table. + let s2 = shadow("[patch.\"https://GitHub.com/Corp/Fork.git/\"]\ncfg-if = { git = \"https://example.com/f\" }\n"); + assert!(s2 + .why( + "https://github.com/corp/fork", + "cfg-if", + Some("vendor/x"), + None + ) + .is_some()); + assert!(s2 + .why( + "sparse+https://github.com/corp/fork", + "cfg-if", + Some("vendor/x"), + None + ) + .is_none()); + // The half-migrated shape: the same path is the same wiring. + assert!(s + .why( + "https://my-corp.example/index", + "cfg-if", + Some("../fork"), + None + ) + .is_none()); + // The identical path-less Socket registry pin in both files is the + // same wiring, not a shadow. + let s3 = shadow("[patch.crates-io]\nserde = { registry = \"socket-patch-aaaaaaaa\" }\n"); + assert!(s3 + .why("crates-io", "serde", None, Some("socket-patch-aaaaaaaa")) + .is_none()); + assert!(s3 + .why("crates-io", "serde", None, Some("socket-patch-bbbbbbbb")) + .is_some()); + // A `[patch.""]` config table patches the same source as the + // manifest's `[patch.]` for a registry defined on that index. + let s4 = shadow( + "[registries.my-corp]\n\ + index = \"https://my-corp.example/index\"\n\ + [patch.\"https://my-corp.example/index\"]\n\ + cfg-if = { git = \"https://example.com/fork\" }\n", + ); + assert!(s4 + .why("my-corp", "cfg-if", Some("vendor/x"), None) + .is_some()); + assert!(s4 + .why("other-corp", "cfg-if", Some("vendor/x"), None) + .is_none()); + } + + /// A `registry-index` declaration is redirected by a + /// `[patch.""]` table for the same index (host case, a + /// trailing `/` and the lock's `registry+` spelling aside; `sparse+` is + /// another source), and + /// one naming the crates.io git index is crates.io. The sparse crates.io + /// index stays another source: whether `[patch.crates-io]` reaches it is + /// not what the files show, so it is withheld rather than risked. + #[tokio::test] + async fn a_registry_index_patch_table_covers_that_index() { + let rel = vendor_path(UUID_A, "cfg-if-1.0.4"); + let table = |key: &str| format!("\n[patch.\"{key}\"]\ncfg-if = {{ path = \"{rel}\" }}\n"); + let decl = + |index: &str| format!("cfg-if = {{ version = \"1\", registry-index = \"{index}\" }}"); + for (member, extra, attested) in [ + ( + decl("https://My-Corp.example/index/"), + table("https://my-corp.example/index"), + true, + ), + ( + decl("sparse+https://my-corp.example/index/"), + table("sparse+https://my-corp.example/index"), + true, + ), + ( + decl("https://github.com/rust-lang/crates.io-index"), + String::new(), + true, + ), + ( + decl("sparse+https://my-corp.example/index/"), + table("https://my-corp.example/index"), + false, + ), + ( + decl("https://other.example/index"), + table("https://my-corp.example/index"), + false, + ), + ( + decl("sparse+https://index.crates.io/"), + String::new(), + false, + ), + ] { + let out = run(&vendored_workspace(&member, &extra, &[])).await; + assert_vendored_attested(&out, attested, &member); + } + } + + /// The remedy follows the evidence — a declaration cannot be fixed by + /// re-vendoring; a crates.io lock entry beside a `[patch.crates-io]` copy + /// is stale (re-lock); another source's entry is some dependency's + /// (`cargo tree -i` finds it). + #[tokio::test] + async fn the_bypass_remedy_follows_its_evidence() { + let out = run(&vendored_workspace( + "cfg-if = { version = \"1\", registry = \"my-corp\" }", + "", + &[], + )) + .await; + let u = bypassed_mark(&out, "crates/a/Cargo.toml"); + assert!( + u.detail + .contains("Re-running `vendor` does not change this"), + "{}", + u.detail + ); + + let out = run(&vendored_workspace( + "cfg-if = \"1\"", + "", + &[("1.0.4", CRATES_IO)], + )) + .await; + let u = bypassed_mark(&out, "Cargo.lock"); + assert!( + u.detail.contains("stale") + && u.detail.contains("cargo update -p cfg-if") + && !u.detail.contains("Re-running `vendor` does not"), + "{}", + u.detail + ); + + let out = run(&vendored_workspace( + "cfg-if = \"1\"", + "", + &[("1.0.4", MY_CORP)], + )) + .await; + let u = bypassed_mark(&out, "Cargo.lock"); + assert!( + u.detail.contains("cargo tree -i cfg-if@1.0.4") && !u.detail.contains("stale"), + "{}", + u.detail + ); + + // A git entry a git `[patch]` table at the copy redirects is stale + // too, but `vendor` (which writes `[patch.crates-io]`) does not + // re-lock it. + let rel = vendor_path(UUID_A, "cfg-if-1.0.4"); + let out = run(&vendored_workspace( + "cfg-if = \"1\"", + &format!( + "\n[patch.\"https://github.com/corp/fork\"]\ncfg-if = {{ path = \"{rel}\" }}\n" + ), + &[("1.0.4", "git+https://github.com/corp/fork#0123456789abcdef")], + )) + .await; + let u = bypassed_mark(&out, "Cargo.lock"); + assert!( + u.detail + .contains("`[patch.\"https://github.com/corp/fork\"]` redirects its source") + && u.detail.contains("cargo update -p cfg-if") + && !u.detail.contains("socket-patch vendor"), + "{}", + u.detail + ); + } + + /// Git URLs compare as cargo's `CanonicalUrl` spells them — a GitHub URL + /// case-insensitively, any host's case aside, one trailing `/` and `.git` + /// dropped — so another path on a non-GitHub host is another repository. + #[tokio::test] + async fn a_git_patch_table_matches_cargos_canonical_url() { + let rel = vendor_path(UUID_A, "cfg-if-1.0.4"); + let table = |key: &str| format!("\n[patch.\"{key}\"]\ncfg-if = {{ path = \"{rel}\" }}\n"); + let decl = |url: &str| format!("cfg-if = {{ git = \"{url}\" }}"); + for (member, extra, attested) in [ + ( + decl("https://GitHub.com/Foo/CFG-IF.git"), + table("https://github.com/foo/cfg-if"), + true, + ), + ( + decl("https://EXAMPLE.com/foo/cfg-if/"), + table("https://example.com/foo/cfg-if"), + true, + ), + ( + decl("https://example.com/Foo/cfg-if"), + table("https://example.com/foo/cfg-if"), + false, + ), + ] { + let out = run(&vendored_workspace(&member, &extra, &[])).await; + assert_vendored_attested(&out, attested, &member); + } + } + /// A member pinned to ANOTHER Socket patch builds that patch's copy. When + /// the lock resolves cfg-if from it, its hosted ref stands beside the + /// vendored one and the CLI's conflict gate refuses both; when the lock + /// does not (stale), the vendored copy is marked bypassed rather than + /// attested alone. + #[tokio::test] + async fn a_member_pinned_to_another_patch_bypasses_the_vendored_copy() { + let p = vendored_workspace(&pinned("cfg-if", "1", UUID_B), "", &[]); + let out = run(&p).await; + assert_refs( + &out, + &[("pkg:cargo/cfg-if@1.0.4", UUID_A, WiringMode::Vendored)], + ); + let u = bypassed_mark(&out, "crates/a/Cargo.toml"); + assert!( + u.detail.contains("crates/a/Cargo.toml declares cfg-if") + && u.detail.contains(UUID_B) + && u.detail.contains("stale Cargo.lock"), + "{}", + u.detail + ); + + // Locked from that patch: the conflict pair, no bypass mark on A. + let b = index(UUID_B); + let p = vendored_workspace(&pinned("cfg-if", "1", UUID_B), "", &[("1.0.4", b.as_str())]); + let out = run(&p).await; + assert_refs( + &out, + &[ + ("pkg:cargo/cfg-if@1.0.4", UUID_B, WiringMode::Hosted), + ("pkg:cargo/cfg-if@1.0.4", UUID_A, WiringMode::Vendored), + ], + ); + assert!( + out.unattested.iter().all(|u| u.uuid != UUID_A), + "{:#?}", + out.unattested + ); + + // Locked from that patch, but the project's registry definition + // indexes another patch: the hosted pass emits no ref for it, so + // nothing pairs with the vendored copy and it is withheld. + let p = vendored_workspace(&pinned("cfg-if", "1", UUID_B), "", &[("1.0.4", b.as_str())]); + p.write( + ".cargo/config.toml", + format!( + "[registries.socket-patch-{UUID_B}]\nindex = \"{}\"\n", + index(UUID_A) + ), + ); + let out = run(&p).await; + assert!(out.refs.iter().all(|r| r.uuid != UUID_B), "{:#?}", out.refs); + bypassed_mark(&out, "crates/a/Cargo.toml"); + + // A requirement that excludes the vendored version says nothing. + let p = vendored_workspace(&pinned("cfg-if", "0.1", UUID_B), "", &[]); + let out = run(&p).await; + assert!( + out.unattested.iter().all(|u| u.uuid != UUID_A), + "{:#?}", + out.unattested + ); + } } diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index f9d8f6d9c..e2a98f0d5 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -430,6 +430,13 @@ pub struct UnlockedPin { /// `version = "…"`); a ledger version must satisfy every one. Empty = /// none to check (nuget: the csproj `Version` is only a minimum). pub version_reqs: Vec, + /// Requirements of the package's OTHER declarations, which carry no + /// pin (cargo: a member's `serde = "0.9"` beside the root's pinned + /// `serde = "1"`). The hosted rewriter pins every declaration whose + /// requirement accepts the patched version, so one that accepts a + /// ledger version means the pin does not reach it: a ledger version must + /// satisfy none of these. + pub excluded_reqs: Vec, /// The Socket index url the pin routes to (the nuget source value, the /// cargo registry's `index`): its grant-token segment is part of this /// pin, not other hosted wiring. @@ -453,16 +460,22 @@ impl UnlockedPin { }; let pinned = canonical_base_purl(&format!("pkg:{}/{}@{version}", self.ecosystem, self.name)); + let parsed = semver::Version::parse(version); + // An unreadable requirement or version answers `None`: it matches + // no pinned requirement and is never proven excluded. + let matches = |req: &str| match (semver::VersionReq::parse(req), &parsed) { + (Ok(req), Ok(version)) => Some(req.matches(version)), + _ => None, + }; pinned == key - && self.version_reqs.iter().all(|req| { - match ( - semver::VersionReq::parse(req), - semver::Version::parse(version), - ) { - (Ok(req), Ok(version)) => req.matches(&version), - _ => false, - } - }) + && self + .version_reqs + .iter() + .all(|req| matches(req) == Some(true)) + && self + .excluded_reqs + .iter() + .all(|req| matches(req) == Some(false)) } } @@ -621,6 +634,21 @@ pub enum UnattestedKind { /// version: the classifier-less pin does not retarget it and the grant /// serves only the main jar, so the build runs the public classifier jar. MavenClassifierUnpatched, + /// A hosted Cargo pin beside another declaration of the crate (the + /// root's or a workspace member's) that is not pinned (no Socket + /// registry or `[patch]` redirect reaches it, or a path / git / + /// `registry-index` overrides its registry) and whose requirement may + /// select the pinned version: cargo resolves that + /// declaration from its own source as a separate package, so the crate + /// declaring it builds an unpatched copy, possibly of this version. + CargoUnpinnedDeclaration, + /// A vendored Cargo copy that a `[patch.]` entry wires (in v5, + /// the root's `[patch.crates-io]`) beside a copy of the crate from a + /// source that entry does not redirect: a declaration (the root's or a + /// workspace member's) from another registry or git whose requirement + /// accepts the patched version, or a same-version `Cargo.lock` entry + /// from any source. Cargo builds that copy unpatched. + CargoPatchBypassed, /// A pnpm lock's pin while any `pnpm-lock..yaml` exists (#556), /// whatever pnpm's `gitBranchLockfile` setting says (discovery reads no /// environment, so the setting cannot be decided), or while the members @@ -846,6 +874,12 @@ pub struct Discovery { /// (#1445) ([`StaleSelector`]). Not refs and not contested wiring: the /// management commands retire them. Sorted, deduped. pub stale_selectors: Vec, + /// The operator's patch-server origins ([`DiscoverOptions::patch_server_origins`]) + /// this discovery classified urls with. Ledger liveness + /// ([`Discovery::redirect_record_live`]) classifies the lock inventory's + /// urls with the same origins, so a record discovery does not claim is + /// judged by the one origin rule ([`crate::origin::hosted_patch_uuid`]). + pub patch_server_origins: Vec, } /// A Socket-hosted yarn berry `resolutions` selector whose lock entry is @@ -1633,6 +1667,7 @@ async fn discover_with_ctx(mut ctx: DiscoverCtx<'_>) -> Discovery { out.unattest_unwired_copies(); out.recognized.extend(ctx.take_recognized()); out.read.extend(ctx.take_read()); + out.patch_server_origins = ctx.origin.origins().to_vec(); out.finalize(); out } @@ -1732,6 +1767,14 @@ impl<'a> DiscoverCtx<'a> { } } + /// Account for `text`, which a walk through [`Self::view`] read from + /// `rel` on this ctx's behalf, exactly as [`Self::read_text`] accounts + /// for its own reads (rule 11 recognition and the read log). + pub(crate) fn adopt_read(&self, rel: &str, text: &str) { + self.log_read(rel, true); + self.recognize_text(rel, text); + } + /// Log a guarded read of `rel` ([`Discovery::read`]). fn log_read(&self, rel: &str, readable: bool) { self.read @@ -2446,8 +2489,15 @@ impl Discovery { /// `source "" do` block still declares the gem /// ([`gem::gemfile_source_block_pins`]): live — bundler re-resolves /// from it whatever the lock says; - /// 1. a lock inventory entry for this purl whose resolved url carries - /// the uuid: live; + /// 1. a lock inventory entry for this purl whose resolved url is a + /// Socket-hosted url for THIS uuid + /// ([`crate::origin::hosted_patch_uuid`] over Socket's host + /// and [`Discovery::patch_server_origins`]): live. Failing that, an + /// entry whose url has the hosted shape for this uuid on an origin + /// this run was not given (a patch server an earlier run was pointed + /// at with `--patch-server-url`, or a host that merely spells the + /// uuid): [`RecordLiveness::Undecided`] — never attested, never + /// judged dead; /// 2. a lock inventory entry that resolves the purl from somewhere ELSE /// (a registry url, a crates.io-checksummed Cargo.lock source): dead /// — the lock positively says the build fetches the unpatched @@ -2458,10 +2508,19 @@ impl Discovery { /// route nothing once the dependency pin is reverted, and reads a /// Gemfile / `gems.rb` with discovery's source-block grammar). /// + /// A record steps 0 or 3 find pinned is [`RecordLiveness::Undecided`] + /// instead of live when its recorded files route the uuid through a + /// hosted-shaped url on an origin this run was not given — the step-1 + /// rule, for the formats whose inventory carries no url + /// ([`foreign_hosted_url_in_files`]). + /// /// `inventory` is the project's EVERY-instance lock inventory /// ([`inventory_project_every_lock`] — step 2 is negative evidence, so a /// collapsed view that dropped the hosted instance must not feed it), /// loaded on first need and shared across calls. + /// + /// The attestation-grade verdict: [`Self::redirect_record_verdict`] + /// is [`RecordLiveness::Live`]. pub async fn redirect_record_live( &self, root: &Path, @@ -2470,8 +2529,23 @@ impl Discovery { recorded_files: &[&str], inventory: &mut Option>, ) -> bool { + self.redirect_record_verdict(root, purl, uuid, recorded_files, inventory) + .await + .attests() + } + + /// [`Self::redirect_record_live`]'s three-way verdict (see + /// [`RecordLiveness`]). + pub async fn redirect_record_verdict( + &self, + root: &Path, + purl: &str, + uuid: &str, + recorded_files: &[&str], + inventory: &mut Option>, + ) -> RecordLiveness { if let Some(live) = self.hosted_claim(purl, uuid) { - return live; + return RecordLiveness::from_bool(live); } // Bundler resolves from the Gemfile / `gems.rb`: when a `source // "" do` block there still pins the uuid but the @@ -2496,7 +2570,7 @@ impl Discovery { continue; }; if gem::gemfile_source_block_pins(&text, uuid, Some(&gem_name)) { - return true; + return self.unless_foreign_hosted(root, recorded_files, uuid).await; } } } @@ -2514,19 +2588,162 @@ impl Discovery { .iter() .filter(|e| lookup(std::slice::from_ref(*e), purl).is_some()) .collect(); - if entries - .iter() - .any(|e| e.resolved.as_deref().is_some_and(|r| r.contains(uuid))) - { - return true; + let resolved = || entries.iter().filter_map(|e| e.resolved.as_deref()); + if resolved().any(|r| { + crate::origin::hosted_patch_uuid(r, &self.patch_server_origins).as_deref() == Some(uuid) + }) { + return RecordLiveness::Live; + } + if resolved().any(|r| crate::origin::hosted_shape_uuid(r).as_deref() == Some(uuid)) { + return RecordLiveness::Undecided; } if entries.iter().any(|e| resolves_elsewhere(e)) { - return false; + return RecordLiveness::Dead; + } + if !hosted_wiring_in_files(root, &recorded, uuid).await { + return RecordLiveness::Dead; + } + self.unless_foreign_hosted(root, recorded_files, uuid).await + } + + /// The verdict for a record whose ledger-recorded wiring still pins + /// `uuid` (steps 0 and 3 of [`Self::redirect_record_live`]): `Live`, + /// unless those files route it through a url with the hosted shape for + /// `uuid` on an origin this run was not given — the same wiring step 1 + /// judges `Undecided` when a lock inventory entry carries the url, here + /// for the formats whose inventory does not (a yarn berry + /// `__archiveUrl=` binding, a `Cargo.lock` registry `source`, a pom + /// ``, a Gemfile `source` block). Fail closed: a pin to a + /// staging server used without `--patch-server-url`, or to a look-alike + /// host, is never attested ([`foreign_hosted_url_in_files`]). + async fn unless_foreign_hosted( + &self, + root: &Path, + recorded_files: &[&str], + uuid: &str, + ) -> RecordLiveness { + if foreign_hosted_url_in_files(root, recorded_files, uuid, &self.patch_server_origins).await + { + RecordLiveness::Undecided + } else { + RecordLiveness::Live + } + } +} + +/// A redirect-ledger record's liveness ([`Discovery::redirect_record_verdict`]). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RecordLiveness { + /// The build resolves the package from the record's patch. + Live, + /// A lock entry resolves the package from a url with the hosted shape + /// for the record's patch, on an origin this run does not accept as a + /// patch server: perhaps the operator's own server, configured on an + /// earlier run (`--patch-server-url`) but not this one; perhaps a host + /// that merely spells the uuid. Never attested, never judged dead — + /// what keeps the wiring (the hosted-wiring advisories) keeps it. + Undecided, + /// The build does not resolve the package from the record's patch. + Dead, +} + +impl RecordLiveness { + fn from_bool(live: bool) -> Self { + if live { + Self::Live + } else { + Self::Dead } - hosted_wiring_in_files(root, &recorded, uuid).await + } + + /// Whether the record may be attested. + pub fn attests(self) -> bool { + self == Self::Live + } + + /// Whether the record's wiring is to be kept (not judged gone). + pub fn retained(self) -> bool { + self != Self::Dead } } +/// Whether any of `files` — the root-relative files a REDIRECT ledger recorded +/// editing — spells a url with the hosted shape for `uuid` +/// ([`crate::origin::hosted_shape_uuid`]) on an origin outside Socket's host +/// and `origins`. Fail closed: one such url is enough, whatever else the files +/// spell (a Socket-host url beside it — a leftover definition, a comment — +/// does not clear it). Each occurrence of the uuid is widened to the url token +/// around it (bounded by whitespace, quotes, markup and `=` / `&`, so a berry +/// `__archiveUrl=` binding value or a `Cargo.lock` `sparse+` source reads +/// whole), then read the way [`crate::origin::hosted_patch_uuid`] reads a url; +/// a scheme-less token (a go.mod `replace` module path, +/// `/gopatch/`) is read as `https://`. Every recorded file counts, +/// registry definitions included: a definition routes nothing on its own, but +/// it names the host the pin routes to. `bun.lockb` is binary and is skipped +/// (its inventory carries the url to step 1). Same path guards as +/// [`hosted_wiring_in_files`]. +async fn foreign_hosted_url_in_files( + root: &Path, + files: &[&str], + uuid: &str, + origins: &[String], +) -> bool { + if !is_canonical_uuid(uuid) { + return false; + } + for file in unique_safe_rel_files(files) { + if file == "bun.lockb" || file.ends_with("/bun.lockb") { + continue; + } + let Ok(text) = crate::utils::fs::read_regular_to_string(&root.join(&file)).await else { + continue; + }; + for token in url_tokens_around(&text, uuid) { + let schemed; + let url = if token.contains("://") || token.contains("%3A%2F%2F") { + token + } else { + schemed = format!("https://{token}"); + &schemed + }; + if crate::origin::hosted_patch_uuid(url, origins).as_deref() != Some(uuid) + && crate::origin::hosted_shape_uuid(url).as_deref() == Some(uuid) + { + return true; + } + } + } + false +} + +/// The url-shaped token around each occurrence of `needle` in `text`: +/// widened in both directions up to whitespace, a quote, markup or a +/// key / binding separator (`=`, `&`, `,`, brackets). +fn url_tokens_around<'a>(text: &'a str, needle: &str) -> Vec<&'a str> { + let stop = |c: char| { + c.is_whitespace() + || matches!( + c, + '"' | '\'' | '`' | '<' | '>' | '(' | ')' | '[' | ']' | '{' | '}' | '=' | '&' | ',' + ) + }; + let mut out = Vec::new(); + let mut from = 0; + while let Some(pos) = text[from..].find(needle) { + let idx = from + pos; + let end = idx + needle.len(); + let start = text[..idx] + .char_indices() + .rev() + .find(|&(_, c)| stop(c)) + .map_or(0, |(i, c)| i + c.len_utf8()); + let stop_at = text[end..].find(stop).map_or(text.len(), |i| end + i); + out.push(&text[start..stop_at]); + from = end; + } + out +} + /// Whether root-relative `file` is one of `ecosystem`'s lockfiles — what an /// install of that ecosystem resolves from: a [`ROOT`]-role row of the /// format registry by basename (`Cargo.lock`, `composer.lock`, `go.mod`, @@ -2601,10 +2818,24 @@ impl<'a> LedgerLiveness<'a> { } } - /// [`Discovery::redirect_record_live`] for this project state. + /// [`Discovery::redirect_record_live`] for this project state: whether + /// the record may be ATTESTED. pub async fn redirect_record(&mut self, purl: &str, uuid: &str) -> bool { + self.redirect_record_verdict(purl, uuid).await.attests() + } + + /// Whether the record's hosted wiring is still in place for a caller + /// that keeps or reports wiring rather than attesting it + /// ([`RecordLiveness::retained`]: an [`RecordLiveness::Undecided`] + /// record is kept). + pub async fn redirect_record_retained(&mut self, purl: &str, uuid: &str) -> bool { + self.redirect_record_verdict(purl, uuid).await.retained() + } + + /// [`Discovery::redirect_record_verdict`] for this project state. + pub async fn redirect_record_verdict(&mut self, purl: &str, uuid: &str) -> RecordLiveness { self.discovery - .redirect_record_live( + .redirect_record_verdict( self.root, purl, uuid, @@ -2625,9 +2856,10 @@ impl<'a> LedgerLiveness<'a> { } } -/// A lock inventory entry (already known NOT to carry the patch uuid in its -/// resolved url) that positively resolves from a non-patch source: an -/// explicit artifact url, or a crates.io Cargo.lock source (the registry +/// A lock inventory entry (already known NOT to resolve from the record's +/// patch: its resolved url, if any, is no Socket-hosted url for that uuid) +/// that positively resolves from somewhere else: an explicit artifact url +/// (another host, or another patch), or a crates.io Cargo.lock source (the registry /// view's [`SourceKind::CratesIo`]; Socket sparse sources carry no url in /// the inventory and so prove nothing either way). An entry with no url is /// the conventional registry for most formats but is also how the inventory @@ -3206,6 +3438,36 @@ mod tests { use super::testing::*; use super::*; + /// A foreign hosted url reads its origin after the same normalization + /// `hosted_patch_uuid` applies: `\\/` escapes, a percent-encoded url + /// and a Cargo `sparse+` / `registry+` prefix. + #[test] + fn hosted_shape_uuid_normalizes_like_hosted_patch_uuid() { + const U: &str = "11111111-2222-4333-8444-555555555555"; + for url in [ + format!("https://mirror.example/patch/{U}/x.tgz"), + format!("https:\\/\\/mirror.example\\/patch\\/{U}\\/x.tgz"), + format!("https%3A%2F%2Fmirror.example%2Fpatch%2F{U}%2Fx.tgz"), + format!("sparse+https://mirror.example/patch/{U}/"), + format!("registry+https://mirror.example/patch/{U}/"), + format!("https://patch.socket.dev@mirror.example/patch/{U}/x.tgz"), + format!("https://user:pw@patch.socket.dev/patch/{U}/x.tgz"), + ] { + assert_eq!( + crate::origin::hosted_shape_uuid(&url).as_deref(), + Some(U), + "{url}" + ); + } + // A url merely ending in the uuid is not a patch server's. + for url in [ + format!("https://artifacts.corp/blobs/{U}/x-1.0.0.tgz"), + format!("https://evil.example/{U}/x.tgz"), + ] { + assert_eq!(crate::origin::hosted_shape_uuid(&url), None, "{url}"); + } + } + /// The read-only parse [`toml_or_diag`] hands extractors reads every /// value the way the `DocumentMut` it replaced did (#1473): escaped, /// literal and multi-line strings decode, quoted and dotted keys @@ -3473,12 +3735,13 @@ mod tests { } /// REGRESSION: step 2's negative evidence reads EVERY lock's instance. - /// `uv.lock` routes six to a hosted wheel on a patch host outside the - /// discovery allowlist (no `hosted_claim`), and a PEP 723 script lock - /// that sorts first resolves the same six from PyPI. The collapsed - /// inventory keeps only the script lock's instance, which made the - /// record look resolved-elsewhere (dead) although the project lock still - /// installs the patch. + /// `uv.lock` routes six to a hosted wheel on the operator's configured + /// patch server, which a discovery that holds no claim on the uuid + /// classifies with the run's origins, and a PEP 723 script lock that + /// sorts first resolves the same six from PyPI. The collapsed inventory + /// keeps only the script lock's instance, which made the record look + /// resolved-elsewhere (dead) although the project lock still installs + /// the patch. #[tokio::test] async fn a_script_lock_registry_instance_does_not_veto_the_hosted_project_lock() { let p = Project::new(); @@ -3537,6 +3800,21 @@ mod tests { None, "host not allowlisted" ); + // Without the origin, the internal host's url is undecided: never + // attested, and not vetoed by the script lock's registry instance + // either. + let mut inventory = None; + assert_eq!( + discovery + .redirect_record_verdict(root, purl, UUID_A, &["uv.lock"], &mut inventory) + .await, + RecordLiveness::Undecided, + "an unconfigured patch host is never attested, nor judged dead" + ); + let discovery = Discovery { + patch_server_origins: vec!["https://patches.internal.example".to_string()], + ..Discovery::default() + }; let mut inventory = None; assert!( discovery @@ -3565,7 +3843,9 @@ mod tests { /// would). #[tokio::test] async fn gemfile_liveness_reads_live_source_blocks_only() { - let index = format!("https://patches.example.com/patch-registry/gem/{TOKEN}/{UUID_A}/"); + // Socket's host: a source block on an origin this run was not given + // is never live (`recorded_wiring_on_a_foreign_origin_is_undecided`). + let index = format!("https://patch.socket.dev/patch-registry/gem/{TOKEN}/{UUID_A}/"); let p = Project::new(); p.write( "Gemfile.lock", @@ -3669,6 +3949,250 @@ mod tests { assert!(!hosted_wiring_in_files(root, &["package-lock.json"], "not-a-uuid").await); } + /// Step 1 classifies a lock entry's resolved url with the one origin rule + /// ([`crate::origin::hosted_patch_uuid`]), never by substring. Only a + /// Socket-host url or a configured origin's url for the record's uuid is + /// LIVE (attested). A url with the hosted shape for the uuid on an origin + /// this run was not given — a patch server an earlier run was pointed at, + /// or a host that merely spells it — is UNDECIDED: never attested, but its + /// wiring is retained. A url that only contains the uuid (a file name, a + /// query) and a Socket url for ANOTHER patch (whose path carries the uuid + /// as its grant token) resolve the package from somewhere else: DEAD. + #[tokio::test] + async fn liveness_matches_the_resolved_url_by_origin_not_substring() { + use RecordLiveness::{Dead, Live, Undecided}; + let origin = "http://127.0.0.1:4545"; + let purl = "pkg:npm/x@1.0.0"; + let elsewhere = + format!("https://evil.example/patch/npm/x/1.0.0/{TOKEN}/{UUID_A}/x-1.0.0.tgz"); + let userinfo = format!( + "https://patch.socket.dev@evil.example/patch/npm/x/1.0.0/{TOKEN}/{UUID_A}/x-1.0.0.tgz" + ); + let in_name = format!("https://evil.example/files/x-{UUID_A}.tgz"); + let in_query = format!("https://evil.example/files/x-1.0.0.tgz?id={UUID_A}"); + let other_patch = + format!("https://patch.socket.dev/patch/npm/x/1.0.0/{UUID_A}/{UUID_B}/x-1.0.0.tgz"); + let socket = hosted_url("npm", "x", "1.0.0", UUID_A, "x-1.0.0.tgz"); + let configured = format!("{origin}/patch/npm/x/1.0.0/{TOKEN}/{UUID_A}/x-1.0.0.tgz"); + let with_origin = Discovery { + patch_server_origins: vec![origin.to_string()], + ..Discovery::default() + }; + let without = Discovery::default(); + for (resolved, discovery, verdict) in [ + (&elsewhere, &with_origin, Undecided), + (&configured, &without, Undecided), + (&userinfo, &without, Undecided), + (&in_name, &with_origin, Dead), + (&in_query, &with_origin, Dead), + (&other_patch, &with_origin, Dead), + (&socket, &without, Live), + (&configured, &with_origin, Live), + ] { + let p = Project::new(); + p.write( + "package-lock.json", + format!( + r#"{{"lockfileVersion":3,"packages":{{"":{{}},"node_modules/x":{{"version":"1.0.0","resolved":"{resolved}"}}}}}}"# + ), + ); + let why = format!("{resolved} (origins {:?})", discovery.patch_server_origins); + let mut inventory = None; + assert_eq!( + discovery + .redirect_record_verdict( + p.root(), + purl, + UUID_A, + &["package-lock.json"], + &mut inventory, + ) + .await, + verdict, + "{why}" + ); + let mut inventory = None; + assert_eq!( + discovery + .redirect_record_live( + p.root(), + purl, + UUID_A, + &["package-lock.json"], + &mut inventory, + ) + .await, + verdict == Live, + "only a live record attests: {why}" + ); + let mut liveness = LedgerLiveness::new(p.root(), discovery, None); + assert_eq!( + liveness.redirect_record(purl, UUID_A).await, + verdict.attests(), + "{why}" + ); + assert_eq!( + liveness.redirect_record_retained(purl, UUID_A).await, + verdict != Dead, + "{why}" + ); + } + } + + /// Steps 0 and 3 (the ledger's own recorded wiring) fail closed the + /// way step 1 does: a pin whose recorded files route the uuid through + /// a hosted-shaped url on an origin this run was not given — here a + /// yarn berry `__archiveUrl=` binding and a `Cargo.lock` `sparse+` + /// source, neither of which the lock inventory carries as a url — is + /// `Undecided`, never attested; the same url on Socket's host or a + /// configured origin is live, and a userinfo look-alike reads as the + /// host after its `@`. + #[tokio::test] + async fn recorded_wiring_on_a_foreign_origin_is_undecided() { + use RecordLiveness::{Live, Undecided}; + let origin = "http://127.0.0.1:4545"; + let berry = |base: &str| { + let url = format!("{base}/patch/npm/x/1.0.0/{TOKEN}/{UUID_A}/x-1.0.0.tgz"); + let encoded = url.replace(':', "%3A").replace('/', "%2F"); + format!( + "__metadata:\n version: 8\n\n\"x@npm:1.0.0\":\n version: 1.0.0\n \ + resolution: \"x@npm:1.0.0::__archiveUrl={encoded}\"\n languageName: node\n \ + linkType: hard\n" + ) + }; + let cargo = |base: &str| { + format!( + "version = 3\n\n[[package]]\nname = \"x\"\nversion = \"1.0.0\"\n\ + source = \"sparse+{base}/patch-registry/cargo/{TOKEN}/{UUID_A}/index/\"\n" + ) + }; + let with_origin = Discovery { + patch_server_origins: vec![origin.to_string()], + ..Discovery::default() + }; + let without = Discovery::default(); + for (file, purl, text, discovery, verdict) in [ + ( + "yarn.lock", + "pkg:npm/x@1.0.0", + berry("https://evil.example"), + &without, + Undecided, + ), + ( + "yarn.lock", + "pkg:npm/x@1.0.0", + berry("https://patch.socket.dev@evil.example"), + &without, + Undecided, + ), + ( + "yarn.lock", + "pkg:npm/x@1.0.0", + berry(origin), + &without, + Undecided, + ), + ( + "yarn.lock", + "pkg:npm/x@1.0.0", + berry(origin), + &with_origin, + Live, + ), + ( + "yarn.lock", + "pkg:npm/x@1.0.0", + berry("https://patch.socket.dev"), + &without, + Live, + ), + ( + "Cargo.lock", + "pkg:cargo/x@1.0.0", + cargo("https://evil.example"), + &without, + Undecided, + ), + ( + "Cargo.lock", + "pkg:cargo/x@1.0.0", + cargo(origin), + &with_origin, + Live, + ), + // A Socket-host spelling elsewhere in the files (a comment, a + // leftover definition) does not clear a foreign pin. + ( + "yarn.lock", + "pkg:npm/x@1.0.0", + format!( + "# https://patch.socket.dev/patch/npm/x/1.0.0/{TOKEN}/{UUID_A}/x-1.0.0.tgz\n{}", + berry("https://evil.example") + ), + &without, + Undecided, + ), + // A go.mod `replace` names a scheme-less module path. + ( + "go.mod", + "pkg:golang/example.com/x@v1.0.0", + format!( + "module example.com/app\n\ngo 1.21\n\nrequire example.com/x v1.0.0\n\n\ + replace example.com/x v1.0.0 => staging.example/gopatch/{UUID_A} v1.0.0\n" + ), + &without, + Undecided, + ), + ( + "go.mod", + "pkg:golang/example.com/x@v1.0.0", + format!( + "module example.com/app\n\ngo 1.21\n\nrequire example.com/x v1.0.0\n\n\ + replace example.com/x v1.0.0 => patch.socket.dev/gopatch/{UUID_A} v1.0.0\n" + ), + &without, + Live, + ), + ] { + let p = Project::new(); + p.write(file, &text); + let why = format!( + "{file}: {text} (origins {:?})", + discovery.patch_server_origins + ); + let mut inventory = None; + assert_eq!( + discovery + .redirect_record_verdict(p.root(), purl, UUID_A, &[file], &mut inventory) + .await, + verdict, + "{why}" + ); + } + } + + /// The url token around each needle occurrence stops at quotes, markup + /// and binding separators, and never splits a multi-byte character. + #[test] + fn url_tokens_around_widen_to_the_url() { + let text = "a=\"sparse+https://h/x/U/i/\" https://h/U\u{a0}https://h/U#f&b"; + assert_eq!( + url_tokens_around(text, "U"), + vec!["sparse+https://h/x/U/i/", "https://h/U", "https://h/U#f"] + ); + } + + /// Discovery records the origins it classified with, for liveness. + #[tokio::test] + async fn discovery_carries_its_patch_server_origins() { + let p = Project::new().with_origin("http://127.0.0.1:4545"); + assert_eq!( + p.discover().await.patch_server_origins, + vec!["http://127.0.0.1:4545".to_string()] + ); + } + /// [`LedgerLiveness`]: the redirect ledger's edit files are sorted and /// deduplicated, verdicts are the [`Discovery`] rule's, and the lock /// inventory loads only when a verdict first needs it and then once per diff --git a/crates/socket-patch-core/src/vex/discover/nuget.rs b/crates/socket-patch-core/src/vex/discover/nuget.rs index 7b4391679..b495b8d16 100644 --- a/crates/socket-patch-core/src/vex/discover/nuget.rs +++ b/crates/socket-patch-core/src/vex/discover/nuget.rs @@ -668,6 +668,7 @@ fn emit_hosted( uuid: src.uuid.clone(), file: cfg_rel.into(), version_reqs: Vec::new(), + excluded_reqs: Vec::new(), index_url: Some(src.value.clone()), }); } diff --git a/crates/socket-patch-core/src/vex/discover/testing/golden.rs b/crates/socket-patch-core/src/vex/discover/testing/golden.rs index 54509be0e..464167b72 100644 --- a/crates/socket-patch-core/src/vex/discover/testing/golden.rs +++ b/crates/socket-patch-core/src/vex/discover/testing/golden.rs @@ -137,6 +137,8 @@ fn render(out: &Discovery, root: &Path) -> Value { unwired_copies: _, // Already folded into `unpatched_copies` by the time a run returns. unidentified_copies: _, + // The run's input (the golden runs with none), not a finding. + patch_server_origins: _, } = out; let render_ref = |r: &PatchedRef| { let PatchedRef { @@ -195,15 +197,22 @@ fn render(out: &Discovery, root: &Path) -> Value { uuid, file, version_reqs, + excluded_reqs, index_url: _, } = p; - json!({ + let mut pin = json!({ "ecosystem": ecosystem, "name": name, "uuid": uuid, "file": path_str(file), "version_reqs": version_reqs, - }) + }); + // Rendered only when present, so goldens without any keep + // their shape. + if !excluded_reqs.is_empty() { + pin["excluded_reqs"] = json!(excluded_reqs); + } + pin }) .collect(); let elsewhere_rendered: Vec = elsewhere diff --git a/crates/socket-patch-core/tests/fixtures/vex-discover-golden/redirect-cargo.json b/crates/socket-patch-core/tests/fixtures/vex-discover-golden/redirect-cargo.json index 99be573f0..c14a317f3 100644 --- a/crates/socket-patch-core/tests/fixtures/vex-discover-golden/redirect-cargo.json +++ b/crates/socket-patch-core/tests/fixtures/vex-discover-golden/redirect-cargo.json @@ -634,7 +634,13 @@ "lockfile_basis_ok": true } ], - "diagnostics": [], + "diagnostics": [ + { + "code": "patched_ref_invalid", + "file": "Cargo.toml", + "detail": "Cargo.toml: serde's Socket patch registry pins nothing there: no manifest inherits that `[workspace.dependencies]` entry" + } + ], "recognized": [ { "uuid": "11111111-1111-1111-1111-111111111111",