Repository navigation
fix: retain dependency evidence for unchanged unlocked requirements - #2553
groupthinking wants to merge 2 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
 |
There was a problem hiding this comment.
🟡 Changes recommended
The bare-name regex accepts archive filenames that pip treats as local references, allowing mutable archives to be incorrectly certified as self-contained, unchanged source.
1 open issue
What changed in this PR
This PR repairs dependency-review evidence recording when unchanged, unlocked Python requirements resolve to different versions, addressing the artifact failures tracked in #2552 without changing vulnerability or license policy.
Changes:
- Adds source-verified classification for paired pip version changes under identical requirements blobs.
- Records schema-3 provenance and reproducibility limits while retaining the complete comparison and original review outcome.
- Adds regressions for artifact preservation, unsupported declarations, malformed rows, source mismatches, and lock-evidence validation.
| File | Description |
|---|---|
tests/unit/test_dependency_review_unlocked_requirements.py |
Adds positive and fail-closed regressions for unlocked requirements evidence. |
scripts/ci/record_dependency_review.py |
Validates unchanged requirements sources and records resolver differences with schema-3 provenance. |
Review effort: Balanced
| if not declarations or not all( | ||
| re.fullmatch(package_name, line) for line in declarations | ||
| ): | ||
| reject("source is not self-contained bare unpinned requirements") |
The dependency evidence recorder rejected unrelated unlocked Python resolver changes, so complete comparisons produced no review artifact. On November 2, the same unchanged prototype requirements file resolved filelock 4.2.6 on the base snapshot and 4.2.7 on the head snapshot, blocking #2401, #2400, #2446 and #2398.
This repair keeps the complete GitHub diff and its original vulnerability/license review outcome. It classifies a narrow resolver difference only when the manifest is the exact same regular Git blob at both commits, contains only bare unpinned package declarations, and its pip rows form paired numeric-release version changes with unchanged scope. Schema version 3 records both blob IDs, the content hash, declared packages, all classified rows and the explicit reproducibility limitation.
Comments are recognized only at line start or after whitespace, matching pip's inspected comment grammar. Attached hashes remain in the declaration and are rejected. Version support is deliberately limited to decimal release segments, such as 4.2.7. Epochs, pre/post/dev releases and local labels remain unsupported and fail closed. Includes, constraints, pins, extras, markers, options, URLs, local references, missing/changed/symlink sources, malformed or ambiguous rows also fail closed. The recorder adds no third-party dependency.
Exact npm/uv version, scope and completeness checks, pagination trust/cycle protection, snapshot-warning rejection, and all security/license settings remain unchanged. A genuine development-to-runtime Vite transition remains in the raw comparison and under the original vulnerability policy. No rows are deleted, no finding is waived, and successful artifact recording never changes a failing review outcome. Only the recorder and its new regression test file change.
Validation at local commit c62c51f, based on main 1b826c1:
The exact recorder has now passed the controlled hosted artifact verification below. The test PR is closed unmerged after verification; its source branches and artifacts are preserved. Original dependency PR checks against main remain separate. This PR is being marked ready for normal review; no main merge, policy waiver or approval is included.
Closes #2552.
Hosted verification — 2026-11-02
Controlled off-main canary #2554 ran this exact recorder at head 87990c3 against base 1b826c1. Dependency run 41530056594, attempt 1, passed full-tree snapshot submission, combined-lock submission, dependency policy review, evidence recording and upload. Head combined snapshot 117439219 reports four manifests and 2,191 packages; exact-base run 41526123217 had already submitted both snapshot types.
Downloaded artifact 13181672366 contains schema-3 review.json with the exact base/head/run/attempt and review_outcome=success. Its 22 raw rows comprise all 20 expected package-lock.json version/scope changes plus two apps/web/package.json changes; the rows contain no reported vulnerability entries. An independent live API read returned HTTP 200, an empty snapshot-warning header and no next page. The entire raw JSON is byte-identical to the uploaded file: SHA-256 0ba4804ac578e6d40500d12e86b8bf766cb5b02df36352ddb26efda5cdad99d7. The downloaded ZIP also matches GitHub's archive digest, SHA-256 2906844dcae21c5523d66454598f522461e3cefaf8b438481fb295034307ff52. The recorded success agrees with the actual policy action; policy settings were not changed.
No fresh unlocked-Python resolver drift occurred in this comparison, so unchanged_requirements_resolutions is empty. The separate captured 68-row production-input replay exercises that repaired branch, retaining every row and its original failing verdict. Both base and head full-tree scanners hit the existing 240-second limit/exit 137 for root requirements.txt and docs/guides/requirements.txt and fell back to declared dependencies without their transitive graph. A warning-free compare is therefore not a claim of complete transitive Python resolution.
Hosted logs confirm all 34 new recorder cases passed in repair Python 3.11 and canary Python 3.12; each inspected unit run passed 3,074 tests with nine deselected, five xpassed and 118 passing subtests. The repair PR's own dependency check correctly used the source-only path and did not produce an artifact. Canary evidence does not establish that each original update PR against main has passed; their exact-head reviews and checks remain separate.
Review status
The meaningful test and dependency jobs passed. Two non-required repository-reconciliation report jobs were cancelled because the repository uses a shared concurrency group; their annotations say a higher-priority waiting reconciliation cancelled them. At the draft checkpoint GitHub therefore reported a failing rollup and UNSTABLE merge-state despite no failing test job. Active ruleset 24057160 requires only
Closing keyword to open issue, which passed. Moving this PR out of draft permits its normal review checks to run; it does not establish approval or merge readiness. CodeRabbit was skipped while draft, and no GitHub approval is claimed.