Repository navigation
chore(toolbox): bump nine pins, and restate the differentiator SkillSpector v2.13.4 changed - #103
Merged
Merged
Conversation
…pector v2.13.4 changed The bump and the docs pass ship together on purpose. v2.13.4 moves the one carrier this gate uniquely covers, so landing the pin alone would leave nine files asserting numbers and a mechanism that the shipped version does not have. ### Bumped | Tool | From | To | |---|---|---| | betterleaks | `1.9.0` | `1.10.0` | | trufflehog | `3.99.7` | `3.99.8` | | syft | `1.55.0` | `1.57.0` | | pip | `26.2.2` | `26.3` | | snyk-agent-scan | `0.6.10` | `0.6.11` | | zizmor | `1.31.1` | `1.32.0` | | semgrep | `1.183.0` | `1.184.0` (both pins) | | checkov | `3.4.2` | `3.4.3` | | skillspector | `v2.13.2` | `v2.13.4` = 54824463814a | skillspector is a hand bump by design: its ARG is a commit SHA, so apply-tool-bumps.py refuses it rather than rewriting the lock while matching nothing in the Dockerfile. ### What v2.13.4 changed, measured The claim in README.md and docs/ was that the memory write "is not modelled" and that neither payload variant blocks. Both halves are now false. Measured on the same fixture across the three tags there are images for: | tag | inert canary to MEMORY.md | literal injection string | |---|---|---| | v2.13.0 | 0/100, exit 0, no finding | 35/100, exit 0, no finding | | v2.13.2 | 0/100, exit 0, no finding | 35/100, exit 0, no finding | | v2.13.4 | 27/100, exit 0, MP3 fires | 63/100, exit 1, blocks | On the inert fixture v2.13.4 reports exactly one finding: HIGH MP3 - Memory Manipulation, 85% confidence, evidence carrier_kind=auto_run destination=MEMORY.md. It names the write, rates it HIGH, and exits 0, because the verdict is a holistic risk_score and 27 is under 50. Decomposed: the write alone is worth 27, the injection prose alone is worth 27, and only the sum crosses the threshold. So the installable variant is the one whose payload does not read like an injection, which is the easier half to author. That is the differentiator now. Not that upstream cannot see the write, but that seeing it does not stop it, and that the payload wording rather than the write decides the verdict. The gate blocks on the write, so wording does not move it. Reframed README.md, docs/threat-model.md, docs/references.md, docs/gecko-vector-walkthrough.md (Step 6 gains the report excerpt and a findings column), docs/threats/developer-execution.md, tests/gate-proof.sh, toolbox/skill-audit/Dockerfile and skill-testfile-gate.sh. gate-proof.sh passes 16 of 16 against the image built from these pins. The other four fixture scores are unchanged: 64/100, 76/100, 100/100, 51/100. Assertion 4's hard check is unchanged and still passes, but it now sits 23 points from flipping rather than 50, and the header says so. ### Corrects the reasoning in 1.7.6, not its conclusion That entry checked NVIDIA/SkillSpector#893 against the commit graph, found the merge commit 1196 commits ahead of the v2.13.2 tag, and concluded the 0/100 figures were stable. The conclusion held for v2.13.2 and the predicted number was exactly right, but the method could not have settled it: git still reports 2de9b387...v2.13.4 as diverged, 183 ahead and 1196 behind, because that project cuts releases from separate branches rather than from main. A merge commit's ancestry in main is not a test of what a release does. Running the release against the fixture is.
garymike
added a commit
that referenced
this pull request
Nov 9, 2026
…blished (#104) The follow-up half of #103, and it covers #102 as well: both landed toolbox changes without moving a pin, so callers have been resolving images built before the nine-tool bump. For skill-audit-toolbox that gap was load-bearing, because #103's docs assert v2.13.4 numbers while the pinned image still carried v2.13.2. Permitted rather than forced. check-image-pins.py reported all 5 of 5 behind before the move, which is the state the mover's guard requires, so no --force was needed: gha-security.yml inputs 5c137c684483 -> f6a4a72e874a iac-security.yml inputs f24b2cabd9b1 -> 4a28bbf3e20f sast.yml inputs de509bd1ace1 -> 547b6bc2feaf security-scan.yml inputs 2aed68cf9f33 -> b4802091e180 skill-audit.yml inputs b011e83c21f6 -> a36fc4fa94cf Verified the usual way: each digest resolved from the per-commit sha-976ef61d7b38 tag rather than latest, and its build-inputs label read back out of the registry and compared with the fingerprint this repo computes now. All five matched before anything was written. check-image-pins.py now reports all five current. Verified one step further, because this is the pin #103's claims rest on. Pulled the skill-audit-toolbox digest callers will now resolve and ran the proof against it, not against a locally built image: SkillSpector v2.13.4, semgrep 1.184.0 gate-proof.sh: 16 of 16 gecko-demo 64/100 exit 1, BLOCKS gecko-hook-demo 76/100 exit 1, BLOCKS config-injection-demo 100/100 exit 1, BLOCKS sibling-config-demo 51/100 exit 1, BLOCKS memory-poisoning-demo 27/100 exit 0, does NOT block The 27/100 in README.md and docs/ is now what a caller resolving this pin actually gets.
garymike
added a commit
that referenced
this pull request
Nov 9, 2026
…after #103 (#105) Both are patch releases published after #103's nine-tool bump, so the weekly drift watcher was red for the right reason. Applied through scripts/apply-tool-bumps.py, which moves tools.lock and the Dockerfile ARG together rather than letting one of them lie: osv-scanner 2.7.0 -> 2.7.1 toolbox/tools.lock, toolbox/base/Dockerfile syft 1.57.0 -> 1.57.1 toolbox/tools.lock, toolbox/base/Dockerfile check-tool-updates.sh --fail-on-drift now exits 0 with all twelve pins current. Pre-flighted the one failure mode a patch bump has here. The base image verifies each download against the release's own checksum file, so a renamed asset fails the build rather than the drift check. Both releases publish the exact names the Dockerfile constructs: osv-scanner_linux_amd64 with osv-scanner_SHA256SUMS, and syft_1.57.1_linux_amd64.tar.gz with syft_1.57.1_checksums.txt. Both pins live in security-toolbox-base, so this rebuilds all five images and the five workflow pins will need moving once it lands.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bump and the docs pass ship together on purpose. v2.13.4 moves the one carrier this gate uniquely covers, so landing the pin alone would leave nine files asserting numbers and a mechanism the shipped version does not have.
Bumped
1.9.01.10.03.99.73.99.81.55.01.57.026.2.226.30.6.100.6.111.31.11.32.01.183.01.184.0(both pins)3.4.23.4.3v2.13.2v2.13.4=54824463814askillspectoris a hand bump by design: its ARG is a commit SHA, soapply-tool-bumps.pyrefuses it rather than rewriting the lock while matching nothing in the Dockerfile.What v2.13.4 changed
The claim in
README.mdanddocs/was that the memory write "is not modelled" and that neither payload variant blocks. Both halves are now false. Measured on the same fixture across the three tags there are images for, rather than inferred from release notes:MEMORY.mdOn the inert fixture v2.13.4 reports exactly one finding:
It names the write, knows the carrier is auto-run, rates it HIGH, and exits 0, because the verdict is a holistic
risk_scoreand 27 is under 50.Decomposed by holding the mechanism fixed and varying the payload:
CLAUDE.md, same stringSKILL.mdprose, no writeThe write alone is worth 27 and the injection prose alone is worth 27. Neither half reaches the threshold; only the sum does. So the installable variant is the one whose payload does not read like an injection, which is the easier half to author.
The restated differentiator: not that upstream cannot see the write, but that seeing it does not stop it, and that the payload wording rather than the write decides the verdict. The gate blocks on the write itself, so wording does not move it.
Proof
gate-proof.shpasses 16 of 16 against the image built from these pins. The other four fixture scores are unchanged (64/100, 76/100, 100/100, 51/100), and the live table reads:Assertion 4's hard check is unchanged and still passes, but it now sits 23 points from flipping rather than 50, and the header records that.
Corrects the reasoning in 1.7.6, not its conclusion
That entry checked NVIDIA/SkillSpector#893 against the commit graph, found the merge commit 1196 commits ahead of the v2.13.2 tag, and concluded the
0/100figures were stable. The conclusion held for v2.13.2 and the predicted number was exactly right (27/100), but the method could not have settled it:gitstill reports2de9b387...v2.13.4asdiverged, 183 ahead and 1196 behind, because that project cuts releases from separate branches rather than frommain. A merge commit's ancestry inmainis not a test of what a release does. Running the release against the fixture is, which is whatgate-proof.shdoes on every build.Follow-up
The five image pins move in a separate PR once this lands, covering both this bump and #102.