Repository navigation
feat(mcp): return the data and judgment procedures from the tools - #671
Open
mlieberman85 wants to merge 7 commits into
Open
mlieberman85 wants to merge 7 commits into
mlieberman85 wants to merge 7 commits into
Conversation
Open
7 of 8 tasks
mlieberman85
force-pushed
the
feat/tool-procedures
branch
2 times, most recently
from
October 25, 2026 23:45
e3d9f6d to
fb0727e
Compare
8 tasks done
7 of 8 tasks
Marc-cn
previously approved these changes
Oct 29, 2026
Marc-cn
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed its own three commits. One source for the procedure text, calls name only the repository and checkout the caller passed (placeholders otherwise, never a detected value), confirm_project_data ends with the next call, skipped lists controls with a stored candidate. One leftover from #670: a PENDING llm_judgment control with no llm_consultation is still absent from requests, remaining and skipped, so an agent cannot tell why it never shows up. Rare, but listing it (with why) would close the gap. Good to merge in order.
A coding agent gets what it judges from get_judgment_requests: each control's prompt, hints, and files, with one shared files map holding exactly the content submit_judgment checks citations against, paged under 40,000 bytes. Summary rows of a PENDING control carry pending.kind and a candidate summary (5.4). Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com>
The json audit output is about 1 MB, because every control waiting on a judgment carries its own copy of its files and all of its evidence. An MCP client refuses a tool result that large, so the comply flow spilled it to a file and parsed it with shell tools. get_judgment_requests audits the checkout the same way submit_judgment does and returns only the prompt, hints, evidence digest, and file keys per control, with each file once in a shared map. Pages stay under 40,000 bytes; 'remaining' names the controls for the next call. Controls with a stored PASS candidate are skipped unless include_judged is set. It records nothing. Summary rows of a PENDING control now carry pending.kind, and a candidate's model, reasoning, and number of cited excerpts, so a client can tell a judgment request from a candidate without the json output. Refs darnitdevorg#644. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com>
…tput darnit-audit and darnit-comply read summary rows, then page through get_judgment_requests and cite from its files. The '~164K' size claims in the skills and the audit tool description are gone. Refs darnitdevorg#644. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com>
…erver darnitdevorg#630 removed create_server_from_dict. The two tests that check the get_judgment_requests registration now write a minimal framework TOML and call create_server, as darnitdevorg#630 did for its own tests. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com>
get_pending_data and get_judgment_requests carry a 'procedure': the ordered steps for the result, with calls that name only the repository the caller named. A confirm_project_data result on a server with get_pending_data ends with the next get_pending_data call. The text has one home in the framework, so skills and clients defer to it. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com>
darnit-comply told the agent to "follow the /darnit-data skill" and "follow the /darnit-audit skill", but nothing loads another skill's text, so the comply run improvised data collection from one line and asked questions differently from a darnit-data run. darnit.server.procedures holds the steps once. get_pending_data returns them as 'procedure' (show candidates labelled unconfirmed, ask every question, per-question skip, one confirm_project_data call with accept_candidates filled only after a yes, the next call, the review of stored values, the summary), get_judgment_requests returns the judging steps on every page, and the baseline confirm_project_data result ends with the next get_pending_data call. Calls name only the owner and repo the caller passed, never ones read from the checkout's remote, and hold no candidate value. The skills defer to the procedures instead of restating them, and no skill tells the agent to follow another one (a test checks). The comply and remediate skills pass owner and repo to remediate_audit_findings, which needs them to find operator-side confirmations (darnitdevorg#612). Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com>
A page now names, in 'skipped', the controls left out because they already have a stored PASS candidate, and the judging procedure says not to request them again. In the darnitdevorg#644 e2e runs the agent called the tool again with those ids and got empty pages back. Refs darnitdevorg#644. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com>
mlieberman85
force-pushed
the
feat/tool-procedures
branch
from
November 1, 2026 15:59
fb0727e to
c874fcb
Compare
This was referenced Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
darnit-comply told the agent to "follow the /darnit-data skill" and "follow the /darnit-audit skill". Nothing loads another skill's text, so the comply run improvised data collection from one line and asked questions differently from a darnit-data run (#654).
darnit.server.proceduresholds the steps once:get_pending_datareturns them asprocedure;get_judgment_requestsreturns the judging steps on every page;confirm_project_dataresult ends with the nextget_pending_datacall.Calls name only the owner and repo the caller passed, never ones read from the checkout's remote, and hold no candidate value.
skipped: aget_judgment_requestspage names the controls it left out because they already have a stored PASS candidate, and the procedure says not to request them again. In the audit_openssf_baseline JSON output is about 1 MB and exceeds the MCP client's result limit #644 e2e runs the agent asked for them again and got empty pages.Skills:
remediate_audit_findings.Spec: 5.4 and 7.x.
Refs #654.
Type of Change
Framework Changes Checklist
docs/architecture/framework-design.md)uv run python scripts/validate_sync.py --verboseand it passesTesting
uv run pytest tests/on the whole rebased stack (18bd63e): 5286 passed, 18 skippeduv run ruff check .End-to-end
darnit-e2e: headless/darnit:darnit-audit openssf-baselinethrough Claude Code. Confirm tools are disallowed and the repository is named in the prompt.get_judgment_requestswas larger than Claude Code's tool-result limit and was saved to a file. The agent's attempt to read it with Bash was denied. This is the oversized single-request follow-up.AI assistance
Claude Code (Claude Opus) wrote the change, tests and this description; I reviewed them and ran the e2e.
Drafted with AI assistance (Claude), reviewed and filed by me. Measurements quoted were executed and verified.