[mlieberman85]

Stacked on #671. Review the last 3 commits: the question selectors, the skills naming the repository, and the flow test.

Summary

Questions. Since #552, _build_ask_user_params returned no selector for free-text questions or for choice questions with more than 4 values. So one batch was split between Claude Code's question tool and plain text. Every question now has a selector:

• A long choice list shows its first 4 values and names all of them in the question, each typeable under Other (FR-015).
• A free-text question offers "Skip for now" and "Answer in chat", never a value.

This builds on #657, which already merged the word-boundary headers and the yes/no hint in the question.

Repository naming. The audit and comply skills called the audit without owner and repo. Confirmed project data and stored PASS candidates apply only to a named repository, so they were never applied: after answering the data questions, a re-audit showed the same results (#646). Every skill now:

• asks the person to confirm the repository, suggesting the origin remote;
• names it on every call.

Comply re-runs the audit after the data is confirmed. A test checks every skill.

Spec: 7.x and the data procedure. The flow test now checks that every question is in ask_user_batch, with at most 4 per round.

Refs #650, #646, #655.

Type of Change

• Bug fix (non-breaking change fixing an issue)

Framework Changes Checklist

• Updated framework spec (docs/architecture/framework-design.md, 7.x)
• Ran uv run python scripts/validate_sync.py --verbose and it passes

Testing

• uv run pytest tests/ on the whole rebased stack (18bd63e): 5286 passed, 18 skipped
• e2e: see the table below. On wtf-frontend, the confirmed answers make QA-02.02, QA-04.01 and QA-04.02 N/A, which they did not when the repository was unnamed.
• uv run ruff check .

End-to-end

darnit-e2e: headless /darnit:darnit-audit openssf-baseline through Claude Code. Confirm tools are disallowed and the repository is named in the prompt.

darnit	darnit repo	wtf-frontend	cosign-installer
main (2026-10-20; cosign-installer on 603bdda)	387s, 44 turns	415s, 47 turns	297s
#653 only (2026-10-21)	299s, 57 turns	250s, 55 turns	185s, 39 turns
full stack, rebased on 8070949 (2026-10-23)	282s, 50 turns, 30 judgments	145s, 23 turns, 15 judgments	141s, 30 turns

• Results: per-control results match main. The only differences are controls the model judged, and the effect of operator-side answers on wtf-frontend, which needs the repository named. wtf-frontend also shows QA-03.01 and QA-07.01 passing because their platform settings changed on GitHub.
• Stored candidates skew the comparison. A control with a stored PASS candidate for unchanged evidence is skipped, so run-to-run times depend on how many are stored. darnit's HEAD moved to 8070949, which invalidated its stored candidates, so it was judged in full (30 controls).
• Denials:
  • darnit: one page of get_judgment_requests was larger than Claude Code's tool-result limit and was saved to a file. The agent's attempt to read it with Bash was denied. This is the oversized single-request follow-up.
  • wtf-frontend: the agent tried to run zizmor itself, and that was denied.

AI assistance

• No AI assistance was used
• AI assistance was used

Claude Code (Claude Opus) wrote the change, tests and this description; I reviewed them and ran the e2e.

---

Drafted with AI assistance (Claude), reviewed and filed by me. Measurements quoted were executed and verified.

[Marc-cn]

Reviewed its own three commits. Every question gets a selector, long choice lists show four and point to Other, free text offers Skip / Answer in chat, and the skills name the repository on every call (the #646 fix). Two things to check, not blockers: (1) for a choice with more than 4 values the code adds 'Choose Other to type a value not listed below' but does not name values 5+ in the question; fine if prompt already lists them, otherwise they are visible only in allowed_values. (2) the free-text value_map maps Skip and Answer-in-chat to the strings "SKIP" and "ASK_USER_FOR_VALUE"; an agent applying answer_mapping mechanically could pass "SKIP" as the answer. null, or leaving those labels out of value_map, would be safer. Good to merge in order.

[Marc-cn]

Re-read the head of fix/question-picker, own commits only. Routing every question through the question tool is the right fix for #655, and the enum vocabulary check keeps "type it under Other" safe for choice keys. One hole it opens, and two doc/skill inconsistencies:

1. The two labels added to every free-text key, "Skip for now" and "Answer in chat" (tools.py:1058-1062), and the value_map sentinels "SKIP" / "ASK_USER_FOR_VALUE" (tools.py:872), are accepted by the server as confirmed values: context_writes.py:189-193 refuses only None and out-of-vocabulary enum values, and free-text keys have no vocabulary. If the agent forwards the chosen label — which is what AskUserQuestion returns — confirm_project_data(security_contact="Skip for now", ...) is recorded as a person-confirmed value (maintainers becomes ["Skip for now"]). Procedure prose is the only guard, on a server whose premise is that the agent is untrusted. Please refuse those four strings (and inside list values) in record_value_confirmation with a clear reason, define them once in darnit.server.procedures so core and baseline share them, and add a test. (Follows up the value_map note in my first review.)
2. specs/042-candidate-integrity/contracts/context-confirmation-tools.md:27 still says selector options are omitted above 4; framework-design 1748 (changed here) says the opposite and names that contract as the definition. The sentinels and the free-text selector should be documented there too.
3. darnit-comply/SKILL.md:42 now says to call the audit again after confirming data; line 85 still says "Do NOT run a separate audit before calling remediate_audit_findings". Reword 85.

Not blocking: "name the repository on every call" is enforced only by skill text plus a test that inspects two lines (darnit-remediate:110 already names none); a warnings entry from get_pending_data / the audit when target is None would make #655 a real fix. The agent-flow test takes answers from the ANSWERS dict, not from ask_user_batch / answer_mapping, so it can't fail on point 1.

[mlieberman85]

Thanks, good catch on point 1. All three points are fixed in two new commits on top: "docs(spec): a selector label or answer_mapping marker is not an answer" and "fix(data): refuse a selector label or marker as a confirmed value". Rebasing on the updated #671 conflicted in procedures.py and framework-design 7.10, where both PRs edit the same step and paragraph. I resolved it by keeping both sets of changes in the first commit; the other two commits are unchanged.

1. record_value_confirmation refuses "Skip for now", "Answer in chat", "SKIP" and "ASK_USER_FOR_VALUE", alone or inside a list, both before and after coercion. The reason it gives is that the value is a question option or answer_mapping marker, not an answer, and to leave the key out or ask for the value. The four strings are defined once in darnit.server.procedures (NOT_AN_ANSWER), and the selector, answer_mapping and the procedure use them. Tests:

   • every string, as a string and inside a list, for security_contact and maintainers;
   • the operator-side path;
   • a value that only contains "SKIP" is still accepted.

   The agent-flow test now forwards the "Skip for now" label for a free-text question over MCP and requires refused. With the check disabled, all 14 new checks fail.

2. The 042 contract no longer says selectors are omitted above 4 values. It documents the free-text selector, the two markers, the candidate "No" marker, and the refusal. Framework-design 7.10 states the refusal too.

3. The comply gotcha now reads "Do NOT run an audit just to feed remediate_audit_findings: it audits internally. The one audit after collecting data (step 4) is for showing the person what changed."

Not done, from your non-blocking notes: a warning from get_pending_data or the audit when no repository is named. That changes what those tools return, so I'd rather do it in its own PR.

The headless comply run on #693's head, which includes this branch, is in my reply on #693.

---

Drafted with AI assistance (Claude), reviewed and filed by me. Measurements quoted were executed and verified.

[Marc-cn]

Approved on c77cb47. The server now refuses 'Skip for now', 'Answer in chat', 'SKIP' and 'ASK_USER_FOR_VALUE' as a confirmed value, alone or in a list, before and after coercion and on the operator path, with the strings defined once in darnit.server.procedures and used by the selector, answer_mapping and the procedure; the agent-flow test forwards the label over MCP and requires refused. The 042 contract documents the free-text selector and the markers, and darnit-comply line 85 now says what the one extra audit is for. On the VM against main: tests/darnit/server + darnit_baseline + config + the MCP integration suite 1983 passed, ruff and validate_sync clean.