[mlieberman85]

Stacked on #666 (the replay script). Review the last 2 commits: the spec change and the fix.

Summary

Since feature 041 a pattern miss is INCONCLUSIVE. Six documentation controls therefore went to a model judgment with only the README and stayed PENDING. Remediation acts only on FAIL, so it never created the document (#665).

• Spec 3.4: with fail_on_miss, a pattern step also concludes FAIL when none of the files it lists exists. That is the strongest miss. Without fail_on_miss, the result is INCONCLUSIVE as before.
• TOML: OSPS-SA-01.01, DO-03.02, DO-04.01, DO-05.01, DO-06.01 and QA-06.02 set fail_on_miss.

Corpus: every new FAIL conclusion is correct (10 of 10), with no incorrect conclusion, and the gate passes. The empty fixture labels the five release-gated controls NOT_PASS.

Replay (scripts/comply_replay.py, wtf-frontend at 939e838):

	FAIL before	PASS before	PASS after
main	9	17	18
this PR	15	17	18

Remediation now writes ARCHITECTURE.md, SUPPORT.md, docs/DEPENDENCIES.md, docs/TESTING.md and docs/RELEASE-VERIFICATION.md, and all six controls move to PENDING. PASS does not change: what remediation writes is a template that still needs a judgment and a person's confirmation (#662).

Known gap: five of these requirements apply "when the project has made a release". darnit does not gate them on has_releases, and has_releases is not asked (#526). On a project without releases this FAIL is conservative, not N/A.

Fixes #665.

Type of Change

• Bug fix (non-breaking change fixing an issue)
• New feature (non-breaking change adding functionality)
• Breaking change (fix or feature causing existing functionality to change)
• Documentation update
• Refactoring (no functional changes)

Framework Changes Checklist

• Updated framework spec (docs/architecture/framework-design.md, 3.4)
• Ran uv run python scripts/validate_sync.py --verbose and it passes

Control/TOML Changes Checklist

• Control metadata defined in TOML (not Python code)
• SARIF fields (description, severity, help_url) included where appropriate (unchanged)
• Ran validation to confirm TOML schema compliance

Testing

• uv run pytest tests/: 5244 passed, 18 skipped
• New handler test: no listed file exists -> INCONCLUSIVE, or FAIL with fail_on_miss
• scripts/corpus_report.py: gate passes
• Replay as above
• uv run ruff check .

AI assistance

• No AI assistance was used
• AI assistance was used

Claude Code (Claude Opus) wrote the change, tests and this description; I reviewed them and ran the corpus and the replay.

---

Drafted with AI assistance (Claude), reviewed and filed by me. Measurements quoted were executed and verified.

[Marc-cn]

The handler change and spec 3.4 are right, and the corpus/replay evidence is convincing. One thing to fix before this lands:

fail is terminal, so with fail_on_miss these six controls now conclude FAIL before the model step when the pattern misses. Four of them carry the README.rst/.txt/no-extension variants from #402, but OSPS-DO-06.01 and OSPS-QA-06.02 list only README.md (plus CONTRIBUTING.md and a docs file). On a project with README.rst (tqdm and most Sphinx projects) none of the listed files exist, so "no files resolved" becomes a hard FAIL and the model never sees the README that may document dependencies/testing — that's the #402 case; these two controls just weren't in that batch. Please add README.rst, README.txt, README to those two files lists here.

Not blocking: a pattern miss on an existing README is now a definitive FAIL too, so these six regexes carry more weight than before. The corpus gate (0 incorrect of 10) covers that for now; worth keeping an eye on as the corpus grows.

Tests, validate_sync and the corpus gate pass locally.

[mlieberman85]

Added README.rst, README.txt and README to OSPS-DO-06.01 and OSPS-QA-06.02. OSPS-SA-01.01 had README.rst but not README.txt or README, so it gets those two as well. A new test requires every README variant in each fail_on_miss pattern step in openssf-baseline.toml that lists README.md. It fails on the old lists (DO-06.01, QA-06.02, SA-01.01) and passes now. It wouldn't catch a step that drops README.md altogether.

I also found a bug in my commit. DO-06.01's remediation creates docs/DEPENDENCIES.md, which its step didn't list, so on a case-sensitive volume the control stayed FAIL after remediation. The step now lists it, and in the wtf-frontend replay DO-06.01 goes FAIL to PENDING on both case-sensitive and case-insensitive volumes. A second test requires each fail_on_miss step to list the files its remediation creates.

On a repository with only this README.rst, darnit audit:

Example
=======

Dependencies
------------
We manage dependencies with pip-tools; each library is pinned in
requirements.txt and updated weekly by Renovate after review.

Testing
-------
To run the tests: ``pytest``. CI runs the test suite on every pull request.

	main	before this commit	now
QA-06.02	PENDING	FAIL ("No files found matching")	PENDING
DO-06.01	PENDING	FAIL ("No files found matching")	FAIL ("Pattern not found in any file")

DO-06.01 now reads the README.rst, but its regex needs one of the nouns followed by a keyword on the same line. "dependencies" isn't followed by one here, and "updated" wraps onto the line after "library". So it concludes FAIL, although the README documents the policy. On main both controls are PENDING with empty file_contents, so the model wasn't given the README either.

Your non-blocking point reaches further than I'd assumed. DO-03.02, DO-04.01, DO-05.01 and SA-01.01 also go PENDING to FAIL on this fixture, and these shapes fail too, with main PENDING for all of them:

repository	controls that FAIL now
README.rst, with the docs in CONTRIBUTING.rst	all six
README.md, with the docs in .github/CONTRIBUTING.md	all six
README.adoc containing the docs	all six
README.md, with the policy in docs/dependency-policy.md	DO-06.01 is a false FAIL (the other five have nothing documented in that fixture)

Remediation then acts on those FAILs: with the docs in .github/CONTRIBUTING.md it writes docs/DEPENDENCIES.md, docs/TESTING.md, docs/RELEASE-VERIFICATION.md, SUPPORT.md and ARCHITECTURE.md beside them, leaving the existing file alone. The handler change also reaches community-spec: CSL-06.01 goes WARN to FAIL on an empty repo, correctly, and on a repo with only README.txt, which it doesn't list.

These are false negatives, which our conservative rule prefers to false passes, and the #665 fix depends on a miss being a FAIL, so I've left the regexes and these lists alone in this PR. I've filed #702 to cover them. Say if you'd rather block on them.

Also rebased onto c52fe5a. The only conflict was in CHANGELOG.md, and a5e29ba is gone because it landed as #666. On 075556e merged with this branch, which is clean: full suite 5142 passed, ruff and validate_sync pass.

---

Drafted with AI assistance (Claude), reviewed and filed by me. Measurements quoted were executed and verified.