Repository navigation
fix(baseline): use declared fail_exit_codes = [0] for OSPS-BR-01.02 grep step (fixes #647) - #656
Conversation
Marc-cn
left a comment
There was a problem hiding this comment.
Thanks for the clear write-up and the tests (91 pass here). I compared three runs on a repository whose workflow uses github.head_ref:
- main: WARN,
error_class: unexpected_exit(the bug in #647); - main with one line added to the step,
fail_exit_codes = [0]: WARN, no error class, the match recorded as evidence; - this branch: WARN, no error class.
The step already has concludes = [], so a declared exit 0 is recorded as evidence and concludes nothing, which is what #647 asks for. That makes inconclusive_exit_codes a framework-wide setting plus a spec change for a control that upstream has retired. I'd rather keep the TOML fix and leave a new handler setting to a separate design discussion with @mlieberman85, if there is a second case that needs it.
#647 also asks that other exec grep steps be checked for the same mapping; a sweep of those with the same one-line fix would make this PR complete.
|
Thanks @Marc-cn! Kept the TOML fix as recommended in 7a8bd3c:
|
Marc-cn
left a comment
There was a problem hiding this comment.
This is the fix: one TOML line, and the match case is tested end to end through the orchestrator. One small thing, no need to block on it: the no-match test asserts the same WARN and no error class as the match test, so it does not show that exit 1 was taken as a pass (its docstring says PASS). Asserting pass_history[0].result.outcome == PassOutcome.PASS there would make the two tests distinguish the two cases
|
Approved, but it now conflicts with main (likely CHANGELOG). Could you rebase? Then it can merge. |
…rep step (fixes darnitdevorg#647) - Declare fail_exit_codes = [0] in openssf-baseline.toml for OSPS-BR-01.02; since concludes = [] is declared, exit 0 records matching evidence without concluding and prevents unexpected_exit error_class. - Add integration tests in test_secure_branch_names.py verifying fail_exit_codes and distinguishing FAIL (exit 0 match) vs PASS (exit 1 clean). - Document in CHANGELOG.md under Fixed. Fixes darnitdevorg#647 Signed-off-by: Paradox <159635503+i-am-paradox@users.noreply.github.com>
7a8bd3c to
98ca0bf
Compare
|
Rebased cleanly onto latest |
Marc-cn
left a comment
There was a problem hiding this comment.
This is the fix I asked for: fail_exit_codes = [0] makes a grep match a non-concluding FAIL with the file list as evidence, no unexpected_exit; the two integration tests cover match and no-match. One thing: the PR description still describes the earlier inconclusive_exit_codes design, which is gone from the diff. Please update it to match. Needs a rebase, then good to merge. Note #685 adds a step before this one in the same control.
Summary
Resolves #647 where OpenSSF Baseline control
OSPS-BR-01.02reported a grep match as an environmental failure (error_class="unexpected_exit").In
packages/darnit/src/darnit/sieve/builtin_handlers.py,exec_handlertreats exit codes outsidepass_exit_codesandfail_exit_codesas undeclared, assigning an error class (defaulting to"unexpected_exit"). InOSPS-BR-01.02,grep -rln github.head_ref .github/workflows/declarespass_exit_codes = [1]. Whengithub.head_refis present, grep exits 0. Because 0 was undeclared, it was classified as an unexpected exit rather than recording the found match as evidence.Changes:
inconclusive_exit_codes: list[int] | Nonesetting toexec_handler. Whenproc.returncode in inconclusive_exit_codes, returnsHandlerResultStatus.INCONCLUSIVEwitherror_class=Noneand preserves stdout/stderr in evidence."inconclusive_exit_codes"inregister_builtin_handlers()settings schema.inconclusive_exit_codes = [0]toOSPS-BR-01.02inpackages/darnit-baseline/src/darnit_baseline/openssf-baseline.toml.inconclusive_exit_codesand updated undeclared exit code rules indocs/architecture/framework-design.md.tests/darnit/sieve/test_builtin_handlers.pyand integration tests forOSPS-BR-01.02intests/darnit_baseline/controls/test_secure_branch_names.py.CHANGELOG.mdunder### Fixed.Type of Change
Framework Changes Checklist
docs/architecture/framework-design.md) if behavior changeduv run python scripts/validate_sync.py --verboseand it passesControl/TOML Changes Checklist
Testing
uv run pytest tests/darnit/sieve/ tests/darnit_baseline/controls/ -v)uv run ruff check .)AI assistance
Assisted-by: Antigravity:eni-gemini-2.5-pro
Used for diagnosing handler exit-code routing, implementing handler settings and test fixtures, and updating framework design documentation.
Additional Notes
Fixes #647.