Skip to content

feat: add governed selection outcome monitoring plan - #42

Draft
seonghobae wants to merge 117 commits into
developfrom
feat/selection-outcome-monitoring-plan
Draft

feat: add governed selection outcome monitoring plan#42
seonghobae wants to merge 117 commits into
developfrom
feat/selection-outcome-monitoring-plan

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Buyer-visible outcome

Adds a governed, aggregate-only selection-outcome monitoring plan for post-hire criterion evidence while keeping high-impact employment decisions human-controlled. The packet binds tenant and Job scope, exact population/outcome snapshot evidence, monitoring window, protected-attribute/small-sample/statistical-plan provenance, accountable actor/reviewer/purpose/reason, evidence version, and opaque governed references into deterministic canonical evidence. LLM/model output is not an employment decision authority.

Current authority — 2026-09-09

Protected base is develop@eb9757f8649aaad026a9865508d9aad50c1a7a4f. Current exact head is 155c9cdc9586ac29bdc4ace8a718fba3ab6304a5; the PR remains open · Draft · mechanically mergeable. Predecessor checks and reviews are causal evidence only and do not authorize this head.

The retired .github/workflows/selection-monitoring-quality.yml stays deleted. Canonical .github/workflows/foundation-ci.yml remains the repository owner path; this feature must not resurrect a leaf workflow or copy mutable Foundation source into siblings.

Governed evidence contract retained

SelectionOutcomeMonitoringPlan remains aggregate-only, Job-scoped, human-review-only evidence. It carries no candidate identity, protected-attribute value, individual assessment score, individual employment decision, or free-form model output. Tenant and packet references remain opaque and purpose-bound; every reference must be re-resolved within the exact tenant before accountable review. The package does not calculate selection rates, apply a four-fifths pass/fail rule, infer discrimination, or authorize an employment-process change.

Construction continues to require exact built-in trust-bearing text/date/time/numeric/boolean evidence and freezes generated_at into a detached built-in UTC instant. A process-local HMAC seal binds each live issued plan to its construction-time canonical bytes, with single-use seal registration and fail-closed export when issuance evidence is missing or semantically changed.

#273 post-issuance runtime-type repair

Issue #273's representation-integrity gap is already repaired on this exact head. HMAC proves canonical-byte identity, but bytes alone do not prove that the live Python field still has the exact inert runtime type validated at construction. After issuance, low-level object.__setattr__ could otherwise substitute a behavior-bearing str subtype carrying the same underlying text, preserving canonical JSON and therefore the HMAC while retaining caller-owned executable runtime state. A date subtype could similarly reach overridden rendering behavior if canonical export did not re-establish the exact type boundary first.

Test-first 06d8d2fbf8822f26758b810004dadbcf218c83e0 added regressions for a same-value str subtype and an executable date subtype. That short-lived test-only head produced no materialized hosted run, so no hosted RED is claimed. Ordinary successor c2a157d37f3a0bf6ac04f3ca2101a6607087e8f7 added _assert_canonical_runtime_evidence(...): every serialized trust-bearing text field and each date/datetime/int/bool field must still be its exact built-in type before date/timestamp rendering, JSON serialization, or HMAC comparison. HMAC remains the semantic-byte tamper check; the exact-type gate closes only the representation-preserving runtime-type blind spot.

Forward documentation successors keep README.md, CHANGELOG.md, ADR 0016, and selection-monitoring TRACEABILITY aligned with that behavior. The final current head is 155c9cdc9586ac29bdc4ace8a718fba3ab6304a5. Net product behavior for valid plans and the public canonical JSON schema are unchanged. Fresh review enumeration has no qualifying APPROVED review, and all currently returned inline review threads are resolved; historical COMMENTED findings remain review provenance, not merge approval.

Shared Foundation dependency

The package-quality contract remains reached through canonical Foundation ownership. Issue #258 / Draft PR #259 remains the canonical repository-level Foundation compatibility solution: package-neutral compatibility, exact runner/interpreter evidence, reviewed hash-locked tooling, complete dependency-file provenance binding, retired-leaf non-resurrection, and deterministic manifest sealing.

Required owner order remains #258/#259 normal protected integration first, then #42 non-force adoption of that protected Foundation truth while preserving the complete Selection Monitoring product/runtime delta. Do not copy this mutable shared-dispatcher source into sibling product branches.

Current exact-head acceptance

Fresh re-read of the workflows bound to exact 155c9cdc9586ac29bdc4ace8a718fba3ab6304a5 supersedes the stale queued snapshot previously recorded here:

  • Foundation CI 34140109476 — terminal SUCCESS.
  • SAST Semgrep 34140109425 — terminal SUCCESS.
  • Security Scan 34140109689 — terminal FAILURE solely because dependency-review fails closed at Check dependency review support after exact-head checkout. Trivy and OSV jobs succeeded. Central owner .github#810 still owns the public non-fork dependency-comparison availability/configuration incident; sibling scanner GREEN is not promoted as Dependency Review evidence.
  • CodeQL PR 34140109611 — terminal FAILURE. The actions compatibility consumer failed enforcement at 2026-09-07T16:33:50Z and the Python consumer at 16:34:36Z; the same run's Dispatch current-head CodeQL scan job did not start until 16:51:21Z and then dispatched successfully at 16:51:25Z. This is the same central producer/consumer ordering class now reproduced on the combined central successor .github#2040@d7bb95f6d6ca705725596df5170d6e1345080535; it is not repaired by changing Selection Monitoring source or by repeating this immutable head.

The combined central #2040 successor is still open · Draft and its own exact CodeQL run 34244658739 reproduces the ordering RED: compatibility enforcement failed before the later dispatch job began and successfully published. Do not start a no-op #42 rerun until that canonical owner reaches exact-head GREEN, integrates normally to protected .github/main, and the protected workflow source is re-read.

No predecessor result transfers to this head. There is no qualifying independent APPROVED review. Keep Draft until #259 is protected truth and all then-current exact-head technical/governance gates materially authorize integration.

Merge discipline

Do not restore package-local CI, transfer predecessor checks, copy mutable sibling Foundation source, self-approve, fabricate reviewer/verdict evidence, use routine administrator bypass, force-push/destructively rebase, weaken a gate, or Close the valid monitoring delta. If a current-head RED materializes in owned product code, repair its causal owner and reacquire exact-head evidence; central dependency-review/CodeQL failures remain central owner-path work.

@coderabbitai

coderabbitai Bot commented Aug 18, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1791dcc7-645c-4cd2-96c8-fcdcd628be71

📥 Commits

Reviewing files that changed from the base of the PR and between fca4041 and fb03c08.

📒 Files selected for processing (8)
  • .github/workflows/selection-monitoring-quality.yml
  • docs/adr/0016-governed-selection-outcome-monitoring-plan.md
  • docs/traceability/selection-outcome-monitoring.md
  • packages/selection-monitoring/CHANGELOG.md
  • packages/selection-monitoring/README.md
  • packages/selection-monitoring/src/orgmetra_selection_monitoring/plan.py
  • packages/selection-monitoring/tests/test_issuance_integrity.py
  • packages/selection-monitoring/tests/test_quality_workflow_trigger.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/selection-monitoring/CHANGELOG.md

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

SelectionOutcomeMonitoringPlan 계약과 Python 패키지를 추가했다. 테넌트 범위, UUID 참조, 증거 버전, canonical JSON, SHA-256 다이제스트, 발행 시각 및 인간 검토 상태를 검증한다. 테스트와 GitHub Actions 품질 검사를 추가했다.

Changes

선택 결과 모니터링

Layer / File(s) Summary
거버넌스 계약 및 패키지 공개
docs/adr/0016-governed-selection-outcome-monitoring-plan.md, docs/doctoring/selection-outcome-monitoring-references.md, docs/traceability/selection-outcome-monitoring.md, packages/selection-monitoring/README.md, packages/selection-monitoring/CHANGELOG.md, packages/selection-monitoring/pyproject.toml, packages/selection-monitoring/src/orgmetra_selection_monitoring/__init__.py
집계 전용 모니터링 계획의 필드와 거버넌스 제약을 정의했다. 테넌트 범위 재확인, actor 분리, 발행 시각 증거 및 공개 API를 문서화했다.
계획 검증 및 증거 직렬화
packages/selection-monitoring/src/orgmetra_selection_monitoring/plan.py
운영 UUID, UUIDv4 참조, 다이제스트, 시간, 기간, actor 분리와 고정 상태를 검증한다. 비노출 repr, canonical JSON, SHA-256 다이제스트, HMAC 봉인 및 검증 빌더를 제공한다.
계약 테스트 및 품질 워크플로
packages/selection-monitoring/tests/*, .github/workflows/selection-monitoring-quality.yml
계획 생성·대체·불변성·개인정보·참조 무결성·증거 버전·고정 거버넌스·시간 정규화·발행 무결성을 검증한다. 고정 해시 도구, 컴파일, 전체 커버리지, 정확한 체크아웃 및 clean tree 검사를 실행한다.

Estimated code review effort: 4 (복잡) | ~45분

Merge Risk: 🟡 Moderate · up to fb03c

This PR adds aggregate-only monitoring evidence with deterministic integrity checks and human-controlled employment decisions, but it should not merge until all required current-head reviews, validations, and independent approval pass; integrating hosts must also enforce tenant ownership, actor separation, purpose authorization, and durable audit to prevent evidence from being attributed to the wrong tenant or unauthorized purpose.

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant build_selection_outcome_monitoring_plan
  participant SelectionOutcomeMonitoringPlan
  Caller->>build_selection_outcome_monitoring_plan: 계획 입력 전달
  build_selection_outcome_monitoring_plan->>SelectionOutcomeMonitoringPlan: 입력 검증 및 계획 생성
  SelectionOutcomeMonitoringPlan-->>Caller: HMAC 봉인된 계획 반환
  Caller->>SelectionOutcomeMonitoringPlan: canonical_json() 호출
  SelectionOutcomeMonitoringPlan-->>Caller: 검증된 canonical JSON 반환
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 96.74% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 92 functions across 12 files. (5 skipped: 5…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 PR의 주요 변경 사항인 governed selection outcome monitoring plan 추가를 정확하고 간결하게 설명합니다.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/selection-outcome-monitoring-plan

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review August 18, 2026 19:00

Copy link
Copy Markdown
Contributor Author

@opencode-agent Please independently review exact current head 0aea81f768235d62974686f02b9b0a93a30266be against protected develop f815feedcb0bac264d9814b2932338c512276110. Re-read the full diff and current thread state rather than predecessor evidence. Verify the aggregate-only selection-outcome monitoring boundary, protected-attribute minimization, human-review-only authority, deterministic evidence, exact 100% owned coverage evidence, and security/governance claims. If this unchanged head satisfies the repository review standard, submit a formal APPROVE; otherwise leave only source-backed actionable findings tied to this exact head. Do not modify or merge the PR.

@seonghobae
seonghobae marked this pull request as draft August 19, 2026 02:12
@seonghobae
seonghobae marked this pull request as ready for review August 19, 2026 06:35

Copy link
Copy Markdown
Contributor Author

Current-head control-evidence correction for fb03c0837b38424412fa774576a8ded0f9847896 against develop@9e3e4847510e1e612b48474ba42b177b8ed824df:

The PR body’s statement that no authenticated exact-head OpenCode review exists is stale. A post-.github#1497 exact-head review does exist: review 5087978427, submitted 2026-08-31T08:18:17Z, state COMMENTED, verdict COVERAGE_BLOCKED. Central run 33371635973 materialized the exact head/base; dispatch and review publication succeeded.

The first current OpenCode boundary is central coverage double-instrumentation, not missing dispatch and not uncovered Orgmetra code. In coverage-source-tree job 99424380243, the conventional detector runs this exact HRIS kernel source/tests and gets 119 passed, 397/397 statements, 224/224 branches = 100%. The config-root detector then wraps the same pytest suite in outer coverage run while packages/hris-kernel/pyproject.toml already enables pytest-cov. That nested instrumentation produces module-not-measured / no-data-collected, artificial 0% pytest-cov, then outer module-not-imported / No data to report; coverage-evidence job 99424380250 consequently fails. The final OpenCode verifier correctly remains non-passing.

The exact RED/GREEN acceptance canary has been routed through the existing central owner path .github#624, comment 5481179472: keep the 100% threshold, ensure exactly one coverage controller, and require unchanged-head conventional/config-root evidence to agree before treating OpenCode as substantive. Until then this PR remains not merge-ready; no predecessor verdict, self-approval, bypass, or synthetic no-op retrigger is acceptable.

@opencode-agent

opencode-agent Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Scheduled review-feedback autofix for this PR head.

  • Head SHA: fb03c0837b38424412fa774576a8ded0f9847896

@opencode-agent

opencode-agent Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Scheduled review-feedback autofix for this PR head.

  • Head SHA: fb03c0837b38424412fa774576a8ded0f9847896

@opencode-agent

opencode-agent Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Scheduled review-feedback autofix for this PR head.

  • Head SHA: fb03c0837b38424412fa774576a8ded0f9847896

@opencode-agent

opencode-agent Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Scheduled review-feedback autofix for this PR head.

  • Head SHA: fb03c0837b38424412fa774576a8ded0f9847896

@seonghobae
seonghobae marked this pull request as draft September 5, 2026 20:59
Adopt protected develop without resurrecting the retired package-local workflow. Preserve selection-monitoring 100% statement/branch coverage through a hash-locked Foundation-owned contract and add executable non-resurrection/ownership regression.

Signed-off-by: Seongho Bae <me@seonghobae.me>
Replace the selection-monitoring-specific dispatcher edit with a stable Foundation-owned discovery convention so later package artifact contracts can be added without competing edits to the shared dependency-hygiene script. Keep the retired leaf workflow deleted, preserve hash-locked 100% coverage, compile before tests, and clean the isolated venv on exit.

Signed-off-by: Seongho Bae <me@seonghobae.me>

Copy link
Copy Markdown
Contributor Author

Canonical prerequisite #258 now has an active protected-parent implementation in Draft PR #259 at exact head bde2bd5472175ab7263ff27ff56a09c9d619e0c5 (parent develop@eb9757f8649aaad026a9865508d9aad50c1a7a4f). #259 moves package-neutral execution and declared-Python compatibility into Foundation itself, so this branch's mutable shared-dispatcher implementation must remain feature-branch evidence rather than become a sibling dependency. Keep #42 Draft and unchanged until #259 is normally integrated; then non-force adopt protected Foundation truth, preserve the Selection Monitoring domain/scientific delta, drop any now-redundant shared-owner implementation, and reacquire exact-head evidence. Current #259 gates are fresh and queued; no predecessor GREEN transfers.

Copy link
Copy Markdown
Contributor Author

New current-head repair, refs #273.

Fresh review found a gap not covered by the existing issuance HMAC: a post-issuance exact built-in field could be replaced via object.__setattr__ by a behavior-bearing subtype carrying the same canonical value. Because HMAC binds bytes, not Python runtime type, a same-value str subtype could preserve the issued JSON/HMAC while the live packet retained caller-owned executable state; a date subtype could also reach overridden .isoformat() before a post-issuance exact-type gate.

Test-first 06d8d2fbf8822f26758b810004dadbcf218c83e0 adds same-value text and executable-date regressions. No terminal hosted RED is claimed for that short-lived head. Ordinary successor/current head c2a157d37f3a0bf6ac04f3ca2101a6607087e8f7 adds _assert_canonical_runtime_evidence(...) before canonical rendering. It requires exact built-in runtime types for every serialized trust-bearing text field and the date/datetime/int/bool evidence fields before .isoformat(), timestamp rendering, JSON encoding, or HMAC comparison. Existing HMAC semantics remain the value-change detector; valid canonical bytes/schema are unchanged.

Net delta from ec35dc87...: 2 commits ahead / 0 behind, only plan.py plus the dedicated regression file. New exact-head runs are Foundation 34139613554, Security 34139613394, SAST 34139613429, CodeQL 34139613460; all were queued at the first successor read. The PR remains Draft and still depends on #258/#259 becoming protected Foundation truth before shared-dispatcher reconciliation. No predecessor GREEN, self-approval, bypass, or leaf workflow restoration transfers to this head.

Copy link
Copy Markdown
Contributor Author

Fresh exact-head gate authority for 155c9cdc9586ac29bdc4ace8a718fba3ab6304a5 after the #273 representation-integrity successor:

  • Foundation 34140109476: terminal SUCCESS.
  • SAST 34140109425: terminal SUCCESS.
  • Security 34140109689: terminal FAILURE only at dependency-review; exact checkout succeeds, Trivy/OSV and the other material scanners succeed, while Check dependency review support fails before the pinned Dependency Review action. Canonical shared owner remains .github#810; do not substitute another scanner.
  • CodeQL 34140109611: terminal FAILURE in the same central terminal-verdict ordering path as other unchanged Orgmetra canaries. Detect-languages succeeds; Actions/Python compatibility readers successfully read current-head state then fail at Release runner or enforce current-head CodeQL verdict; only afterwards does job 101811654938 successfully dispatch the current-head scan. Canonical owner remains .github#1929 / its active repair lane, not this package.
  • Fresh formal reviews remain COMMENTED-only; there is no qualifying APPROVED review. Current returned review threads were already resolved at the preceding fresh thread sweep.

Keep Draft. The #273 runtime-type repair has current-head Foundation/SAST execution evidence, but required Security/CodeQL/review admission is still non-passing. Do not no-op retrigger, self-approve, transfer predecessor checks, or use administrator bypass.

Copy link
Copy Markdown
Contributor Author

Fresh exact-head acceptance correction for unchanged 155c9cdc9586ac29bdc4ace8a718fba3ab6304a5 (the PR body still records all four lanes as queued).

  • Foundation CI 34140109476: terminal SUCCESS.
  • SAST Semgrep 34140109425: terminal SUCCESS.
  • Security Scan 34140109689: terminal FAILURE at the centrally owned Dependency Review support boundary; this is not a Selection Monitoring source finding and sibling scanners are not semantic substitutes.
  • CodeQL PR 34140109611: terminal FAILURE at the current-head terminal-verdict handoff/ordering boundary already tracked by .github#1929, not a package SARIF/source RED.
  • Review-thread sweep remains resolved; no qualifying independent APPROVED review is established.

#273's representation-integrity source repair is therefore current-head Foundation/SAST GREEN and should not be churned for central control-plane failures. Keep #42 Draft until its shared Foundation prerequisite is protected truth and then-current central/security/review governance is satisfied; no predecessor transfer, no leaf workflow resurrection, no no-op retrigger.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant