Skip to content

task: Complete formal performance testing and decouple profiling in service-reuse campaigns #513

Open

Description

@lirx-pd

Current state: The scope of #513 is accepted as a follow-up to the November 19 owner decision in #467 and its merged implementation in #480. Implementation and its acceptance checks remain pending. Scope decision for #513.

Decision needed: Assign implementers to the accepted delivery slices.

Next action: @lirx-pd coordinates slice claims in this thread and records the implementers as they are assigned.

Goal and acceptance

  • Problem / desired outcome: Treat profiling as supplemental evidence for a specific round. Per-round acceptance, ranking, parent selection, progression, and final follow the campaign's existing functional prerequisites and performance and quality rules. Profiling has its own execution limits and consumes none of the original campaign budget.
  • Completion criteria:
    • Complete formal performance testing. Screening is optional and diagnostic. Whether it passes, fails its threshold, returns an error, or is absent, run and retain every configured formal scenario and repetition if the managed service remains valid and the required evaluation budget permits it. Functional checks and service readiness remain prerequisites.

      Formal performance Quality Round verdict Inline profiling
      Valid evidence, gate failed Skipped FAIL Attempt if configured and the same service is usable
      Gate passed Gate passed PASS, eligible for normal ranking Attempt if configured and the same service is usable
      Gate passed Valid evidence, gate failed FAIL Skipped
      Required evidence missing, invalid, or interrupted Skipped INCONCLUSIVE Skipped
      Gate passed Required evidence missing, invalid, or interrupted INCONCLUSIVE Skipped
    • Persist the complete evaluation before profiling. Before starting capture, durably save the formal performance samples or their retained evidence references, the aggregate results, the applicable quality result and score or recorded skip, and the gate assessment and verdict. Bind that checkpoint to the original attempt and its revision, effective configuration, conditions, service, and run. Track capture and cleanup separately. Once cleanup is verified, recover settles the checkpoint as recorded, including a performance FAIL, without repeating performance or quality.

    • Keep profiling out of every decision. Apply the same rule at round settlement, next-round admission, and final admission and settlement. Capture availability, failure, timeout, interruption, or subsequent loss of a profile leaves evaluation, ranking, and selection governed by the retained performance and quality evidence. Verify the revision, effective configuration, conditions, original attempt binding, and required test evidence as before. Unverified cleanup blocks new service starts for both ordinary evaluation and profile-only capture, including after an interrupted backfill.

    • Use a separate profiling time limit. Add an optional positive sampling.profile_timeout_seconds, defaulting to sampling.stage_timeout_seconds. It bounds inline capture or, for standalone capture, startup, readiness, any required warmup, and capture together. Cleanup uses the existing separate bounded cleanup window. A timeout records timed_out in capture diagnostics and makes that capture unavailable. An operator interruption records interrupted. Exclude time spent solely on profiling from the round and campaign evaluation budgets and preserve the time reserved for final, round count, and optimization usage. Persist the capture ID and start time before capture starts, and settle the excluded interval in a durable campaign accounting entry keyed by that ID. Repeated recovery applies each entry once. Preserve elapsed_seconds for round wall time and wall_elapsed_seconds for campaign wall time. Inline capture holds the campaign lock and adds wall time. External job and agent deadlines still apply as configured.

      • Apply the same budget accounting to admission and execution. Use the adjusted budget for status and its remaining-time fields, can_finalize, round and final admission, and the internal deadlines used for evaluation and its stages. Keep the configured final reserve at its original value. Wall-time fields continue to show actual elapsed time.
      • Recover only substantiated profiling time. If the process crashes before writing the end receipt or accounting entry, use a durable end receipt for the same capture when available. Otherwise, exclude only elapsed profiling-only time established by durable progress or completed-stage receipts for that capture. A start record alone supports a zero-second exclusion. Any unverified interval, including crash downtime and time spent waiting for manual recovery, remains charged to the original clock. The recovery timestamp and the configured timeout provide no evidence of actual capture duration. Finalize the adjustment once under the capture ID and preserve the original final reserve.
    • Expose a backward-compatible round summary. Add profiling to the CLI JSON and dashboard/report round output without a campaign schema version bump. Emit {"profiling":{"available":false}} when no usable capture exists. For a validated, readable capture, emit available: true plus campaign-relative result and artifacts paths. The summary prefers the latest usable backfill and otherwise uses the usable inline capture. Derive legacy availability at read time from the same validation, including false for partial, invalid, or unreadable legacy evidence. Retain the original verdict. Keep failure details in the capture execution record and stage logs.

    • Return profiling failures to the Agent. For failed, timed-out, or interrupted inline captures and backfills, return an Agent-facing command diagnostic with the failed stage, the available error details, and the capture log path when available. If a crash prevents the capture command from responding, return the recorded diagnostic during recover and identify any missing error details. Keep this diagnostic outside the persisted round's profiling object. An inline capture failure alone leaves the round command's evaluation result and selection unchanged. Standalone capture reports its own result via the exit codes below.

    • Provide a profile-only command for settled rounds. Proposed CLI inferrsi campaign profile --session <campaign> --round <round-number>. Resolve the round number to its last settled attempt with PASS or FAIL, including after retries. A round with only historical INCONCLUSIVE attempts is ineligible. Allow capture after final or exhaustion of the original budget. Use the existing nonblocking campaign lock and fail immediately if another operation holds it or cleanup is unresolved. Verify the recorded Git revision, retained protocol and effective configuration, and compatible workload, model, precision, device topology, runtime, and timing scope. Reuse the existing service lifecycle. Campaigns may omit the profiler command. A standalone capture requires a usable profiler in the retained protocol. The command runs only capture and its prerequisites. Exit 0 when a new usable capture is published, 2 with a command-level diagnostic for an ineligible round, a busy or unresolved campaign, unreproducible inputs, or failed or timed-out capture, and 130 on operator interruption after cleanup is attempted.

    • Publish backfill evidence separately. Give each backfill its own capture directory and service/run IDs, linked to the original attempt, revision, configuration, and conditions. Preserve the inline profile pointer and receipt under their original attempt/service binding, as well as the original measurements, observation, selection, and final. After validation and verified cleanup, atomically replace a separate backfill index for that attempt under the campaign lock. CLI JSON and dashboards read the same index. Retain unpublished capture directories for diagnosis and recovery. Failed or interrupted backfills leave the previous published references usable.

    • Add automated regression coverage. Exercise the verdict table, including screening rejection and script error followed by complete formal measurements. Verify timeout handling, cleanup guards for both commands, and fail-fast lock contention. Check both availability states in CLI JSON and the report, including legacy partial and unreadable evidence.

      • Test recovery and decision evidence. Inject an interruption after the complete evaluation checkpoint is saved but before profiling begins, and again during capture. Cover a performance-gate FAIL and a combined PASS, and verify that both settle from the retained results after cleanup without a new performance or quality run. A deleted or unreadable profile changes only availability across settlement, the next round, and final. Missing or invalid required candidate or evaluation evidence still blocks the corresponding decision.
      • Test backfill and historical records. Backfill a rejected round after the original deadline and an accepted, already-finalized round with a valid inline profile. Verify that CLI JSON and the report expose the separately attributed capture, with the original inline binding, selection, and final preserved. Confirm the command targets the final settled attempt of a round with retries, leaves earlier attempts untouched, rejects an INCONCLUSIVE-only round, and preserves an existing usable profile on failure.
      • Test a crash before budget settlement. Crash after the start record is durable and before the end receipt or accounting entry is written. Delay recovery, then run recover twice after confirming cleanup. Test both a start-only record and one with a durable progress receipt. The first excludes zero seconds and the second excludes exactly the proven profiling duration. Assert that the downtime remains charged, there is only one settled entry for the capture, and the second recovery grants no extra budget or final reserve.
      • Test effective deadlines. Use a controlled clock to verify status, round and final admission, and execution time limits when proven inline capture time crosses the original wall-clock deadline with evaluation budget still available. Verify that standalone capture remains outside the original budget and the configured final reserve is identical in both cases.
      • Test Agent-facing diagnostics. Fail inline profiling after both gates pass and fail a standalone backfill. Assert that the Agent receives the error in both cases, the profiling object keeps its availability-only shape when no usable capture exists, and the original verdict and selection are preserved. Verify that recover returns the diagnostic for a capture interrupted by a crash.
      • Test final after a condition change. Refresh both references and verify that final remains INCONCLUSIVE without a new accepted candidate, even if an older round has a usable backfill. A candidate that passes performance and quality under the new conditions satisfies admission without a profile. final runs its independent evaluation without collecting a profile.

Scope and constraints

  • In scope / out of scope: Applies to ordinary managed project campaigns using opt-in service_reuse. The accepted task: Complete formal performance testing and decouple profiling in service-reuse campaigns #513 scope replaces the screening short-circuit and the performance-rejection profiling skip. It also supersedes the November 19 amendment's profiling-dependent verdict, pause, full retry, and final requirements. The accepted fresh-candidate requirement remains in force. Refresh the references after a condition change and require a candidate accepted under the new conditions before a conclusive final. Under task: Complete formal performance testing and decouple profiling in service-reuse campaigns #513, performance and quality establish that acceptance. An old round's backfill remains evidence for its recorded conditions. AB/BA, benchmark sessions, gate thresholds, and reference refresh policy are outside the scope.
  • Relevant dependencies, shared interfaces or limits (if any): Use the existing campaign lock, readiness, cleanup, and artifact validation. A later change to the candidate revision, effective configuration, or test conditions requires its own evaluation before it can support a new acceptance.

Ownership and authorization

  • Proposed by / actual requester: lirx-pd.
  • Status: Accepted (scope). Decision recorded by Dong-bot on behalf of @Dong1017.
  • Accountable parent (Issue-level owner, if assigned): @lirx-pd coordinates delivery.
  • Slice implementer(s) / acting contributor (if assigned): @lirx-pd is the acting contributor and coordinator. The three implementation slices remain unassigned pending claims in this thread.
  • Goal-acceptance owner: @Dong1017 for the repository decision, with lirx-pd confirming the requested behavior.
  • Accepted decision or scoped delegation: The #513 scope decision accepts the completion criteria above and appoints @lirx-pd as accountable parent. The decision supersedes only the provisions identified above in the November 19 owner decision in #467.

Inputs and deliverables

  • Relevant sources, revisions or linked issues: src/inferrsi/harness/project_service.py, src/inferrsi/harness/optimization.py, and tests/test_project_service_reuse.py. Parent issue #467 and implementation PR #480.
  • Expected artifacts: Managed campaign changes, round output and profile-only CLI documentation, and automated regression results.

Completion and closure

  • Who accepts completion and may close: The goal-acceptance owners confirm the criteria above. The repository owner confirms closure authority.
  • Remaining uncertainty / next decision: Record implementers as they claim the verdict and recovery, output and budget, and backfill CLI slices. Each slice carries its own regression tests. Completion requires implementation evidence against the accepted criteria.

Activity

  1. added
    kind:taskBounded delivery task with a delegated slice
    on Nov 19, 2026
  2. Dong1017 commented on Nov 19, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

    Assessment: The proposal is directionally sound and mostly well specified. It makes profiling supplemental evidence instead of an acceptance gate, and it adds a backfill path for already settled rounds. Before anyone accepts it or delegates implementation, I'd settle the scope boundary and several underspecified points below. I read only the issue text. I did not check project_service.py, optimization.py, #467 or #480 against it.

    What it asks

    • Make profiling non-gating. Profile availability must not affect acceptance, ranking, parent selection, progression, or final.
    • Persist the test conclusion before capture so recovery can reuse it.
    • Run profiling outside the evaluation, round and campaign budgets, with its own bounded window.
    • Record profiling.available on every round.
    • Add inferrsi campaign profile --session <campaign> --round <n> to capture a profile for a settled round after final or after the budget is exhausted.
    • Add regression coverage for these cases.

    Scope concerns (should be settled before acceptance)

    1. The change is wider than the title says. The first criterion reverses Add service-reuse evaluation for project campaigns #480 screening semantics. Today a failed screening result skips formal performance testing. The proposal makes it diagnostic only and always runs formal measurement. It also reverses the rule that a failed performance gate skips profiling.
    2. Supersession bookkeeping is missing. After acceptance, task: Support service-reuse and multi-scenario optimization in managed campaigns #467 needs a pointer or edit. Otherwise two issues will carry conflicting rules about pause, full retry and profile requirements.
    3. The first criterion bundles too much. It covers screening, perf FAIL, INCONCLUSIVE, quality FAIL and inline profiling in one checkbox. A verdict/profiling decision table would be easier to implement and test: screening result × formal perf × quality → verdict, and whether inline profiling runs.

    Gaps and risks

    • Profiling window. "Bounded profiling time window" has no default, no configuration key, and no timeout behavior. It does not say whether a timeout counts as failed or interrupted.
    • Wall-clock effect. Inline profiling runs under the campaign lock and delays the next round and final. Excluding it from the budget is fine, but the doc should confirm that external or session deadlines are not affected. "Report actual elapsed time" needs a concrete field name.
    • No failure reason. {"available":false} carries no reason. The proposal should say where operators find the cause, such as logs or a separate diagnostic artifact, so that debugging isn't lost.
    • Schema and legacy rounds.
      • Is this an additive field?
      • Is there a schema or version bump?
      • How are legacy rounds with partial or unreadable profiles derived, and is the derivation done at read time or by migration?
    • Profile-only command.
      • What happens if a campaign is currently running? The proposal mentions the managed lock but not whether the command waits, fails fast, or uses a distinct exit code.
      • Which retained inputs make a capture "reproducible", and how does it behave when they are gone? Cover candidate revision, effective config, protocol, and the hardware or environment drift that reuse assumes.
      • Exit codes for "inputs not reproducible", "capture failed" and "interrupted" are not specified.
      • "Both output formats" is ambiguous. Name them (presumably JSON and text/report).
    • Atomic reference commit. "Single recoverable update" needs a mechanism, such as write-then-rename of the manifest or a journal. It also needs a rule for orphaned capture directories after an interrupted backfill: retain or garbage-collect.
    • Cleanup interplay. Unverified cleanup blocks a new managed service. The proposal should say that this also blocks profile-only backfill, and that a backfill interrupted mid-capture goes through the same check.
    • Accounting exactly once. "Recovery accounts for an interrupted capture once" is hard to test. Specify where the accounting is recorded.

    Test additions I'd suggest

    • Profile-only on an accepted round, not just a rejected one.
    • Profile-only while the campaign lock is held.
    • Legacy-round availability derivation.
    • Profile-window timeout.
    • Round and campaign budget numbers unchanged after inline and standalone capture.

    Ownership and process

    • The parent owner and implementer are still TBD, and the acceptance status is "Proposed".

    • For @Dong1017: the issue asks for scope confirmation and implementer delegation. The decision hinges on the supersession scope above. I'd accept the profiling-decoupling and backfill parts as written. The screening and perf-fail semantic change should either be explicitly included in the supersession or split out.

    • Suggested slicing for delegation:

      1. Verdict and inline-profiling flow, plus persist-before-profile.
      2. profiling output object, legacy derivation and budget exclusion.
      3. campaign profile CLI, including atomic publish and recovery.

      Each slice would carry its own regression tests.

    • For lirx-pd: please answer the open questions above (window default and config, lock behavior, exit codes, output formats) in the body, then tag the decision owner for acceptance.

    Dong-bot (authorized delegate).

  3. david6666666 commented on Nov 19, 2026

    @david6666666
    Collaborator

    Please use #467’s November 19 amendment as the baseline, including the already-accepted fresh candidate requirement after a condition change. For backfill, please preserve the original inline profile’s attempt/service binding separately from the new capture’s binding. The current verifier requires an inline profile to match the original attempt and service, so replacing that pointer cannot faithfully describe a new service. An acceptance test could backfill an already-finalized round that has a valid inline profile, then assert the old final, selection, and inline receipt are unchanged while the CLI/report expose the separately attributed capture.

  4. zengchuang-hw commented on Nov 19, 2026

    @zengchuang-hw
    Contributor

    PASS(方案方向检视,不代表实现验收或范围授权)

    支持把 profiling 从性能/质量结论中解耦,并允许对已结算轮次单独补采。静态核对了当前 main 9b418d7fe16f3979b67f74f290949629ef103e8f 的 project_service.py、optimization.py 及相关测试。建议把以下具体边界写进实现和回归验收:

    1. 采样前的持久化检查点必须包含完整测试结论,而不只是 PASS 标记。 目前 profile 调用 先于最终 PASS,measurements、quality_score 又到 函数末尾 才统一写入;recover 会将仍为 running 的执行改判 INCONCLUSIVE。新检查点应同时保存正式样本/汇总、适用的质量结果、结论和原 attempt 绑定,单独保留 profiling 与 cleanup 状态。测试应在“结论落盘后、profile 启动前”和“采样中”分别中断;确认清理后可原样结算,而不是重新测量或丢失性能 FAIL。

    2. 只解除 profile 对选择的约束,保留其他证据完整性约束。 当前 final 证据检查、结算 和 下一轮准入 都会检查 profile。需要一起调整,不能只修改执行阶段。已有 profile/configuration 损坏测试 应拆开预期:profile 缺失只改变 availability;候选版本、有效配置、测试条件或原测试证据不可核实时,仍不能据此放行。

    3. 预算排除要作用于所有实际准入和执行时限。 当前 status/final、启动准入 依赖绝对 deadline,阶段执行 另有 evaluation timeout。仅在报表减去 profile 时长仍可能耗尽下一轮/final。建议持久化每次 capture 的可排除区间和结算标识,回归连续 recover 两次,证明不会重复返还预算;同时明确中断后的停机/人工处理时间如何计费,不能一并误算成 profiling。

    4. 补采不能追溯改判旧轮次。 对旧规则下因 profile 失败而记录为 INCONCLUSIVE 的 attempt,即使后来补采成功,也应按本提案“preserve recorded verdict”保持原结论与资格;同样不能满足新条件下的新候选验收要求。补充一个有多次 retry 的轮次用例,确认 --round 解析到规定的最终 attempt,不修改早期 attempt、已完成 final、排名或计数。

    另请更新授权段落对 #467 的历史描述:当前有效基线是 11 月 19 日 owner 修订,其中 profile 失败为 INCONCLUSIVE。本提案仍是待 owner 接受的规则变更。

    本次只做静态方案与代码核对,未运行测试、服务或采样,也未以 CI 结果作依据。

  5. added
    area:harnessHarness module: profile, A/B loop, quality gates, evidence reports
    status:proposedWaiting for the decision named in the status block
    on Nov 19, 2026
  6. lirx-pd commented on Nov 19, 2026

    @lirx-pd
    ContributorAuthor

    Thanks for the reviews. I updated the issue to include the screening change and profiling after a valid performance rejection in the proposed scope, and added a verdict table. The November 19 amendment to #467 is the baseline. It requires a fresh accepted candidate after a condition change. Under this follow-up, passing performance and quality in the new conditions satisfies that requirement, and an old round's backfill applies only to its recorded conditions. Once #513 is accepted, I'll cross-link the decision from #467.

    The draft now proposes sampling.profile_timeout_seconds, defaulting to sampling.stage_timeout_seconds, and spells out timeout, interruption, fail-fast lock behavior, exit codes, and where diagnostics live. Profiling time is excluded from the internal evaluation budget. Wall-clock reporting and external deadlines still reflect the actual elapsed time. CLI JSON and the report derive the additive availability field from validated evidence, including for older rounds.

    Each backfill has its own service and run IDs and publishes through a separate index. The original inline receipt and binding, selection, and final remain intact. The regression checklist includes an already-finalized accepted round with a valid inline profile, a rejected round after the deadline, lock contention, legacy records, timeouts, and budget and cleanup recovery.

  7. zengchuang-hw commented on Nov 19, 2026

    @zengchuang-hw
    Contributor

    PASS(增量方案检视)

    已核对这次正文更新和作者说明。相对我的上一轮意见,现在已明确判定表、独立 profiling 时间窗、按 capture ID 幂等记账、仅定位最后一个已结算的 PASS/FAIL attempt,以及单独发布 backfill index。历史 verdict、inline 绑定和已完成 final 保持不变,新条件仍要求新的候选验收,这些边界一致。

    补充一个预算恢复验收点:开始时间落盘后进程崩溃、尚未写入结束时间或记账记录。不能直接把恢复时刻减去开始时刻全部算作 profiling,否则停机/人工处理时间也会被返还。请明确缺失结束凭据时如何确定可排除区间,并加入“崩溃后延迟恢复,再连续 recover 两次”的用例,验证停机时间未被误排除、同一 capture 只记账一次、final reserve 未被重复增加。这里只是落实既定预算边界的实现建议,不阻塞方案方向。

    本轮为静态增量设计检视,未运行代码或测试、未用 CI 作依据;实现与范围接受仍待指定 owner 决定。

  8. changed the title [-]task: Make profiling optional and backfillable in service-reuse campaigns[/-] [+]task: Complete formal performance testing and decouple profiling in service-reuse campaigns[/+] on Nov 19, 2026
  9. lirx-pd commented on Nov 19, 2026

    @lirx-pd
    ContributorAuthor

    Thanks for the follow-up. I added regression cases for interruptions both before profiling starts and during capture. After verified cleanup, recovery must reuse the complete evaluation checkpoint and preserve either a PASS or a performance FAIL. The existing checks for candidate identity, test conditions, and required evidence still apply to later rounds and final.

    If a crash leaves no durable end receipt, recovery excludes only the profiling time supported by records for that capture. A start record alone excludes zero seconds. The test delays recovery and runs recover twice to verify that downtime remains charged, the adjustment is applied once, and the original final reserve is preserved. The adjusted budget applies to admission and execution. Profiling errors are returned to the Agent through the command or recover, while the round summary keeps only availability and artifact paths.

    The backfill tests now cover round selection after retries and preservation of the original inline binding, historical verdicts, and completed final results. @Dong1017, could you confirm the revised scope and assign an implementer? The fresh-candidate requirement after a condition change stays in place under the November 19 decision in #467. Once #513 is accepted, I’ll link the decision from #467.

  10. Dong1017 commented on Nov 19, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

    Accepted. Recording this as the scope decision for #513, with the revised body and thread as the accepted text. This is a design/scope acceptance only, not an implementation acceptance. I did not verify it against the code or run any tests. (Dong-bot, authorized delegate.)

    Clauses accepted

    • Formal performance testing: Screening is optional and diagnostic. Every configured formal scenario and repetition runs whenever the service is valid and budget allows. The verdict and inline-profiling table is accepted as written.
    • Checkpoint before profiling: The complete evaluation is saved and bound to the original attempt before capture starts. Capture and cleanup are tracked separately. After verified cleanup, recover settles from the checkpoint, including a performance FAIL, without re-measuring.
    • Profiling stays out of decisions: This applies at settlement, next-round admission, and final. Candidate revision, effective configuration, conditions, and required test evidence are still verified. Unverified cleanup blocks new service starts for both ordinary evaluation and profile-only capture.
    • Separate time limit: sampling.profile_timeout_seconds defaults to sampling.stage_timeout_seconds. Profiling-only time is excluded from the round and campaign budgets through a durable accounting entry keyed by capture ID.
      • The adjusted budget applies to admission and execution, and the final reserve is unchanged.
      • After a crash, only time proven by an end receipt or durable progress receipts is excluded. A start record alone excludes zero seconds. Downtime and manual-recovery time stay charged.
    • Round summary: The profiling object is additive, with no schema bump. Availability for legacy rounds is derived at read time.
    • Failure diagnostics: Profiling failures are returned to the Agent as a command diagnostic, or through recover after a crash. They stay outside the persisted profiling object.
    • inferrsi campaign profile --session <campaign> --round <n>: A profile-only command for settled rounds.
      • It targets the last settled PASS or FAIL attempt, and rounds with only INCONCLUSIVE attempts are ineligible.
      • It fails fast on lock contention or unresolved cleanup.
      • Exit codes are 0, 2 and 130, as specified.
    • Backfill publication: Each backfill has its own directory and IDs, and publishes through a separate atomically replaced index. The original inline receipt and binding, selection, and final are unchanged, and a failed backfill leaves the previous references usable.
    • Regression coverage: All listed test groups are accepted as the completion bar.

    Supersession

    Implementation assignment

    • Accountable parent and coordinator: lirx-pd, as proposer and acting contributor. I'm recording this from the thread because no other owner is named. If someone else should own it, say so here.
    • Suggested slices, each with its own regression tests:
      1. Verdict and recovery: the verdict flow, the checkpoint before profiling, and crash recovery.
      2. Output and budget: profiling output, legacy availability derivation, and budget accounting for admission and execution.
      3. Backfill CLI: the campaign profile command and publication of backfill evidence.
    • Slice implementers: still unassigned. Please claim a slice in this thread.
    • Review: zengchuang-hw's and david6666666's boundary points are folded into the accepted text and should be the review checklist.

    Non-blocking next steps

  11. zengchuang-hw commented on Nov 19, 2026

    @zengchuang-hw
    Contributor

    PASS

    增量设计检视,不代表实现验收。

    上一轮指出的崩溃记账问题已解决:只有开始记录时排除零秒,只扣除同一 capture 的持久化结束或进度凭据能证明的 profiling 时长;停机时间仍计入预算。延迟恢复并连续 recover 两次的回归能同时检验停机时间、幂等记账和 final reserve。

    我也对照了 #480 的现有实现:原流程在 profiling 后才持久化 PASS,恢复可能把缺少完整结算记录的轮次判为 INCONCLUSIVE。现在要求先持久化完整评估,以核实清理为前提,恢复时直接结算原 PASS 或性能 FAIL,不重跑测量;调整后的预算同时约束准入和执行。重试后只定位最近已结算的 PASS/FAIL,backfill 不改写原 inline 绑定、历史 verdict 或 final。这些要求覆盖了本次变更的关键边界,无新增设计阻塞。

    范围决定已记录,正文状态和分片负责人待同步。未查 CI、未运行代码或测试。

  12. added
    status:acceptedThe decision owner accepted it in a recorded comment
    and removed
    status:proposedWaiting for the decision named in the status block
    on Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:harnessHarness module: profile, A/B loop, quality gates, evidence reportskind:taskBounded delivery task with a delegated slicestatus:acceptedThe decision owner accepted it in a recorded comment

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions