Repository navigation
Trace CANNBot round knowledge in managed optimization - #523
jiangkuaixue123 wants to merge 3 commits into
Conversation
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com> Agent-assisted-by: Codex (implementer)
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com> Agent-assisted-by: Codex (implementer)
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
首轮检视,c573f40a43ceef23ef08507fca662c95ffb65390 → 226f8d0f5fb114443ed76311879e78f24b4b6180,覆盖全部 14 个变更文件,按 #522 的软件交付范围核对。
实际追踪了 CLI → managed session → Harness → CANNBot 渠道与 worker。未开启时不调用外部渠道;搜索和读取分开,读取要求当前轮 locator 并校验 revision、digest 和 page。正文不写入留存 trace;候选只能引用已送达的本地条目或外部读取回执,搜索命中和未引用的读取均不算采用。Dashboard 读取同一记录,分别展示来源、状态、引用及 evaluator 分数,没有更改候选选择。
批准前必做:无。已静态阅读覆盖查找、读取、引用、不可用及留存最小化的用例;未运行代码、测试、浏览器或生产 grant/Ascend 集成,不据此宣称硬件验收或性能收益。
david6666666
left a comment
There was a problem hiding this comment.
Reviewed the managed retrieval and Dashboard attribution paths on this head. Two findings below.
| reference = {"path": path.relative_to(root).as_posix(), "sha256": fingerprint, | ||
| "kind": "delivered_context"} | ||
| if number is not None: | ||
| reference["round"] = number |
There was a problem hiding this comment.
This adds round to the returned reference even with CANNBot off. capture_delivery.managed_envelopes() still reconstructs the exact three-field {path, sha256, kind} response, so the trace verifier can no longer match these genuine local searches. Both Ready jobs fail four citation regressions. Please keep round on the retained state reference only, or version the envelope and verify both old and new traces.
| "status": facts, "evaluation": judged}, | ||
| usage=record["observations"]["usage"], | ||
| measured_score=_number(assessment.get("score")), | ||
| knowledge_uses=[citations[ref] for ref in record.get("knowledge_refs") or [] |
There was a problem hiding this comment.
This resolves mutable state.json citations against the current session-wide context map. _load() verifies the observation file's hash but doesn't compare its recorded citations and context snapshot with state. A completed round can therefore display a later receipt as cited while its verified observation stays unchanged. Please resolve completed-round citations from the verified observation and submitted context snapshot, or check that state matches both, with a regression for later receipts.
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com> Agent-assisted-by: Codex (implementer)
|
@zengchuang-hw @david6666666 麻烦重新评审当前提交 当前提交的 Dashboard 浏览器检查已通过,Python 3.10 和 3.11 的 full-check 测试仍有失败。 CI 结果 如果复核后认可当前改动,请在 GitHub 提交 Approve,正文第一行单独写 |
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
增量检视 226f8d0f5fb114443ed76311879e78f24b4b6180 → 8bbc58ccbb05fee2b0ad2f44209a7badea9732b2,承接上轮检视。本轮只有一个提交,覆盖控制器及回归测试。读取已完成候选时,现在先核对原始 observation 摘要,再核对原始引用和当时的 context 快照;已引用的 receipt 不能被后续轮次悄悄换掉。留存时仍保存 round,本地调用恢复原先的精简响应字段。已核对新检查与真实 settle 写入一致,也确认两边都没有 knowledge_refs 的旧记录不会因此失败。
批准前必做:无。合入协调:独立于本轮检视,#526 若先合,旧 apps.cannbot 兼容层不导出 SCORE_SCALE。后合入的一方应把 round_knowledge.py 及相关导入迁到 apps.integrations.cannbot,核对安装包 worker 和 managed attribution smoke;未做组合验证。 本轮未执行代码、测试或实机验证。
david6666666
left a comment
There was a problem hiding this comment.
One remaining Dashboard metadata validation concern.
| # The digest checks file integrity, not whether the record's | ||
| # fields are safe to disclose. Expose only known view metadata. | ||
| search = { | ||
| **{key: record.get(key) for key in ( |
There was a problem hiding this comment.
The metadata field allowlist does not validate the values. public_value() preserves nested objects and lists, so a digest-valid retained record can still carry card content or diagnostics inside an allowed field. The existing regression only covers extra top-level fields. Please validate the expected scalar types and formats for search and read metadata before projection, and cover malformed values.
|
Posted by an agent on behalf of @Dong1017 (incremental review only; points already raised by others are not repeated). DO NOT PASS — reviewed head Blocking
Non-blocking
Scope checked: the |
Governing Issue and delivery slice
Refs #522
Add opt-in CANNBot round retrieval to managed benchmark sessions and AB/BA campaigns. The user authorized the scoped implementation and a PR.
start --cannbot-platform <platform>freezes the source and Ascend platform.knowledgeretains a local context and a separate external result under the next round.knowledge-readopens a card page pinned to a hit's revision and digest. Retained search and read records contain metadata only.round --knowledge-refrecords explicit item and read-receipt citations. Round decisions shows both sources, reads, the candidate's declared use, and the evaluator's score and verdict. Read receipts and candidate citations are distinct.Checks and dependencies
ruff check src tests tools. The preceding revision passed 304 tests and 287 subtests in the wider optimization, channel, Dashboard and module-boundary selection. The last revision changes only external Dashboard projection and its regression coverage.Review follow-up
Commit
8bbc58ccpreserves the three-field localknowledgereturn used by the Codex trace verifier. The round number remains on the retained context reference for Dashboard. All four previously failing citation regressions now pass locally. Return format reviewCompleted-round attribution now checks citations and the submitted context snapshot against the digest-verified observation. Every cited delivery must still match its submitted reference. Regressions cover a later receipt added to an earlier candidate, a changed snapshot, and a cited receipt replaced at the same path. Attribution review
Local checks on this revision passed 320 tests across citation, optimization, delivery, comparison and Dashboard suites, plus
ruff check src tests tools. Two Playwright-dependent offline HTML cases were skipped locally. GitHub checks run on the new head.Remaining work and Issue closure
An independent Codex reviewer checked head
226f8d0fagainst basec573f40a. The reviewer found the Dashboard projection issue above, rechecked the correction and reported no blocking findings on the current head. This was a code review within the implementation session. GitHub maintainer approvals and repository checks are tracked on this PR. The requester owns Issue acceptance and closure.