Repository navigation
Unify inference knowledge in skill_library with seven task routes and direct search - #516
david6666666 wants to merge 28 commits into
Conversation
Signed-off-by: David Chen <530634352@qq.com> Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author)
Signed-off-by: David Chen <530634352@qq.com> Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author)
Signed-off-by: David Chen <530634352@qq.com> Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author)
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Skill evaluationPLANNED: not executed; there are no task outcomes yet.
Author verification steps (nonpaid; not yet run)
Evidence and logs · Retention: 30 days. Plan bindings, budget, frozen exam and advisory detailsAdvisory: obtain missing evidenceRetained evidence availability: planned / unexecuted. No approval or merge authorization is issued.
Model/settings:
Automatic execution / cumulative PR budget: {"reason":"Trusted configuration or pricing missing, invalid or expired","remaining_usd":"unknown","reserved_or_unresolved_usd":"unknown","spent_or_legacy_usd":"unknown","state":"blocked"}Projection check selection (not an executed result): {"reason":"No verified ancestor source pin for automatic command selection","status":"unavailable"}Provider/settings: Uncovered / unsupported paths (1791)
|
david6666666
left a comment
There was a problem hiding this comment.
The exact-head Python 3.11 check reproduces three migration regressions. The path and schema changes need to be covered together.
david6666666
left a comment
There was a problem hiding this comment.
One separate gap in the contract keeping navigation out of automatic evidence ranking.
Signed-off-by: David Chen <530634352@qq.com> Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author)
There was a problem hiding this comment.
PR review verdict — DO NOT PASS
Reviewed base 9b81c9ba1d4d1f7d45bd039a7c02ac2f01859426 and head 5a8c98cd9f21b8354c9a99e7795102ca0ab35c82.
Ready full-check fails on tip (44473700718) with library-layout migration fallout. Blockers before re-review:
-
Broken campaign-plan link —
evidence/_template/plan/campaign-plan.mdstill points at.../skill_library/methods/performance-ceiling-analysis/references/acceptance-criteria.md, which no longer exists after the move toplaybooks/measurement/....tests/test_campaign_plan_template.pyasserts the target exists (AssertionErroron 3.10/3.11). -
Missing
provenance_readsetson some skill metadata —tests/test_knowledge_contract.py/_skill_selectionrequire the key; tip raisesKeyError: 'provenance_readsets'for at least one migratedmetadata.json. Every skill metadata under the new layout needs the field (empty list OK). -
Undeclared skill reference — Ready reports
ValueError: Undeclared skill reference: ../inference-optimization-casebook/cases/admission-drain-occupancy.md. Update remaining relative refs to thecasebooks/inference/...location (or declare them) so skill-reference checks pass.
Re-review after Ready full-check is green on the new tip. Technical/third-party verify only (cursor[bot] permission none); not a write+ merge gate claim.
…ews on read Signed-off-by: David Chen <530634352@qq.com> Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author) Agent-reviewed-by: Claude Code / claude-fable-5-2 [effort=xhigh] (design review)
Signed-off-by: David Chen <530634352@qq.com> Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author)
Signed-off-by: David Chen <530634352@qq.com> Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author)
david6666666
left a comment
There was a problem hiding this comment.
I found a replay issue caused by hashing the new physical root as a single bundle dependency.
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
首次独立检视:9b81c9ba1d4d1f7d45bd039a7c02ac2f01859426 → 4739b6dc7d899a7d451ac0a838c9f1ae5fc8d31e(范围 #515)。
批准前必做:迁移知识视图生成逻辑时保留原有的保护范围。 _skill_view.py 现在生成对外可见的来源链接、文件指纹、推荐和验证标识。此前这些字段来自受保护目录中的索引记录。knowledge-model-policy.json 仍覆盖旧数据路径及 catalog,但是没有覆盖新投影实现。按仓库既有规则,迁移不应让同一知识内容失去保护。请让维护者通过单独的策略变更纳入该实现后再更新本 PR,或把决定这些字段的投影留在已有保护范围;不要仅凭本次有受保护文件就认为迁移后仍受保护。
已静态核对完整 Git 树、注册表及打包投影,48 条原始记录保留 ID、来源层级和来源 pin;七个任务入口的 28 个 seed 均可定位,router 不计入 Find 的检索语料。这些是结构和直接检索的核对,不是 #515 的受控 pilot 验收。未运行安装、测试或实验,未查 CI。
Signed-off-by: David Chen <530634352@qq.com> Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author)
Signed-off-by: David Chen <530634352@qq.com> Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author)
…nt shared ranking Signed-off-by: David Chen <530634352@qq.com> Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author) Agent-reviewed-by: Claude Code / claude-fable-5-2 [effort=xhigh] (design review)
|
Please promote the seven task cards to top-level skills and make all seven part of the main InferRSI agent entrypoint's required skill set. For each inference task, the entrypoint should select and load the primary skill for the user's goal before searching for supporting evidence. Currently, the instruction to choose a route lives inside the optional router, and the main plugin directs the agent straight to Please add an entrypoint test that checks all seven skills are available by default and shows, for example, that an accuracy-regression request loads the accuracy skill before its first evidence search. |
|
Please turn the seven route cards into seven top-level skills available to the main InferRSI entrypoint. Each skill should include the knowledge needed for its goal and explain when to load each supporting sub-skill. Use the actual run to make those choices. Capture the hardware vendor, device and backend, model specification, run script and flags, inference engine and version, benchmark and acceptance criteria, current diagnosis, and allowed search space. For each decision, say what observation triggers a sub-skill, what evidence it returns, and how that evidence determines the next step. Each skill should give the agent a practical routine for reaching its own goal. For example,
Each routine should say when the evidence satisfies the user's goal. If it falls short, identify the next test that would distinguish the remaining explanations, or record the blocker when the search budget is exhausted. |
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
增量检视:4739b6dc7d899a7d451ac0a838c9f1ae5fc8d31e → f49cd4f7f9d896b5bc123fbeb7f0307e619b70d1,上轮意见。
上轮阻塞已解决:skill_views 和推荐投影归入受保护的 catalog.py,旧的独立实现已移除,构建脚本与测试的导入已同步;新增测试检查函数实际定义文件确实匹配现有保护范围,而不是只比较配置字符串。输出投影未发现有实质语义变化。当前 main 的 5 个 case backend 值在改名后均保留:4 个 CUDA、1 个 Ascend。任务入口仍不计入 Find 语料,原 case ID 继续可用。
已静态核对 Git 树、实现、保护路径测试及 case 文件;未运行安装、测试或 CI。批准限这次结构与兼容性变更,#515 的路由检索收益仍需独立 pilot。
Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author) Signed-off-by: David Chen <530634352@qq.com>
|
Reviewed Two things to fix before merging
What this gives InferRSI
With these two fixes, the unified layout and its search integration are worth merging. The task cards can stay optional while the pilot measures their effect. |
…ence Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (author) Signed-off-by: David Chen <530634352@qq.com>
|
Addressed both follow-ups in The main agent selects the primary by the requested deliverable and opens its full procedure with Each primary captures the run configuration, current diagnosis, acceptance criteria and allowed search space. The seven procedures include 28 observation → supporting skill → returned evidence → next decision branches, covering 20 distinct supporting skills. Each concludes with evidence for acceptance or the smallest remaining discriminating test. Accuracy procedure · All seven procedures A clean wheel returned every complete primary procedure without Git or PyYAML. The installed resource inventory matched all 868 tracked files. All 36 fixed queries matched the preceding PR head in ordered top five, scores and term explanations. Measuring whether a host selects the correct task from ambiguous prompts is the separate blinded acceptance pilot. The issue and this PR body both include the updated layer, physical tree, search and migration diagrams. |
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
增量检视 58c022fda7484c30b42144ba56fc423283f39d3a → ebe5e6836c8269e3cea9e82821c912929bd1c960,上一轮 检视。新提交只更新四个已有证据文件,未改运行逻辑、检索、验证器或来源 pin。旧的 42 条硬件会话记录保持原样,新增 9 条。
我复核了有原始样本的会话中位数。两组最终对照方向相反,汇总仅约 +0.072%,且像素相同;B1/B2 的托管复跑均未通过像素门槛。最终选回基线,1.000× 和 performance_gate_not_met 的表述与证据一致。这个 PASS 是对本轮改动和证据口径的判断,不是性能或知识布局收益验收。
批准前必做:无。只做静态 diff 与留存数据复算,未重跑代码、测试或 GPU 实验。
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
david6666666
left a comment
There was a problem hiding this comment.
Installed-wheel validation on the metadata-projection update.
|
Addressed the fixed-query replay review. The final diff keeps the search comparison and the three migrated template paths. The standalone GPU trial and its offline UUID comparison are outside this PR after scope was narrowed; the historical version is available at Search parity passed on the frozen workload. Ordered top-five results matched on all 49 queries; all 245 hits retained read commands. On Windows with Python 3.12.6, fresh-process offline Find took 2.583 s on main and 1.809 s on the candidate (median, 28 observations per arm, 26/28 paired observations faster). Neither version produced an empty response on the 19 evidence-gap queries. Results measure this search command and workload. The JSON below includes exact Git revisions, per-query ranks/readability, all timings in seconds and the predefined gates. SHA-256 of the UTF-8 compact JSON (no trailing newline) is Complete machine-readable search report{"revisions":{"base":"a3e33851981f1ecdc8b73d7b62633cc598e5d5ae","candidate":"f6502a6d1c7c8e8cf56bf8b648f1493483d9c09a"},"platform":"Windows","python":"3.12.6","gate":{"match_all_ordered_top5":true,"max_lost_readable_hits":0,"max_candidate_to_base_median_cli_latency_ratio":1.0},"passed":true,"quality":{"matching_ordered_top5":49,"queries":49,"lost_readable_hits":0,"base":{"hit_at_3_by_expected_kind":15,"labelled_positive":17,"empty_results_on_labelled_gap":0,"labelled_gap":19},"candidate":{"hit_at_3_by_expected_kind":15,"labelled_positive":17,"empty_results_on_labelled_gap":0,"labelled_gap":19}},"latency":{"arms":{"base":{"median_s":2.5829681999748573,"p95_s":3.8193758999696},"candidate":{"median_s":1.8093878999934532,"p95_s":2.4154819000978023}},"pooled_median_ratio":0.7005072304068884,"paired_block_bootstrap_95pct":[0.6858961722436299,0.7409815889118484]},"samples_s_base_candidate":{"day-0":[[2.391219099983573,1.8099357000319287],[2.4550408999202773,1.8113031999673694],[2.5764376999577507,2.0859145999420434],[2.350265400018543,2.0120835000416264]],"performance":[[4.034709700034,1.755700399982743],[3.40615809999872,1.8088400999549776],[3.4876103000715375,1.7819966999813914],[3.266684800037183,2.049753600033]],"debug":[[2.3987034000456333,1.6594283999875188],[2.73624340002425,1.5055827000178397],[2.6919579000677913,1.4996519999112934],[2.1565635999431834,1.8066733999876305]],"accuracy":[[2.0905057000927627,1.5482305000768974],[2.3040007000090554,1.3917638000566512],[2.2935276000062004,2.3142674999544397],[2.1001248999964446,1.8570974000031129]],"version-alignment":[[2.615225499961525,1.7773917000740767],[2.589498699991964,1.9469111000653356],[2.6196744000772014,1.6930601000785828],[2.6359644999029115,2.2105346999596804]],"derived-engine":[[1.9499901999952272,1.351433200063184],[1.7808496000943705,1.9931260000448674],[1.7685084000695497,1.353363799978979],[1.92963360005524,1.4185320000397041]],"independent-engine":[[3.1002288999734446,2.3213082998991013],[3.458147400058806,2.4154819000978023],[3.8193758999696,2.600055600050837],[3.5432915999554098,2.271029300056398]]},"rankings":[{"query":"BF16 QK normalization complex RoPE fusion numerical order","expected_kind":"record","base":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.qwen-b300-rejected-directions",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.qwen-b300-rejected-directions",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true]]},{"query":"Has Qwen Image QK normalization and complex RoPE fusion been tried on four B300s, and what happened to speed and full-image quality?","expected_kind":"case","base":[["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","inference-benchmark-design",true]],"candidate":[["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","inference-benchmark-design",true]]},{"query":"QK norm RoPE B300","expected_kind":"case","base":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.admission-drain-occupancy",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.admission-drain-occupancy",true]]},{"query":"图像生成里把 QK 归一化和旋转位置编码融合后,如何验证算子、整图质量和端到端时延?","expected_kind":"skill","base":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","kernel-writing-guide",true],["skill","vllm-ascend",true],["skill","ascend-operator-profiling",true],["record","triton.common.kernel.bandwidth_fusion_resource_limits",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","kernel-writing-guide",true],["skill","vllm-ascend",true],["skill","ascend-operator-profiling",true],["record","triton.common.kernel.bandwidth_fusion_resource_limits",true]]},{"query":"Has RoPE cosine and sine caching been tried for Qwen Image on one Ascend 910B1 with BF16 eager execution, and what happened?","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","ascend-graph-mode",true],["skill","ascend-model-adaptation",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","ascend-graph-mode",true],["skill","ascend-model-adaptation",true]]},{"query":"Qwen Image single 910B1 BF16 eager eight-step RoPE cosine sine cache","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.drama3-rejected-directions",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.drama3-rejected-directions",true]]},{"query":"910B1 RoPE cache benchmark","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","inference-benchmark-design",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","inference-benchmark-design",true]]},{"query":"四卡 B300 上的融合经验能直接推断单卡 910B1 缓存三角值的效果吗?","expected_kind":"none","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","vllm-ascend",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.drama3-rejected-directions",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","vllm-ascend",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.drama3-rejected-directions",true]]},{"query":"operator test passes but image output fails the confirmed quality gate","expected_kind":"none","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","ascend-operator-tuning",true],["skill","ascend-operator-profiling",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","ascend-operator-tuning",true],["skill","ascend-operator-profiling",true]]},{"query":"Can a candidate that failed the confirmed full-image quality gate be accepted because its operator test passed?","expected_kind":"none","base":[["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","ascend-operator-tuning",true]],"candidate":[["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","ascend-operator-tuning",true]]},{"query":"image quality gate failure","expected_kind":"none","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["case","case.drama3-rejected-directions",true],["skill","audit-sd-cpp-residency",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["case","case.drama3-rejected-directions",true],["skill","audit-sd-cpp-residency",true]]},{"query":"算子测过了但整图验收没过,这个版本还能入选吗,接下来怎么办?","expected_kind":"none","base":[["skill","kernel-writing-guide",true],["skill","ascend-operator-tuning",true],["skill","ascend-operator-profiling",true],["skill","ascendc-tiling-design",true],["case","case.drama3-rejected-directions",true]],"candidate":[["skill","kernel-writing-guide",true],["skill","ascend-operator-tuning",true],["skill","ascend-operator-profiling",true],["skill","ascendc-tiling-design",true],["case","case.drama3-rejected-directions",true]]},{"query":"continuously fed service has low full-run average decode batch below C64","expected_kind":"case","base":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.qwen-b300-rejected-directions",true]],"candidate":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.qwen-b300-rejected-directions",true]]},{"query":"Does a low full-run average decode batch prove that the active batch stayed far below 64 while requests were still being admitted?","expected_kind":"none","base":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["case","case.vllm-gdn-source-contract",true],["case","case.sglang-gdn-source-contract",true]],"candidate":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["case","case.vllm-gdn-source-contract",true],["case","case.sglang-gdn-source-contract",true]]},{"query":"decode batch drain occupancy","expected_kind":"case","base":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.vllm-gdn-source-contract",true],["case","case.sarathi-chunk-budget",true]],"candidate":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.vllm-gdn-source-contract",true],["case","case.sarathi-chunk-budget",true]]},{"query":"怎么分开统计持续接收请求和尾部排空时的活跃批量,同时检查整个请求集的吞吐?","expected_kind":"skill","base":[["case","case.admission-drain-occupancy",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["record","qwen3_tts_engine.ascend.audio.graph_batching_state_ownership",true]],"candidate":[["case","case.admission-drain-occupancy",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["record","qwen3_tts_engine.ascend.audio.graph_batching_state_ownership",true]]},{"query":"fused kernel passed unit checks but the finished image failed the verified acceptance test","expected_kind":"none","base":[["case","case.qwen-b300-rejected-directions",true],["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","kernel-writing-guide",true]],"candidate":[["case","case.qwen-b300-rejected-directions",true],["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","kernel-writing-guide",true]]},{"query":"Should I keep an operator-tested fusion as accepted when its confirmed full-image quality check failed?","expected_kind":"none","base":[["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true]],"candidate":[["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true]]},{"query":"fusion quality acceptance failure","expected_kind":"none","base":[["case","case.qwen-ascend-rejected-directions",true],["skill","draft-acceptance-issue",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["skill","inference-benchmark-design",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true]],"candidate":[["case","case.qwen-ascend-rejected-directions",true],["skill","draft-acceptance-issue",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["skill","inference-benchmark-design",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true]]},{"query":"融合版本的单算子测试合格,最终图片却没达到验收标准,该如何处理?","expected_kind":"none","base":[["skill","kernel-writing-guide",true],["case","case.drama3-rejected-directions",true],["skill","vllm-ascend",true],["skill","ascend-operator-tuning",true],["case","case.qwen-ascend-rejected-directions",true]],"candidate":[["skill","kernel-writing-guide",true],["case","case.drama3-rejected-directions",true],["skill","vllm-ascend",true],["skill","ascend-operator-tuning",true],["case","case.qwen-ascend-rejected-directions",true]]},{"query":"candidate ran at 360 ms with diagnostics running versus an old 277 ms result on a quiet host","expected_kind":"none","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-b300-rejected-directions",true],["skill","assess-serving-preemption",true],["skill","inference-benchmark-design",true],["skill","torchnpu-pinned-host-memory-lifecycle",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-b300-rejected-directions",true],["skill","assess-serving-preemption",true],["skill","inference-benchmark-design",true],["skill","torchnpu-pinned-host-memory-lifecycle",true]]},{"query":"Do unpaired latency measurements under different host loads establish that a code change caused a regression?","expected_kind":"none","base":[["case","case.qwen-b300-rejected-directions",true],["skill","weight-loading-optimization",true],["case","case.qwen-ascend-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","host-overhead-optimization",true]],"candidate":[["case","case.qwen-b300-rejected-directions",true],["skill","weight-loading-optimization",true],["case","case.qwen-ascend-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","host-overhead-optimization",true]]},{"query":"unpaired latency host contention","expected_kind":"none","base":[["skill","torchnpu-pinned-host-memory-lifecycle",true],["case","case.qwen-ascend-rejected-directions",true],["skill","host-overhead-profiling",true],["skill","host-memory-management",true],["skill","host-overhead-optimization",true]],"candidate":[["skill","torchnpu-pinned-host-memory-lifecycle",true],["case","case.qwen-ascend-rejected-directions",true],["skill","host-overhead-profiling",true],["skill","host-memory-management",true],["skill","host-overhead-optimization",true]]},{"query":"主机忙闲不一致时,旧基线与新改动的耗时差能归因于代码吗?","expected_kind":"none","base":[["skill","draft-acceptance-issue",true],["skill","ascend-device-health",true],["skill","ascend-operator-tuning",true],["skill","inference-benchmark-design",true],["record","vllm_omni.common.diffusion.measurement_and_profiling",true]],"candidate":[["skill","draft-acceptance-issue",true],["skill","ascend-device-health",true],["skill","ascend-operator-tuning",true],["skill","inference-benchmark-design",true],["record","vllm_omni.common.diffusion.measurement_and_profiling",true]]},{"query":"fixed-request check passes but baseline and async candidate both fail speech-recognition quality in full service","expected_kind":"case","base":[["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["skill","audit-fixed-workload-claim",true],["case","case.sarathi-chunk-budget",true]],"candidate":[["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["skill","audit-fixed-workload-claim",true],["case","case.sarathi-chunk-budget",true]]},{"query":"If full-service speech recognition fails for both baseline and candidate, does that prove async overlap caused the failure?","expected_kind":"none","base":[["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","cann-attribution-overlap",true],["skill","dcp-async-save-performance",true],["skill","assess-serving-preemption",true]],"candidate":[["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","cann-attribution-overlap",true],["skill","dcp-async-save-performance",true],["skill","assess-serving-preemption",true]]},{"query":"ASR baseline candidate failure","expected_kind":"skill","base":[["skill","inference-benchmark-design",true],["skill","draft-acceptance-issue",true],["skill","optimization-search-policy",true],["skill","ascend-device-health",true],["skill","ascend-operator-tuning",true]],"candidate":[["skill","inference-benchmark-design",true],["skill","draft-acceptance-issue",true],["skill","optimization-search-policy",true],["skill","ascend-device-health",true],["skill","ascend-operator-tuning",true]]},{"query":"异步服务的基线和候选语音识别都不达标,怎样用最少对照排查共同问题?","expected_kind":"skill","base":[["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["skill","assess-serving-preemption",true],["record","sglang_omni.common.audio.serving_and_streaming_quality",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true]],"candidate":[["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["skill","assess-serving-preemption",true],["record","sglang_omni.common.audio.serving_and_streaming_quality",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true]]},{"query":"quality checks passed but candidate service is 4307 frames/s versus 4371 baseline with longer drain","expected_kind":"none","base":[["case","case.drama3-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.admission-drain-occupancy",true],["case","case.sarathi-chunk-budget",true]],"candidate":[["case","case.drama3-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.admission-drain-occupancy",true],["case","case.sarathi-chunk-budget",true]]},{"query":"Can I claim a full-service throughput improvement when the candidate measured 4307 frames/s and the baseline 4371 frames/s in one run?","expected_kind":"none","base":[["skill","inference-benchmark-design",true],["case","case.sarathi-chunk-budget",true],["case","case.drama3-rejected-directions",true],["skill","audit-fixed-workload-claim",true],["case","case.qwen-b300-rejected-directions",true]],"candidate":[["skill","inference-benchmark-design",true],["case","case.sarathi-chunk-budget",true],["case","case.drama3-rejected-directions",true],["skill","audit-fixed-workload-claim",true],["case","case.qwen-b300-rejected-directions",true]]},{"query":"throughput comparison drain mismatch","expected_kind":"none","base":[["case","case.drama3-rejected-directions",true],["case","case.admission-drain-occupancy",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.sarathi-chunk-budget",true],["skill","dense-performance-evidence",true]],"candidate":[["case","case.drama3-rejected-directions",true],["case","case.admission-drain-occupancy",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.sarathi-chunk-budget",true],["skill","dense-performance-evidence",true]]},{"query":"两边语音质量检查都过了但候选帧吞吐更低,能宣布优化成功吗?","expected_kind":"none","base":[["case","case.sarathi-chunk-budget",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["record","sglang.common.llm.tuning_and_validation",true],["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true]],"candidate":[["case","case.sarathi-chunk-budget",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["record","sglang.common.llm.tuning_and_validation",true],["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true]]},{"query":"Qwen Image 910B1 BF16 eager 1280x704 interleaved RoPE VAE fastpath fused attention","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","video-diffusion-optimization",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","video-diffusion-optimization",true]]},{"query":"How should I isolate and then combine interleaved RoPE, VAE fastpath and fused attention when this build has not run the combination?","expected_kind":"skill","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","video-diffusion-optimization",true],["case","case.vllm-gdn-source-contract",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","video-diffusion-optimization",true],["case","case.vllm-gdn-source-contract",true]]},{"query":"910B1 RoPE VAE attention","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","video-diffusion-optimization",true],["record","vllm_omni.cuda.diffusion.minimax_h3_vae_stacked_tiling",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","video-diffusion-optimization",true],["record","vllm_omni.cuda.diffusion.minimax_h3_vae_stacked_tiling",true]]},{"query":"昇腾单卡上这些 RoPE、VAE 和注意力改动有没有在相同配置组合试过,速度和整图质量怎样?","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["skill","ascend-video-diffusion-optimization",true],["skill","ascend-model-adaptation",true],["skill","video-diffusion-optimization",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["skill","ascend-video-diffusion-optimization",true],["skill","ascend-model-adaptation",true],["skill","video-diffusion-optimization",true]]},{"query":"MiniMax H3 Image 4K BF16 single B300 six steps hot image latency","expected_kind":null,"base":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_adaln_schedule_cache",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_adaln_schedule_cache",true]]},{"query":"MiniMax H3 B300 diffusion VAE tile decode batching overlap memory","expected_kind":null,"base":[["record","vllm_omni.cuda.diffusion.minimax_h3_vae_stacked_tiling",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["record","vllm_omni.ascend.diffusion.minimax_h3_sp_head_buckets",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_vae_stacked_tiling",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["record","vllm_omni.ascend.diffusion.minimax_h3_sp_head_buckets",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true]]},{"query":"MiniMax H3 B300 packed queries keys RMS normalization rotary embeddings fusion","expected_kind":null,"base":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["skill","audit-h3-runtime-residency",true],["skill","audit-h3-metal-fasth3-semantics",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["skill","audit-h3-runtime-residency",true],["skill","audit-h3-metal-fasth3-semantics",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true]]},{"query":"diffusion T2I redundant conditioning across denoising steps BF16 prompt reuse","expected_kind":null,"base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","audit-h3-runtime-residency",true],["skill","ascend-video-diffusion-optimization",true],["skill","local-dream",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","audit-h3-runtime-residency",true],["skill","ascend-video-diffusion-optimization",true],["skill","local-dream",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true]]},{"query":"MiniMax H3 denoising time modulation projections cache timestep schedule","expected_kind":null,"base":[["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["record","vllm_omni.cuda.diffusion.minimax_h3_adaln_schedule_cache",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true],["skill","audit-h3-metal-fasth3-semantics",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["record","vllm_omni.cuda.diffusion.minimax_h3_adaln_schedule_cache",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true],["skill","audit-h3-metal-fasth3-semantics",true]]},{"query":"inference benchmark design warm requests baseline latency quality control","expected_kind":null,"base":[["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["record","sglang.common.llm.tuning_and_validation",true],["case","case.sarathi-chunk-budget",true],["skill","ascendc-tiling-design",true]],"candidate":[["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["record","sglang.common.llm.tuning_and_validation",true],["case","case.sarathi-chunk-budget",true],["skill","ascendc-tiling-design",true]]},{"query":"Day zero port of an omni model with text and audio stages to vLLM Ascend NPU","expected_kind":null,"base":[["skill","ascend-model-adaptation",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["record","sglang_omni.common.audio.serving_and_streaming_quality",true],["record","vllm_omni.common.audio.streaming_serving_measurement",true],["skill","ascend-video-diffusion-optimization",true]],"candidate":[["skill","ascend-model-adaptation",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["record","sglang_omni.common.audio.serving_and_streaming_quality",true],["record","vllm_omni.common.audio.streaming_serving_measurement",true],["skill","ascend-video-diffusion-optimization",true]]},{"query":"Single GPU diffusion denoising latency baseline warm requests profiler","expected_kind":null,"base":[["skill","ascend-video-diffusion-optimization",true],["record","vllm_omni.common.diffusion.measurement_and_profiling",true],["skill","inference-benchmark-design",true],["skill","video-diffusion-optimization",true],["skill","host-overhead-optimization",true]],"candidate":[["skill","ascend-video-diffusion-optimization",true],["record","vllm_omni.common.diffusion.measurement_and_profiling",true],["skill","inference-benchmark-design",true],["skill","video-diffusion-optimization",true],["skill","host-overhead-optimization",true]]},{"query":"ROCm GPU hang during CUDA graph capture first failing operator minimal reproduction","expected_kind":null,"base":[["case","case.qwen-b300-rejected-directions",true],["skill","ascend-graph-mode",true],["record","pytorch.cuda.compiler.cudagraph_shapes_and_lifetimes",true],["record","flashinfer.cuda.llm.paged_decode_dispatch",true],["skill","ascend-aclgraph-capture-replay",true]],"candidate":[["case","case.qwen-b300-rejected-directions",true],["skill","ascend-graph-mode",true],["record","pytorch.cuda.compiler.cudagraph_shapes_and_lifetimes",true],["record","flashinfer.cuda.llm.paged_decode_dispatch",true],["skill","ascend-aclgraph-capture-replay",true]]},{"query":"TTS 新版本音质下降 对齐参考实现 逐层精度回归","expected_kind":null,"base":[["skill","inference-benchmark-design",true],["record","qwen3_tts_engine.ascend.audio.fixed_customvoice_prompt",true],["record","qwen3_tts_engine.ascend.audio.codec_constants_and_state",true],["record","qwen3_tts_engine.ascend.audio.graph_batching_state_ownership",true],["skill","torch-npu",true]],"candidate":[["skill","inference-benchmark-design",true],["record","qwen3_tts_engine.ascend.audio.fixed_customvoice_prompt",true],["record","qwen3_tts_engine.ascend.audio.codec_constants_and_state",true],["record","qwen3_tts_engine.ascend.audio.graph_batching_state_ownership",true],["skill","torch-npu",true]]},{"query":"Align vllm-omni vllm-ascend and afd-plugin versions after upgrading vLLM","expected_kind":null,"base":[["skill","vllm-ascend",true],["skill","vllm-metal",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true],["skill","ascend-model-adaptation",true],["skill","audit-afd-p2p-exchanges",true]],"candidate":[["skill","vllm-ascend",true],["skill","vllm-metal",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true],["skill","ascend-model-adaptation",true],["skill","audit-afd-p2p-exchanges",true]]},{"query":"Bring a vLLM forked inference engine up on DeepInfer and reuse scheduling components","expected_kind":null,"base":[["record","deepinfer.common.llm.zero_overhead_deferred_scheduling",true],["record","deepinfer.common.llm.bounded_windowed_replay",true],["skill","vllm-metal",true],["skill","host-overhead-optimization",true],["record","deepinfer.common.llm.speculative_verification_budget",true]],"candidate":[["record","deepinfer.common.llm.zero_overhead_deferred_scheduling",true],["record","deepinfer.common.llm.bounded_windowed_replay",true],["skill","vllm-metal",true],["skill","host-overhead-optimization",true],["record","deepinfer.common.llm.speculative_verification_budget",true]]},{"query":"Build an independent inference engine for an on-device model on Windows XPU","expected_kind":null,"base":[["skill","inference-benchmark-design",true],["skill","project-maya",true],["skill","weight-loading-optimization",true],["record","strata.common.llm.engine_validation_contract",true],["skill","cann-attribution-overlap",true]],"candidate":[["skill","inference-benchmark-design",true],["skill","project-maya",true],["skill","weight-loading-optimization",true],["record","strata.common.llm.engine_validation_contract",true],["skill","cann-attribution-overlap",true]]}]}Final-head independent replay. Reran the identical protocol on submitted head Final-head replay data (rankings identical to the complete table above){"revisions":{"base":"a3e33851981f1ecdc8b73d7b62633cc598e5d5ae","candidate":"d03bbb70cc331e3e5eb82779301ebfe060ef0ed0"},"platform":"Windows","python":"3.12.6","gate":{"match_all_ordered_top5":true,"max_lost_readable_hits":0,"max_candidate_to_base_median_cli_latency_ratio":1.0},"passed":true,"quality":{"matching_ordered_top5":49,"queries":49,"lost_readable_hits":0,"base":{"hit_at_3_by_expected_kind":15,"labelled_positive":17,"empty_results_on_labelled_gap":0,"labelled_gap":19},"candidate":{"hit_at_3_by_expected_kind":15,"labelled_positive":17,"empty_results_on_labelled_gap":0,"labelled_gap":19}},"latency":{"arms":{"base":{"median_s":2.450632099993527,"p95_s":3.5098809000337496},"candidate":{"median_s":1.7620492999558337,"p95_s":2.4414378999499604}},"pooled_median_ratio":0.7190182891836306,"paired_block_bootstrap_95pct":[0.6938502291107025,0.7443015974743318]},"samples_s_base_candidate":{"day-0":[[2.369068400003016,1.8458788999123499],[2.403489400050603,1.788920999970287],[2.395974399987608,1.8003886999795213],[2.395831499947235,1.841221599956043]],"performance":[[2.838575500063598,1.7266202999744564],[3.0498232000973076,2.241837800014764],[2.8669772000284865,1.719941999996081],[2.8060538000427186,1.8455796999624]],"debug":[[2.177445499924943,1.4933963000075892],[2.300707899965346,1.732280099997297],[2.500795599888079,1.4517854999285191],[2.20038390008267,1.538377699907869]],"accuracy":[[2.156967499991879,1.3902248999802396],[2.571789300069213,2.958849100046791],[2.236723499954678,1.3866061998996884],[2.157224400085397,1.3705458999611437]],"version-alignment":[[2.497774799936451,1.8301607000175864],[2.9632743999827653,2.4414378999499604],[2.6574457000242546,1.7351775999413803],[2.641268100007437,1.9267131000524387]],"derived-engine":[[1.933899600058794,1.3395896999863908],[1.7875837000319734,1.365114500047639],[2.041042200056836,1.3364445000188425],[1.7456860999809578,1.3440201999619603]],"independent-engine":[[4.231474600033835,2.2838791999965906],[3.1375084000173956,2.2027933000354096],[3.0638347000349313,2.2682601999258623],[3.5098809000337496,2.3110838000429794]]},"rankings_compact_sha256":"c1644ef09dab59134b3d6a8e9b20dd2bc3fd944ad8b0b71c4aaf7945930eed72"} |
Signed-off-by: David Chen <530634352@qq.com>
…ents Signed-off-by: David Chen <530634352@qq.com>
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
增量检视 ebe5e6836c8269e3cea9e82821c912929bd1c960 → d03bbb70cc331e3e5eb82779301ebfe060ef0ed0,承接上轮检视。用完整 Git 树核对了 304 个变动的 blob 路径,补齐 compare 接口省略的四个工具文件。107 个独立 metadata.json 的结构化字段逐项比对一致;剥离新元数据块后,104 个原正文逐字不变,另三处为文档调整。已读元数据解析、捕获范围与哈希生成、编译 pin、离线比较器及回归测试。上一轮来源 pin 修复仍然成立。
首轮实测候选与本 head 只差五个测试文件。复算作者公开的两组数据,离线 Find 中位耗时比为 0.7005 和 0.7190;首轮 49 条排序一致。这支持限定环境的回归门槛,不代表 Agent 任务收益或服务性能提升。B300 资料保留在上一轮固定版本,不作为本轮验收替代。
批准前必做:无。本轮做静态代码、数据及完整文件树比对,未执行代码、测试或平台实验。
|
@amy-why-3459 @Dong1017 ptal thx |
Dong1017
left a comment
There was a problem hiding this comment.
PR review verdict — DO NOT PASS
Reviewed base a3e33851981f1ecdc8b73d7b62633cc598e5d5ae and head d03bbb70cc331e3e5eb82779301ebfe060ef0ed0. This is an evidence-limited result, not a defect finding. I'm withholding an Approve because I could not see enough of the change.
Why I'm not passing
- The diff I could review is incomplete. The seeded diff is assembled from per-file patches and is cut off partway through the 1095 changed files (
docs/skill-layout-migration.csv). GitHub returned no patch fordocs/reference-matrix.md,docs/searchspace-matrix.mdanddocs/skill-coverage.md. I saw none of the code and test changes:catalog.py,_skill_view.py,_skill_selection.py,knowledge_find.py,tools/build_skill_index.py,tools/skill_eval_pr.py,tools/compare_find_layouts.py, the migratedSKILL.mdfiles,_compiled/*,records/*, or the plugin entrypoint and tests. The claims that matter most here (typedmetadata.inferrsiparity, byte ranges and digests, the exact-read validation scope, and primary skills excluded from Find) are in those unseen files. - The CHANGES_REQUESTED from @cursor[bot] is still active. It was filed on
5a8c98cd. The author's later comments say the three items were fixed (campaign-plan link,provenance_readsets, undeclared skill reference). I could not confirm that from the visible diff. I also have no current-head CI result for the Ready full-check. - The skill-eval bot reports PLANNED, not executed. It lists no verified ancestor source pin for automatic command selection and no selected comparisons. It lists 1790 uncovered paths and a blocked paid-execution budget. The author supplies manual evidence in the PR body in place of those checks.
What I did check
- @zengchuang-hw has an APPROVED review on this exact head. It reports a static full-tree comparison with 107
metadata.jsonobjects matching field by field and 104 original bodies unchanged. I weigh that as independent evidence, but I can't reproduce it from what I was given. - The visible diff is consistent with the PR description. Live links move to the new
systems/,playbooks/,workflows/andcasebooks/paths. The docs describe theskills enterprimary-first flow. The layout docs carry the 135-owner Find scope, and the contribution workflow documents the single-file authoring model. - The author reports an ordered top-five match on 49/49 queries across two runs. Find p50 latency ratios were 0.7005 and 0.7190. The author describes the final head as the same as the measured candidate apart from five test files. I did not rerun the comparator.
Not verified here
- I ran no tests, wheel build, comparator or CI.
- I did not audit the knowledge-model-policy protected paths against the moved implementation.
- Agent task-outcome benefit stays INCONCLUSIVE, as the PR itself says.
To clear this
- Provide a way to see the complete diff, or point me at the code, test and tool files that matter. A re-review on the next push or a fresh @Dong1017 ping would let me cover them.
- Show green current-head CI, or confirm that @cursor[bot]'s three items are resolved.
None of this contradicts the existing Approve, and I found no concrete defect in the visible portion. I'll re-read when the author pushes or asks again.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base a3e33851981f1ecdc8b73d7b62633cc598e5d5ae and head d03bbb70cc331e3e5eb82779301ebfe060ef0ed0.
Clears prior cursor Request changes on 5a8c98cd… (5970763702). Re-checked those blockers on tip:
evidence/_template/plan/campaign-plan.mdnow points atplaybooks/measurement/performance-ceiling-analysis/references/acceptance-criteria.md(path exists).- No remaining
metadata.jsonwithoutprovenance_readsets. - Skill refs use
casebooks/inference/...(no undeclared../inference-optimization-casebook/...).
Also verified author follow-ups for exact-read isolation / OS metadata / root-README boundary / search-parity scope narrowing. Local targeted knowledge suite: 123 passed, 2 skipped. Ready green on tip including declarations + full-check 3.10/3.11 (44525067478, declarations 44526425704).
Eligibility: technical/third-party verify only (permission none). Not write+ eligible / dual-tip / merge-ready. Merge left to Dong always-on / write+ humans (zengchuang-hw tip Approve present; Dong1017 tip COMMENT noted evidence-limited non-Approve — steward/write+ still owns merge).
Holding formal Approve per user direction (2026-11-19): large PR; another agent review still in progress. Prior cursor Approve should not be treated as current steward/verify signal.
There was a problem hiding this comment.
Holding formal Approve on this PR (user direction, 2026-11-19).
- Do not treat the prior cursor[bot] Approve as a current steward/verify signal.
- That review has been dismissed: #516 (review)
- Reason: PR is large; another agent review is still in progress.
- This is a hold, not Request changes — no new blocker found in this note.
- Formal cursor Approve stays on hold until user/agent review finishes.
Technical/third-party note only; collaborator permission none — not a write+ eligible vote under docs/agent-collaboration.md.
|
Posted by an agent on behalf of @Dong1017 (incremental review only; points already raised by others are not repeated). DO NOT PASS — reviewed base The layout decision, the migration manifest, the stable-ID retention and the search-parity result all hold up under my own replay: ten free-text queries return byte-identical ordered top-fives on the base and on this head, Blocking1. The single-file decision inflated the delivered The authoring change moved every For This is the same class of regression the PR already treats as a gate: the Collect case in the description dropped from 125,623 back to 77,206 bytes once machine metadata stopped counting as required delivery. The 2.
Issue #500 §3 defines L0 as “one line plus coordinates plus evidence level plus next-layer size”. This PR's hit envelope is neither that nor the previous 400-character card: there is still no next-layer size field, and one field is now large enough to dominate the result. Please cap Verify before the layout is declared final3. Twenty-two procedure files lost a top-level host frontmatter key.
The PR states the annotation is “string-valued” and that hosting semantics were raised in review, so this may be a decision I cannot see the record of. If it is intentional, please state where a consumer of Relation to issue #500 (§3 layering and §4 search space)#500 and #516 are complementary and I am not asking for L0/L1 here; the maintainer decision already recorded in #500 makes #515 a baseline dependency, not a parent. Three facts from the comparison are still worth pinning, because they set what #500 has to pay for:
Non-blocking
Scope checkedFull tree at Ran: ten free-text Not run: Windows, any device or accelerator, the author's timing protocol, wheel build, full suite, and CI. Four |
Signed-off-by: David Chen <530634352@qq.com>
…reads Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
|
Addressed the latest incremental review on
Current-head search replay. On Windows with Python 3.12.6, all 49 ordered top-fives matched current main and all 245 hits retained read commands. The seven task families each had four paired fresh-process Find measurements. Pooled median latency was 2.028 → 1.478 s (27.13% lower), p95 was 2.996 → 2.576 s, and 27/28 pairs were faster. The descriptive paired bootstrap interval for the median ratio was [0.7160, 0.7472]. Both arms put the expected material kind in the first three on 15/17 positive questions and returned some material on all 19 labelled evidence gaps. Only the pinned baseline SHA changed in the replay fixture. The first post-fix run on 30e69b5 (same source, tests and workload; the two later changes are documentation only) matched all 49 rankings and measured 2.522 → 1.733 s median, 4.649 → 2.852 s p95, 27/28 faster. The two rounds are reported separately. Claude Code cross-review. Claude Code (Fable 5.2, Extra) challenged nested ownership, snapshot drift, the raw fallback and compatibility imports. We checked the 151 source files against their Git blobs, reconstructed the projected bytes under both newline conventions and verified that Markdown comments and fenced examples survive. The four old workflow exports resolve to the same function objects as the new implementation. The follow-up review accepted these findings and requested an explicit size-estimate contract and a mutable-checkout regression. We documented the distinction, verified that the CLI only displays the hint, and added a test that edits the source after Find and successfully reads the longer content. The complete seven-skill response median remains 374 bytes (2.76%) above the pre-refactor median, including the new proof and migrated paths. This discussion covered the engineering evidence; formal approval belongs to the maintainers. Verification on the submitted tip. All 333 targeted local regressions passed. The Windows run used a short temporary directory for the deep copy fixture. Both Python 3.10 and 3.11 full checks passed, along with installed-package, dashboard, host-compatibility and draft checks. Ready-lane CI for this commit. Issue and diagrams · Current implementation and diagrams · Checks on the submitted head Replication commands and exact timing / ranking dataRun from the InferRSI repository in a clean environment with Python 3.12 and Git 2.45 or later. The original run used Python 3.12.6 and Git 2.46.0 on Windows. The following PowerShell commands place the two worktrees and the output outside the repository. $pilot = New-Item -ItemType Directory -Path ../find-516-replay
$before = '493416520f537590eb8f2a4e74f276f82e488307'
$after = '7c199747d4c2820a65fbc20b51eba8e33db43668'
git worktree add --detach "$pilot/main" $before
git worktree add --detach "$pilot/head" $after
Set-Location $pilot
@'
from pathlib import Path
for rel in ("tools/compare_find_layouts.py", "tools/git_read.py",
"tests/fixtures/find/labelled-queries.json", "tests/fixtures/find/layout-queries.json"):
data = (Path("head") / rel).read_bytes()
if rel.endswith("layout-queries.json"):
original = b"a3e33851981f1ecdc8b73d7b62633cc598e5d5ae"
assert data.count(original) == 1
data = data.replace(original, b"493416520f537590eb8f2a4e74f276f82e488307")
target = Path("current-main-protocol") / rel
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(data)
'@ | python -
python current-main-protocol/tools/compare_find_layouts.py --base main --candidate head --candidate-sha $after --output compare-4934165-7c19974.jsonThe two arms use the same 49 ranking probes, the same 7 timed task prompts and all 28 paired CLI timings. The JSON report contains the read availability and ordering for each top five as well as every elapsed time. Both worktrees must pass the comparator's clean/pin checks before warmup and after timing. Relative to the committed protocol, only the expected baseline SHA changes. Main advanced through #512 and #517; its knowledge tree and externally labelled fixture retained their Git contents. SHA-256 of compact UTF-8 JSON below (without a trailing newline): {"revisions":{"base":"493416520f537590eb8f2a4e74f276f82e488307","candidate":"7c199747d4c2820a65fbc20b51eba8e33db43668"},"platform":"Windows","python":"3.12.6","gate":{"match_all_ordered_top5":true,"max_lost_readable_hits":0,"max_candidate_to_base_median_cli_latency_ratio":1.0},"passed":true,"quality":{"matching_ordered_top5":49,"queries":49,"lost_readable_hits":0,"base":{"hit_at_3_by_expected_kind":15,"labelled_positive":17,"empty_results_on_labelled_gap":0,"labelled_gap":19},"candidate":{"hit_at_3_by_expected_kind":15,"labelled_positive":17,"empty_results_on_labelled_gap":0,"labelled_gap":19}},"latency":{"arms":{"base":{"median_s":2.0283780000172555,"p95_s":2.995844900025986},"candidate":{"median_s":1.477994300017599,"p95_s":2.575650499900803}},"pooled_median_ratio":0.7286582185396536,"paired_block_bootstrap_95pct":[0.7160119665454457,0.7471776628743312]},"samples_s_base_candidate":{"day-0":[[2.389305099961348,1.575170399970375],[1.9635412000352517,1.5117092999862507],[2.0799075999530032,1.4702238000463694],[1.9768484000815079,1.4597122999839485]],"performance":[[3.1685649999417365,2.6753507999237627],[2.3102471999591216,1.4140096000628546],[2.492789200041443,1.6347440000390634],[2.2768295999849215,1.5585590000264347]],"debug":[[1.8541565000778064,1.2197818000568077],[1.758906599949114,1.224371699965559],[1.7480468999128789,1.6097693999763578],[1.7672519999323413,1.2622252999572083]],"accuracy":[[1.775253799976781,1.2390348999761045],[1.7621616000542417,1.149778800085187],[1.6825041000265628,1.4603548999875784],[1.8034555000485852,1.172932600020431]],"version-alignment":[[2.116836000001058,1.48676300002262],[2.1841870000353083,1.5370573999825865],[2.1648655000608414,1.4786319000413641],[2.1861760000465438,1.477356699993834]],"derived-engine":[[1.4739426999585703,1.1661527999676764],[1.3866616000887007,1.0947862999746576],[1.4667155999923125,1.5661349999718368],[1.5003457999555394,1.1853340999223292]],"independent-engine":[[2.597824100055732,1.8522548000328243],[2.995844900025986,1.9332707000430673],[2.4847789000486955,1.8622851999243721],[2.6658460999606177,2.575650499900803]]},"rankings":[{"query":"BF16 QK normalization complex RoPE fusion numerical order","expected_kind":"record","base":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.qwen-b300-rejected-directions",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.qwen-b300-rejected-directions",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true]]},{"query":"Has Qwen Image QK normalization and complex RoPE fusion been tried on four B300s, and what happened to speed and full-image quality?","expected_kind":"case","base":[["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","inference-benchmark-design",true]],"candidate":[["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","inference-benchmark-design",true]]},{"query":"QK norm RoPE B300","expected_kind":"case","base":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.admission-drain-occupancy",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.admission-drain-occupancy",true]]},{"query":"图像生成里把 QK 归一化和旋转位置编码融合后,如何验证算子、整图质量和端到端时延?","expected_kind":"skill","base":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","kernel-writing-guide",true],["skill","vllm-ascend",true],["skill","ascend-operator-profiling",true],["record","triton.common.kernel.bandwidth_fusion_resource_limits",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","kernel-writing-guide",true],["skill","vllm-ascend",true],["skill","ascend-operator-profiling",true],["record","triton.common.kernel.bandwidth_fusion_resource_limits",true]]},{"query":"Has RoPE cosine and sine caching been tried for Qwen Image on one Ascend 910B1 with BF16 eager execution, and what happened?","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","ascend-graph-mode",true],["skill","ascend-model-adaptation",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","ascend-graph-mode",true],["skill","ascend-model-adaptation",true]]},{"query":"Qwen Image single 910B1 BF16 eager eight-step RoPE cosine sine cache","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.drama3-rejected-directions",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.drama3-rejected-directions",true]]},{"query":"910B1 RoPE cache benchmark","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","inference-benchmark-design",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","inference-benchmark-design",true]]},{"query":"四卡 B300 上的融合经验能直接推断单卡 910B1 缓存三角值的效果吗?","expected_kind":"none","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","vllm-ascend",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.drama3-rejected-directions",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","vllm-ascend",true],["case","case.qwen-ascend-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["case","case.drama3-rejected-directions",true]]},{"query":"operator test passes but image output fails the confirmed quality gate","expected_kind":"none","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","ascend-operator-tuning",true],["skill","ascend-operator-profiling",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","ascend-operator-tuning",true],["skill","ascend-operator-profiling",true]]},{"query":"Can a candidate that failed the confirmed full-image quality gate be accepted because its operator test passed?","expected_kind":"none","base":[["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","ascend-operator-tuning",true]],"candidate":[["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","ascend-operator-tuning",true]]},{"query":"image quality gate failure","expected_kind":"none","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["case","case.drama3-rejected-directions",true],["skill","audit-sd-cpp-residency",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["case","case.drama3-rejected-directions",true],["skill","audit-sd-cpp-residency",true]]},{"query":"算子测过了但整图验收没过,这个版本还能入选吗,接下来怎么办?","expected_kind":"none","base":[["skill","kernel-writing-guide",true],["skill","ascend-operator-tuning",true],["skill","ascend-operator-profiling",true],["skill","ascendc-tiling-design",true],["case","case.drama3-rejected-directions",true]],"candidate":[["skill","kernel-writing-guide",true],["skill","ascend-operator-tuning",true],["skill","ascend-operator-profiling",true],["skill","ascendc-tiling-design",true],["case","case.drama3-rejected-directions",true]]},{"query":"continuously fed service has low full-run average decode batch below C64","expected_kind":"case","base":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.qwen-b300-rejected-directions",true]],"candidate":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.qwen-b300-rejected-directions",true]]},{"query":"Does a low full-run average decode batch prove that the active batch stayed far below 64 while requests were still being admitted?","expected_kind":"none","base":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["case","case.vllm-gdn-source-contract",true],["case","case.sglang-gdn-source-contract",true]],"candidate":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["case","case.vllm-gdn-source-contract",true],["case","case.sglang-gdn-source-contract",true]]},{"query":"decode batch drain occupancy","expected_kind":"case","base":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.vllm-gdn-source-contract",true],["case","case.sarathi-chunk-budget",true]],"candidate":[["case","case.admission-drain-occupancy",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.vllm-gdn-source-contract",true],["case","case.sarathi-chunk-budget",true]]},{"query":"怎么分开统计持续接收请求和尾部排空时的活跃批量,同时检查整个请求集的吞吐?","expected_kind":"skill","base":[["case","case.admission-drain-occupancy",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["record","qwen3_tts_engine.ascend.audio.graph_batching_state_ownership",true]],"candidate":[["case","case.admission-drain-occupancy",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.drama3-rejected-directions",true],["case","case.sarathi-chunk-budget",true],["record","qwen3_tts_engine.ascend.audio.graph_batching_state_ownership",true]]},{"query":"fused kernel passed unit checks but the finished image failed the verified acceptance test","expected_kind":"none","base":[["case","case.qwen-b300-rejected-directions",true],["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","kernel-writing-guide",true]],"candidate":[["case","case.qwen-b300-rejected-directions",true],["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","kernel-writing-guide",true]]},{"query":"Should I keep an operator-tested fusion as accepted when its confirmed full-image quality check failed?","expected_kind":"none","base":[["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true]],"candidate":[["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.drama3-rejected-directions",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true]]},{"query":"fusion quality acceptance failure","expected_kind":"none","base":[["case","case.qwen-ascend-rejected-directions",true],["skill","draft-acceptance-issue",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["skill","inference-benchmark-design",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true]],"candidate":[["case","case.qwen-ascend-rejected-directions",true],["skill","draft-acceptance-issue",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["skill","inference-benchmark-design",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true]]},{"query":"融合版本的单算子测试合格,最终图片却没达到验收标准,该如何处理?","expected_kind":"none","base":[["skill","kernel-writing-guide",true],["case","case.drama3-rejected-directions",true],["skill","vllm-ascend",true],["skill","ascend-operator-tuning",true],["case","case.qwen-ascend-rejected-directions",true]],"candidate":[["skill","kernel-writing-guide",true],["case","case.drama3-rejected-directions",true],["skill","vllm-ascend",true],["skill","ascend-operator-tuning",true],["case","case.qwen-ascend-rejected-directions",true]]},{"query":"candidate ran at 360 ms with diagnostics running versus an old 277 ms result on a quiet host","expected_kind":"none","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-b300-rejected-directions",true],["skill","assess-serving-preemption",true],["skill","inference-benchmark-design",true],["skill","torchnpu-pinned-host-memory-lifecycle",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-b300-rejected-directions",true],["skill","assess-serving-preemption",true],["skill","inference-benchmark-design",true],["skill","torchnpu-pinned-host-memory-lifecycle",true]]},{"query":"Do unpaired latency measurements under different host loads establish that a code change caused a regression?","expected_kind":"none","base":[["case","case.qwen-b300-rejected-directions",true],["skill","weight-loading-optimization",true],["case","case.qwen-ascend-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","host-overhead-optimization",true]],"candidate":[["case","case.qwen-b300-rejected-directions",true],["skill","weight-loading-optimization",true],["case","case.qwen-ascend-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","host-overhead-optimization",true]]},{"query":"unpaired latency host contention","expected_kind":"none","base":[["skill","torchnpu-pinned-host-memory-lifecycle",true],["case","case.qwen-ascend-rejected-directions",true],["skill","host-overhead-profiling",true],["skill","host-memory-management",true],["skill","host-overhead-optimization",true]],"candidate":[["skill","torchnpu-pinned-host-memory-lifecycle",true],["case","case.qwen-ascend-rejected-directions",true],["skill","host-overhead-profiling",true],["skill","host-memory-management",true],["skill","host-overhead-optimization",true]]},{"query":"主机忙闲不一致时,旧基线与新改动的耗时差能归因于代码吗?","expected_kind":"none","base":[["skill","draft-acceptance-issue",true],["skill","ascend-device-health",true],["skill","ascend-operator-tuning",true],["skill","inference-benchmark-design",true],["record","vllm_omni.common.diffusion.measurement_and_profiling",true]],"candidate":[["skill","draft-acceptance-issue",true],["skill","ascend-device-health",true],["skill","ascend-operator-tuning",true],["skill","inference-benchmark-design",true],["record","vllm_omni.common.diffusion.measurement_and_profiling",true]]},{"query":"fixed-request check passes but baseline and async candidate both fail speech-recognition quality in full service","expected_kind":"case","base":[["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["skill","audit-fixed-workload-claim",true],["case","case.sarathi-chunk-budget",true]],"candidate":[["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.qwen-b300-rejected-directions",true],["skill","audit-fixed-workload-claim",true],["case","case.sarathi-chunk-budget",true]]},{"query":"If full-service speech recognition fails for both baseline and candidate, does that prove async overlap caused the failure?","expected_kind":"none","base":[["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","cann-attribution-overlap",true],["skill","dcp-async-save-performance",true],["skill","assess-serving-preemption",true]],"candidate":[["case","case.drama3-rejected-directions",true],["skill","inference-benchmark-design",true],["skill","cann-attribution-overlap",true],["skill","dcp-async-save-performance",true],["skill","assess-serving-preemption",true]]},{"query":"ASR baseline candidate failure","expected_kind":"skill","base":[["skill","inference-benchmark-design",true],["skill","draft-acceptance-issue",true],["skill","optimization-search-policy",true],["skill","ascend-device-health",true],["skill","ascend-operator-tuning",true]],"candidate":[["skill","inference-benchmark-design",true],["skill","draft-acceptance-issue",true],["skill","optimization-search-policy",true],["skill","ascend-device-health",true],["skill","ascend-operator-tuning",true]]},{"query":"异步服务的基线和候选语音识别都不达标,怎样用最少对照排查共同问题?","expected_kind":"skill","base":[["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["skill","assess-serving-preemption",true],["record","sglang_omni.common.audio.serving_and_streaming_quality",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true]],"candidate":[["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["skill","assess-serving-preemption",true],["record","sglang_omni.common.audio.serving_and_streaming_quality",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true]]},{"query":"quality checks passed but candidate service is 4307 frames/s versus 4371 baseline with longer drain","expected_kind":"none","base":[["case","case.drama3-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.admission-drain-occupancy",true],["case","case.sarathi-chunk-budget",true]],"candidate":[["case","case.drama3-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","inference-benchmark-design",true],["case","case.admission-drain-occupancy",true],["case","case.sarathi-chunk-budget",true]]},{"query":"Can I claim a full-service throughput improvement when the candidate measured 4307 frames/s and the baseline 4371 frames/s in one run?","expected_kind":"none","base":[["skill","inference-benchmark-design",true],["case","case.sarathi-chunk-budget",true],["case","case.drama3-rejected-directions",true],["skill","audit-fixed-workload-claim",true],["case","case.qwen-b300-rejected-directions",true]],"candidate":[["skill","inference-benchmark-design",true],["case","case.sarathi-chunk-budget",true],["case","case.drama3-rejected-directions",true],["skill","audit-fixed-workload-claim",true],["case","case.qwen-b300-rejected-directions",true]]},{"query":"throughput comparison drain mismatch","expected_kind":"none","base":[["case","case.drama3-rejected-directions",true],["case","case.admission-drain-occupancy",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.sarathi-chunk-budget",true],["skill","dense-performance-evidence",true]],"candidate":[["case","case.drama3-rejected-directions",true],["case","case.admission-drain-occupancy",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["case","case.sarathi-chunk-budget",true],["skill","dense-performance-evidence",true]]},{"query":"两边语音质量检查都过了但候选帧吞吐更低,能宣布优化成功吗?","expected_kind":"none","base":[["case","case.sarathi-chunk-budget",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["record","sglang.common.llm.tuning_and_validation",true],["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true]],"candidate":[["case","case.sarathi-chunk-budget",true],["record","vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap",true],["record","sglang.common.llm.tuning_and_validation",true],["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true]]},{"query":"Qwen Image 910B1 BF16 eager 1280x704 interleaved RoPE VAE fastpath fused attention","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","video-diffusion-optimization",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","video-diffusion-optimization",true]]},{"query":"How should I isolate and then combine interleaved RoPE, VAE fastpath and fused attention when this build has not run the combination?","expected_kind":"skill","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","video-diffusion-optimization",true],["case","case.vllm-gdn-source-contract",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","video-diffusion-optimization",true],["case","case.vllm-gdn-source-contract",true]]},{"query":"910B1 RoPE VAE attention","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","video-diffusion-optimization",true],["record","vllm_omni.cuda.diffusion.minimax_h3_vae_stacked_tiling",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["case","case.qwen-b300-rejected-directions",true],["skill","video-diffusion-optimization",true],["record","vllm_omni.cuda.diffusion.minimax_h3_vae_stacked_tiling",true]]},{"query":"昇腾单卡上这些 RoPE、VAE 和注意力改动有没有在相同配置组合试过,速度和整图质量怎样?","expected_kind":"case","base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["skill","ascend-video-diffusion-optimization",true],["skill","ascend-model-adaptation",true],["skill","video-diffusion-optimization",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["case","case.qwen-ascend-rejected-directions",true],["skill","ascend-video-diffusion-optimization",true],["skill","ascend-model-adaptation",true],["skill","video-diffusion-optimization",true]]},{"query":"MiniMax H3 Image 4K BF16 single B300 six steps hot image latency","expected_kind":null,"base":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_adaln_schedule_cache",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true],["case","case.qwen-b300-rejected-directions",true],["record","vllm_omni.cuda.diffusion.minimax_h3_adaln_schedule_cache",true]]},{"query":"MiniMax H3 B300 diffusion VAE tile decode batching overlap memory","expected_kind":null,"base":[["record","vllm_omni.cuda.diffusion.minimax_h3_vae_stacked_tiling",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["record","vllm_omni.ascend.diffusion.minimax_h3_sp_head_buckets",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_vae_stacked_tiling",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["record","vllm_omni.ascend.diffusion.minimax_h3_sp_head_buckets",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true]]},{"query":"MiniMax H3 B300 packed queries keys RMS normalization rotary embeddings fusion","expected_kind":null,"base":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["skill","audit-h3-runtime-residency",true],["skill","audit-h3-metal-fasth3-semantics",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope",true],["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["skill","audit-h3-runtime-residency",true],["skill","audit-h3-metal-fasth3-semantics",true],["record","vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache",true]]},{"query":"diffusion T2I redundant conditioning across denoising steps BF16 prompt reuse","expected_kind":null,"base":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","audit-h3-runtime-residency",true],["skill","ascend-video-diffusion-optimization",true],["skill","local-dream",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true]],"candidate":[["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["skill","audit-h3-runtime-residency",true],["skill","ascend-video-diffusion-optimization",true],["skill","local-dream",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true]]},{"query":"MiniMax H3 denoising time modulation projections cache timestep schedule","expected_kind":null,"base":[["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["record","vllm_omni.cuda.diffusion.minimax_h3_adaln_schedule_cache",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true],["skill","audit-h3-metal-fasth3-semantics",true]],"candidate":[["record","vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation",true],["record","vllm_omni.cuda.diffusion.minimax_h3_adaln_schedule_cache",true],["skill","audit-h3-runtime-residency",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true],["skill","audit-h3-metal-fasth3-semantics",true]]},{"query":"inference benchmark design warm requests baseline latency quality control","expected_kind":null,"base":[["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["record","sglang.common.llm.tuning_and_validation",true],["case","case.sarathi-chunk-budget",true],["skill","ascendc-tiling-design",true]],"candidate":[["skill","inference-benchmark-design",true],["case","case.drama3-rejected-directions",true],["record","sglang.common.llm.tuning_and_validation",true],["case","case.sarathi-chunk-budget",true],["skill","ascendc-tiling-design",true]]},{"query":"Day zero port of an omni model with text and audio stages to vLLM Ascend NPU","expected_kind":null,"base":[["skill","ascend-model-adaptation",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["record","sglang_omni.common.audio.serving_and_streaming_quality",true],["record","vllm_omni.common.audio.streaming_serving_measurement",true],["skill","ascend-video-diffusion-optimization",true]],"candidate":[["skill","ascend-model-adaptation",true],["record","vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i",true],["record","sglang_omni.common.audio.serving_and_streaming_quality",true],["record","vllm_omni.common.audio.streaming_serving_measurement",true],["skill","ascend-video-diffusion-optimization",true]]},{"query":"Single GPU diffusion denoising latency baseline warm requests profiler","expected_kind":null,"base":[["skill","ascend-video-diffusion-optimization",true],["record","vllm_omni.common.diffusion.measurement_and_profiling",true],["skill","inference-benchmark-design",true],["skill","video-diffusion-optimization",true],["skill","host-overhead-optimization",true]],"candidate":[["skill","ascend-video-diffusion-optimization",true],["record","vllm_omni.common.diffusion.measurement_and_profiling",true],["skill","inference-benchmark-design",true],["skill","video-diffusion-optimization",true],["skill","host-overhead-optimization",true]]},{"query":"ROCm GPU hang during CUDA graph capture first failing operator minimal reproduction","expected_kind":null,"base":[["case","case.qwen-b300-rejected-directions",true],["skill","ascend-graph-mode",true],["record","pytorch.cuda.compiler.cudagraph_shapes_and_lifetimes",true],["record","flashinfer.cuda.llm.paged_decode_dispatch",true],["skill","ascend-aclgraph-capture-replay",true]],"candidate":[["case","case.qwen-b300-rejected-directions",true],["skill","ascend-graph-mode",true],["record","pytorch.cuda.compiler.cudagraph_shapes_and_lifetimes",true],["record","flashinfer.cuda.llm.paged_decode_dispatch",true],["skill","ascend-aclgraph-capture-replay",true]]},{"query":"TTS 新版本音质下降 对齐参考实现 逐层精度回归","expected_kind":null,"base":[["skill","inference-benchmark-design",true],["record","qwen3_tts_engine.ascend.audio.fixed_customvoice_prompt",true],["record","qwen3_tts_engine.ascend.audio.codec_constants_and_state",true],["record","qwen3_tts_engine.ascend.audio.graph_batching_state_ownership",true],["skill","torch-npu",true]],"candidate":[["skill","inference-benchmark-design",true],["record","qwen3_tts_engine.ascend.audio.fixed_customvoice_prompt",true],["record","qwen3_tts_engine.ascend.audio.codec_constants_and_state",true],["record","qwen3_tts_engine.ascend.audio.graph_batching_state_ownership",true],["skill","torch-npu",true]]},{"query":"Align vllm-omni vllm-ascend and afd-plugin versions after upgrading vLLM","expected_kind":null,"base":[["skill","vllm-ascend",true],["skill","vllm-metal",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true],["skill","ascend-model-adaptation",true],["skill","audit-afd-p2p-exchanges",true]],"candidate":[["skill","vllm-ascend",true],["skill","vllm-metal",true],["record","vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse",true],["skill","ascend-model-adaptation",true],["skill","audit-afd-p2p-exchanges",true]]},{"query":"Bring a vLLM forked inference engine up on DeepInfer and reuse scheduling components","expected_kind":null,"base":[["record","deepinfer.common.llm.zero_overhead_deferred_scheduling",true],["record","deepinfer.common.llm.bounded_windowed_replay",true],["skill","vllm-metal",true],["skill","host-overhead-optimization",true],["record","deepinfer.common.llm.speculative_verification_budget",true]],"candidate":[["record","deepinfer.common.llm.zero_overhead_deferred_scheduling",true],["record","deepinfer.common.llm.bounded_windowed_replay",true],["skill","vllm-metal",true],["skill","host-overhead-optimization",true],["record","deepinfer.common.llm.speculative_verification_budget",true]]},{"query":"Build an independent inference engine for an on-device model on Windows XPU","expected_kind":null,"base":[["skill","inference-benchmark-design",true],["skill","project-maya",true],["skill","weight-loading-optimization",true],["record","strata.common.llm.engine_validation_contract",true],["skill","cann-attribution-overlap",true]],"candidate":[["skill","inference-benchmark-design",true],["skill","project-maya",true],["skill","weight-loading-optimization",true],["record","strata.common.llm.engine_validation_contract",true],["skill","cann-attribution-overlap",true]]}]} |
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
增量 d03bbb70 → 1f74e343(完整 head 1f74e3435e65968815372d12fa4dfc8f5344ff7c),上轮检视。已核对换基线后实际改动:skill 的 --material 现在返回紧凑 procedure,保留源文件和投影的 SHA-256、字节区间;--resource 仍精确返回原文件。Find 增加返回字节数并限制 limitations;未装 YAML 时仍提供可验证的固定版本 --resource 回退,不把它误记成紧凑投影。测试定义覆盖 checkout/store 字节重建、双哈希及 catalog record 的原样读取。Git >=2.45 约束对齐已有 --no-lazy-fetch 用法。
批准前无必修项。非阻塞:后续可为最小安装补一个紧凑投影入口。目前批准的是代码变更;历史 Find 实测不当作本 head 重跑,真实任务收益另验。已静态核对代码、格式契约和测试定义;未执行测试、设备验证或查询 CI。
Dong1017
left a comment
There was a problem hiding this comment.
PR review verdict — DO NOT PASS
Reviewed base 493416520f537590eb8f2a4e74f276f82e488307 and head 1f74e3435e65968815372d12fa4dfc8f5344ff7c (incremental from my d03bbb70cc331e3e5eb82779301ebfe060ef0ed0 review, 5976785459). This is again an evidence-limited result, not a defect allegation against the design. I read the seeded body, thread and formal review state, and the portion of the change the host could seed. No tests, CI, Windows or device runs.
Blocking
1. I still cannot see the code, tests or tools this PR's claims rest on.
meta.json reports diffComplete: false, diffSource: files. The seeded patch is assembled per file and is cut inside docs/skill-layout-migration.csv, and three files carry no patch at all: docs/reference-matrix.md (+126/−126), docs/searchspace-matrix.md (+129/−129), docs/skill-coverage.md (+175/−170). Nothing under src/, tests/, tools/, plugins/, evidence/, records/ or _compiled/, and none of the 151 migrated SKILL.md files are visible to me.
Every load-bearing claim on this head lives exactly in the unseen set: the skill-procedure projection and source_byte_ranges/source_sha256 pair in knowledge_operations.py and _skill_selection.py; the limitations budget and next_read_content_bytes in knowledge_find.py; the 22-file host-key regression; the seven-primary entrypoint test; the Git ≥ 2.45 guards in tools/compare_find_layouts.py and tools/check_skill_layout_migration.py. I could not confirm that the tip is what the author describes, and I could not confirm the statement that 7c199747d4c2820a65fbc20b51eba8e33db43668 and 1f74e343 are byte-identical for runtime code, tools and fixtures. Per the review rules, required evidence insufficient → no PASS. A re-seed of the full diff (or the PR ref fetched and src tests tools reviewed) is all that is needed for this item.
2. docs/contributing.md keeps a root-directory rule this PR invalidates.
The hunk at the skill-library section still says, unchanged:
Keep supporting Markdown and JSON with their owner; the library root contains only
README.mdandskill-index.json.
That is now false for this layout: docs/knowledge-base.md's directory map and docs/knowledge-layout.md → "Directory contract" both place records/, config/, _compiled/ and skill-index.json at the library root. docs/architecture.md carried the identical sentence ("only README.md and skill-index.json remain at the library root") and this diff deletes/replaces it there, so the fix exists and was simply not applied to the contributor guide. Please update it — a contributor reading the guide will conclude config/ and _compiled/ are misplaced.
Prior findings, status at this head
Items 1–3 of my d03bbb70 review are now answered at the documentation-contract level, pending code confirmation under blocker 1:
- Read payload.
docs/knowledge-operations.mdanddocs/knowledge-layout.mdnow specify askill-procedurelocator that delivers the procedure plus host frontmatter withmetadata.inferrsiomitted from context,sha256over the delivered bytes,source_sha256over the full file, half-opensource_byte_ranges, and a pinned--resourceread that returns the original file including the annotation. That is the exclusion I asked for, and it matches the published 13,534 → 13,908 median and the stated 2.76% residual. - Find card size.
docs/knowledge-layout.mdanddocs/operations.mdnow cap limitations at three complete entries inside a 400-byte UTF-8 JSON list withlimitations_withheld, and definenext_read_content_bytesas content bytes at Find time, with the mutable-checkout caveat stated. That closes the uncapped-field finding. - Host frontmatter.
docs/knowledge-layout.mdnow records the decision: the 33 native-flag procedures keepdisable-model-invocationandmetadata.disco-rolein the YAML header, the 22 sidecar-only ones are decoded frommetadata.inferrsi, and promotion would change host invocation behavior. That is the statement I asked for; the asserting regression is in the unseen tests.
Internal consistency of the numbers checks out in the visible layer: 151 = 107 folded pairs (88 registered + 19 nested) + 44 unpaired; 135 evidence owners = 80 skills + 48 findings + 7 cases; catalog 136 = 88 skill views + 48 findings with --limit 136; the seven primaries and the router are excluded from Find before IDF.
Current-head review state (weighed, not disputed)
@zengchuang-hwAPPROVED at1f74e343(5986380777) — write+-eligible and scoped to exactly the material/Find/Git-guard/fallback items. I am not contradicting it; my hold is that I did not personally see the code, not that her approval is wrong.- No outstanding CHANGES_REQUESTED in the latest per-login decision set;
@cursor[bot]'s5a8c98cdRequest changes was cleared atd03bbb70. - The skill-eval bot is still PLANNED with no selected comparisons and "No verified ancestor source pin"; that is an unexecuted check, not a pass.
docs/knowledge-contribution-workflow.mdnow documents the pin limitation, which is honest but does not substitute for an executed result.
Not verified here
No test run, no CI query, no wheel build, no Windows/Git-2.43 reproduction, no device work. I did not re-examine the search-parity numbers beyond the stated SHA and scope limits.
I will re-read as soon as a push lands the contributing.md correction, or immediately if the full diff can be seeded. I will upgrade to PASS on this head or its successor once I can read the implementation and tests against these contracts.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
Implements #515. Current comparison base
493416520f537590eb8f2a4e74f276f82e488307, implementation1f74e3435e65968815372d12fa4dfc8f5344ff7c.Decision
Maintain each skill in one
SKILL.md. The former companion file carried tags, applicability, sources, recommendations and resource declarations. Removing those fields would change search and qualified reads. They now live under the standard string-valued host annotationmetadata.inferrsi; the string contains typed JSON. Existing host fields keep their original types and meanings. Authors update one file for a procedural or metadata change. The two compiled build inputs can be regenerated from the source commit.All 107 paired files from the preceding PR head were folded into their respective procedures (88 registered skills and 19 nested procedures). Main had 99 pairs before the eight task entries were introduced. The remaining 44 unpaired procedure files needed no conversion. The 48 files in
records/each own an independent engineering finding and have their own stable IDs. Cases, aliases, examples and third-party notices continue to serve their individual purposes.The main agent selects one of seven primary skills by the requested deliverable, opens its procedure with
skills enter, then looks for supporting evidence. Default Find searches 135 evidence owners. Hardware, modality, engine, OS, software revisions and workload are applicability conditions. Existing exact IDs and historical snapshots remain available according to the migration manifest. A migrated source file has a new content revision.Architecture and commands · Cutover manifest · Source-authoring workflow
November 20 review
Main moved the campaign implementation into
apps/workflows/campaign.py. The merge retains its original compatibility exports and saves both the projected digest and the source byte ranges in the new implementation. The single physical library, seven task entries, 48 authored findings and 135 searchable owners remain in place.limitations_withheldreports the omitted count. For the TTS regression query, the first hit now shows three limitations (344 bytes) and reports nine withheld.next_read_content_bytesmatched the executed read's 13,166 UTF-8 content bytes. No-PyYAML fallback reports the size of its advertised raw resource. Sizes describe the inspected snapshot and read mode. An unpinned checkout can change before the command runs.knowledge search --limit 136to list the current inventory.The three template path updates are the full
evidence/diff. Seven task owners remain directly addressable. The four diagrams below cover the logical layers, physical directory, search and delivery, and the migration from the old layout. The coordinate index and full L0/L1 design are tracked separately in #500.Current implementation replay. On Windows with Python 3.12.6, all 49 ordered top-fives matched current main and all 245 hits retained read commands. The seven task families each had four paired fresh-process Find measurements. Pooled median latency was 2.028 → 1.478 s (27.13% lower), p95 was 2.996 → 2.576 s, and 27/28 pairs were faster. The descriptive paired bootstrap interval for the median ratio was [0.7160, 0.7472]. Both arms put the expected material kind in the first three on 15/17 positive questions and returned some material on all 19 labelled evidence gaps. The measured SHA was
7c199747d4c2820a65fbc20b51eba8e33db43668; source, tools and benchmark fixtures have identical Git trees on the submitted tip1f74e3435e65968815372d12fa4dfc8f5344ff7c. Raw results and replication commandsIndependent discussion. Claude Code (Fable 5.2, Extra) challenged nested ownership, snapshot drift, the raw fallback and compatibility imports. We checked the 151 source files against their Git blobs, reconstructed the projected bytes under both newline conventions and verified that Markdown comments and fenced examples survive. The four old workflow exports resolve to the same function objects as the new implementation. The follow-up review accepted these findings and requested an explicit size-estimate contract and a mutable-checkout regression. We documented the distinction, verified that the CLI only displays the hint, and added a test that edits the source after Find and successfully reads the longer content. The complete seven-skill response median remains 374 bytes (2.76%) above the pre-refactor median, including the new proof and migrated paths. This discussion covered the engineering evidence; formal approval belongs to the maintainers.
For reviewers using a truncated web diff, fetch the PR ref and inspect the code with
git diff 493416520f537590eb8f2a4e74f276f82e488307 1f74e3435e65968815372d12fa4dfc8f5344ff7c -- src tests tools. Review comment · Material view · Find · Campaign · Focused regression testsSearch acceptance against main
Passed the frozen offline Find gate. Windows, Python 3.12.6, one interpreter, same-depth clean worktrees. The comparator refuses staged, unstaged and untracked changes before either warmup and checks both pins again after sampling. Exact Git commits were
a3e33851981f1ecdc8b73d7b62633cc598e5d5ae(main) andf6502a6d1c7c8e8cf56bf8b648f1493483d9c09a(candidate). The next historical head changed test assertions. The November 20 review subsequently changed material delivery and the Find card. The following numbers identify the exact earlier snapshots measured.The 49 queries include 36 existing labels, the six prior fixed queries and one finisher-oriented question for each of seven task families. Timing covers those seven questions with four alternating AB/BA pairs each. The machine's filesystem was warmed before recording; each measured command started a fresh Python process. The pooled p50 latency ratio was 0.7005 against the predeclared maximum 1.0000 (29.95% lower latency). The descriptive 95% task-stratified paired-block bootstrap interval was [0.6859, 0.7410]. The candidate was faster in 26 of 28 pairs. This is an end-to-end observation for the combined PR; the experiment does not isolate the contribution of each implementation change. All 19 evidence-gap questions still returned material; applicability needs separate evaluation.
Median CLI time by task (seconds)
November 19 independent replay on the previous head. Reran the identical protocol on submitted head
d03bbb70cc331e3e5eb82779301ebfe060ef0ed0with clean pre/post checks. All 49 ordered lists and read-command flags again match main and the first run. The second set of 28 pairs measured p50 2.451 → 1.762 s, p95 3.510 → 2.441 s. The p50 ratio was 0.7190 (28.10% lower, descriptive paired bootstrap interval [0.6939, 0.7443]); 27/28 pairs were faster. The two runs are reported separately. Published limits apply to this Windows host, Python and offline workload.Complete raw timings, all 49 ranked lists and exact gates
Frozen workload · Comparator
Directory contract
1. Task layers and evidence
flowchart TB Q["Inference goal at the main agent entrypoint"] --> P["Select and load ONE primary SKILL.md before evidence search"] P --> A["Day-0 model · performance · debugging · accuracy"] P --> B["Version alignment · derived engine · independent engine"] A & B --> R["Record actual vendor / device / backend / OS, model and stages, engine and revision,<br/>script and flags, benchmark and gates, diagnosis and allowed search space"] R --> D["Primary routine: observation → supporting skill → evidence to collect → next test"] D --> S["systems · 53<br/>platforms / frameworks / engines"] D --> M["playbooks · 17<br/>measurement / planning / optimization"] D --> W["workflows · 8<br/>research / governance / maintenance"] D --> K["casebooks · 2<br/>previous outcomes and retry conditions"] S & M & W & K --> V["Match the evidence to the actual run and test the goal"] V --> O["Gates met: cite artifacts and scope"] V --> N["Open question: run the smallest discriminating test or record the budget blocker"]2. One physical knowledge root
flowchart LR D["src/inferrsi/knowledge/data/"] --> L["skill_library/ · sole child"] L --> T["tasks/"] T --> P["7 primary owners: task-day-0, task-performance, task-debug, task-accuracy,<br/>task-version-alignment, task-derived-engine, task-independent-engine"] T --> H["inference-task-router/ · legacy card paths"] L --> S["80 supporting owners in systems/, playbooks/, workflows/, casebooks/"] P & H & S --> O["One SKILL.md owns procedure and typed fields · 88 registered IDs"] L --> F["records/<stable-id>.json · 48 authored findings"] L --> CFG["config/ · aliases and research profiles"] L --> REG["skill-index.json · stable ID → owner and backend"] L --> BUILD["_compiled/ · inputs.json and _provenance.json"] O & REG -.->|fingerprint resources and compile host headers at the source commit| BUILD3. Search paths and evidence delivery
flowchart TB Q["Main agent receives inference task"] --> P["Choose primary by outcome; skills enter loads its full procedure"] P --> R["Capture actual run, acceptance gates and search budget"] R --> D["Observation triggers a supporting question"] D -->|Known owner| I["Explicit read / resource / capture by ID and revision"] D -->|Unknown owner| F["Find lexical ranker · ranked Collect / campaign capture share selector"] D -->|Need catalog scope| C["knowledge search · filter by repo, backend, scenario or tier"] C -->|Query| F C -->|No query| B["Catalog by ID · 48 findings + 88 skill views<br/>default limit 10 · use --limit 136 for all"] F --> A["Rank the existing searchable fields and procedure text<br/>Exclude 7 primary skills and legacy router before IDF and limits<br/>default 135 evidence owners · 80 skills + 48 findings + 7 cases"] I -->|Authored finding| E["Validate selected record and its dependencies<br/>retain linked-library bundle and pin checks when skill documents are declared"] I -->|Other exact owner| V["Check revision, device, model, flags and measurement against this run"] A --> H["Find card · limited caveats and withheld count<br/>next_read_content_bytes measures content at Find time"] E & H & B --> V V --> Z["Read the procedure and structured metadata, or Collect within budget<br/>Pinned resource read returns original SKILL.md bytes"] Z --> X["Meets gates → cite reproducible evidence and scope"] V --> G["Gap → name missing evidence; test the next distinction or record exhausted budget"] G -.->|budget remains| D4. Before and after
flowchart LR subgraph OLD["main · before #516"] A["skill_library/<br/>hardware / frameworks / repositories / methods / workflows"] B["repo_packs + knowledge_packs<br/>28 contribution + 9 dependency + 11 peer findings"] C["A separate metadata.json beside skill procedures<br/>Generated per-skill JSON records, navigation and shared provenance"] D["data/query_aliases.json + research_profiles/"] end subgraph NEW["PR · knowledge/data/skill_library/"] N["tasks/ · 7 first-class primary owners + 1 compatibility router<br/>systems / playbooks / workflows / casebooks · 53 + 17 + 8 + 2"] R["records/ · 48 authored findings<br/>IDs, tiers, sources and verification retained"] V["Procedure and metadata in one SKILL.md<br/>88 runtime views · 1 registry + 2 generated inputs"] P["config/ · aliases + profiles"] end A -->|move 80 supporting owners and add primary skills| N B -->|retain IDs and migrate linked paths| R C -->|generate views on demand| V D --> PSeven primary skills
The main entrypoint loads the primary that owns the user's acceptance criterion. Record hardware vendor, device, backend and OS, model specification and stages, engine and version, run script and flags, benchmark, acceptance criteria, current diagnosis and allowed search space. State unknowns and obtain the smallest missing input that affects the decision.
For example, when a Windows TTS engine still runs after an upgrade but audio quality falls, load task-accuracy before the first search. Fix the prompt, reference audio or codec tokens, seed, preprocessing, dtype and tolerance. Compare the last-good and target stacks on that input. When only the target stack fails, consult version alignment to isolate the changed component. When both stacks diverge, trace the model's earliest mismatching stage. An Ascend 910B1 result can suggest what to measure; verifying a Windows repair requires the original target reproducer and held-out quality checks. If the reference is missing or the allowed budget runs out, record the missing comparison and its impact.
LLM, Omni, TTS, Diffusion and RLT describe workload conditions. CUDA, ROCm, Ascend and XPU describe backends. Mobile devices and Windows add deployment constraints. Check these dimensions against evidence individually. Use only the filters supported by the chosen CLI reader.
Reading and delivery
Use
inferrsi skills enter task-performancefor the primary procedure. Look up an unknown owner usinginferrsi find "TTS quality regression" --limit 5 --no-external --json, then use the returned read arguments. For an ID you already know, useinferrsi read <id>and a recorded revision. A bare or package-qualified standalone engineering record checks its own inputs. A record with linked library documents retains its package dependency checks. Current owners edit their source once, review applicability and regenerate the derived files from the committed source.The
evidence/change in the final diff against main is limited to three migrated paths in the existing campaign-plan template. The earlier GPU campaign remains available at its historical revision. It has no bearing on the search-parity gate. Earlier runIndependent review and compatibility
Claude Code's Claude Fable 5.2 · Extra independently reviewed the single-file migration through Computer Use. Fable raised host semantics of promoted keys, YAML coercion, ties broken by stable ID, and per-query readability. Codex tested all 107 old objects after parsing, kept the annotation string-valued, and challenged the proposed requirement for identical source-tree fingerprints. Both reviewers agreed to distinguish owner identity, typed metadata, ordered results and the SHA of the exact source bytes.
The 135 searchable
(kind, id)pairs, backends and states match main. Compared with the prior PR revision, all 107 typed objects and their existing host headers match; 44 other SKILL.md files originally had no companion. Directory links and two obsolete authoring instructions changed, so the raw documents do change. The build checks the full YAML interpretation against the base-install decoder. The historical checkout reader still supports separate metadata files.A 100,000-byte Collect test exposed a genuine regression. Including machine metadata made the required set 125,623 bytes. The corrected AFD context contains seven resources and both licenses, 77,076 bytes on main and 77,206 on this PR. The procedure view records a digest of the delivered bytes, the full source digest and reconstructable half-open byte ranges; a pinned resource read still returns the original file. Fable accepted the frozen timing protocol and asked to verify total-budget handling. The frozen benchmark times Find; this Collect case separately tests delivery.
Fable's final review asked whether the fixed queries exercised the two procedures with edited prose. Neither
pytorchnornpu-skill-factoryappeared in their top five. Two supplementary, untimed probes named each procedure's distinctive work. Both versions placed the intended owner first with a working read command, and both top-five lists matched. These probes were chosen after the main run and are excluded from its 49 queries and timing statistics.The installed-wheel check independently compares a pinned raw SKILL.md against the packaged resource, then reconstructs the delivered procedure from its byte ranges and checks both SHA-256 values. The built wheel contains 761 package resources; the previous 868 included the 107 removed companions. The base wheel exposes 135 search owners without PyYAML. Installed-wheel regression
Separate follow-up for agent task outcomes
Compare A (old layout), B (new layout, direct search as an evaluation-only ablation) and C (the shipped primary-skill-first agent entrypoint). Keep source material, hardware, model, tool permissions, token budget and task distribution fixed. Start with four held-out tasks per family, including compound goals and cases with no target evidence. Use paraphrases that omit the task-card names and a blinded assessor. Measure time and steps to the first applicable source, owner top-three hit rate, incorrect evidence use, correct abstention, task completion and cost. Record engineering time, accelerator hours, model spend and ongoing curation. The observed ranking comparison is a regression check. A benefit or service-performance verdict remains INCONCLUSIVE pending the pilot.
The pilot owner is the maintainer-designated evaluation lead; the maintainer should name that person before the pilot. Deployment owners validate private captures on the corresponding retained snapshot before rollout. The proposed primary metric is binary top-three applicable-owner hit rate on answerable tasks. Require zero decrease between paired arms on that metric; a higher false-transfer count fails rollout. Report top-five applicable-source recall as a secondary metric and report time/cost separately. Run four held-out tasks per task type (28 in total), balance A/B/C order and use a fresh session for each arm. Report A→B layout and B→C routing deltas separately. The host selects the primary by the requested deliverable. It opens the complete instructions before the first supporting search. XPU is a target condition only. The caller explicitly reports applicable evidence, a bounded analogue or a gap. This protocol is proposed for approval before the first scored run.
Use an unseen Windows TTS regression as a negative transfer control, paired with a documented Ascend TTS workload as a positive control. Score both unjustified reuse and unjustified abstention. In addition to quality and time-to-evidence, track files edited and time spent merging the two generated files per accepted contribution.
Earlier layout migration, paired retrieval table, historical timings and review
Earlier review decisions are preserved in the discussion. Required task entrypoints · Run-specific decisions · Exact reads and ranking · Projection and search evidence review
Knowledge authorship declaration