Skip to content

fix(content-ops): read image plans as UTF-8 - #7068

Merged
loopx-agent merged 2 commits into
loopx-project:mainfrom
JasonBuildAI:codex/content-ops-image-utf8-z9q4
Oct 22, 2026
Merged

loopx-agent merged 2 commits into
loopx-project:mainfrom
JasonBuildAI:codex/content-ops-image-utf8-z9q4

Conversation

@JasonBuildAI

@JasonBuildAI JasonBuildAI commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Goal And Delivered Outcome

Impact: This fixes a current-main merge-gate regression and restores the shipped content-ops image prepare capability for non-ASCII plans on non-UTF-8 Windows hosts. The shipped-runtime UTF-8 guard fails on main, so the Python test job is red for active contributors until this lands; on a zh-CN Windows host (cp936), non-ASCII image plans can fail before an intent can be prepared, or can be silently mojibaked into the prepared title/prompt. Issue: #7063. Introduced by #6876.

Author Declaration

  • Written by: model_agent (OpenAI Codex) at JasonBuildAI's request, under their direction and review.
  • AI disclosure: this change was drafted with AI assistance at my request, under my direction and review.

Implemented against

Criterion (spec clause) Disposition Symbol / path Test or command
Image plans are decoded as UTF-8 on every host implemented loopx/capabilities/content_ops/media_cli.py:55 uv run --extra test python -m pytest -q tests/test_loopx_text_io_utf8.py; cp936 real-entrypoint reproduction
Keep the adjacent prepare paths and image behavior unchanged implemented same module related content-media suite; real CLI reports prepared rather than submitting a paid provider request
  • Self-check before submission: reviewed the one-line diff and the surrounding prepare paths; verified the guard test, real CLI, related suites, Ruff, diff hygiene, semantic inventory, and premerge selection. No provider submission, public UI, persisted contract, or schema change.

Scope And Continuation

  • Completed scope and remaining work: complete within this scope. The defect is one omitted encoding="utf-8" argument; no successor is needed.
  • Slice boundary / successor: N/A. No broader refactor is required.

Validation

  • Tested revision: bc7f3c37328f060223aa64e5659bd544d4bf1569
  • Run state: finished
  • Input classes: synthetic
Check kind Result Public-safe evidence / limitation
regression_parity passed On latest-at-reproduction main f1f758bdf, tests/test_loopx_text_io_utf8.py failed with ['loopx\\capabilities\\content_ops\\media_cli.py:55'] == []; at the PR head it passes 2 passed. The offending expression was unchanged through final base 795c30a29.
real_entrypoint passed Native Windows, Python 3.14.3, preferred encoding cp936; synthetic UTF-8 plans. Before: the emoji-bearing plan exited 1 with 'gbk' codec can't decode byte 0xad in position 71, while the 你好 plan returned ok:true with both fields corrupted to 浣犲ソ. After: both plans returned ok:true, status:prepared, with exact title/prompt equality. No paid provider call.
unit passed 37 passed, 5 skipped, 1 deselected across content-media image/generation/media, UTF-8 guard, and subprocess UTF-8 tests. The deselected Windows symlink test fails identically at clean base 795c30a29 with WinError 1314 (host lacks symlink privilege).
static passed Ruff on the changed module, py_compile, git diff --check upstream/main...HEAD, and the semantic inventory all passed.
integration blocked Premerge at 265ec9b78 reported 16/17 passed; the only issue was examples/issue-fix-pr-lifecycle-smoke.py timing out under the 240s canary wrapper. The same script passed standalone on both the clean base and PR head (about 2.5 minutes each). The final base moved only unrelated dashboard/docs commits; guard and real CLI were rerun at the final head.
static failed python -m mypy reports 8 Windows-only killpg/SIGKILL/WNOHANG errors in loopx/extensions/process_runtime.py and loopx/control_plane/effect_runtime.py; the identical 8 errors reproduce on clean base 795c30a29.

| static | failed | Exact-head CI kernel-static-checks reports 9 pre-existing E701/E702 errors in unchanged test files. The same 9 errors reproduce on clean base 795c30a29 and at the PR head; checks, pytest, Stage 2C, and merge-gate fail as cascading gates. See the CI note comment for the reproduction command. |

  • Coverage and gaps: the changed behavior is covered by the repository's shipped-runtime guard and a real CLI reproduction on the affected host class. No new test is needed because the existing guard is the failing-before/passing-after regression test. Remaining unavailable checks are the host symlink test, the premerge timeout, and the upstream test-lint CI blocker described above; none touches this path.

Frontend / Visual Evidence

  • UI impact: none
  • Before: N/A
  • After: N/A
  • First viewport and preview: N/A
  • States and viewports shown: N/A
  • Source data: none
  • Attention review: N/A; no UI change.
  • Craft review: N/A; no UI change.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring (no functional changes)
  • Documentation update
  • Test update

LoopX Area

  • Control plane (goals, todos, quota, scheduler, registry, runtime)
  • Benchmark boundary (adapters, runners, verifiers, scoring, evidence)
  • Capability or extension (providers, adapters, skills)
  • Public docs or presentation surface (README, protocols, dashboard)
  • Build, packaging, installer, or CI
  • Host or runtime integration

Technical Direction

  • Direction / acceptance reference, when applicable: N/A; bounded capability bug fix.

Shared-authority RFC fixture impact

  • Production-scale fixture schema: N/A
  • Semantic dimensions changed, or reviewed no-impact rationale: no semantic vocabulary change; advisory inventory reported zero supported candidates.
  • Provider conformance arms run: N/A; no provider contract change.
  • Read-only legacy/file/PostgreSQL three-arm rehearsal (required for promotion, runtime-routing, or compatibility-projection changes): N/A

Boundary Checklist

  • Neither the diff nor this PR body/comments/attachments disclose private state, credentials, raw traces or verifier output, internal links, or local machine paths (including .loopx/, .codex/goals/, and live ACTIVE_GOAL_STATE.md).
  • I did not duplicate maintainer-owned benchmark work unless a maintainer split out a public issue for it.
  • I kept the change scoped to the linked issue/task.
  • Existing English/Chinese RFC pairs (including ledger entries) both have a diff, and the local premerge check python3 scripts/check_rfc_bilingual.py --base origin/main --head HEAD passes, or no paired RFC changed.
  • I completed the visual evidence section for UI changes, or marked UI impact none.
  • Every commit includes a DCO Signed-off-by trailer (git commit -s).

Signed-off-by: JasonBuildAI <jasonbuildai@gmail.com>
@JasonBuildAI

Copy link
Copy Markdown
Contributor Author

CI note: kernel-static-checks is failing before this diff is exercised. Its Ruff command reports 9 pre-existing E701/E702 violations in tests/control_plane/test_quota_acceptance_references.py and tests/presentation/test_linked_goal_sources.py.

I reproduced the same 9 errors with uv run --extra test python -m ruff check tests/control_plane/test_quota_acceptance_references.py tests/presentation/test_linked_goal_sources.py on both untouched base 795c30a29 and PR head bc7f3c3. Neither file is in this PR. The downstream checks, pytest, stage2c-correctness-e2e, and merge-gate failures trace to that upstream failure.

The changed module itself passes Ruff, py_compile, the shipped-runtime UTF-8 guard, the related suite, and the cp936 real-entrypoint reproduction. I am keeping this PR scoped and will rebase when upstream main is green rather than fixing unrelated test lint here.

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max

APPROVE。当前精确头 bc7f3c37328f060223aa64e5659bd544d4bf1569 没有阻断发现。 真实非 UTF-8 locale 的 CLI 对照同时覆盖了“报错”和“成功但保存乱码”,修复后响应及磁盘记录均保留原文。

动机

使用既有图片准备命令、在非 UTF-8 系统中提交中文或表情提示词的操作者。同一份 UTF-8 计划在旧入口可能报解码错误,也可能成功保存乱码;修改后按 UTF-8 读取,并在准备响应与重新读取的记录中保留原文。真实 CLI 对照已复现 ISO8859-1 下的静默乱码和 ASCII locale 下的硬失败,当前头两者均原文准备成功,重复准备及独立 inspect 保持同一 revision 和内容。本轮不声称已独立运行原生 Windows,不执行付费图片提交、发布、安装或合并,也不把准备成功当作生成质量或长程模型收益。

这不是把默认 UTF-8 开关写到测试里就能证明的修复:读取后的乱码仍可能是合法 JSON,后续计划校验也会正常通过,最终准备出错误提示词。已有用户输入和准备权限足够完成同一任务,不需要新增编码选项或确认轮次。

改动思路

复用现有 Path 文本读取和 ImageWorkspace/TypeScript 计划校验,只在文件边界明确既有 UTF-8 编码;相邻 media、generation 已采用同样做法,无需检测编码、容错替换或新增配置。本 PR 的界限是修复既有 image prepare 入口的一次文本解码;计划大小、来源快照、不可变 intent、付费提交和私有产物所有者保持原有职责。

最有力的反对意见是:只有静态扫描通过,未必说明真正保存了正确输入。因此我从真实入口读取同一份中文和表情计划,经过原 TypeScript 计划校验、私有记录写入、重复准备,再启动独立 inspect 读取;比较的是原始 title/prompt 内容,而不是只看成功码。此次位置是文件 I/O 适配层,类型化计划与付费 provider 的职责没有转移,继续抽取通用编码框架没有收益。

具体改动

规格依据是 #7063 中 Expected behavior,其复现主干版本为 f1f758bdf56118428f1f2f83220fe5020b03d8b0;另读不可变基线 795c30a2903638cf4f990adccb4cdceae355c70b 的 tests/test_loopx_text_io_utf8.py,要求所有 shipped runtime 文本 I/O 明确 UTF-8。该项在当前头实现:合法 UTF-8 计划保持原文,真实准备成功,原静态守卫通过。没有把本次补丁当作自己的规格。

关键代码讲解

  • handle_media_command 的 image/prepare 分支先做原有 256KiB 大小检查,再以 read_text(encoding="utf-8") 交给 JSON 解析。唯一生产差异在55行;相邻 generation/media 分支本来就显式使用同一编码,未添加 BOM 猜测、替换错误字符或系统 locale 回退。
  • ImageWorkspace.prepare 仍调用现有 image_plan owner,核计划摘要、锁定 intent、快照显式来源,并以原 UTF-8 原子 writer 保存。相同 id 和摘要重复准备仍为 revision1;inspect 从真实记录返回 title/prompt,prepare 不调用付费提交。

实际14组 base/head 输入对照中,ISO8859-1 基线成功却保存乱码,C locale 基线退出1;当前头二者均保留中文/表情。UTF-8 Unicode 与 Latin1 ASCII 对照的完整结果除 volatile updated_at 外相同,成功案例的重复 prepare 与独立进程 inspect 均相同。非法 UTF-8、坏 JSON 和超过256KiB 在两边均退出1,且没有创建 intent 记录。

对主干的风险

既有五个相关测试模块 38通过、5跳过;跳过来自既有可选 provider/环境条件,不算本轮云端生成资格。不可变基线单独运行 UTF-8 守卫得到1失败、1通过,唯一违规点正是 media_cli55;当前同一守卫通过,能区分旧缺陷,未增加镜像实现的测试。

仓库 native standard premerge 的6项直接检查、9项 catalog canary、8项风险 smoke全部通过,无人工 hold。没有查询或等待远端 CI,也未把作者的 Windows cp936、lint 或 CI 故障声明算作我的独立执行。我本轮在实际 POSIX 非 UTF-8 locale 上验证了同一解码原因,未独立取得原生 Windows 执行证据。

此改动没有默认关闭功能、额外状态、字段、权限或通用义务,不依赖发现 provider 才生效;paid submit、来源范围、不可变摘要和 publication=false 保持原规则。现有 CLI 是修复的真实用户入口,无 UI 变更。回滚一行即可撤销代码,但会重新暴露原编码缺陷;已被旧版写坏的历史 intent 不会被本补丁自动改写。

我的整体评价

这是一项完整且与问题相称的既有能力修复。长期续作的不可变准备与重试边界保持,操作者可在原入口可靠使用合法非 ASCII 提示词,且无需增加交互轮次。它复用原词汇和所有者,代码量已经是最小实际修复,进一步重构无必要。当前没有未解决的代码阻断项;原生 Windows、真实付费图片质量及安装后采用不在本轮独立资格内。发布评审与合并授权、原生 readiness 继续分开。

English verdict: APPROVE at bc7f3c37328f060223aa64e5659bd544d4bf1569. The real CLI reproduces silent Latin1 corruption and ASCII decoding failure on the immutable base; the reviewed head preserves the exact UTF-8 plan through persistence, repeated prepare and cold inspect. Existing UTF-8/ASCII controls and malformed/oversize rejection remain intact.38 focused tests pass with5 optional skips; native local premerge passes. No own Windows, paid generation, installed adoption or merge is claimed.

@loopx-agent
loopx-agent merged commit 601ce4a into loopx-project:main Oct 22, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: content-ops image prepare decodes its UTF-8 plan with the host locale, failing non-ASCII plans on cp936 and the shipped UTF-8 guard on main

2 participants