Skip to content

fix(harness): keep a block comment open across lines, and save no prompt from unconfirmed lines - #289

Merged
khushalsonawat merged 6 commits into
feat/TH-8804-source-subfolderfrom
fix/th-8953-review-followups
Nov 11, 2026
Merged

khushalsonawat merged 6 commits into
feat/TH-8804-source-subfolderfrom
fix/th-8953-review-followups

Conversation

@khushalsonawat

@khushalsonawat khushalsonawat commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Summary

Follow-up to #268, which is merged, for two review comments.

  • A model inside a multi-line block comment (model_citation.py). An unclosed /* or <!-- now stays open across lines until its */ or -->, so a model written inside one is refused. Before, the comment ended with its own line, so a model on line 2 of a /* ... */ block was accepted.
    • block_open_before tracks triple quotes, */ and -->.
    • It passes the SQL -- and INI ; comment settings to the scan, so -- see /* or ; <!-- opens no false block.
    • model_problems always passes the open block.
    • Text after a closer, and blocks closed on one line, behave as before.
  • Unconfirmed lines are no longer saved as the prompt (tools.py). When the named lines fail twice against a quote of fewer than 16 letters, no prompt is saved (system_prompt_excerpt is empty, system_prompt_from is cleared), and the lines are kept only in unconfirmed_prompt_from. A quote of 16 letters or more is kept as before.
    • The replies, and the understand-agent skill text, say so.
    • One behaviour changes: a placeholder quote with the right lines used to save those lines, and now saves no prompt.

Linear: TH-8953.

Verification

  • uv run ruff check .: no new findings in the changed files (22 before, 22 after)
  • Focused pytest target: tests/harness/test_contract_model.py tests/harness/test_seed_rows.py tests/test_harness.py, on Python 3.12.3.
    • The failures are identical by name before and after: six pre-existing failures in test_harness.py.
    • 5 new test functions cover a model inside a block over lines (refused), one after a closer (accepted), no false blocks from strings, URLs, globs, SQL or INI comments, linear time on 1 MiB files, and the short-quote outcome.
    • Each of the deliberate breaks tried was caught by a named test.
  • Full uv run pytest -q when touching shared runtime, CLI, SDK, or release gates (not run; the change is confined to the model citation check and the prompt branch)
  • TypeScript build/test when touching typescript/ (not touched)
  • agent-learn release-proof when touching release-candidate behavior (not touched)

Known limits:

  • A block comment is carried to later lines only when its closer (*/ or -->) is on a later line of the file. <!-- is carried in every file but known code languages and configuration. So a stray /* that isn't a comment (a regex like /\/*/g) still hides the lines up to a later */ anywhere in the file. A <!-- inside a Vue or Svelte <script> block carries to a later -->. In both cases a correct model is refused, then left out: it fails safe.
  • XML written under .conf or .cfg, and markup named like configuration by itself (Dockerfile.xml, .env.xml), are read as configuration. There a model inside a real multi-line <!-- --> comment is accepted. No such file was found in our repos.
  • test_a_crafted_python_file_does_not_hold_up_the_conversation fails on CI (a 0.38-0.40 s stall against its 0.25 s limit) on this branch's base too, and on its other PRs. It comes from the base, not from this change.
  • Nothing on the platform reads unconfirmed_prompt_from yet.

Checklist

  • Public API names stay under agent_learning.*, agent-learn, or @future-agi/agent-learning-kit
  • New user-facing workflow has docs or an example (no new workflow)
  • New release claim has executable gate coverage (no release claim)
  • No hosted-service dependency was added to release-gated examples
  • Security-sensitive output is redacted or justified

…mpt from unconfirmed lines

A model written inside a multi-line /* */ or <!-- --> block was accepted,
because the scan closed the comment at the end of its first line. The
open block is now carried across lines until its closer.

When lines named twice don't hold a quote too short to stand for the
prompt, no prompt is saved; the lines are kept only as unconfirmed,
since the platform would otherwise show them as the agent's prompt.

Refs TH-8953

Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…-review-followups

Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>

@cdileep23 cdileep23 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check the comments.

Comment thread src/fi/alk/harness/model_citation.py Outdated
open none, outside a block, is passed over without a scan; inside one, only its closer is
looked for."""
openers = ['"""', "'''"] if triple else []
openers += ["/*", "<!--"] if block_comments else ["<!--"]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3. An opener that isn't a comment now hides the rest of the file, so an honest model below it is refused and then left out. <!-- is carried across lines in every file, and /* in every block-comment language, even where the line holds no comment.

Before this change each of these kept gpt-4o on the next line as the model; now it is refused:

  • a.js: const isComment = /<!--/.test(s); (a regex literal)
  • a.js: const re = /\/*/g;
  • c.yaml: banner: <!-- top
  • .env: BANNER=<!--
  • a.sh: a heredoc holding <!-- header
  • a.ts: a multi-line template literal holding /* todo with no closer

Fix: carry a block to later lines only when its closer (*/ or -->) appears on a later line, since an unterminated comment doesn't compile anyway. Also carry <!-- only in markup files (.html, .xml, .vue, .svelte), where it is a comment.

… and <!-- only in markup

An opener that isn't a comment (a regex, a YAML value, a heredoc, a template
literal) no longer hides the lines after it. A block is carried to later lines
only when its closer is on a later line of the file, and <!-- runs on over lines
only in markup files (.html, .htm, .xhtml, .xml, .svg, .vue, .svelte).

Answers Dileep's P3 on #289 (TH-8953).

Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…uration

Restricting the carry to a short list of markup suffixes let a model inside a
real multi-line <!-- --> comment through in XML under other suffixes (.config,
.csproj, .xaml) and in PHP or JSP. Now <!-- is a comment only to its line's end
in known code languages and configuration, and runs on to its --> everywhere
else. Adds tests for those files and for a comment after an earlier closed one.

Review round 1 of the P3 fix on #289 (TH-8953).

Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
env.xml and Dockerfile.xml took the configuration rule and accepted a model
inside a real multi-line <!-- --> comment. Configuration is now decided by the
suffix or the whole file name, so they carry <!-- again. Adds tests for the
configuration half of the rule, by suffix and by name.

Review round 2 of the P3 fix on #289 (TH-8953).

Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
Review round 3 of the P3 fix on #289 (TH-8953).

Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
@khushalsonawat
khushalsonawat merged commit e9d18ad into feat/TH-8804-source-subfolder Nov 11, 2026
9 of 11 checks passed
@khushalsonawat
khushalsonawat deleted the fix/th-8953-review-followups branch November 11, 2026 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants