Repository navigation
fix(harness): keep a block comment open across lines, and save no prompt from unconfirmed lines - #289
Merged
khushalsonawat merged 6 commits intoNov 11, 2026
Conversation
…mpt from unconfirmed lines A model written inside a multi-line /* */ or <!-- --> block was accepted, because the scan closed the comment at the end of its first line. The open block is now carried across lines until its closer. When lines named twice don't hold a quote too short to stand for the prompt, no prompt is saved; the lines are kept only as unconfirmed, since the platform would otherwise show them as the agent's prompt. Refs TH-8953 Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…-review-followups Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
cdileep23
approved these changes
Nov 11, 2026
| open none, outside a block, is passed over without a scan; inside one, only its closer is | ||
| looked for.""" | ||
| openers = ['"""', "'''"] if triple else [] | ||
| openers += ["/*", "<!--"] if block_comments else ["<!--"] |
Member
There was a problem hiding this comment.
P3. An opener that isn't a comment now hides the rest of the file, so an honest model below it is refused and then left out. <!-- is carried across lines in every file, and /* in every block-comment language, even where the line holds no comment.
Before this change each of these kept gpt-4o on the next line as the model; now it is refused:
a.js:const isComment = /<!--/.test(s);(a regex literal)a.js:const re = /\/*/g;c.yaml:banner: <!-- top.env:BANNER=<!--a.sh: a heredoc holding<!-- headera.ts: a multi-line template literal holding/* todowith no closer
Fix: carry a block to later lines only when its closer (*/ or -->) appears on a later line, since an unterminated comment doesn't compile anyway. Also carry <!-- only in markup files (.html, .xml, .vue, .svelte), where it is a comment.
… and <!-- only in markup An opener that isn't a comment (a regex, a YAML value, a heredoc, a template literal) no longer hides the lines after it. A block is carried to later lines only when its closer is on a later line of the file, and <!-- runs on over lines only in markup files (.html, .htm, .xhtml, .xml, .svg, .vue, .svelte). Answers Dileep's P3 on #289 (TH-8953). Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…uration Restricting the carry to a short list of markup suffixes let a model inside a real multi-line <!-- --> comment through in XML under other suffixes (.config, .csproj, .xaml) and in PHP or JSP. Now <!-- is a comment only to its line's end in known code languages and configuration, and runs on to its --> everywhere else. Adds tests for those files and for a comment after an earlier closed one. Review round 1 of the P3 fix on #289 (TH-8953). Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
env.xml and Dockerfile.xml took the configuration rule and accepted a model inside a real multi-line <!-- --> comment. Configuration is now decided by the suffix or the whole file name, so they carry <!-- again. Adds tests for the configuration half of the rule, by suffix and by name. Review round 2 of the P3 fix on #289 (TH-8953). Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
Review round 3 of the P3 fix on #289 (TH-8953). Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
khushalsonawat
merged commit Nov 11, 2026
e9d18ad
into
feat/TH-8804-source-subfolder
9 of 11 checks passed
This was referenced Nov 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #268, which is merged, for two review comments.
model_citation.py). An unclosed/*or<!--now stays open across lines until its*/or-->, so a model written inside one is refused. Before, the comment ended with its own line, so a model on line 2 of a/* ... */block was accepted.block_open_beforetracks triple quotes,*/and-->.--and INI;comment settings to the scan, so-- see /*or; <!--opens no false block.model_problemsalways passes the open block.tools.py). When the named lines fail twice against a quote of fewer than 16 letters, no prompt is saved (system_prompt_excerptis empty,system_prompt_fromis cleared), and the lines are kept only inunconfirmed_prompt_from. A quote of 16 letters or more is kept as before.Linear: TH-8953.
Verification
uv run ruff check .: no new findings in the changed files (22 before, 22 after)tests/harness/test_contract_model.py tests/harness/test_seed_rows.py tests/test_harness.py, on Python 3.12.3.test_harness.py.uv run pytest -qwhen touching shared runtime, CLI, SDK, or release gates (not run; the change is confined to the model citation check and the prompt branch)typescript/(not touched)agent-learn release-proofwhen touching release-candidate behavior (not touched)Known limits:
*/or-->) is on a later line of the file.<!--is carried in every file but known code languages and configuration. So a stray/*that isn't a comment (a regex like/\/*/g) still hides the lines up to a later*/anywhere in the file. A<!--inside a Vue or Svelte<script>block carries to a later-->. In both cases a correct model is refused, then left out: it fails safe..confor.cfg, and markup named like configuration by itself (Dockerfile.xml,.env.xml), are read as configuration. There a model inside a real multi-line<!-- -->comment is accepted. No such file was found in our repos.test_a_crafted_python_file_does_not_hold_up_the_conversationfails on CI (a 0.38-0.40 s stall against its 0.25 s limit) on this branch's base too, and on its other PRs. It comes from the base, not from this change.unconfirmed_prompt_fromyet.Checklist
agent_learning.*,agent-learn, or@future-agi/agent-learning-kit