Skip to content

Fix ordered component context in Command eval (#47) - #48

Merged
triggeredcode merged 1 commit into
mainfrom
fix/command-eval-context
Oct 20, 2026
Merged

triggeredcode merged 1 commit into
mainfrom
fix/command-eval-context

Conversation

@triggeredcode

@triggeredcode triggeredcode commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

Summary

Closes #47 by fixing the targeted-context gap found by the first valid post-roadmap live eval.

  • resolves explicit component ordinals such as first table, second diagram, and last chart
  • expands only the named component in full
  • preserves existing cursor, recent-touch, whole-document, and unresolved-ambiguity behavior
  • keeps plain requests such as the table ambiguous when several tables exist

Verification

  • full local CI: 1,519 passed, 22 skipped
  • focused context/prompt tests: 9/9 passed
  • exact live regression: 1/1 passed with zero repairs
  • canonical live Command eval with qwen/qwen3.8-27b at 16K context: 35/35 passed
  • strict full live suite with utterance variants: 98/99 passed (99%); every category met its bar
  • reference resolution: 6/6 passed
  • no pipeline errors, cancellations, or unrelated-block edits
  • the only full-suite miss safely refused nonexistent block 9 without changing the document

Eval reports remain local and uncommitted.

@triggeredcode
triggeredcode marked this pull request as ready for review October 20, 2026 04:30
@triggeredcode
triggeredcode merged commit 8387281 into main Oct 20, 2026
4 checks passed
@triggeredcode
triggeredcode deleted the fix/command-eval-context branch October 20, 2026 04:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Release: bring the qwen3.5-9b Command eval to threshold

1 participant