Repository navigation
feat: add text_label bulk labeling tool for the analyst, surface provider stream errors - #1536
Conversation
…neratedError` `fullStream` error parts were drained and discarded, so a provider 400 surfaced as `NoOutputGeneratedError`. Add a shared `readStreamText` helper and use it in `skill-selector`, `enhance-prompt` and `ai-review` so the real provider message propagates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Labels every row of a scratch query against a fixed label set in bounded, concurrent batches outside the model context, writes results to a scratch table, and reports coverage (rows_in / labeled / failed / unparseable) with failed ids retrievable. Optional agreement spot-check. Allowed for the `analyst`; `ALTIMATE_TEXT_LABEL=0` disables it. The `data-question` skill now points bulk labeling at `text_label`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 0bf2c354-3786-4597-9535-54ae491ad745) |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Docstring Coverage |  |
Docstring coverage is 47.06% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 11 files. (1 skipped:… | Write docstrings for the functions missing them to satisfy the coverage threshold. |
-
Check name Status Explanation Linked Issues check Issue #1535is directly linked and has coding requirements. The PR addstext_labelwith bounded concurrent batching, fixed labels with descriptions, scratch-table persistence for labeled and failed …Out of Scope Changes check The changed files support Issue #1535. The agent permission and registry changes expose or disabletext_label. Thedata-questionchange documents correct use and coverage reporting. The tests vali…Title check The title clearly identifies both primary changes: the new bulk text_label tool and provider stream error propagation. It is specific and related to the changeset. Description check The description includes all required template sections, explains the motivation and implementation, documents verification results, marks the checklist, and identifies screenshots as not applicable. … Full details: Linked Issues check
-
Explanation
Issue
#1535is directly linked and has coding requirements. The PR addstext_labelwith bounded concurrent batching, fixed labels with descriptions, scratch-table persistence for labeled and failed rows, coverage counts, failed-ID SQL retrieval, and optional agreement spot-checking. The PR registers the tool for the analyst and honorsALTIMATE_TEXT_LABEL=0. Thedata-questionguidance points bulk labeling to the tool. The sharedreadStreamTexthelper consumesfullStreamand throws provider error parts.skill-selector,enhance-prompt, andai-reviewuse the helper. The reported regression tests cover labeling and stream-error behavior.
Full details: Out of Scope Changes check
-
Explanation
The changed files support Issue
#1535. The agent permission and registry changes expose or disabletext_label. Thedata-questionchange documents correct use and coverage reporting. The tests validate the new tool, schema behavior, persistence, retries, and stream-error propagation. The stream helper changes are limited to the three paths named by the issue. No unrelated change is established.
Full details: Description check
-
Explanation
The description includes all required template sections, explains the motivation and implementation, documents verification results, marks the checklist, and identifies screenshots as not applicable. The additional generated summaries are lengthy but remain relevant.
-
Full details: Docstring Coverage
-
Explanation
Docstring coverage is 47.06% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 11 files. (1 skipped: 1 unsupported.)
Resolution
Write docstrings for the functions missing them to satisfy the coverage threshold.
- Fix all pre-merge checks with AI
✨ Finishing Touches: 💡 1
-
📝 Generate docstrings: 💡
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
A rabbit checks each row in flight
Then sorts the labels neat and right
The stream brings errors into view
The scratch table stores each clue
And carrots wait when batches are through
Comment @coderabbitai help to get the list of available commands.
|
@codex review |
…pulations - Stop the whole run immediately only on 401/403/404, or when permanent 4xx errors repeat with no successful batch; a batch-specific 4xx fails only that batch. - Mark NULL source text as failed with an explicit error instead of labeling "". - Validate id/text columns and reject an empty population before creating a table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 04a4d44a-9171-4bbf-b8d6-3b101f8dc57d) |
|
Reviews: independent review (Opus) and local Codex review at head 979ff37: no P0/P1 from Codex; independent findings fixed in 91281f9 (scoped fatal 4xx, NULL text, empty population). Codex P2s (UHUGEINT ids, query deadline, truncated-output retry at same batch size) left as follow-ups. Declined: session-model small options on Gemini 2.5 Pro (pre-existing ProviderTransform behavior; error now surfaces as incomplete coverage). @codex review |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Files Reviewed (12 files)
Previous Review Summary (commit 91281f9)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 91281f9)Status: No Issues Found | Recommendation: Merge Files Reviewed (12 files)
Reviewed by gemini-4-argon · Input: 126.8K · Output: 3.3K · Cached: 684.6K Review guidance: REVIEW.md from base branch |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 91281f9ee3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
5 issues found across 12 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/test/altimate/tools/text-label.test.ts">
<violation number="1" location="packages/opencode/test/altimate/tools/text-label.test.ts:364">
P3: This test does not verify its “without creating a table” claim. Query the scratch catalog after both failure paths so accidental target-table creation cannot pass unnoticed.</violation>
</file>
<file name="packages/opencode/src/altimate/tools/text-label.ts">
<violation number="1" location="packages/opencode/src/altimate/tools/text-label.ts:198">
P2: The batch-size guard does not include the rubric and instructions in `systemPrompt`. A valid maximum-sized label set can consume over 100,000 characters before any row text, exceeding smaller session-model context windows and making the entire run fail; bound the combined system-plus-batch prompt against the selected model's context or reject an oversized rubric before labeling.</violation>
<violation number="2" location="packages/opencode/src/altimate/tools/text-label.ts:507">
P2: This gate authorizes the output table but not the arbitrary scratch query that runs before labeling. Request the SQL-read permission for `args.sql` (or otherwise restrict the query to the intended scratch population) before executing it, so approving `text_label` cannot silently authorize unrelated data disclosure.</violation>
<violation number="3" location="packages/opencode/src/altimate/tools/text-label.ts:546">
P2: Record usage from each hidden labeling stream in session accounting; the current text-only read omits these provider-billed calls from session totals.</violation>
</file>
<file name=".opencode/skills/data-question/SKILL.md">
<violation number="1" location=".opencode/skills/data-question/SKILL.md:142">
P2: This promises LLM classification for every row, but `text_label` intentionally skips NULL/oversized inputs and can leave rows failed or unparseable after retries. Say it attempts every eligible row and persists a status for every input, so analysts do not interpret `rows_in` as successful model coverage.</violation>
</file>
Requires human review: Auto-approval blocked because this review re-detected 2 unresolved issues already reported by Cubic.
View guided diff | Turn on auto-fix | Re-trigger cubic
| "Labels are model judgments; confidence and agreement do not establish accuracy.", | ||
| parameters, | ||
| async execute(args, ctx) { | ||
| await ctx.ask({ |
There was a problem hiding this comment.
P2: This gate authorizes the output table but not the arbitrary scratch query that runs before labeling. Request the SQL-read permission for args.sql (or otherwise restrict the query to the intended scratch population) before executing it, so approving text_label cannot silently authorize unrelated data disclosure.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/tools/text-label.ts, line 507:
<comment>This gate authorizes the output table but not the arbitrary scratch query that runs before labeling. Request the SQL-read permission for `args.sql` (or otherwise restrict the query to the intended scratch population) before executing it, so approving `text_label` cannot silently authorize unrelated data disclosure.</comment>
<file context>
@@ -0,0 +1,550 @@
+ "Labels are model judgments; confidence and agreement do not establish accuracy.",
+ parameters,
+ async execute(args, ctx) {
+ await ctx.ask({
+ permission: "text_label",
+ patterns: [args.target_table],
</file context>
| Treat inferred labels as new data, not ground truth. Define a fixed rubric, allowed labels and uncertainty policy before classifying. For synthetic `messages(message_id, body)`, a rubric might classify a request to reverse a charge as `refund`, a question about shipping progress as `delivery`, and ambiguous text as `uncertain`. Specify tie-breaking for multi-topic messages. | ||
|
|
||
| Inspect text, then define the complete population of stable IDs. If `task` and read-only `data-explorer` are available, assign disjoint ID ranges or catalog-defined batches with the same rubric/version. Each child returns exactly one `{message_id, label, rubric_version}` record per assigned ID, including `uncertain`; request records, not a guessed count. Otherwise label manageable batches directly using the same contract. Review uncertain cases consistently; batching makes counting reproducible, not classification inherently correct. | ||
| Inspect text, then define the complete population of stable IDs. When `text_label` is available, materialize that population in `scratch` with `sql_materialize`, then call `text_label` with SQL over scratch, the stable ID and text column names, a fixed label set with a description for each label, and any rubric/tie-breaking instructions. It classifies every row through the session's LLM in bounded batches and persists the results in scratch; do not substitute regex or keyword guesses for semantic labels. Check `rows_in`, `labeled`, `failed` and `unparseable`, and retrieve failed IDs before claiming complete coverage. Optional spot-check agreement measures repeatability, not correctness. Review uncertain cases consistently; batching makes counting reproducible, not classification inherently correct. |
There was a problem hiding this comment.
P2: This promises LLM classification for every row, but text_label intentionally skips NULL/oversized inputs and can leave rows failed or unparseable after retries. Say it attempts every eligible row and persists a status for every input, so analysts do not interpret rows_in as successful model coverage.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .opencode/skills/data-question/SKILL.md, line 142:
<comment>This promises LLM classification for every row, but `text_label` intentionally skips NULL/oversized inputs and can leave rows failed or unparseable after retries. Say it attempts every eligible row and persists a status for every input, so analysts do not interpret `rows_in` as successful model coverage.</comment>
<file context>
@@ -139,9 +139,9 @@ Dialect references: [DuckDB query_table](https://www.duckdb.org/docs/current/gui
Treat inferred labels as new data, not ground truth. Define a fixed rubric, allowed labels and uncertainty policy before classifying. For synthetic `messages(message_id, body)`, a rubric might classify a request to reverse a charge as `refund`, a question about shipping progress as `delivery`, and ambiguous text as `uncertain`. Specify tie-breaking for multi-topic messages.
-Inspect text, then define the complete population of stable IDs. If `task` and read-only `data-explorer` are available, assign disjoint ID ranges or catalog-defined batches with the same rubric/version. Each child returns exactly one `{message_id, label, rubric_version}` record per assigned ID, including `uncertain`; request records, not a guessed count. Otherwise label manageable batches directly using the same contract. Review uncertain cases consistently; batching makes counting reproducible, not classification inherently correct.
+Inspect text, then define the complete population of stable IDs. When `text_label` is available, materialize that population in `scratch` with `sql_materialize`, then call `text_label` with SQL over scratch, the stable ID and text column names, a fixed label set with a description for each label, and any rubric/tie-breaking instructions. It classifies every row through the session's LLM in bounded batches and persists the results in scratch; do not substitute regex or keyword guesses for semantic labels. Check `rows_in`, `labeled`, `failed` and `unparseable`, and retrieve failed IDs before claiming complete coverage. Optional spot-check agreement measures repeatability, not correctness. Review uncertain cases consistently; batching makes counting reproducible, not classification inherently correct.
-Use an available permitted ingestion path to turn these records into a relation `labels(message_id, label, rubric_version)`. When `scratch` and `sql_materialize` are available, small audited annotation batches can be expressed as a SELECT relation and materialized there. For example (DuckDB/SQLite SELECT syntax):
</file context>
| Inspect text, then define the complete population of stable IDs. When `text_label` is available, materialize that population in `scratch` with `sql_materialize`, then call `text_label` with SQL over scratch, the stable ID and text column names, a fixed label set with a description for each label, and any rubric/tie-breaking instructions. It classifies every row through the session's LLM in bounded batches and persists the results in scratch; do not substitute regex or keyword guesses for semantic labels. Check `rows_in`, `labeled`, `failed` and `unparseable`, and retrieve failed IDs before claiming complete coverage. Optional spot-check agreement measures repeatability, not correctness. Review uncertain cases consistently; batching makes counting reproducible, not classification inherently correct. | |
| Inspect text, then define the complete population of stable IDs. When `text_label` is available, materialize that population in `scratch` with `sql_materialize`, then call `text_label` with SQL over scratch, the stable ID and text column names, a fixed label set with a description for each label, and any rubric/tie-breaking instructions. It attempts every eligible row through the session's LLM in bounded batches and persists a result status for every input in scratch; do not substitute regex or keyword guesses for semantic labels. Check `rows_in`, `labeled`, `failed` and `unparseable`, and retrieve failed IDs before claiming complete coverage. Optional spot-check agreement measures repeatability, not correctness. Review uncertain cases consistently; batching makes counting reproducible, not classification inherently correct. |
| Math.max(0, input.deadline - Date.now()), | ||
| ) | ||
| const signal = AbortSignal.any([input.abort, controller.signal]) | ||
| const system = systemPrompt(input.labels, input.instructions) |
There was a problem hiding this comment.
P2: The batch-size guard does not include the rubric and instructions in systemPrompt. A valid maximum-sized label set can consume over 100,000 characters before any row text, exceeding smaller session-model context windows and making the entire run fail; bound the combined system-plus-batch prompt against the selected model's context or reject an oversized rubric before labeling.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/tools/text-label.ts, line 198:
<comment>The batch-size guard does not include the rubric and instructions in `systemPrompt`. A valid maximum-sized label set can consume over 100,000 characters before any row text, exceeding smaller session-model context windows and making the entire run fail; bound the combined system-plus-batch prompt against the selected model's context or reject an oversized rubric before labeling.</comment>
<file context>
@@ -0,0 +1,550 @@
+ Math.max(0, input.deadline - Date.now()),
+ )
+ const signal = AbortSignal.any([input.abort, controller.signal])
+ const system = systemPrompt(input.labels, input.instructions)
+ let next = 0
+ let batches = 0
</file context>
| retries: 0, | ||
| messages: [{ role: "user", content: user }], | ||
| }) | ||
| return readStreamText(stream) |
There was a problem hiding this comment.
P2: Record usage from each hidden labeling stream in session accounting; the current text-only read omits these provider-billed calls from session totals.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/tools/text-label.ts, line 546:
<comment>Record usage from each hidden labeling stream in session accounting; the current text-only read omits these provider-billed calls from session totals.</comment>
<file context>
@@ -0,0 +1,550 @@
+ retries: 0,
+ messages: [{ role: "user", content: user }],
+ })
+ return readStreamText(stream)
+ }
+ return labelScratch(args, { abort: ctx.abort, sessionID: ctx.sessionID }, labeller)
</file context>
| }) | ||
|
|
||
| test("an empty population or wrong column names is rejected without creating a table", async () => { | ||
| const empty = await labelScratch({ ...input, sql: `${input.sql} WHERE 1 = 0` }, { abort: never }, answer) |
There was a problem hiding this comment.
P3: This test does not verify its “without creating a table” claim. Query the scratch catalog after both failure paths so accidental target-table creation cannot pass unnoticed.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/tools/text-label.test.ts, line 364:
<comment>This test does not verify its “without creating a table” claim. Query the scratch catalog after both failure paths so accidental target-table creation cannot pass unnoticed.</comment>
<file context>
@@ -0,0 +1,489 @@
+ })
+
+ test("an empty population or wrong column names is rejected without creating a table", async () => {
+ const empty = await labelScratch({ ...input, sql: `${input.sql} WHERE 1 = 0` }, { abort: never }, answer)
+ expect(empty.metadata).toMatchObject({ ok: false, persisted: false })
+ const wrong = await labelScratch({ ...input, sql: `${input.sql} WHERE 1 = 0`, text_column: "nope" }, { abort: never }, answer)
</file context>
- Register `text_label` only for `analyst` and `data-explorer` agents, not globally - Oversized-input error and `data-question` skill now describe splitting into disjoint id ranges and verifying combined coverage - Schema test rejects any enum containing an empty string - Model-resolution test persists the assistant message through the legacy session API so its assertions execute Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 684d471a-6e56-44be-b4df-cece6167b39c) |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
@codex review |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
1 issue found across 5 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/test/altimate/tools/tool-schema-anthropic.test.ts">
<violation number="1" location="packages/opencode/test/altimate/tools/tool-schema-anthropic.test.ts:160">
P3: This new schema assertion never validates `text_label` for `data-explorer`, although the registry exposes it to that agent. Add `data-explorer` to the provider matrix and expect the tool for both permitted agents so schema regressions in delegated analysis are covered.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| expect(tools.map((tool) => tool.id)).toEqual(expect.arrayContaining(["sql_check", "sql_execute", "read"])) | ||
| // altimate_change start — validate the labeling schema with each Anthropic provider. | ||
| expect(tools.some((tool) => tool.id === "text_label")).toBe( | ||
| name === "analyst" && process.env.ALTIMATE_TEXT_LABEL !== "0", |
There was a problem hiding this comment.
P3: This new schema assertion never validates text_label for data-explorer, although the registry exposes it to that agent. Add data-explorer to the provider matrix and expect the tool for both permitted agents so schema regressions in delegated analysis are covered.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/test/altimate/tools/tool-schema-anthropic.test.ts, line 160:
<comment>This new schema assertion never validates `text_label` for `data-explorer`, although the registry exposes it to that agent. Add `data-explorer` to the provider matrix and expect the tool for both permitted agents so schema regressions in delegated analysis are covered.</comment>
<file context>
@@ -156,7 +156,9 @@ describe("Anthropic tool JSON schemas", () => {
// altimate_change start — validate the labeling schema with each Anthropic provider.
- expect(tools.some((tool) => tool.id === "text_label")).toBe(process.env.ALTIMATE_TEXT_LABEL !== "0")
+ expect(tools.some((tool) => tool.id === "text_label")).toBe(
+ name === "analyst" && process.env.ALTIMATE_TEXT_LABEL !== "0",
+ )
// altimate_change end
</file context>
1 participant
Issue for this PR
Closes #1535
Type of change
What does this PR do?
Why. Questions that need classifying many free-text rows (topic, intent, sentiment) had no tool, so the
analystfell back to keyword/regex guesses or read rows into the conversation. Separately,streamTextfullStreamerror parts were discarded, so a provider 400 surfaced only asNoOutputGeneratedError.Before / after.
skill-selector,enhance-prompt,ai-reviewhidden.text_labellabels every row of a scratch query against a fixed label set (with descriptions) in bounded concurrent batches outside the model context, writes a scratch table, and reports coveragerows_in / labeled / failed / unparseable(failed ids retrievable via SQL). Optionalspot_checkmeasures repeat-label agreement (stability, not accuracy). Non-transient 4xx stops retries; transient errors retry with backoff. Unlabeled rows are never dropped: they are stored with a status and error. Thedata-questionskill now points bulk labeling attext_labeland keeps the rule against claiming labels that were not computed. The real provider error now propagates via a sharedaltimate/llm/stream-text.ts.Flag.
ALTIMATE_TEXT_LABEL=0disables the tool (default on,analystonly).Two commits:
fix:for the stream-error helper (separable),feat:for the tool.Gemini 4 live use depends on #1534 (thinkingLevel fix) and #1532 (empty-enum tool schema) merging; neither is included here.
How did you verify your code works?
bun testontext-label,text-label-model,stream-text,tool-schema-anthropic,enhance-prompt,skill-filtering,review,release-v0.8.1-adversarial: 240 pass, 0 fail.bun run typecheckclean.text_labelis now scoped to the analyst and data-explorer agents (builder does not receive it); the oversized-input error anddata-questionskill state the 30,000 row limit and the split-and-verify-coverage procedure; the schema test rejects any enum containing an empty string; the model-resolution test now persists the assistant message through the legacy session API so its assertions execute (verified it fails when the asserted behavior is broken). Text-label, model, schema andtest/agentsuites: 205 pass, 0 fail; typecheck clean.altimate_changeblocks).altimate-code run --agent analyst -m google-vertex/gemini-4-argonon 30 short texts intopraise/complaint/question,spot_check: 10. Result: rows_in 30, labeled 30, failed 0, unparseable 0; SQL counts 10 / 10 / 10..textpath gaveNoOutputGeneratedError: No output generated;readStreamTextgaveAI_APICallError: ... thinking_budget is only supported when thinking is enabled.Screenshots / recordings
N/A
Checklist
🤖 Generated with Claude Code
Note
Medium Risk
New analyst-facing tool issues many LLM calls and writes session scratch tables; misconfiguration or partial runs could mislead counts if coverage checks are skipped, though failures are persisted explicitly.
Overview
Adds a
text_labeltool so the analyst (and read-only data-explorer) can classify every row from a scratch SQL query against a fixed label rubric via the session model, without pulling rows into chat or using keyword heuristics. Labeling runs in bounded batches with concurrency limits, transient retries, deadlines, and strict JSON parsing; results land in a new scratch table with coverage stats (rows_in/labeled/failed/unparseable), optional spot_check agreement, and SQL helpers for failed IDs and counts. Registration is analyst/data-explorer scoped and can be turned off withALTIMATE_TEXT_LABEL=0; permissions allowtext_labelon the read-only data agents.Introduces shared
readStreamTextso auxiliary LLM streams surface real provider errors fromfullStreaminstead of masking them asNoOutputGeneratedError; enhance-prompt, skill-selector, and ai-review now use it, andtext_labeluses it for model calls.The data-question skill now directs large semantic labeling through
text_label+sql_materializeand tightens rules against claiming labels that were not persisted.Reviewed by Cursor Bugbot for commit 7254d08. Bugbot is set up for automated code reviews on this repo. Configure here.
Summary by cubic
Adds a
text_labeltool that labels every row of a scratch query against a fixed label set, and surfaces the real provider error fromstreamTextinstead of a genericNoOutputGeneratedError.New tool
rows_in/labeled/failed/unparseable) with failed IDs retrievable via SQL.spot_checkmeasures repeat-label agreement (stability, not accuracy).analystanddata-exploreragents only; disabled globally viaALTIMATE_TEXT_LABEL=0. Capped at 30,000 rows per call, with larger populations split across disjoint ID ranges. Thedata-questionskill now points bulk labeling attext_label.Stream errors
skill-selector,enhance-prompt, andai-reviewnow sharereadStreamText, which propagates the provider's error instead ofNoOutputGeneratedError.Written for commit 7254d08. Summary will update on new commits.
Summary by CodeRabbit