Reporter overhaul (4/6): opt-in reporters, selection, StreamCollator replacement - #5868
Merged
Merged
Conversation
Sean Larkin (TheLarkInn)
requested review from
Ian Clanton-Thuon (iclanton) and
Pete Gonzalez (octogonz)
as code owners
July 15, 2026 03:13
This was referenced Jul 15, 2026
Sean Larkin (TheLarkInn)
force-pushed
the
reporter/4-opt-in-reporters
branch
from
July 15, 2026 04:53
beb5166 to
291cbc2
Compare
Sean Larkin (TheLarkInn)
force-pushed
the
reporter/4-opt-in-reporters
branch
from
August 19, 2026 14:08
291cbc2 to
58a8fd2
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Adds standalone opt-in reporter selection, rendering, logging, semantic operation events, problem matching, and legacy compatibility.
Changes:
- Adds environment-based reporter planning and log-level controls.
- Implements interactive, plaintext, JSON, AI, file, and legacy reporters.
- Adds uncollated operation events, diagnostic recovery, and legacy error bridging.
Reviewed changes
Copilot reviewed 51 out of 51 changed files in this pull request and generated 17 comments.
Show a summary per file
| File | Description |
|---|---|
research/progress.txt |
Records phase progress. |
research/feature-list.json |
Marks phase features complete. |
libraries/reporter/src/test/ReporterSelection.test.ts |
Tests reporter selection. |
libraries/reporter/src/test/ProblemMatchers.test.ts |
Tests problem matching. |
libraries/reporter/src/test/PlaintextReporter.test.ts |
Tests plaintext output. |
libraries/reporter/src/test/OperationStreamEmitter.test.ts |
Tests semantic event emission. |
libraries/reporter/src/test/LogLevelFilter.test.ts |
Tests log filtering. |
libraries/reporter/src/test/LegacyReporter.test.ts |
Tests legacy rendering. |
libraries/reporter/src/test/LegacyErrorBridge.test.ts |
Tests error bridging. |
libraries/reporter/src/test/JsonAiReporter.test.ts |
Tests machine reporters. |
libraries/reporter/src/test/FileReporter.test.ts |
Tests file logging. |
libraries/reporter/src/test/DefaultInteractiveReporter.test.ts |
Tests interactive rendering. |
libraries/reporter/src/test/AutomaticReporterMatrix.test.ts |
Tests automatic planning. |
libraries/reporter/src/test/__snapshots__/PlaintextReporter.test.ts.snap |
Captures plaintext output. |
libraries/reporter/src/test/__snapshots__/LegacyReporter.test.ts.snap |
Captures legacy output. |
libraries/reporter/src/scheduler/OperationStreamEmitter.ts |
Emits uncollated operation events. |
libraries/reporter/src/scheduler/OperationOutputGrouping.ts |
Regroups operation output. |
libraries/reporter/src/reporters/ReporterRedaction.ts |
Redacts secret fields. |
libraries/reporter/src/reporters/PlaintextReporter.ts |
Implements plaintext reporting. |
libraries/reporter/src/reporters/LegacyReporter.ts |
Implements legacy reporting. |
libraries/reporter/src/reporters/JsonReporter.ts |
Implements NDJSON reporting. |
libraries/reporter/src/reporters/InteractiveRendering.ts |
Provides live-region helpers. |
libraries/reporter/src/reporters/FileReporter.ts |
Implements retained file logs. |
libraries/reporter/src/reporters/DefaultInteractiveReporter.ts |
Implements interactive output. |
libraries/reporter/src/reporters/AiReporter.ts |
Implements bounded AI output. |
libraries/reporter/src/matchers/ProblemMatcherRunner.ts |
Recovers diagnostics from output. |
libraries/reporter/src/matchers/ProblemMatcherRegistry.ts |
Selects applicable matchers. |
libraries/reporter/src/matchers/ProblemMatcher.ts |
Defines matcher contracts. |
libraries/reporter/src/matchers/AnsiNormalization.ts |
Normalizes ANSI output. |
libraries/reporter/src/index.ts |
Exports new APIs. |
libraries/reporter/src/diagnostics/templates/operation.ts |
Adds external-tool text. |
libraries/reporter/src/diagnostics/RushDiagnosticCodeRegistry.ts |
Registers external-tool diagnostics. |
libraries/reporter/src/config/ReporterSelection.ts |
Resolves reporter controls. |
libraries/reporter/src/config/ReporterNames.ts |
Defines reporter names and levels. |
libraries/reporter/src/config/OutputControl.ts |
Parses output destinations. |
libraries/reporter/src/config/LogLevelFilter.ts |
Classifies event visibility. |
libraries/reporter/src/config/AutomaticReporterMatrix.ts |
Builds environment-specific plans. |
libraries/reporter/src/config/AgentDetection.ts |
Detects agents and CI. |
libraries/reporter/src/compat/LegacyErrorBridge.ts |
Correlates legacy sentinels. |
common/reviews/api/rush-reporter.api.md |
Updates the API report. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-40-55.json |
Documents error bridging. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-36-33.json |
Documents problem matchers. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-30-02.json |
Documents operation events. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-24-04.json |
Documents legacy reporting. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-18-53.json |
Documents file reporting. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-12-29.json |
Documents machine reporters. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-05-03.json |
Documents plaintext reporting. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-00-00.json |
Documents interactive reporting. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-50-08.json |
Documents reporter planning. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-45-15.json |
Documents log filtering. |
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-41-00.json |
Documents reporter selection. |
Suppressed comments (1)
libraries/reporter/src/reporters/AiReporter.ts:312
- After all three arrays are emptied, the record can still exceed
maxBytesbecausecommandName, the log path, category/count maps, and other fixed fields remain unbounded. The method then writes the oversized record anyway, violating the documented maximum. Add a final size check and a bounded fallback record or bound the remaining fields.
for (const target of trimTargets) {
while (Buffer.byteLength(JSON.stringify(record), 'utf8') > this._maxBytes && target.get().length > 0) {
target.set(target.get().slice(0, target.get().length - 1));
record.truncated = true;
}
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Sean Larkin (TheLarkInn)
force-pushed
the
reporter/4-opt-in-reporters
branch
from
August 27, 2026 00:26
22f4f49 to
4702ee1
Compare
Resolve the reporter and log level from controls for @rushstack/reporter (#5858). - Add the built-in reporter names and log levels with validation guards - Add agent and CI detection, including COPILOT_CLI and configured agent environment variables and the active-value semantics - Parse the --output control into reporter, target, and params - Add resolveReporterSelection: the primary reporter runs from explicit CLI through RUSH_REPORTER, agent, CI, and TTY down to generic non-TTY plaintext, and always pairs with the file reporter - Resolve the log level independently, mapping --quiet, --verbose, and --debug aliases and rejecting contradictions - Keep command-specific --json distinct from the json reporter and fail explicit unsupported reporter or log-level requests - Cover precedence, aliases, contradictions, outputs, and detection with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add per-reporter log-level filtering for @rushstack/reporter (#5858). - Classify each event to a minimum log level and add shouldRenderAtLogLevel and filterEventsForLogLevel so each reporter applies quiet, normal, verbose, or debug filtering independently - Keep diagnostic severity separate from the reporter log level: severity sets the minimum level while the reporter's configured level gates rendering - Default the full-detail file reporter to debug - Cover ranking, classification, per-level rendering, monotonicity, and the severity separation with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Resolve the automatic reporter plan by environment for @rushstack/reporter (#5858). - Add planAutomaticReporters, which pairs ai with file for an agent, detailed plaintext with file for CI, default with file for an interactive TTY, and concise plaintext with file otherwise - Give machine reporters (ai, json) exclusive stdout and route human progress to stderr, with emergency diagnostics always on stderr - Add describeReporterPlan to record the selection reason and reporters in the detailed log - Cover the full matrix, machine stdout ownership, and the plan description with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the concise default reporter for @rushstack/reporter (#5858). - Add interactive rendering helpers: a spinner, color resolution honoring NO_COLOR and FORCE_COLOR, width-aware active projects with +N more, and a three-row live region renderer that truncates before coloring - Add DefaultInteractiveReporter, which paints the live region at no more than 10 Hz, reacts to terminal width, hides and restores the cursor, leaves at most three stable lines on success, appends a bounded diagnostic block and log path on failure, and appends one summary per completed watch cycle - Cover rendering helpers and reporter behavior with an injected terminal and clock Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the plaintext reporter for @rushstack/reporter (#5858). - Add PlaintextReporter, an append-only reporter that never moves the cursor and disables color by default - Emit the start line, meaningful operation state changes, diagnostics, and the final result - Add a compact 30-second heartbeat for long non-TTY and CI sessions - Retain StreamCollator-like operation grouping in the detailed CI variant by grouping each operation's output under a header - Add stable concise and detailed plaintext snapshot tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the machine reporters for @rushstack/reporter (#5858). - Add JsonReporter, which emits the complete versioned NDJSON event stream on exclusive stdout and replaces an oversized record with a valid marker - Add AiReporter, a bounded projection that emits a status record and a final record with the result, scope, error codes and categories, structured remediation, aggregate counts, log reference, and artifact completeness - Cap the AI record at 64 KiB and 20 detailed diagnostics, represent warnings by count when failures exist, and exclude raw output and stacks - Keep the absolute log path in AI output while telemetry continues to exclude it - Add stdout-purity tests for both reporters and a telemetry path cross-check Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the full-detail file reporter for @rushstack/reporter (#5858). - Add FileReporter, which writes a debug NDJSON invocation log to <commonTempFolder>/rush-logs/<UTC timestamp>-<pid>-<action>.log with owner-only permissions and redacts fields classified as secret - Maintain a latest.log pointer for both successful and failed commands - Buffer events until the first flush and fall back to the OS temp folder - Delete logs older than 14 days and cap retention at 20 sessions - Treat failure at both paths as nonfatal, emitting an emergency warning and marking the artifact unavailable - Cover writing, permissions, redaction, retention, fallback, and failure with tests against real temp directories Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the legacy reporter for @rushstack/reporter (#5858). - Add LegacyReporter, which reproduces the current Rush output: the start line, the parallelism line, StreamCollator-style operation headers with grouped output, and a success or failure summary with durations - Add isLegacyEmergencyFallbackRequested and confirm RUSH_REPORTER=legacy selects the legacy reporter as an emergency fallback - Validate the reporter against the frozen legacy output snapshots Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the uncollated operation event stream for @rushstack/reporter (#5858). - Add OperationStreamEmitter so the scheduler emits operation registration, status transitions, raw output chunks, and the aggregate command result - Emit output chunks immediately in call order and never collate them, so the concise reporter derives activity without buffering while the detailed and file reporters own grouping - Add iterateExternalOutput and regroupOperationOutput so problem matchers consume the uncollated stream and reporters reconstruct StreamCollator-parity grouping - Cover emission, chunking, uncollated ordering, and reporter parity with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Recover diagnostics from raw external output for @rushstack/reporter (#5858). - Preserve raw stdout and stderr as ordered chunks and run matchers over an ANSI-normalized copy so the raw evidence and process status are never modified - Add a tool- and version-scoped problem matcher registry gated on default enablement, routing older Heft versions through the version predicate - Add runProblemMatchers, which reassembles lines split across chunks, links recovered diagnostics to the operation and source location, preserves unmatched text, and caps duplicate diagnostics - Cover ANSI stripping, scoping, recovery, split chunks, the duplicate cap, and a representative corpus with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Bridge the legacy sentinel error pattern for @rushstack/reporter (#5858). - Deprecate AlreadyReportedError so new usage is prohibited now that structured diagnostics and RushError are available - Add LegacyErrorBridge, which observes emitted diagnostics, correlates legacy sentinels with them, and suppresses duplicate rendering of failures that are already represented - Document the bridge removal criteria for a later major - Cover sentinel detection, suppression, correlation, and the removal criteria with tests Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
- OperationStreamEmitter no longer sets 'required' (manager derives it from the event type) - Problem matcher diagnostics use the kind-tagged source union and the registered RUSH_EXTERNAL_TOOL_PROBLEM code with its template Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve streamed output, enforce reporter privacy and lifecycle contracts, restore legacy compatibility, and correct reporter change metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Harden reporter filtering, protocol bounds, terminal rendering, file streaming, matcher recovery, and legacy summaries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GitHub continued to report conflicts even though the current base is the direct second parent of the PR head. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sean Larkin (TheLarkInn)
force-pushed
the
reporter/4-opt-in-reporters
branch
from
August 27, 2026 12:04
4702ee1 to
502b0b3
Compare
Mo Jazayeri (mojaza)
approved these changes
Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📚 Reporter Overhaul PR stack (merge bottom-up)
mainEach PR's diff is scoped to its phase; review independently, merge from #5865 upward.
👉 This is PR 4 of 6.
Phase 4 — Opt-in reporters (RFC §8.1)
Adds the actual reporters plus selection/config, and replaces StreamCollator only on the new reporter path.
What's included
--reporter/RUSH_REPORTER/agent/CI/TTY), independent per-reporter log levels, and the automatic environment matrix.defaultinteractive (3-row live region),plaintext/non-TTY,json, boundedai, full-detailfile(owner-only perms, retention, OS-temp fallback), andlegacy(selectable + emergency fallback).AlreadyReportedErrorusage; correlate legacy sentinels with emitted diagnostics.Validation
rush build --to @rushstack/reporter✅rush test --only @rushstack/reporter✅ (adds ReporterSelection, LogLevelFilter, AutomaticReporterMatrix, the reporter suites, OperationStreamEmitter, ProblemMatchers, LegacyErrorBridge)@beta.Scope note
Standalone package; not yet wired into the live CLI. Independently releasable/revertible per RFC §8.1.