Skip to content

Reporter overhaul (4/6): opt-in reporters, selection, StreamCollator replacement - #5868

Merged
Sean Larkin (TheLarkInn) merged 27 commits into
mainfrom
reporter/4-opt-in-reporters
Aug 27, 2026
Merged

Reporter overhaul (4/6): opt-in reporters, selection, StreamCollator replacement#5868
Sean Larkin (TheLarkInn) merged 27 commits into
mainfrom
reporter/4-opt-in-reporters

Conversation

@TheLarkInn

@TheLarkInn Sean Larkin (TheLarkInn) commented Jul 15, 2026

Copy link
Copy Markdown
Member

📚 Reporter Overhaul PR stack (merge bottom-up)

# PR Phase Base
1 #5865 Contracts & baselines (package) main
2 #5866 Bootstrap & compatibility #5865
3 #5867 Shadow structured emission #5866
4 #5868 Opt-in reporters #5867
5 #5869 Heft protocol track #5868
6 #5870 Perf budgets & default flip #5869

Each PR's diff is scoped to its phase; review independently, merge from #5865 upward.

👉 This is PR 4 of 6.

Phase 4 — Opt-in reporters (RFC §8.1)

Adds the actual reporters plus selection/config, and replaces StreamCollator only on the new reporter path.

What's included

  • Selection & config — reporter selection precedence (--reporter/RUSH_REPORTER/agent/CI/TTY), independent per-reporter log levels, and the automatic environment matrix.
  • Reportersdefault interactive (3-row live region), plaintext/non-TTY, json, bounded ai, full-detail file (owner-only perms, retention, OS-temp fallback), and legacy (selectable + emergency fallback).
  • StreamCollator replacement — raw semantic operation events in the primary output path.
  • External output — lossless preservation + diagnostic recovery via problem matchers.
  • Legacy error bridge — prohibit new AlreadyReportedError usage; correlate legacy sentinels with emitted diagnostics.

Validation

  • rush build --to @rushstack/reporterrush test --only @rushstack/reporter ✅ (adds ReporterSelection, LogLevelFilter, AutomaticReporterMatrix, the reporter suites, OperationStreamEmitter, ProblemMatchers, LegacyErrorBridge)
  • All exports @beta.

Scope note

Standalone package; not yet wired into the live CLI. Independently releasable/revertible per RFC §8.1.

@github-project-automation github-project-automation Bot moved this to Needs triage in Bug Triage Jul 15, 2026
@TheLarkInn
Sean Larkin (TheLarkInn) force-pushed the reporter/4-opt-in-reporters branch from beb5166 to 291cbc2 Compare July 15, 2026 04:53
@mojaza
Mo Jazayeri (mojaza) requested a balanced review from Copilot August 24, 2026 21:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds standalone opt-in reporter selection, rendering, logging, semantic operation events, problem matching, and legacy compatibility.

Changes:

  • Adds environment-based reporter planning and log-level controls.
  • Implements interactive, plaintext, JSON, AI, file, and legacy reporters.
  • Adds uncollated operation events, diagnostic recovery, and legacy error bridging.

Reviewed changes

Copilot reviewed 51 out of 51 changed files in this pull request and generated 17 comments.

Show a summary per file
File Description
research/progress.txt Records phase progress.
research/feature-list.json Marks phase features complete.
libraries/reporter/src/test/ReporterSelection.test.ts Tests reporter selection.
libraries/reporter/src/test/ProblemMatchers.test.ts Tests problem matching.
libraries/reporter/src/test/PlaintextReporter.test.ts Tests plaintext output.
libraries/reporter/src/test/OperationStreamEmitter.test.ts Tests semantic event emission.
libraries/reporter/src/test/LogLevelFilter.test.ts Tests log filtering.
libraries/reporter/src/test/LegacyReporter.test.ts Tests legacy rendering.
libraries/reporter/src/test/LegacyErrorBridge.test.ts Tests error bridging.
libraries/reporter/src/test/JsonAiReporter.test.ts Tests machine reporters.
libraries/reporter/src/test/FileReporter.test.ts Tests file logging.
libraries/reporter/src/test/DefaultInteractiveReporter.test.ts Tests interactive rendering.
libraries/reporter/src/test/AutomaticReporterMatrix.test.ts Tests automatic planning.
libraries/reporter/src/test/__snapshots__/PlaintextReporter.test.ts.snap Captures plaintext output.
libraries/reporter/src/test/__snapshots__/LegacyReporter.test.ts.snap Captures legacy output.
libraries/reporter/src/scheduler/OperationStreamEmitter.ts Emits uncollated operation events.
libraries/reporter/src/scheduler/OperationOutputGrouping.ts Regroups operation output.
libraries/reporter/src/reporters/ReporterRedaction.ts Redacts secret fields.
libraries/reporter/src/reporters/PlaintextReporter.ts Implements plaintext reporting.
libraries/reporter/src/reporters/LegacyReporter.ts Implements legacy reporting.
libraries/reporter/src/reporters/JsonReporter.ts Implements NDJSON reporting.
libraries/reporter/src/reporters/InteractiveRendering.ts Provides live-region helpers.
libraries/reporter/src/reporters/FileReporter.ts Implements retained file logs.
libraries/reporter/src/reporters/DefaultInteractiveReporter.ts Implements interactive output.
libraries/reporter/src/reporters/AiReporter.ts Implements bounded AI output.
libraries/reporter/src/matchers/ProblemMatcherRunner.ts Recovers diagnostics from output.
libraries/reporter/src/matchers/ProblemMatcherRegistry.ts Selects applicable matchers.
libraries/reporter/src/matchers/ProblemMatcher.ts Defines matcher contracts.
libraries/reporter/src/matchers/AnsiNormalization.ts Normalizes ANSI output.
libraries/reporter/src/index.ts Exports new APIs.
libraries/reporter/src/diagnostics/templates/operation.ts Adds external-tool text.
libraries/reporter/src/diagnostics/RushDiagnosticCodeRegistry.ts Registers external-tool diagnostics.
libraries/reporter/src/config/ReporterSelection.ts Resolves reporter controls.
libraries/reporter/src/config/ReporterNames.ts Defines reporter names and levels.
libraries/reporter/src/config/OutputControl.ts Parses output destinations.
libraries/reporter/src/config/LogLevelFilter.ts Classifies event visibility.
libraries/reporter/src/config/AutomaticReporterMatrix.ts Builds environment-specific plans.
libraries/reporter/src/config/AgentDetection.ts Detects agents and CI.
libraries/reporter/src/compat/LegacyErrorBridge.ts Correlates legacy sentinels.
common/reviews/api/rush-reporter.api.md Updates the API report.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-40-55.json Documents error bridging.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-36-33.json Documents problem matchers.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-30-02.json Documents operation events.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-24-04.json Documents legacy reporting.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-18-53.json Documents file reporting.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-12-29.json Documents machine reporters.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-05-03.json Documents plaintext reporting.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-01-00-00.json Documents interactive reporting.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-50-08.json Documents reporter planning.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-45-15.json Documents log filtering.
common/changes/@rushstack/rush-reporter/docs-rush-reporter-overhaul-spec_2026-07-15-00-41-00.json Documents reporter selection.
Suppressed comments (1)

libraries/reporter/src/reporters/AiReporter.ts:312

  • After all three arrays are emptied, the record can still exceed maxBytes because commandName, the log path, category/count maps, and other fixed fields remain unbounded. The method then writes the oversized record anyway, violating the documented maximum. Add a final size check and a bounded fallback record or bound the remaining fields.
    for (const target of trimTargets) {
      while (Buffer.byteLength(JSON.stringify(record), 'utf8') > this._maxBytes && target.get().length > 0) {
        target.set(target.get().slice(0, target.get().length - 1));
        record.truncated = true;
      }

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread libraries/reporter/src/config/LogLevelFilter.ts Outdated
Comment thread libraries/reporter/src/config/LogLevelFilter.ts
Comment thread libraries/reporter/src/reporters/DefaultInteractiveReporter.ts
Comment thread libraries/reporter/src/reporters/InteractiveRendering.ts Outdated
Comment thread libraries/reporter/src/reporters/JsonReporter.ts Outdated
Comment thread libraries/reporter/src/scheduler/OperationStreamEmitter.ts
Comment thread libraries/reporter/src/reporters/DefaultInteractiveReporter.ts Outdated
Comment thread libraries/reporter/src/reporters/FileReporter.ts Outdated
Comment thread libraries/reporter/src/reporters/AiReporter.ts Outdated
Comment thread libraries/reporter/src/config/ReporterSelection.ts Outdated
Base automatically changed from reporter/3-shadow-emission to main August 27, 2026 02:42
Resolve the reporter and log level from controls for @rushstack/reporter (#5858).

- Add the built-in reporter names and log levels with validation guards
- Add agent and CI detection, including COPILOT_CLI and configured agent
  environment variables and the active-value semantics
- Parse the --output control into reporter, target, and params
- Add resolveReporterSelection: the primary reporter runs from explicit CLI
  through RUSH_REPORTER, agent, CI, and TTY down to generic non-TTY plaintext,
  and always pairs with the file reporter
- Resolve the log level independently, mapping --quiet, --verbose, and --debug
  aliases and rejecting contradictions
- Keep command-specific --json distinct from the json reporter and fail explicit
  unsupported reporter or log-level requests
- Cover precedence, aliases, contradictions, outputs, and detection with tests

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add per-reporter log-level filtering for @rushstack/reporter (#5858).

- Classify each event to a minimum log level and add shouldRenderAtLogLevel and
  filterEventsForLogLevel so each reporter applies quiet, normal, verbose, or
  debug filtering independently
- Keep diagnostic severity separate from the reporter log level: severity sets
  the minimum level while the reporter's configured level gates rendering
- Default the full-detail file reporter to debug
- Cover ranking, classification, per-level rendering, monotonicity, and the
  severity separation with tests

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Sean Larkin (TheLarkInn) and others added 23 commits August 27, 2026 12:02
Resolve the automatic reporter plan by environment for @rushstack/reporter (#5858).

- Add planAutomaticReporters, which pairs ai with file for an agent, detailed
  plaintext with file for CI, default with file for an interactive TTY, and
  concise plaintext with file otherwise
- Give machine reporters (ai, json) exclusive stdout and route human progress to
  stderr, with emergency diagnostics always on stderr
- Add describeReporterPlan to record the selection reason and reporters in the
  detailed log
- Cover the full matrix, machine stdout ownership, and the plan description with
  tests

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the concise default reporter for @rushstack/reporter (#5858).

- Add interactive rendering helpers: a spinner, color resolution honoring
  NO_COLOR and FORCE_COLOR, width-aware active projects with +N more, and a
  three-row live region renderer that truncates before coloring
- Add DefaultInteractiveReporter, which paints the live region at no more than
  10 Hz, reacts to terminal width, hides and restores the cursor, leaves at most
  three stable lines on success, appends a bounded diagnostic block and log path
  on failure, and appends one summary per completed watch cycle
- Cover rendering helpers and reporter behavior with an injected terminal and
  clock

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the plaintext reporter for @rushstack/reporter (#5858).

- Add PlaintextReporter, an append-only reporter that never moves the cursor and
  disables color by default
- Emit the start line, meaningful operation state changes, diagnostics, and the
  final result
- Add a compact 30-second heartbeat for long non-TTY and CI sessions
- Retain StreamCollator-like operation grouping in the detailed CI variant by
  grouping each operation's output under a header
- Add stable concise and detailed plaintext snapshot tests

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the machine reporters for @rushstack/reporter (#5858).

- Add JsonReporter, which emits the complete versioned NDJSON event stream on
  exclusive stdout and replaces an oversized record with a valid marker
- Add AiReporter, a bounded projection that emits a status record and a final
  record with the result, scope, error codes and categories, structured
  remediation, aggregate counts, log reference, and artifact completeness
- Cap the AI record at 64 KiB and 20 detailed diagnostics, represent warnings by
  count when failures exist, and exclude raw output and stacks
- Keep the absolute log path in AI output while telemetry continues to exclude it
- Add stdout-purity tests for both reporters and a telemetry path cross-check

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the full-detail file reporter for @rushstack/reporter (#5858).

- Add FileReporter, which writes a debug NDJSON invocation log to
  <commonTempFolder>/rush-logs/<UTC timestamp>-<pid>-<action>.log with
  owner-only permissions and redacts fields classified as secret
- Maintain a latest.log pointer for both successful and failed commands
- Buffer events until the first flush and fall back to the OS temp folder
- Delete logs older than 14 days and cap retention at 20 sessions
- Treat failure at both paths as nonfatal, emitting an emergency warning and
  marking the artifact unavailable
- Cover writing, permissions, redaction, retention, fallback, and failure with
  tests against real temp directories

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the legacy reporter for @rushstack/reporter (#5858).

- Add LegacyReporter, which reproduces the current Rush output: the start line,
  the parallelism line, StreamCollator-style operation headers with grouped
  output, and a success or failure summary with durations
- Add isLegacyEmergencyFallbackRequested and confirm RUSH_REPORTER=legacy selects
  the legacy reporter as an emergency fallback
- Validate the reporter against the frozen legacy output snapshots

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Add the uncollated operation event stream for @rushstack/reporter (#5858).

- Add OperationStreamEmitter so the scheduler emits operation registration,
  status transitions, raw output chunks, and the aggregate command result
- Emit output chunks immediately in call order and never collate them, so the
  concise reporter derives activity without buffering while the detailed and
  file reporters own grouping
- Add iterateExternalOutput and regroupOperationOutput so problem matchers
  consume the uncollated stream and reporters reconstruct StreamCollator-parity
  grouping
- Cover emission, chunking, uncollated ordering, and reporter parity with tests

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Recover diagnostics from raw external output for @rushstack/reporter (#5858).

- Preserve raw stdout and stderr as ordered chunks and run matchers over an
  ANSI-normalized copy so the raw evidence and process status are never modified
- Add a tool- and version-scoped problem matcher registry gated on default
  enablement, routing older Heft versions through the version predicate
- Add runProblemMatchers, which reassembles lines split across chunks, links
  recovered diagnostics to the operation and source location, preserves
  unmatched text, and caps duplicate diagnostics
- Cover ANSI stripping, scoping, recovery, split chunks, the duplicate cap, and a
  representative corpus with tests

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Bridge the legacy sentinel error pattern for @rushstack/reporter (#5858).

- Deprecate AlreadyReportedError so new usage is prohibited now that structured
  diagnostics and RushError are available
- Add LegacyErrorBridge, which observes emitted diagnostics, correlates legacy
  sentinels with them, and suppresses duplicate rendering of failures that are
  already represented
- Document the bridge removal criteria for a later major
- Cover sentinel detection, suppression, correlation, and the removal criteria
  with tests

Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
Assistant-model: GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 897dcf7e-e6e8-4a84-85ca-34b93fa29be3
- OperationStreamEmitter no longer sets 'required' (manager derives it from
  the event type)
- Problem matcher diagnostics use the kind-tagged source union and the
  registered RUSH_EXTERNAL_TOOL_PROBLEM code with its template

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve streamed output, enforce reporter privacy and lifecycle contracts, restore legacy compatibility, and correct reporter change metadata.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Harden reporter filtering, protocol bounds, terminal rendering, file streaming, matcher recovery, and legacy summaries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GitHub continued to report conflicts even though the current base is the direct second parent of the PR head.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@TheLarkInn
Sean Larkin (TheLarkInn) merged commit d0129d8 into main Aug 27, 2026
10 checks passed
@github-project-automation github-project-automation Bot moved this from Needs triage to Closed in Bug Triage Aug 27, 2026
@TheLarkInn
Sean Larkin (TheLarkInn) deleted the reporter/4-opt-in-reporters branch August 27, 2026 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Closed

Development

Successfully merging this pull request may close these issues.

3 participants