[dfrankland]

Stacked on #1253. Part of TRUNK-19074.

Adds the hidden github-annotations command that trunk-io/analytics-uploader#153 calls. It reads the summary --summary-output-file wrote (#1246) and prints GitHub Actions annotations saying why the run failed. The CLI words the failure, and the action never reads the summary's format.

Interface (what #153 assumes).

• trunk-analytics-cli github-annotations --summary-file <path>, hidden from --help.
• Workflow commands on stdout, escaped per GitHub's rules (%, CR and LF in messages; also : and , in file/title).
• Exits 0 only when it printed something. A summary with nothing to say, a missing file, or one it can't read exits 1, and the action falls back to its own message. Read errors go to stderr, which the action discards.
• Runs before Sentry, logging and the renderer start, so stdout carries only the workflow commands. To do that, Cli::parse() moves to the top of main(), before Sentry init. Parsing spawns nothing, so the "init Sentry before threads" rule still holds.

What it says. It follows render_upload:

• An error that blocks quarantining (unauthorized, git repository, interrupted) is the only annotation, as the text summary stops there too. For unauthorized, that includes the settings-page hint.
• Any other upload error becomes a ::warning:: before the outcome.
• No tests found with a non-zero exit code: Say when no tests were found but the exit code isn't 0 #1253's "The test step appears to have failed without running tests" line.
• No failures but a non-zero exit code: the existing "some tests were not run" line.
• Failures that weren't quarantined: a headline with the count and exit code, plus why quarantining didn't apply (lookup failed, --disable-quarantining, or turned off in Trunk). Then one annotation per failed test, with file= when known, title=suite/name, up to 20 lines of the failure message and its Trunk link. At most 9 tests are annotated, since GitHub shows 10 errors per step; the headline says when there are more.
• A run that failed outright (type: "ERROR"): the same lines as render_error, minus the exit code line.
• Passing runs, all-quarantined runs and no-tests-with-exit-0 print nothing.

The wording comes from constants in report/text.rs that the text summary now uses too, so the two can't drift.

Reading the summary back.

• The report types derive Deserialize. ExitCode reads a number back as its named code, or PassedThrough for anything else.
• json::from_str / json::read_file return an owned json::Summary. A schemaVersion newer than the CLI knows is an error rather than a guess.
• A test checks that a summary read back serializes to exactly the same JSON.

Choices worth a look.

• "Nothing to say" exits with ExitCode::Failure (1), not a new documented code. The command is hidden and only the action calls it, so a new variant would only add noise to --help and the schema.
• Annotations attach to a file but not a line, since the summary has no line numbers.

Testing.

• 14 new tests in cli/tests/summary/github.rs: each outcome, escaping, the 9-annotation cap, both error paths, the schema-version check, the round trip, and two that run the binary (prints and exits 0; exits 1 with empty stdout for a passing run and for a missing file).
• cargo test -p trunk-analytics-cli -p test_report -p constants: all pass except the three Swift xunit tests in tests/upload.rs, which fail the same way on the base branch locally.
• cargo clippy -p trunk-analytics-cli --tests: no new warnings. cargo fmt --all --check: clean.

🤖 Generated with Claude Code

[trunk-staging-io]

   

Failed Test	Failure Summary	Logs
pending_quarantine_test should be quarantined when run with variant	A test marked as pending was expected to fail but unexpectedly passed.	Logs ↗︎
variant_quarantine_test should be quarantined when run with variant	The quarantined variant test failed because it expected 2 + 2 to equal 5, but the result was 4.	Logs ↗︎
multiple_exception_error_test nested context with multiple failures multiple expectations that fail	The failure aggregation test reports three failed expectations and an additional RuntimeError raised inside the aggregation block.	Logs ↗︎
multiple_exception_error_test with before and after hook failures test execution with hook errors	The test failed because both its before hook and after hook raised errors, preventing RSpec from matching the failure to a backtrace line.	Logs ↗︎

... and 3 more

View Full Report ↗︎ ⋅ Docs

[codecov-commenter]

Codecov Report

❌ Patch coverage is 95.31915% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.62%. Comparing base (46db108) to head (0b056b4).

Files with missing lines	Patch %	Lines
cli/src/summary_output/github.rs	96.59%	5 Missing ⚠️
cli/src/main.rs	83.33%	2 Missing ⚠️
cli/src/summary_output/json.rs	96.00%	2 Missing ⚠️
cli/src/summary_output/text.rs	81.81%	2 Missing ⚠️

Additional details and impacted files

@@                           Coverage Diff                            @@
##           dylan/trunk-19074-no-tests-exit-code    #1265      +/-   ##
========================================================================
+ Coverage                                 82.40%   85.62%   +3.21%     
========================================================================
  Files                                        85       86       +1     
  Lines                                     18919    19235     +316     
========================================================================
+ Hits                                      15591    16470     +879     
+ Misses                                     3328     2765     -563     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[TylerJang27]

nit: would like to see an example annotation of this in action

[dfrankland]

@TylerJang27 here's what github-annotations prints for a summary with two unquarantined failures. This is straight from annotates_each_failure_that_was_not_quarantined:

::error::2 tests failed and were not quarantined, using exit code: 1
::error file=src/math.rs,title=Math%2C Tests%3A unit/subtracts::Failed%0Ahttps://app.trunk.io/test-org/flaky-tests/test/subtracts-id?repo=trunk-io%252Fanalytics-cli
::error title=MathTests/divides::expected 1, got 2%0A100%25 wrong%0Ahttps://app.trunk.io/test-org/flaky-tests/test/divides-id?repo=trunk-io%252Fanalytics-cli

On GitHub this renders as one run-level error, then one error per failed test. Each test error is titled suite/name, attached to its file when we know it, and its body is the first lines of the failure followed by the test's Trunk link.

Roughly how it renders (a mockup of the Annotations box on the run's summary page, with the escapes decoded)

Annotations
3 errors

❌ <job name>
2 tests failed and were not quarantined, using exit code: 1

❌ Math, Tests: unit/subtracts
_src/math.rs
Failed
https://app.trunk.io/test-org/flaky-tests/test/subtracts-id?repo=trunk-io%2Fanalytics-cli

❌ MathTests/divides
expected 1, got 2
100% wrong
https://app.trunk.io/test-org/flaky-tests/test/divides-id?repo=trunk-io%2Fanalytics-cli

• An annotation without a title (the run-level one) is headed by the job's name.
• %0A becomes a line break, %2C/%3A become ,/: in the title, and %25 becomes %, so the link's own %2F survives.
• Annotations with a file also list that path. With no line, GitHub doesn't place them inline in the PR's Files changed view.
• The step's log shows the same annotations as red Error: lines.

[trunk-io]

😎 Stack merged successfully - details.