Skip to content

test: Check an older CLI against a newer agent - #5

Merged
nfebe merged 4 commits into
mainfrom
feat/full-agent-coverage
Aug 15, 2026
Merged

test: Check an older CLI against a newer agent#5
nfebe merged 4 commits into
mainfrom
feat/full-agent-coverage

Conversation

@nfebe

@nfebe nfebe commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

A CLI built before the shared list shape existed still has to read a new agent's answer, and until
now nothing checked it: the staging agent runs an older build, so every comparison ran against a
response that predates the shape.

This pins it with the bytes the agent actually sends, where a collection answers under items and
under the name it used to answer by, plus the field it carried alongside. The released 0.2.0 binary
reads that correctly, and so does the current one.

The doc lines say what the output does when it is piped, which is nothing: deployment list | grep running keeps working, and --json is how the raw answer is asked for.

nfebe added 4 commits August 15, 2026 11:19
Commands that had always printed raw JSON started printing tables, which reads better on screen and
breaks anything parsing the output, including every workflow using this CLI today.

A table is now for a terminal. A pipe, a redirect or a CI step gets the raw answer, which is what it
got before.
Printing the raw answer whenever the output was not a terminal was too broad: it turned a
deployment's restart confirmation, and anything else that answers with a sentence, into JSON in a CI
log. Only a table is now held back, since something on the other end of a pipe would rather parse
the answer than read columns.
Holding a table back from a pipe broke the thing it was meant to protect: `deployment list` has
printed a table since it existed, so anything grepping one in a workflow would have got JSON
instead. No command that shipped ever printed differently depending on where its output went.

A table prints wherever it was asked for, and --json is how the raw answer is asked for, which is
what every command already did.
The shared list shape writes the rows under items and under the name the collection used to answer
by, so a client built before that shape existed keeps working. This pins it with the bytes the
agent actually sends, since a staging agent on an older build cannot show it.
@sourceant

sourceant Bot commented Aug 15, 2026

Copy link
Copy Markdown

Code Review Summary

This PR introduces critical test coverage for backward compatibility between the CLI and newer agent responses that use the items field alongside legacy specific fields (e.g., deployments). It also ensures that the CLI maintains its human-readable table formatting even when output is piped, while preserving the --json flag as the mechanism for raw data.

🚀 Key Improvements

  • Added TestOldCommandsReadANewAgentsAnswer to verify that the CLI correctly handles agent responses containing both legacy specific fields and the new items field without duplicating output.
  • Added TestOutputDoesNotChangeWhenPiped to ensure consistent output behavior across terminals and pipes.
  • Clarified the fallback layout logic in the documentation and changelog for responses not explicitly described by the agent.

💡 Minor Suggestions

  • Verify the content of the decoded JSON in TestJSONIsHowYouAskForTheAnswer to ensure data integrity.
  • Add a clarifying comment to the strings.Count assertion in TestOldCommandsReadANewAgentsAnswer regarding field duplication.

@sourceant sourceant Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review complete. No specific code suggestions were generated. See the overview comment for a summary.

@nfebe
nfebe merged commit e7d944a into main Aug 15, 2026
7 checks passed
@nfebe
nfebe deleted the feat/full-agent-coverage branch August 21, 2026 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant