Skip to content

fix(preview): say that previews need a preview URL and a simulator URL - #285

Open
angeloashmore wants to merge 7 commits into
mainfrom
claude/prismic-cli-273-review-uo7f1o
Open

fix(preview): say that previews need a preview URL and a simulator URL#285
angeloashmore wants to merge 7 commits into
mainfrom
claude/prismic-cli-273-review-uo7f1o

Conversation

@angeloashmore

@angeloashmore angeloashmore commented Sep 4, 2026

Copy link
Copy Markdown
Member

Resolves: #273

Description

An agent asked to configure previews for a deployed website sets the production preview URL and stops. Live previews in the Page Builder keep loading from localhost, so writers lose them after a deploy and nobody is told.

Nothing the agent reads says that previews have two settings. It reads prismic --help first, and the help named one. The help now says what previews are and names both settings. The preview command list names them too, in place of "preview configurations", which matches neither kind of preview.

The eval passed 0 of 3 in the last recorded run and 15 of 16 local trials on this branch. The other evals held. The result does not depend on prismicio/prismic-docs-v4#3951.

The eval fixture also starts from a repository that prismic init configured, so the eval measures the update after a deploy and not a first-time setup.

Checklist

  • If my changes require tests, I added them.
  • If my changes affect backward compatibility, it has been discussed.
  • If my changes require an update to the CONTRIBUTING.md guide, I updated it.

Preview

$ prismic --help
...
PREVIEWS
  Writers use previews to see draft content before it is published.
  Previews need two settings. After a deploy, set both:
    prismic preview add https://example.com/api/preview
    prismic preview set-simulator https://example.com
  A repository can hold multiple preview URLs but only one simulator URL,
  which the Page Builder loads live slice previews from.
  Run `prismic docs view previews` for details.

How to QA 1

$ node --run build
$ node dist/index.mjs --help
$ node dist/index.mjs preview --help
$ EVAL_TRIALS=5 PRISMIC_ALLOW_EVALS=true node --run evals -- \
    evals/configure-repositories.eval.ts evals/check-before-acting.eval.ts \
    -t "updates previews for production|sets up a content preview|consults the docs before"

🤖 Generated with Claude Code

https://claude.ai/code/session_01CgKyNmSz1frkfF8TqfL47F

Footnotes

  1. Please use these labels when submitting a review:
    ❓ #ask: Ask a question.
    💡 #idea: Suggest an idea.
    ⚠️ #issue: Strongly suggest a change.
    🎉 #nice: Share a compliment.

Agents asked to set up previews for a deployed website added the
production preview URL and stopped. The root help did not mention the
slice simulator URL, so the agent did not know a second setting exists
or that a deploy changes it.

The root help has a PREVIEWS section with both commands. The preview
row and the set-simulator row name the simulator URL. The
set-simulator argument says a website URL is accepted, which the
command already appends /slice-simulator to.

The production preview eval now sets a local simulator URL in its
fixture, the same as prismic init does. The eval measured "set the
simulator URL" instead of "update" it, and preview list showed no
stale value for the agent to notice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgKyNmSz1frkfF8TqfL47F
The set-simulator command row said "for live previews", and the url
argument said "Website or simulator URL". The root help now states
both facts, and the set-simulator description already says that the
command appends /slice-simulator to a website URL.

The two command strings are back to their earlier text. The three
evals that this branch affects still pass 5/5 each.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgKyNmSz1frkfF8TqfL47F
The PREVIEWS help held the commands, the purpose of the simulator URL,
the count of each setting, and the deploy step. The previews docs page
holds all of it too, so the same facts lived in two places and could
drift apart.

The help now names the two settings and sends the reader to the docs.
Agents read the page: one of five trials read it when the help was
long, and five of five read it now.

This makes the CLI depend on the docs page. Merge and deploy
prismicio/prismic-docs-v4#3951 before this branch, because the evals
workflow reads the docs at prismic.io.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgKyNmSz1frkfF8TqfL47F
The preview help called both settings "preview configurations". That
name matches neither kind of preview that the docs describe, and it
did not say that a repository holds preview URLs and a simulator URL.

The command list now names the preview URL and the simulator URL. An
EXAMPLES section shows both commands, in the form that other commands
in this CLI use.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgKyNmSz1frkfF8TqfL47F
The PREVIEWS section started with the two settings. A reader who does
not know what a preview is got instructions with no context.

One sentence now says what previews do. The rest of the section is
unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgKyNmSz1frkfF8TqfL47F
@angeloashmore
angeloashmore marked this pull request as ready for review September 5, 2026 04:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

evals: agent does not update the simulator URL after a production deploy

2 participants