Skip to content

run-with-vercel-deployment #5

Closed
1 of 1 issue completed
Closed

Description

@zskbot

Help me set up Endform so this repository's Playwright tests run on GitHub Actions against the Vercel preview of every pull request.
Endform is a drop-in replacement for the Playwright test runner: npx endform@latest test replaces playwright test and runs every test on its own machine. The tests stay as they are; setup adds a few settings to playwright.config.ts.
Guide me through the setup, make the changes you can, and ask me when you need information or an action from me. Ask when something is unclear instead of guessing, and tell me what you changed at each step.

  1. Find the Playwright project:
    • If this repository already has one, use it and leave its tests as they are.
    • If not, add @playwright/test as a dev dependency, create playwright.config.ts with testDir set to ./tests, forbidOnly: !!process.env.CI, and a single chromium project, and add test-results/ and playwright-report/ to .gitignore. Leave out workers and reporter settings; Endform handles those. Then ask which three or four pages or flows matter most, and write one small test for each with stable locators such as roles and labels. Then check they pass with npx endform@latest test --organization-id YthnM7vx.
    • In both cases, do not install browsers; Endform provides them. Run Endform from the directory that contains playwright.config.ts, and in CI set the job's working directory there.
  2. Merge these settings into playwright.config.ts and keep everything else:
    export default defineConfig({
      fullyParallel: true,
      retries: 2,
      use: {
        baseURL: process.env.BASE_URL || "<the URL the suite uses today>",
        trace: "retain-on-failure",
      },
    });
    • fullyParallel: true lets Endform run every test in parallel. Endform only reads fullyParallel per project, so test.describe.configure({ mode: "serial" }) and test.describe.serial do not keep tests together. If tests in a file share beforeAll state or worker fixtures, move those files to a project with fullyParallel: false and exclude them from the other projects with testIgnore. Each file in that project then runs on one machine, in order. A retry reruns only the failed tests, so setup that one test leaves for the next belongs in beforeAll or a fixture.
    • Endform retries only as often as retries allows. Keep the value if the config already sets one.
    • If the config starts a dev server through webServer, leave webServer and baseURL as they are, unless the tests will run against Vercel previews. Otherwise read baseURL from BASE_URL, falling back to the URL the suite uses today, and if none is in the repo, ask whether to add a webServer that starts the local dev server or to use a deployed URL such as staging or a Vercel preview.
    • trace: "retain-on-failure" gives failed tests a trace in the Endform dashboard.
  3. Some suites need an Endform config for one of these reasons. Endform transfers imported files and the environment variables tests reference automatically, so do not add one up front. Add the narrowest setting that fixes an observed failure. npx endform@latest test --debug --organization-id YthnM7vx shows which files and environment variables are sent to the runners:
    • a test reads files without importing them: additionalFiles (globs).
    • a test needs environment variables it does not reference through process.env: environmentVariables (regexes).
    • the tests target a localhost server that is not started through webServer: proxyNetworkHosts: ["<loopback>"], which sends that traffic through the local machine.
    • tests fail or time out under load because the target cannot handle many tests at once, such as a local server, a staging database, or a payment sandbox: concurrentTestLimits, for example [{ scope: "within-suite-run", limit: 40 }], or with label: "tag:@checkout" to limit only the tests that hit the bottleneck.
    • To add a setting, use the repository's existing Endform config, in this directory or a parent up to the repository root.
    • Convert a legacy endform.jsonc or endform.json with npx endform@latest migrate-config first, since a new endform.config.ts next to it silently replaces it.
    • If there is none, add endform as a dev dependency, create endform.config.ts next to playwright.config.ts using defineEndformConfig from "endform", and read https://endform.dev/docs/_llms-txt/run-and-configure-endform.txt for its options.
  4. The tests run against the Vercel preview, so baseURL must read process.env.BASE_URL, even when webServer starts a dev server. Then start webServer only when BASE_URL is not set, so local runs keep working.
  5. Add the endformdev/actions/run-with-vercel-deployment action with set-url-env-var: BASE_URL before npx endform@latest test, as shown in the Vercel section of https://endform.dev/docs/_llms-txt/github-actions-ci-and-vercel.txt. Endform reports on the Vercel check only when the workflow uses this action.
    • If a workflow already runs playwright test, directly or through a package script such as pnpm test:e2e, add the action before it and replace that command with npx endform@latest test, leaving the package script as it is. Keep the other steps, and drop what Endform replaces: browser installs and caches, --shard matrix jobs, and the job that merges their blob reports. Otherwise add a workflow as the docs show.
    • The workflow must run on pull requests, since previews exist only for them.
    • Take the Vercel project name from .vercel/project.json, or ask for it.
    • The action authenticates only through the Endform GitHub App with OIDC: add id-token: write to the workflow's permissions and keep contents: read, since listing any permission removes the ones not listed. Do not add an API key.
  6. Make sure the Endform GitHub App and Vercel are connected with npx endform@latest connect github --organization-id YthnM7vx and npx endform@latest connect vercel --organization-id YthnM7vx. Each says so when the integration is already connected. Otherwise it prints a URL where the install is finished in the browser and tells you which command to run next, so show the URL and follow its output until it is connected. These commands need a login: if one says you are not logged in, run npx endform@latest login, show the URL it prints, follow its output until the login completes, then run the command again.
  7. If the tests read other environment variables, such as app credentials, list them and ask for them to be added as GitHub Actions secrets. Reference them in the workflow, never write their values.
  8. Do not commit or push. When you are done, tell me what is working, list the changed files and ask me to commit and push them so CI runs Endform. Explain anything I still need to do. If the workflow has not run yet, say that setup is ready but still needs verification.

Do not add API keys or secrets to the repository.
These steps cover the whole setup, so do not read the docs up front. Only if a command fails in a way these steps do not explain, read https://endform.dev/docs/llms.txt.

Activity

  1. self-assigned this
    on Oct 31, 2026
  2. added
    bugSomething isn't working
    documentationImprovements or additions to documentation
    enhancementNew feature or request
    wontfixThis will not be worked on
    on Oct 31, 2026
  3. added theissue type on Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingdocumentationImprovements or additions to documentationenhancementNew feature or requestgood first issueGood for newcomerswontfixThis will not be worked on

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions