fix(create-app): design contracts for local Docker start failure and transient UVE 403 (#37262) - #37264
Conversation
|
Claude finished @fmontes's task in 1m 43s —— View job ReviewThe diff is four design docs plus one Bash script; the only executable code is New Issues
Nothing blocking. · |
244a69d to
38e8ae1
Compare
2a8895a to
b010dd4
Compare
Phase 1 design artifacts from /speckit-plan. plan.md, research.md and quickstart.md stay local per .gitignore — this repo tracks only spec.md, data-model.md and contracts/. Research corrected four assumptions carried in the issue: - "Six siblings use condition: service_healthy" — actually three, and NONE of them gates dotcms on opensearch being healthy (all use service_started). Gating on both deviates from every precedent; justified because this stack is driven by an unattended CLI, using os-migration's proven opensearch probe. - restart: unless-stopped does NOT restart an unhealthy container — Compose restart policies react to exit, not health. The feared "flapping" cannot happen; the real risk is the opposite, `--wait` blocking on a bad probe. - The management port is authorized purely by arrival port — no credential check, no IP allowlist. Publishing 8090 on 0.0.0.0 would expose /dotmgt/health and /dotmgt/metrics to the local network, so the contract requires 127.0.0.1:8090:8090. - The Jest harness already exists (jest.config.ts, tsconfig.spec.json, @nx/jest/plugin); only spec files are missing. node_modules is absent in this worktree, so pnpm install is a prerequisite for the Red gate. Also records a compatibility constraint the compose edit must not break: the file must keep a line matching the CUSTOM_STARTER_URL regex in updateDockerComposeStarterUrl, which throws on no match in every installed CLI. Refs #37262 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…gn findings Adds verify-cold-start.sh (T005-T008) — the executable form of the compose acceptance checks, accepted at the T009 gate as the recorded Principle V substitute for automated coverage, since the behavior needs a real Docker daemon and a multi-minute starter import. Adds cli-design-decisions.md covering three questions the contracts left open, and withdraws contract X1's implementation note, which was wrong: it required a `finally`-equivalent position, but `finally` does not run on process.exit() and there are 17 such call sites, 13 inside a single try. Measurement changed the diagnosis. On a clean boot there is NO settling window: the UVE endpoint is usable at 46s, two seconds BEFORE /dotmgt/readyz goes green, because the starter import and ES reindex complete inside Tomcat startup and the connector accepts no traffic until after them. Reproducing the reporter's actual path instead — kill dotcms mid starter-import, then hand-start it — reproduces the 403 exactly, and it is PERMANENT: 193 consecutive attempts over ~7 minutes, zero successes. The server reports the admin user lacking READ permission on demo.dotcms.com; the interrupted import never wrote the site's permission rows and a restart does not repair them. So the read-before-write gate polls forever against a condition that never clears, the poll budget question is moot, and the planned "configure UVE manually" warning is wrong advice — manual setup fails identically. Fixing the crash removes the 403 entirely. Refs #37262 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…osis /speckit-analyze flagged three CRITICAL inconsistencies: the polled-GET contract outlived the diagnosis that justified it, and two of the three contradicting files are tracked in this PR — so a reviewer was reading a contract that contradicted the spec in the same stack. contracts/cli-exit-contract.md X3 and data-model.md's UVEAppConfig precondition both said "poll GET until 200, retry on 401/403/5xx". Measurement showed a 403 here is terminal, not transient: 193 consecutive failures over ~7 minutes after an interrupted starter import, because the site's permission rows were never written. Polling would spin forever. Both now specify a single GET probe, retry on 5xx only, and no retry on 403. data-model.md gains a status-to-message table making the terminal-403 path explicit: on 403 the CLI must tell the user to recreate the instance with `docker compose down -v`, and must NOT offer manual UVE setup steps, which fail identically for the same missing permissions. Titles corrected from "transient UVE 403" to "permanent". Refs #37262, #37268 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ation
D3 decided: silent auto-reuse on CI only; otherwise ask, and let the user stop
right there. Two points that settles — the prompt must offer abort, not just
reuse, since someone who did not expect a dotCMS on 8082 needs to stop and look;
and even the CI path prints a notice, because "silent" means no prompt, not no
output. A scripted run quietly attaching to an unknown instance is the failure
this is meant to avoid. No TTY without a CI env var is treated as CI: there is
nobody to answer, so blocking is the worst option.
D1 recommendation corrected. An earlier draft said "Option A for the guarantee,
Option B for the UVE path", which was imprecise — the UVE site does not need
throwing or catching at all. X2 requires the run to CONTINUE, so that
process.exit(1) is simply deleted and replaced with ordinary control flow. The
whole change is one process.on('exit') handler plus one deleted exit; there is
no 13-site refactor.
Refs #37262
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ng the shared example
Resolves all eight open design decisions and rescopes the compose work.
The original plan hardened
docker/docker-compose-examples/single-node-demo-site/docker-compose.yml, which is
fetched from main at runtime and also used directly by README readers. Every
hardening step we wanted was therefore a behavior change shipped unversioned to
consumers who never asked for it. Gating on opensearch health was the sharpest
case: it introduces a way for dotCMS to NEVER start if that probe later breaks —
an opensearch:1 -> :2 bump invalidating admin:admin would do it — where today the
container starts regardless.
So the CLI now ships its own compose file, bundled in the npm package, and the
shared example is left untouched. Nothing else reads the CLI's file, so it can be
strict at no cost to anyone: both services gated on service_healthy, livez
healthcheck with start_period 120s (~2.5x the measured 46s boot), restart
policies, and 8090 published loopback-only.
Accepted consequence: users on <=1.2.5 keep fetching the old shared file and are
not repaired. This starts fresh local instances rather than serving CI, no known
users have it in CI, and `npx @dotcms/create-app` resolves to latest anyway — only
a warm npx cache stays behind.
Bundling also removes downloadFile's missing timeout, absent redirect handling and
lack of retry from the default path. A ComposeSource interface keeps remote
fetching one env var away (DOTCMS_COMPOSE_URL) so a field hotfix needs no release.
Other decisions recorded: X1 emits via a synchronous process.on('exit') handler
(finally does not run on process.exit, and there are 17 such call sites); .env is
always named .env, written if absent; port reuse prompts on a TTY offering reuse
or abort, and auto-reuses with a printed notice on CI or no TTY; --wait-timeout is
600s conditional on continuous feedback for the whole wait, since ten minutes of
frozen spinner is the failure this issue was reported for.
Deliberately still open: the image tag stays `latest`, so the drift the report
flagged and ADR-0019 alignment are deferred, not resolved.
verify-cold-start.sh moves to core-web/libs/sdk/create-app/scripts/ and targets
the CLI's own file.
Refs #37262
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
4877b92 to
c3243d6
Compare
Proposed Changes
PR 2 of 2 for #37262, stacked on
#37263 (the spec). Draft — design artifacts only so far; implementation follows.
specs/37262-create-app-docker-uve/data-model.md— run state, compose health/dependency topology, UVE payloadspecs/37262-create-app-docker-uve/contracts/compose-service-contract.md— what the demo compose file guarantees its consumersspecs/37262-create-app-docker-uve/contracts/cli-exit-contract.md— what the CLI guarantees on every exit pathplan.md,research.mdandquickstart.mdare gitignored by this repo's spec-kit setup (process-only, kept local), so they aren't in the diff.Two experiments (M5/64GB host, dotCMS constrained to 2 CPUs / 4G) disproved the "transient startup race" diagnosis:
Clean boot has no settling window. UVE endpoint usable at 46s — two seconds before
/dotmgt/readyzgoes green. The starter import (T+20s) and ES reindex (T+44s) both complete inside Tomcat startup; the connector accepts no traffic until after them.The reporter's real path reproduces it, permanently. Kill dotCMS mid starter-import, hand-start it (reproduction step 4), and you get the reported log exactly — then 403 on 193 consecutive attempts over ~7 minutes, zero successes. Server side:
User 'Admin User' does not have READ permissions on Site 'demo.dotcms.com'. The interrupted import never wrote the site's permission rows and the restart does not repair them.Consequences for this PR's design:
GETprobe; retry restricted to5xx; 403 explicitly does not retry.docker compose down -v.cli-design-decisions.md.Two CLI decisions still open (D1, D3)
cli-design-decisions.mddocuments them. D1 matters: contract X1 said to emit recoverable state "from afinally-equivalent position", which is not implementable —finallydoes not run onprocess.exit(), and there are 17 such call sites, 13 inside a singletry. X1's implementation note is withdrawn in this PR pending that decision; the guarantee stands, the mechanism does not.Research corrected four assumptions from the issue
Worth reading before reviewing the contracts — each of these changed the design:
"Six siblings already use
condition: service_healthy" — actually three, and none of them gatesdotcmson OpenSearch being healthy; every one usesdb: service_healthy+opensearch: service_started. Gating on both, as the issue asks, deviates from every precedent in the repo. Kept it anyway — this stack is driven by an unattended CLI, so an OpenSearch that is up-but-not-ready is a failure nobody is present to diagnose — but usingsingle-node-os-migration's already-proven probe (-kfor the self-signed cert,-u admin:adminbecause the stack setsDOT_ES_AUTH_BASIC_PASSWORD) rather than a new one.restart: unless-stoppeddoes not restart an unhealthy container. Compose restart policies react to container exit, not health status (health-driven restart is a Swarm feature). So the "it will flap" risk I raised on the spec PR cannot happen. The real failure mode of a bad probe is the opposite and worse for a CLI:docker compose up --waitblocks until timeout. That reframes review question 2 — the question isn't flapping, it's whetherstart_period: 180sclears a cold starter import.The management port is unauthenticated.
InfrastructureManagementFilterauthorizes purely by the port a request arrived on — no credential check, no IP allowlist. A bare8090:8090binds0.0.0.0, putting/dotmgt/healthand/dotmgt/metricson the local network. The contract therefore requires127.0.0.1:8090:8090— stricter than what the issue asked for and stricter than both precedent compose examples. That answers review question 1 with a recommendation instead of leaving it open.The Jest harness already exists —
jest.config.ts,tsconfig.spec.json, and@nx/jest/pluginare all in place; only spec files are missing. So there's no harness to build. (node_modulesis absent in my worktree, sopnpm installis a prerequisite before the Red gate can be demonstrated.)One compatibility constraint the compose edit must not break
updateDockerComposeStarterUrl(src/index.ts:487) rewrites the compose file withand throws if there's no match. Converting that key to a block scalar, an anchor, or
- KEY=valuelist form would break--starterfor every already-installed CLI, with no release able to reach them. Recorded as contract C5 with a regression step in the quickstart.Sequencing
The compose change is independently shippable and lands first: an old CLI running
docker compose up -dagainst the new file gets a dotCMS that actually starts and a restart policy, and ignores 8090 entirely. It fixes every installed CLI with no release. The CLI resilience work (non-fatal UVE, always-emit recoverable state,try/finally, the truthy-Resultfix) has no dependency on it, so a slow review of the highest-risk file doesn't block the rest.Checklist
0.0.0.0Additional Info
com.dotmarketing.*, no DB/ES/REST contract change — not rollback-unsafe.Refs #37262
This PR fixes: #37262