Skip to content

fix(create-app): design contracts for local Docker start failure and transient UVE 403 (#37262) - #37264

Draft
fmontes wants to merge 5 commits into
issue-37262-create-app-docker-uve-specfrom
issue-37262-create-app-docker-uve
Draft

fix(create-app): design contracts for local Docker start failure and transient UVE 403 (#37262)#37264
fmontes wants to merge 5 commits into
issue-37262-create-app-docker-uve-specfrom
issue-37262-create-app-docker-uve

Conversation

@fmontes

@fmontes fmontes commented Aug 28, 2026

Copy link
Copy Markdown
Member

Proposed Changes

PR 2 of 2 for #37262, stacked on
#37263 (the spec). Draft — design artifacts only so far; implementation follows.

  • specs/37262-create-app-docker-uve/data-model.md — run state, compose health/dependency topology, UVE payload
  • specs/37262-create-app-docker-uve/contracts/compose-service-contract.md — what the demo compose file guarantees its consumers
  • specs/37262-create-app-docker-uve/contracts/cli-exit-contract.md — what the CLI guarantees on every exit path

plan.md, research.md and quickstart.md are gitignored by this repo's spec-kit setup (process-only, kept local), so they aren't in the diff.

⚠️ Measurement overturned the issue's root cause 2

Two experiments (M5/64GB host, dotCMS constrained to 2 CPUs / 4G) disproved the "transient startup race" diagnosis:

Clean boot has no settling window. UVE endpoint usable at 46stwo seconds before /dotmgt/readyz goes green. The starter import (T+20s) and ES reindex (T+44s) both complete inside Tomcat startup; the connector accepts no traffic until after them.

The reporter's real path reproduces it, permanently. Kill dotCMS mid starter-import, hand-start it (reproduction step 4), and you get the reported log exactly — then 403 on 193 consecutive attempts over ~7 minutes, zero successes. Server side: User 'Admin User' does not have READ permissions on Site 'demo.dotcms.com'. The interrupted import never wrote the site's permission rows and the restart does not repair them.

Consequences for this PR's design:

  • The read-before-write gate (X3/US3) as specified would poll forever. Replaced with a single GET probe; retry restricted to 5xx; 403 explicitly does not retry.
  • The warning text was wrong. "Configure UVE manually" fails identically — the CLI must say the instance is unrecoverable and to run docker compose down -v.
  • D2 (poll budget) is resolved as moot — see cli-design-decisions.md.
  • Fixing the compose file removes the 403, so US1 is the actual fix rather than just the MVP.
  • Backend half filed separately as An interrupted first boot silently bricks the instance: starter import leaves site permissions unwritten and every Apps API call 403s forever #37268 — an interrupted first boot silently bricks any instance, not just CLI-created ones.

Two CLI decisions still open (D1, D3)

cli-design-decisions.md documents them. D1 matters: contract X1 said to emit recoverable state "from a finally-equivalent position", which is not implementablefinally does not run on process.exit(), and there are 17 such call sites, 13 inside a single try. X1's implementation note is withdrawn in this PR pending that decision; the guarantee stands, the mechanism does not.

Research corrected four assumptions from the issue

Worth reading before reviewing the contracts — each of these changed the design:

  1. "Six siblings already use condition: service_healthy" — actually three, and none of them gates dotcms on OpenSearch being healthy; every one uses db: service_healthy + opensearch: service_started. Gating on both, as the issue asks, deviates from every precedent in the repo. Kept it anyway — this stack is driven by an unattended CLI, so an OpenSearch that is up-but-not-ready is a failure nobody is present to diagnose — but using single-node-os-migration's already-proven probe (-k for the self-signed cert, -u admin:admin because the stack sets DOT_ES_AUTH_BASIC_PASSWORD) rather than a new one.

  2. restart: unless-stopped does not restart an unhealthy container. Compose restart policies react to container exit, not health status (health-driven restart is a Swarm feature). So the "it will flap" risk I raised on the spec PR cannot happen. The real failure mode of a bad probe is the opposite and worse for a CLI: docker compose up --wait blocks until timeout. That reframes review question 2 — the question isn't flapping, it's whether start_period: 180s clears a cold starter import.

  3. The management port is unauthenticated. InfrastructureManagementFilter authorizes purely by the port a request arrived on — no credential check, no IP allowlist. A bare 8090:8090 binds 0.0.0.0, putting /dotmgt/health and /dotmgt/metrics on the local network. The contract therefore requires 127.0.0.1:8090:8090 — stricter than what the issue asked for and stricter than both precedent compose examples. That answers review question 1 with a recommendation instead of leaving it open.

  4. The Jest harness already existsjest.config.ts, tsconfig.spec.json, and @nx/jest/plugin are all in place; only spec files are missing. So there's no harness to build. (node_modules is absent in my worktree, so pnpm install is a prerequisite before the Red gate can be demonstrated.)

One compatibility constraint the compose edit must not break

updateDockerComposeStarterUrl (src/index.ts:487) rewrites the compose file with

/^(\s*["']?CUSTOM_STARTER_URL["']?\s*:\s*).+$/m

and throws if there's no match. Converting that key to a block scalar, an anchor, or - KEY=value list form would break --starter for every already-installed CLI, with no release able to reach them. Recorded as contract C5 with a regression step in the quickstart.

Sequencing

The compose change is independently shippable and lands first: an old CLI running docker compose up -d against the new file gets a dotCMS that actually starts and a restart policy, and ignores 8090 entirely. It fixes every installed CLI with no release. The CLI resilience work (non-fatal UVE, always-emit recoverable state, try/finally, the truthy-Result fix) has no dependency on it, so a slow review of the highest-risk file doesn't block the rest.

Checklist

  • Tests — not yet; TDD gate is next. Every behavior change has an assigned Jest spec, with one declared exception (cold-start compose behavior needs a real Docker daemon and a multi-minute starter import) that needs explicit sign-off per constitution Principle V
  • Translations — n/a
  • Security Implications Contemplated — finding 3 above: the design binds the unauthenticated management port to loopback rather than publishing it on 0.0.0.0

Additional Info

  • ADR gate: ADR-0019 (date-lockstep SDK versioning, accepted) is binding — this CLI fix ships through a dotCMS release, not a standalone SDK publish, and it's the reason image-tag pinning is deferred to a P2 follow-up rather than folded in here. ADR-0016 was read in full: it's an unfilled template and imposes nothing. No conflicts.
  • Two ADRs proposed (not created — spec-kit never authors ADRs): "CLI readiness means the operation you're about to perform succeeds", and "the management port is unauthenticated and must never be published beyond loopback."
  • No Java, no com.dotmarketing.*, no DB/ES/REST contract change — not rollback-unsafe.

Refs #37262

This PR fixes: #37262

@fmontes fmontes changed the title docs(create-app): add design contracts and data model for #37262 fix(create-app): design contracts for local Docker start failure and transient UVE 403 (#37262) Aug 28, 2026
@claude

claude Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Claude finished @fmontes's task in 1m 43s —— View job


Review

The diff is four design docs plus one Bash script; the only executable code is verify-cold-start.sh. The compose file it targets (../assets/docker-compose.yml) doesn't exist yet — that's the intended TDD Red state, not a defect. Review below is scoped to the script.

New Issues

  • 🟡 Medium: core-web/libs/sdk/create-app/scripts/verify-cold-start.sh:96-102 — the service_healthy assertion is claimed for dotcms but the fast grep half isn't scoped to it. grep -A3 -E '^\s+db:' matches any service's depends_on: db: {condition: service_healthy} (and the top-level db: block). Because it's OR'd before the correctly-scoped python3 check, a stack where some other service gates on db healthy — but dotcms does not — short-circuits to a false PASS. The python clause already does exactly what the assertion says; the grep half only weakens it. Also: \s is a GNU extension and never matches under BSD grep on the macOS hosts this script clearly targets (ipconfig getifaddr en0, line 251), so on macOS the grep half is dead and only python runs — harmless, but the grep half provides no value on either platform. Recommend dropping the grep clause and keeping only the scoped python check. Fix this →

  • 🟡 Medium: core-web/libs/sdk/create-app/scripts/verify-cold-start.sh:249-272 — the T007 negative check (LAN address must refuse 8090) can pass vacuously on the exact platform it targets. Assumption: on Docker Desktop for macOS/Windows, published ports are reached through a VM/proxy and the host's LAN address (ipconfig getifaddr en0) is typically not the reachable path even for a 0.0.0.0 binding. What to verify: run this against a compose file that publishes 8090:8090 (wildcard) on a Docker Desktop Mac — if it still reports PASS, the check isn't actually detecting wildcard exposure and gives false assurance for the finding-3 security guarantee. The loopback-only property is more reliably asserted from the compose text (the static half at line 136 already does this), so the runtime negative check is the weaker of the two and shouldn't be the thing the security claim rests on.

  • 🟡 Medium: core-web/libs/sdk/create-app/scripts/verify-cold-start.sh:155-160 — warm-start detection hardcodes dotcms/dotcms:latest. If the compose file this PR will ship pins a specific tag (ADR-0019 defers image-tag pinning to P2, so latest is plausible for now — but not guaranteed), the "warm start" note silently never fires and a warm run is misreported as cold. Low impact (informational output only, doesn't affect pass/fail), but derive the image ref from compose config rather than hardcoding it. Fix this →

Nothing blocking. set -uo pipefail, the -a/-q distinction in health_of/state_of, the false-green guard at line 174, and the vacuous-pass guard at line 263 are all handled correctly. The design docs read as internally consistent with the corrected 403 diagnosis described in the PR body; no code contract to verify against them yet.

· issue-37262-create-app-docker-uve

@fmontes
fmontes force-pushed the issue-37262-create-app-docker-uve branch from 244a69d to 38e8ae1 Compare August 28, 2026 11:59
@fmontes
fmontes force-pushed the issue-37262-create-app-docker-uve branch 3 times, most recently from 2a8895a to b010dd4 Compare August 28, 2026 15:17
fmontes and others added 5 commits August 28, 2026 14:37
Phase 1 design artifacts from /speckit-plan. plan.md, research.md and
quickstart.md stay local per .gitignore — this repo tracks only spec.md,
data-model.md and contracts/.

Research corrected four assumptions carried in the issue:

- "Six siblings use condition: service_healthy" — actually three, and NONE of
  them gates dotcms on opensearch being healthy (all use service_started).
  Gating on both deviates from every precedent; justified because this stack is
  driven by an unattended CLI, using os-migration's proven opensearch probe.

- restart: unless-stopped does NOT restart an unhealthy container — Compose
  restart policies react to exit, not health. The feared "flapping" cannot
  happen; the real risk is the opposite, `--wait` blocking on a bad probe.

- The management port is authorized purely by arrival port — no credential
  check, no IP allowlist. Publishing 8090 on 0.0.0.0 would expose
  /dotmgt/health and /dotmgt/metrics to the local network, so the contract
  requires 127.0.0.1:8090:8090.

- The Jest harness already exists (jest.config.ts, tsconfig.spec.json,
  @nx/jest/plugin); only spec files are missing. node_modules is absent in this
  worktree, so pnpm install is a prerequisite for the Red gate.

Also records a compatibility constraint the compose edit must not break: the
file must keep a line matching the CUSTOM_STARTER_URL regex in
updateDockerComposeStarterUrl, which throws on no match in every installed CLI.

Refs #37262

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…gn findings

Adds verify-cold-start.sh (T005-T008) — the executable form of the compose
acceptance checks, accepted at the T009 gate as the recorded Principle V
substitute for automated coverage, since the behavior needs a real Docker
daemon and a multi-minute starter import.

Adds cli-design-decisions.md covering three questions the contracts left open,
and withdraws contract X1's implementation note, which was wrong: it required a
`finally`-equivalent position, but `finally` does not run on process.exit() and
there are 17 such call sites, 13 inside a single try.

Measurement changed the diagnosis. On a clean boot there is NO settling window:
the UVE endpoint is usable at 46s, two seconds BEFORE /dotmgt/readyz goes green,
because the starter import and ES reindex complete inside Tomcat startup and the
connector accepts no traffic until after them.

Reproducing the reporter's actual path instead — kill dotcms mid starter-import,
then hand-start it — reproduces the 403 exactly, and it is PERMANENT: 193
consecutive attempts over ~7 minutes, zero successes. The server reports the
admin user lacking READ permission on demo.dotcms.com; the interrupted import
never wrote the site's permission rows and a restart does not repair them.

So the read-before-write gate polls forever against a condition that never
clears, the poll budget question is moot, and the planned "configure UVE
manually" warning is wrong advice — manual setup fails identically. Fixing the
crash removes the 403 entirely.

Refs #37262

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…osis

/speckit-analyze flagged three CRITICAL inconsistencies: the polled-GET
contract outlived the diagnosis that justified it, and two of the three
contradicting files are tracked in this PR — so a reviewer was reading a
contract that contradicted the spec in the same stack.

contracts/cli-exit-contract.md X3 and data-model.md's UVEAppConfig precondition
both said "poll GET until 200, retry on 401/403/5xx". Measurement showed a 403
here is terminal, not transient: 193 consecutive failures over ~7 minutes after
an interrupted starter import, because the site's permission rows were never
written. Polling would spin forever.

Both now specify a single GET probe, retry on 5xx only, and no retry on 403.
data-model.md gains a status-to-message table making the terminal-403 path
explicit: on 403 the CLI must tell the user to recreate the instance with
`docker compose down -v`, and must NOT offer manual UVE setup steps, which fail
identically for the same missing permissions.

Titles corrected from "transient UVE 403" to "permanent".

Refs #37262, #37268

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ation

D3 decided: silent auto-reuse on CI only; otherwise ask, and let the user stop
right there. Two points that settles — the prompt must offer abort, not just
reuse, since someone who did not expect a dotCMS on 8082 needs to stop and look;
and even the CI path prints a notice, because "silent" means no prompt, not no
output. A scripted run quietly attaching to an unknown instance is the failure
this is meant to avoid. No TTY without a CI env var is treated as CI: there is
nobody to answer, so blocking is the worst option.

D1 recommendation corrected. An earlier draft said "Option A for the guarantee,
Option B for the UVE path", which was imprecise — the UVE site does not need
throwing or catching at all. X2 requires the run to CONTINUE, so that
process.exit(1) is simply deleted and replaced with ordinary control flow. The
whole change is one process.on('exit') handler plus one deleted exit; there is
no 13-site refactor.

Refs #37262

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ng the shared example

Resolves all eight open design decisions and rescopes the compose work.

The original plan hardened
docker/docker-compose-examples/single-node-demo-site/docker-compose.yml, which is
fetched from main at runtime and also used directly by README readers. Every
hardening step we wanted was therefore a behavior change shipped unversioned to
consumers who never asked for it. Gating on opensearch health was the sharpest
case: it introduces a way for dotCMS to NEVER start if that probe later breaks —
an opensearch:1 -> :2 bump invalidating admin:admin would do it — where today the
container starts regardless.

So the CLI now ships its own compose file, bundled in the npm package, and the
shared example is left untouched. Nothing else reads the CLI's file, so it can be
strict at no cost to anyone: both services gated on service_healthy, livez
healthcheck with start_period 120s (~2.5x the measured 46s boot), restart
policies, and 8090 published loopback-only.

Accepted consequence: users on <=1.2.5 keep fetching the old shared file and are
not repaired. This starts fresh local instances rather than serving CI, no known
users have it in CI, and `npx @dotcms/create-app` resolves to latest anyway — only
a warm npx cache stays behind.

Bundling also removes downloadFile's missing timeout, absent redirect handling and
lack of retry from the default path. A ComposeSource interface keeps remote
fetching one env var away (DOTCMS_COMPOSE_URL) so a field hotfix needs no release.

Other decisions recorded: X1 emits via a synchronous process.on('exit') handler
(finally does not run on process.exit, and there are 17 such call sites); .env is
always named .env, written if absent; port reuse prompts on a TTY offering reuse
or abort, and auto-reuses with a printed notice on CI or no TTY; --wait-timeout is
600s conditional on continuous feedback for the whole wait, since ten minutes of
frozen spinner is the failure this issue was reported for.

Deliberately still open: the image tag stays `latest`, so the drift the report
flagged and ADR-0019 alignment are deferred, not resolved.

verify-cold-start.sh moves to core-web/libs/sdk/create-app/scripts/ and targets
the CLI's own file.

Refs #37262

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@fmontes
fmontes force-pushed the issue-37262-create-app-docker-uve branch from 4877b92 to c3243d6 Compare August 28, 2026 20:38
@github-actions github-actions Bot added Area : Frontend PR changes Angular/TypeScript frontend code Area : SDK PR changes SDK libraries labels Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area : Frontend PR changes Angular/TypeScript frontend code Area : SDK PR changes SDK libraries

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

create-app: local Docker run never starts dotCMS, then a transient UVE 403 aborts the CLI and discards the project

1 participant