[selul]

What

A new optimole export command group that starts an export of the whole library on the dashboard and downloads its ZIP archives:

Command	Does
export start [-o DIR] [--domain D]... [-c N] [--no-wait] [--timeout DUR] [--json]	Starts an export (or follows the one already running), waits, downloads each archive as soon as it is listed
export download [ID|latest] [same flags]	Downloads or resumes an existing export
export status [ID|latest] [--json], export list [--json], export delete ID	Inspect / free one of the 5 export slots

Files land in DIR/optimole-export-ID/DOMAIN/images-N.zip, plus an export.json manifest (no URLs).

How it works

• Follows the export while it runs (internal/export). One coordinator goroutine checks GET /api/asset-exports/{id} (5 s, then 15 s, then 30 s) and runs up to -c downloads. Archives download while the export is still building the next ones.

• Safe downloads.

  • Each archive goes to images-N.zip.<listed size>.part. After a cut it resumes with Range; if storage ignores the range it starts over, and a 416 empties the part file. A 2-minute stall restarts the download.
  • It is renamed into place only when its size matches the listing and every file in it reads back with its checksum. That check also catches a resume that joined two same-size versions of a rewritten archive. A failed check is downloaded once more from scratch.
  • An archive with no size from either the listing or storage is refused.

• Fresh state from the server.

  • A 403 from storage gets a fresh URL by checking the export again (at most 5 times).
  • An archive whose listed size changes is downloaded again, whether mid-run or on a later run.
  • Part files left from another size are removed.

• The export folder is protected.

  • One command at a time per export folder, held by an OS lock (flock / LockFileEx) on .optimole-export.lock. The manifest is only written while holding it.
  • Links planted in the folder aren't followed: domain folders must be plain folders, a link at a part file is removed and the file opened with O_NOFOLLOW, and export.json goes through a fresh O_EXCL file.

• Untrusted domain names. Folder names keep only letters, digits and .-_, at most 100 bytes, with no leading or trailing dots and no Windows device names. Names that collide case-insensitively get -2, -3… in sorted order, so a rerun maps the same way.

• Conflicts on start. The dashboard's new codes are used (Codeinwp/optimole-service#1919):

  • export_in_progress + export_id: follow that export;
  • export_limit_reached: say so and name the oldest export to delete.

  Servers without codes (production until #1919 ships) are handled through the export list. Below the limit, a bare 409 can only mean the newest export, which is followed even if it has finished since: staging hit that race live. If the create call may have reached the server but got no answer, it isn't repeated, and a running export is followed.

• Same rules as uploads. Download URLs never get the API key, are never printed or written to the manifest, must be https (or localhost), and redirects are refused.

• Interrupting is safe. Ctrl-C (exit 130), --timeout (exit 1) and a failed status check all stop the downloads in flight. Their partial files stay for the next run, and the command prints a ready-to-paste optimole export download ID -o … with the path quoted for the platform's shell. It says the export "continues on the server" only when the export hadn't finished.

Reviews

Round	Head	Result
Codex 1 (gpt-6-astra, high), full	0e571c5	10 findings + 3 missing tests, all fixed in fd86179 / 54e0b1c
Codex 2, focused delta	54e0b1c	4 findings, fixed in 92762a5
Codex 3, focused delta	92762a5	2 resume-hint quoting findings, fixed in f9e974a
Codex 4, focused delta	f9e974a	Earlier quoting findings closed. @ % = now quoted (9c335fc, trivial follow-up). Accepted: cmd.exe and PowerShell both expand %/$ inside double quotes, and PowerShell keeps a doubled trailing backslash (same folder). No single Windows form fits both shells; noted in shellQuote
Codex 5, focused delta	8990156	No production issue. The new test cancelled on a 200 ms timer that could fire before the CLI had read the export; now it cancels on the archive request (4a6daef, test-only). 4e6fe82 (test-only) makes the parallel-download test count overlapping requests with a barrier instead of a 30 ms delay, after Windows CI caught the timing
pirate-bot	—	Not requested (owner's call)

Tests

Level	Result
go test -race ./... (3 runs)	PASS
gofmt -l, go vet (darwin, linux, windows, freebsd)	PASS
New tests: internal/api/exports_test.go, internal/export/export_test.go, internal/cli/export_test.go. The fake gained export endpoints and an archive store with Range, cuts, stalls, ignored ranges, expiring URLs, rewrites, size-less listings and a download delay	PASS
Go 1.23.0 floor, go test ./...	PASS
Mutation check: guards broken one at a time	60/60 caught (56 earlier, plus both resume-hint branches and the download limit one too high / one too low)
CI on 3 OSes × Go 1.23.x / stable	PASS on 92762a5, f9e974a and 9c335fc and 4e6fe82 (Windows had caught the resume-hint quoting, and on 4a6daef the timing in the parallel-download test)
Live on staging (fresh optimole-e2e+… accounts, erased after): upload 3 files → export start --no-wait → export start (bare 409 → follows) → zip holds the 3 files byte for byte → re-download skips → status/list/delete	PASS (runs 2 and 3; run 1 found the 409 race fixed in fd86179). A real staging archive passes the per-file checksum check
Live multi-part on staging (CLI 9c335fc): 48 random 24 MiB files (1.21 GB) → export start	PASS. Export 27 had 2 parts: 1,006,947,324 bytes (40 files) and 201,389,500 bytes (8 files), as the 20-file batch and 1 GB rule predicts. Part 1 was listed and downloading 31 s before the export finished, and part 2 downloaded alongside it. Both passed the per-file checksum check, and all 48 files match byte for byte. Part 1 took 32 minutes (the link to S3 ran at ~460 KB/s at the time) without a false stall
Live resume on staging: Ctrl-C during the download of export 28 (144 MiB, 1 part)	PASS. Exit 130 at 51,546,453 bytes, .part kept, hint printed. The rerun fetched exactly the missing 99,495,683 bytes with a Range request to S3, and all 6 files match byte for byte. This run found the "continues on the server" wording for a finished export, fixed in 8990156

Not proven

• A part over 1 GB (the first batch of 20 files always goes in, whatever its size) and exports over a few GB: covered by the fake only.
• The coded 409 path against a real server: it waits for optimole-service#1919 to deploy.
• On Windows, the link tests skip when symlinks can't be created. The lock is tested in CI.

🤖 Generated with Claude Code