Skip to content

pg_walserver: standalone replication-protocol server (WAL/basebackup serve + push/pull archiving) - #1193

Open
dimitri wants to merge 130 commits into
mainfrom
pg_walsender-server
Open

dimitri wants to merge 130 commits into
mainfrom
pg_walsender-server

Conversation

@dimitri

@dimitri dimitri commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

pg_walserver (renamed from pg_walsender partway through review — see "Naming" below): a standalone binary speaking the real PostgreSQL replication wire protocol, hardened like a real network-facing server rather than a manual-testing toy, that can serve pg_basebackup, pg_receivewal and a real standby's own walreceiver directly out of a directory tree of WAL segments and base backups it owns — no live postmaster behind it, no pg_auto_failover monitor or archiver integration in this PR at all. It is a standalone, independently useful building block (works outside pg_auto_failover entirely — see docs/ref/pg_walserver.rst's full worked example with plain PostgreSQL core tooling), and the intended foundation the follow-up archiving-integration PR builds on, not a replacement for it.

What it serves

On the wire: IDENTIFY_SYSTEM, SHOW, BASE_BACKUP, TIMELINE_HISTORY, CREATE/READ/DROP_REPLICATION_SLOT, START_REPLICATION — exactly as a real primary — plus three project-specific extensions: FETCH_FILE (a restore_command-friendly one-shot file fetch), and CHECK_FILE/ARCHIVE_FILE (the push-side counterpart, idempotent and overwrite-safe, used by the archive-wal client below).

Sub-commands

  • serve — the accept loop (default command; pg_walserver [options] with no sub-command name works too).
  • scram-secret — print an archiver-passwd line for a role.
  • setup — write/validate a pg_walserver.ini route (--route/--path/--upstream/--hostname), fetch the upstream system identifier, optionally take the first base backup synchronously (--with-basebackup), and enable the embedded WAL capturer by default (--no-capture to opt out).
  • fetch-systemid / basebackup — the two steps setup calls internally, also usable standalone.
  • create-cert — create (or --force-rotate) a self-signed TLS certificate for --pgdata; setup calls this automatically the moment a second named route needs TLS for SNI-based routing.
  • archive-wal %p %f / restore-wal %f %p — drop-in archive_command/restore_command clients, using CHECK_FILE/ARCHIVE_FILE and FETCH_FILE respectively (named -wal rather than a bare archive/restore, matching pgBackRest's archive-push/archive-get and WAL-G's wal-push/wal-fetch convention, since basebackup already exists in this same CLI).
  • internal service pg-receivewal — hidden, pg_autoctl-style re-exec entry point the embedded capturer uses (see below).

Routing: dbname, TLS SNI, and a PgBouncer-style wildcard

A route key is an opaque string (pg_auto_failover's own <formation>/<group> convention is one choice among many, never given filesystem meaning), matched first by exact dbname, then — for a real physical standby, whose walreceiver always sends the literal dbname=replication regardless of primary_conninfo (libpqrcv_connect()'s own unconditional override) — by TLS SNI hostname, then by a "*" wildcard route. More than one named route requires TLS; pg_walserver refuses to start otherwise. See docs/ref/pg_walserver.rst's "Routing more than one cluster by name: TLS SNI" section for the full mechanism and its DNS/CNAME prerequisite.

The embedded pull capturer

A route (on by default via setup, capture = pull in pg_walserver.ini) gets its own long-lived, supervised child running the vendored pg_receivewal (src/bin/common/vendor/pg_receivewal/, shared with pg_autoctl) against upstream, captured directly into the route's own directory — no external pg_receivewal process required. Supervision reuses this codebase's own pattern for long-lived services rather than a bespoke loop: fork()+execv() of pg_walserver itself, re-entering as the hidden internal service pg-receivewal sub-command, matching pg_autoctl's own internal service postgres|listener|node-active shape (and its live-upgrade-under-PID-1 rationale). archive-wal remains a fully independent, monitor-agnostic defense-in-depth backstop on top of this, not a replacement for it.

Shared infrastructure

  • src/bin/common/process_supervisor.c/.h: the generic Erlang-style MaxR/MaxT restart-backoff ring buffer and PID-1-safe orphan-reap classification, now genuinely shared — pg_autoctl/supervisor.c's own internals call into it too (previously a symbol-renamed parallel implementation; consolidated in this PR, kept as an isolated first commit so it can be split into its own preliminary PR the same way src/bin/common/vendor/pg_receivewal/'s relocation was in pg_autoctl: relocate vendored pg_receivewal to src/bin/common #1194).
  • src/bin/common/fetch_client.c/.h: the FETCH_FILE client, used in-process by restore-wal (and, later, by pg_autoctl restore_command.c once the archiving-integration PR adds it).
  • HBA (archiver-hba.conf, host/hostssl/hostnossl, CIDR/hostname/samehost/samenet, trust/scram-sha-256/reject), SCRAM-SHA-256 auth, TLS, absolute per-connection auth deadlines, and a per-route replication-slot cap.
  • pgaftest (this project's own docker-compose-driven test DSL) gained postgres <name> / pg_walserver <name> cluster-level node sugar (a bare, unmanaged Postgres or pg_walserver box, deliberately distinct from pg_auto_failover's own future archiver node kind — see the sibling archiving-integration branch), plus a per-node command "..." override and hostname alias support (for TLS SNI testing) and init: true support.

Naming

Renamed pg_walsender → pg_walserver mid-PR once the scope grew to include receiving (the embedded capturer, ARCHIVE_FILE) alongside sending — "sender" undersold half the job. See src/bin/pg_walserver/README.md's own "Naming" section for the full survey against PostgreSQL's own glossary and external tools (pgBackRest, Barman, WAL-G).

Tests

Five tests/tap/specs/pg_walserver_*.pgaf specs, 22 steps total: standalone (7 — setup wizard, IDENTIFY_SYSTEM/SHOW/FETCH_FILE, real pg_receivewal START_REPLICATION, grammar edge cases, wildcard routing, a real pg_basebackup + real standby promoted via core tooling alone), sni_routing (4 — two real routes disambiguated purely by TLS SNI, auto-cert-creation, the no-TLS-with-2-routes hard failure), archive_command (5 — CHECK_FILE missing/matches/differs, overwrite-safety, create-cert, restore-wal), capture (3 — the embedded capturer runs, restarts on kill, stops cleanly with the server), pid1 (3 — the same capturer as a container's real PID 1, orphan-reaped, orderly-shutdown-not-SIGKILL). citus_indent/ci/banned.h.sh clean; PG17 build clean, zero warnings.

What's still out of scope for this PR

No pg_auto_failover archiver node kind, no monitor schema, no pg_autoctl restore command/archive command with quorum/full-archiver-node participation — all of that is the follow-up archiving-integration PR's job, building on top of what's here.

@dimitri
dimitri force-pushed the pg_walsender-server branch 2 times, most recently from 41d702d to 70c09b1 Compare September 26, 2026 19:03
@dimitri
dimitri force-pushed the pg_walsender-server branch from 70c09b1 to ea06c7b Compare September 27, 2026 02:15
dimitri added a commit that referenced this pull request Sep 27, 2026
…, rewrite restore_command.c to call fetch_client in-process

The archiving-integration-base merge just landed brought in pg_walsender's
review-round rewrite (flex/bison grammar, HBA tokenizer, CommandLine-based
CLI, common/fetch_client.c), which along the way dropped the "monitor" HBA
keyword and the refresher/monitor_hosts machinery behind it: PR #1193 has
no monitor extension of its own to query, so that feature was deliberately
lifted out of it to be reintroduced here, where the SQL (get_group_hosts/
get_group_hosts_hash) actually exists.

This commit does that:

  - hba.c/hba.h: restore the "monitor" ADDRESS keyword and hba_lookup()'s
    routePath/monitorUriPath/refreshSockPath parameters, on top of the new
    tokenizer (ported from the pre-merge tree, not re-derived).
  - monitor_hosts.c/.h, refresher.c/.h: restored from their last reviewed
    state (the refresher already uses the project's PGSQL wrapper and a
    retry policy, per an earlier review round, before being deleted along
    with the rest of the monitor feature).
  - auth.c/.h: WsAuthConfig grows monitorUriPath/refreshSockPath back;
    hba_lookup() is called with the route's path so "monitor" can match.
  - accept_loop.c/.h: the parent forks/supervises/restarts the refresher
    child and creates its datagram socket, mirroring the postmaster/
    refresher split documented in monitor_hosts.h.
  - cli_root.c: `serve` derives archiver-monitor.uri/archiver-refresh.sock
    from --pgdata, same as the other archiver-*  paths.
  - defaults.h, ws_util.c/.h: PG_AUTOCTL_ARCHIVER_NODES_FILE and
    ws_auth_deadline_remaining_ms() restored (both were only removed
    because the feature using them was gone).
  - Makefile: build monitor_hosts.c/refresher.c again.

Also, per review: `pg_autoctl restore command` (restore_command.c) used to
execv() into `pg_walsender fetch-file`, which no longer exists (pg_walsender
is a server binary, plus scram-secret, nothing else -- the FETCH_FILE
client moved to src/bin/common/fetch_client.c in the pg_walsender-server
PR). restore_command_run() now calls ws_fetch_file_client() directly,
in-process; docs (archiving-details.rst, pg_autoctl_restore_command.rst)
and the leftover stale src/bin/pg_walsender/fetch_client.c/.h copy (dead,
unreferenced by the Makefile since the move) are updated/removed to match.

Four pre-existing archiver test specs drove the raw `pg_walsender
fetch-file` CLI directly (archiver_walsender_auth, archiver_lifecycle,
archiver_walsender_protocol) or only had a stale comment about it
(archiver_warm_standby); updated to go through `pg_autoctl restore
command` instead, matching what a real restore_command line actually
runs.

Verified: PG17 docker build clean (zero warnings), monitor installcheck
20/6 tests green, citus_indent + banned.h.sh clean, and every archiver-*
spec touched by this change re-run green: archiver_walsender_auth (all 8
steps, including the "monitor" keyword and node2's registration/drop
follow-along), archiver_lifecycle, archiver_walsender_protocol,
archiver_warm_standby, archiver_fsm_quorum, archiver_fsm_maintenance,
archiver_archive_command, archiver_wal_segsize, and pg_walsender_standalone
(#1193's own suite, unaffected).
@dimitri
dimitri force-pushed the pg_walsender-server branch from 96cf12e to 599c702 Compare September 27, 2026 19:54
@dimitri dimitri changed the title pg_walsender: new standalone replication-protocol server pg_walserver: standalone replication-protocol server (WAL/basebackup serve + push/pull archiving) Sep 27, 2026
Additions to the existing src/bin/common/ shared library, adapted from
PostgreSQL's own source rather than reimplemented, prep for a real
replication-protocol server (a later PR) to authenticate clients by HBA
address/network and by SCRAM, the same way postgres's own postmaster does,
instead of accepting connections unauthenticated. Nothing here is used by
pg_autoctl today.

- ifaddr.c/.h (new): local interface address enumeration (getifaddrs()-
  based), the same code Postgres's own libpq/backend HBA matching uses to
  decide whether a connecting client's address falls inside a 'samehost'
  or 'samenet' pg_hba.conf entry.
- scram.c/.h + scram_compat.h (new): a server-side SCRAM-SHA-256 verifier
  (parse/build a stored SCRAM secret, run the exchange), built on top of
  Postgres's own common/scram-common.c, common/hmac.c, common/cryptohash.c
  and common/saslprep.c -- scram_compat.h is the small adapter layer
  translating between those internal APIs and what a frontend-side caller
  (not a backend) can actually link against.
- ipaddr.c/.h (extended): five new peer-address-matching entry points
  (ipaddrHostMatchesAddress, ipaddrInCIDR, ipaddrIsSameHostOrNet,
  ipaddrFindHostnamesFromAddress) ported from postgres's own
  src/backend/libpq/hba.c, for matching a numeric client address against
  an HBA file's host/CIDR/samehost/samenet entries.
- file_utils.c/.h (extended): write_file_atomic(), a rename()-into-place
  atomic write for a file another process might read concurrently (a
  config/state file another running service polls).

Makefile.common gains -lssl -lcrypto (scram.c's own crypto primitives)
and -lresolv (ifaddr.c/ipaddr.c's own resolver use).

Verified: PG17 build (pg_autoctl continues to link and behave identically;
everything here is compiled but unreferenced) + monitor installcheck
(19/19, unaffected); citus_indent --check and ci/banned.h.sh pass with no
exceptions needed (all of it already follows this project's own style).
A new binary (src/bin/pg_walsender/, 48 files) speaking the real Postgres
replication wire protocol: IDENTIFY_SYSTEM, SHOW, BASE_BACKUP, TIMELINE_
HISTORY, CREATE/READ/DROP_REPLICATION_SLOT, START_REPLICATION, and this
project's own FETCH_FILE (a restore_command-friendly one-shot file fetch,
alongside the standard commands). Hardened like a real network-facing
server rather than a manual-testing toy:

- pg_hba.conf-style HBA file (host/hostssl/hostnossl, CIDR/hostname/
  samehost/samenet address forms, trust/scram-sha-256/reject), built on
  the common/ipaddr.c address-matching this PR adds.
- SCRAM-SHA-256 authentication (common/scram.c) and TLS (OpenSSL),
  matching Postgres's own defaults.
- Absolute per-connection auth deadline (--auth-timeout), oversize pre-
  auth message rejection, a per-route replication-slot cap -- the same
  postmaster-like hardening a network-facing server needs that a one-off
  script doesn't.
- A routes file (archiver-routes.ini) mapping "<formation>/<group>" to a
  local storage root, and a small vendored ustar writer (vendor/tar.c,
  reformatted to this project's own style, unlike pg_receivewal's
  deliberately-untouched vendoring) for BASE_BACKUP's tar stream.
- A single-writer "nodes list" refresher process + read-only children,
  the same nodehost cache design a real postmaster uses for pg_hba.conf's
  own cached DNS resolution, so a lookup never blocks a connection.

Nothing in this PR calls pg_walsender from pg_autoctl yet: it is a fully
functional, independently testable server on its own, with no archiver/
monitor integration -- that wiring is a later PR's job. Includes the two
src/bin/common/ additions (ifaddr.c, scram.c/scram_compat.h) and
extensions (ipaddr.c's five new address-matching entry points, file_
utils.c's write_file_atomic()) this server needs, since nothing else
uses them yet either.

Build wiring: src/bin/Makefile builds pg_walsender alongside pg_autoctl/
pgaftest; Dockerfile copies the binary with a bracket-glob COPY (kept
optional so tests/upgrade can still build the Dockerfile against an old
release's source tree that predates this file); top-level Makefile's
docker-check/docker-indent now run correctly from a git worktree and
also run ci/banned.h.sh (ci-lint).

New test: tests/tap/specs/pg_walsender_standalone.pgaf -- a hand-crafted
routes file, HBA rule, and archiver-systemid, real WAL captured off node1
by the stock OS pg_receivewal (not this project's own vendored copy,
which is a separate PR), pg_walsender started directly by hand (no
pg_autoctl archiver integration exists yet in this PR's stack). Covers
IDENTIFY_SYSTEM, SHOW wal_segment_size, FETCH_FILE (byte-for-byte match
against the captured segment), and START_REPLICATION (a second, real
pg_receivewal streaming from pg_walsender itself, byte-for-byte match
against what was captured).

Verified: PG17 build + monitor installcheck (19/19, unaffected);
citus_indent --check and ci/banned.h.sh pass; pg_walsender_standalone.pgaf
passes end to end (4/4 steps) on PG17.
- Strip the incorrect "Copyright (c) Microsoft Corporation" line from
  every genuinely new file in this PR (pg_walsender's own code, plus the
  scram.c/.h/scram_compat.h additions to src/bin/common/); none of it is
  Microsoft's. Genuinely vendored files (ifaddr.c/.h, byte-for-byte from
  PostgreSQL) keep their real upstream PostgreSQL copyright, untouched.
- Remove ws_write_file_atomic (ws_util.c/.h) entirely: pg_walsender
  already links src/bin/common/, so duplicating write_file_atomic() had
  no Makefile-split justification (unlike the WAL-segment-arithmetic
  case, which genuinely can't link across the pg_autoctl/pg_walsender
  binary boundary). Ported its extra safety into the shared function
  instead of dropping it: write_file_atomic() now uses a pid-qualified
  tmp name (safe when two processes write the same path concurrently,
  which cmd_replication_slot.c's per-connection children can do),
  fsync()s the file before rename(), and fsync()s the containing
  directory afterward -- strictly more durable than its previous
  "tmp + rename" behavior, with no signature change, so every existing
  pg_autoctl caller benefits automatically. All 3 pg_walsender call
  sites (refresher.c x2, cmd_replication_slot.c) now call it directly.

Verified: PG17 build + monitor installcheck (19/19, unaffected);
citus_indent --check and ci/banned.h.sh pass; pg_walsender_standalone.pgaf
passes end to end (4/4 steps).
…r.l/repl_gram.y), and fscanf-based read_backup_label

Task 1 (highest-value review item): repl_command.c's hand-rolled
recursive-descent parser for the replication command language is
replaced by a real flex/bison pair adapted from PostgreSQL's own
src/backend/replication/repl_scanner.l and repl_gram.y:

- repl_scanner.l keeps the same token set, the same xq/xd exclusive
  states for single-quoted strings and double-quoted identifiers (with
  '' / "" escaping), and the same UCONST/RECPTR patterns as the real
  scanner. Backend-only bits are swapped for this project's own
  conventions: ereport() becomes log_error() plus a parse-failed flag,
  StringInfo becomes PQExpBuffer, and there is no bison-bridge/reentrant
  scanner -- pg_walsender forks one child per connection (accept_loop.c),
  so plain global yylex()/yyparse() state is safe, the same way this
  project's own pgaftest test_spec_scan.l/test_spec_parse.y already work.
  One addition beyond the real scanner: an exclusive <fname> state for
  this project's own FETCH_FILE extension, whose argument is a WAL
  segment filename (digits and letters, case-sensitive) that fits neither
  the UCONST nor the IDENT pattern.

- repl_gram.y keeps the same grammar shape (command alternatives,
  generic_option_list for BASE_BACKUP/CREATE_REPLICATION_SLOT options,
  create_slot_legacy_opt for the pre-9.4 EXPORT_SNAPSHOT/RESERVE_WAL/etc
  keywords) but the semantic actions fill in a plain WsCommand struct
  directly (repl_command.h), with a fixed-size WsCommandOption array
  standing in for a backend DefElem List -- no Node*/makeNode()/palloc(),
  none of which exist in this frontend project. ALTER_REPLICATION_SLOT,
  UPLOAD_MANIFEST and logical START_REPLICATION are dropped: pg_walsender
  never implements them. FETCH_FILE is added as this project's own
  extension, with no Postgres equivalent.

This also lets two pieces of ad hoc text-scanning be deleted, now that
their input arrives already parsed:
- cmd_base_backup.c's scan_options() (BASE_BACKUP's own option-list
  tokenizer) is gone; cmd_base_backup() now takes the parsed
  WsCommandOption array directly.
- cmd_replication_slot.c's parse_slot_name() is gone; slot names arrive
  pre-parsed via the grammar's IDENT token, which already implements the
  same quoting/case-folding PostgreSQL's own replication clients expect
  (case-preserved when double-quoted, folded to lower case otherwise --
  the old hand-rolled version never folded case on a bare name, which was
  a latent bug against real Postgres semantics).
- cmd_start_replication.c's own SLOT/PHYSICAL/TIMELINE/LSN token-eating
  is gone in favor of the grammar's parsed slotName/startLsn/timeline
  fields.

Build infra follows the exact pattern already used for
src/bin/pgaftest/test_spec_parse.y/test_spec_scan.l: repl_gram.c/
repl_gram.h/repl_scanner.c are generated once with bison/flex (both
already present in the Docker build stage, used for pgaftest) and
committed, `make generate` regenerates them, and `.gitattributes` marks
the generated files -citus-style so citus_indent leaves them alone. The
Dockerfile's existing "touch bison/flex generated files" step is
extended to cover pg_walsender's new files too, so the Docker build
never needs to invoke bison/flex itself (mirrors pgaftest exactly).

Task 3: cmd_base_backup.c's read_backup_label is reimplemented following
PostgreSQL's own read_backup_label() in
src/backend/access/transam/xlogrecovery.c: the same fscanf()-based,
fixed-format-string parsing of "START WAL LOCATION" and "START TIMELINE",
instead of the previous ad hoc line-splitting version. Unlike the
backend's ereport(FATAL, ...), a parse failure here is logged and the
function returns false, matching the existing caller's own fallback
behavior. Fields the backend also extracts but no caller here needs
(BACKUP METHOD, BACKUP FROM, START TIME, LABEL, INCREMENTAL FROM LSN)
are not ported, since nothing here uses them; the function's signature
(LSN string + timeline int) is unchanged.

Verification:
- `docker build --target run` (PG17): clean build, "All 19 tests
  passed." (monitor installcheck).
- citus_indent --check && ci/banned.h.sh: clean (fscanf/fopen/fclose
  marked IGNORE-BANNED, same convention as elsewhere in this file).
- tests/tap/specs/pg_walsender_standalone.pgaf: all steps pass,
  including a new test_004_grammar_edge_cases step added as permanent
  coverage for two protocol edge cases the port is specifically meant to
  get right: a double-quoted slot name with an embedded escaped quote
  (rejected at the *semantic* layer, not a syntax error, proving the
  <xd> lexer state extracted the identifier correctly) and a garbage
  command producing a clean ErrorResponse with the connection still
  usable afterwards. A BASE_BACKUP-with-quoted-option-value case was
  also verified manually against a standalone --insecure instance (reaches
  the semantic "no base backup configured" error, not a syntax error) but
  was not added to the permanent spec, to avoid the multi-layer shell-
  quoting fragility that would introduce into the .pgaf file for no
  additional grammar coverage beyond what test_004 already exercises.

Not included in this commit: Task 2 (hba.c tokenizer rewrite), Task 4
(main.c CommandLine-tree restructuring), Task 5 (refresher.c retry
policy / PGSQL wrapper), and Task 6 (missing-header-comment audit) are
deferred -- out of scope for the time available in this pass.
…ext_token()

hba_parse used to split each line on plain whitespace with strtok_r and
strip everything after the first '#' unconditionally, so a quoted field
containing a space or a literal '#', a trailing '# comment' after real
tokens, and a line-continuation backslash all either broke tokenizing
or were not supported at all.

Ported the tokenizing mechanism (not the feature set) of PostgreSQL's
own src/backend/libpq/hba.c next_token()/tokenize_auth_file():

  - next_hba_token() replaces strtok_r's per-field split: a field can be
    wrapped in double quotes (so it may contain spaces or a literal
    '#'), a doubled "" inside a quoted field is a literal '"', and an
    unquoted '#' starts a comment that runs to the end of the line.

  - hba_read_logical_line()/HbaLineReader join physical lines ending in
    a trailing backslash into one logical line before any field
    splitting happens, the same order of operations PostgreSQL uses
    (continuation joining happens first, so a comment or quote that
    spans a continued line behaves the same way it does in Postgres).

This project's own HBA feature set (documented in hba.h) is unchanged:
still exactly 5 whitespace/quote-delimited fields per logical line, no
@-include expansion, no comma-separated lists, no regular expressions.
A malformed logical line still rejects the whole file, matching
PostgreSQL's fail-closed behavior for a bad pg_hba.conf.

Rule fields are now individually strdup'd (freed by hba_rule_free_fields)
rather than pointing into the file's own buffer, since the tokenizer no
longer null-terminates fields in place inside a reusable per-line buffer.

Verified: full docker build (17 tests + monitor installcheck, no
warnings), citus_indent + banned.h.sh clean, the standalone pgaftest
spec pg_walsender_standalone.pgaf (5/5), and manual checks of a
comment-only line, a trailing '# comment', a quoted field with an
embedded space, and a backslash line continuation all parsing as one
correct rule set (confirmed against a deliberately unquoted variant of
the same line, which still triggers the 'Malformed HBA line' rejection
as before).
… framework

main.c used to hand-roll its own argv[1] dispatch (string-comparing
against "fetch-file"/"scram-secret") and then a manual getopt_long
loop for whichever mode matched. Replaced that with a CommandLine tree
(src/bin/lib/subcommands.c/commandline.h), the same framework
pg_autoctl/cli_root.c and pgaftest/cli_root.c already use, split into a
new cli_root.c following pgaftest's own file layout (main.c stays a
thin entry point; cli_root.c holds the command table and every
sub-command's getopt/run pair) since pg_walsender is a similarly-sized
standalone binary, not pg_autoctl's own much larger tree.

Three sub-commands: serve (the accept loop), fetch-file, scram-secret.
The framework itself has no notion of an optional/default sub-command,
so pg_walsender_default_argv() (cli_root.c) is a small, explicit shim:
when argv[1] isn't a known sub-command name (or --help/-h, which
commandline_run() already intercepts on its own), it splices "serve"
into argv before main() calls commandline_run(), so bare
"pg_walsender --port ... [--pgdata ... | --insecure] ..." keeps
working exactly as before -- only fetch-file and scram-secret require
typing their name.

Every flag is unchanged: --port/--pgdata/--insecure/--auth-timeout/
--ssl-cert-file/--ssl-key-file for serve, fetch-file's own
--host/--port/--route/--filename/--output/--user, and scram-secret's
--user (password still via PGPASSWORD, never the command line). Only
the dispatch mechanism moved; "pg_walsender --help" output is
cosmetically different (the framework's own generated format) as
expected.

Makefile: added cli_root.c to LOCAL_SRC and lib-commandline.o (built
from src/bin/lib/subcommands.c/commandline.c by Makefile.common's own
generic rule, already used by pgaftest) to OBJS.

Verified: full docker build (17 tests + monitor installcheck, no
warnings), citus_indent + banned.h.sh clean, the standalone pgaftest
spec pg_walsender_standalone.pgaf (5/5, including the flex/bison
grammar edge cases), and manual checks of "pg_walsender --help",
"pg_walsender fetch-file --help", "pg_walsender scram-secret --help",
a real scram-secret run, and an invalid flag on the default (serve)
path producing a clear error plus usage on stderr and exit code 1.
…licy

connect_to_monitor() used a single bare PQconnectdbParams() attempt, and
refresh_route() ran two PQexecParams() calls and parsed PQgetvalue()
results inline, instead of this project's own PGSQL struct + the
pgsql_execute_with_params()-with-callback pattern used throughout
monitor.c (src/bin/common/pgsql.c/pgsql.h).

Connection lifetime: a single long-lived PGSQL connection (monitorPgsql)
is now kept open across every tick of ws_refresher_main()'s own loop and
every route it validates on a given tick, rather than reconnected once
per query. The refresher already runs its own periodic loop (a
per-route coalescing timer plus a WS_REFRESH_HEARTBEAT_MS sweep) and a
negative cache on failure (mark_failure()); a short-lived
connect/query/disconnect cycle on every single check would only add
extra TCP and SCRAM round-trips for no benefit, since the outer loop
already provides the long-horizon retry/backoff a periodic background
process needs. monitor_ensure_connection() is the only place that
(re)connects: it reuses the existing connection unless the monitor URI
file changed or PQstatus() shows the connection died since the previous
tick, in which case it reconnects via pgsql_init()/pgsql_finish().

Retry policy: the connection uses pgsql_set_interactive_retry_policy()
(a bounded, backed-off handful of attempts within pgconnect_timeout
seconds -- 2s by default) instead of a single bare connect(), so one
transient hiccup no longer immediately marks the route as failed;
verified manually below.

Split into the same connect/run-one-query/parse-its-result shape
monitor.c's own monitor_get_*() functions use:
  - monitor_ensure_connection(): connect-or-reuse.
  - monitor_get_group_hosts_hash() / monitor_get_group_hosts(): each runs
    one query via pgsql_execute_with_params(), with its own small parse
    callback (parse_hosts_hash_result / parse_hosts_list_result) modeled
    on monitor.c's SingleValueResultContext/parseSingleValueResult, sized
    for this query's own one- and two-column results.
  - refresh_route() itself is now just the control flow over those,
    with no raw libpq calls left.

Makefile/defaults.h: pgsql.h needs pg_autoctl's own defaults.h (for
POSTGRES_PING_RETRY_TIMEOUT and friends, used inside pgsql.c, and
MAXIMUM_BACKUP_RATE_LEN, used directly in a pgsql.h struct field), so
-I../pg_autoctl was added to this Makefile's CFLAGS. pg_walsender's own
defaults.h (its own trimmed, standalone-binary substitute for
pg_autoctl's) sits earlier in the same -I search path and so still
shadows pg_autoctl's file by name for every other pg_walsender source;
MAXIMUM_BACKUP_RATE_LEN is therefore also duplicated into it, the same
way PG_AUTOCTL_REPLICA_USERNAME already was, so pgsql.h's own struct
definitions still compile for translation units that only see this
project's local defaults.h.

Verified: full docker build (17 tests + monitor installcheck, no
warnings), citus_indent + banned.h.sh clean, the standalone pgaftest
spec pg_walsender_standalone.pgaf (5/5) -- which does NOT exercise
refresher.c/monitor connectivity at all, since it never creates an
archiver-monitor.uri file (ws_accept_loop only forks the refresher once
that file exists). Confirmed separately, by hand, against a real
pgaftest cluster (pgaftest cluster setup on this same spec) with
pg_walsender started manually against a pgdata pointed at that
cluster's real monitor:
  - a reachable but schema-mismatched monitor: the connection succeeds
    (proving the retry-policy connect path works end to end), the query
    fails server-side ("function pgautofailover.get_group_hosts_hash
    does not exist" -- a pre-existing gap in the monitor extension,
    unrelated to this change: the old bare-libpq code ran the exact same
    SQL text and would fail identically; flagged separately as a
    follow-up, out of scope here), and mark_failure() correctly writes
    the .err sentinel;
  - a deliberately unreachable monitor (a non-routable address): the log
    shows "retrying until the server is ready" and "after 2 attempts in
    2007 ms", i.e. real bounded retries within the interactive policy's
    window, not an instant single-attempt give-up, before mark_failure()
    runs.
Audited every function in src/bin/pg_walsender/*.c and src/bin/common/
scram.c against the previous review pass's own starting list,
re-verifying each item against the current tree (some had already been
resolved by the flex/bison grammar port and by Tasks 2/4/5 of this same
pass, and were skipped here since those rewrites already carry fresh
comments). Added a genuine explanatory comment -- parameters, return
semantics, protocol/behavior context matching the density already used
elsewhere in each file -- wherever one was missing; left every existing,
adequate comment untouched.

Files touched: accept_loop.c, auth.c, cmd_fetch_file.c,
cmd_identify_system.c, cmd_show.c, cmd_timeline_history.c, framing.c
(most of its ws_send_*/ws_read_* wire-message functions had none),
hba.c (parse_method only -- the rest was rewritten fresh by Task 2),
monitor_hosts.c, refresher.c (key_is_plausible only -- the rest was
rewritten fresh by Task 5), routes.c, startup.c, tar_stream.c, tls.c,
wal_dir_scan.c, ws_util.c, and common/scram.c (scram_parse_verifier,
scram_mock_init). main.c and cli_root.c are untouched here: Task 4
already gave them fresh, complete comments.

Verified: full docker build (17 tests + monitor installcheck, no
warnings -- one added comment initially tripped -Wcomment on a stray
"*/" substring in prose, fixed), citus_indent + banned.h.sh clean, and
the standalone pgaftest spec pg_walsender_standalone.pgaf (5/5).
…ILE client to common/

Task 1: remove the "monitor" HBA address keyword and its supporting
machinery entirely. It depended on two monitor-side SQL functions
(pgautofailover.get_group_hosts_hash, pgautofailover.get_group_hosts) that
do not exist anywhere in this PR's own dependency chain (no monitor
extension is part of this PR at all); the feature only becomes real once a
later archiving PR adds that schema, so it is reintroduced there instead.

Removed:
 - refresher.c/refresher.h (the nodes-list refresher child process and its
   PGSQL-wrapper-based monitor queries)
 - monitor_hosts.c/monitor_hosts.h (the local cache reader/matcher)
 - hba.c's "monitor" address-keyword special case: an HBA line using it
   is now tokenized as an ordinary address token (resolved via DNS like
   any other hostname), no new rejected-keyword path was added
 - accept_loop.c's forking/reaping/stopping of the refresher child, its
   datagram socket, and the archiver-monitor.uri existence check that
   triggered it
 - cli_root.c's monitorUriPath/refreshSockPath construction
 - WsAuthConfig's monitorUriPath/refreshSockPath fields and the matching
   hba_lookup()/rule_address_matches() parameters, threaded back down to
   what is actually used
 - the now-dead ws_auth_deadline_remaining_ms() (its only caller was
   monitor_hosts.c's request_refresh())
 - refresher.c/monitor_hosts.c from the Makefile's source list

The default HBA file pg_walsender writes now documents an explicit rule
per host instead of a "monitor" line that would silently do nothing
useful. tests/tap/specs/pg_walsender_standalone.pgaf did not reference any
of this and needed no changes; it still passes 5/5.

Task 2: move the FETCH_FILE *client* out of this binary. It never had any
dependency on pg_walsender's own wire-framing code (framing.h) or any other
pg_walsender-internal header -- it is a plain libpq client (PQconnectdbParams
+ PQexec("FETCH_FILE ...") + PQgetCopyData()) -- so it moves as-is to
src/bin/common/fetch_client.c/fetch_client.h, which both pg_autoctl and
pg_walsender already link. This lets a future caller (pg_autoctl restore
command, in the later archiving PR) call ws_fetch_file_client() directly,
in-process, with no execv()/subprocess indirection.

pg_walsender's own "fetch-file" CLI sub-command is removed from cli_root.c
(and its --help text): pg_walsender is now a server binary (serve, the
default command) plus its scram-secret utility, nothing else.
cmd_fetch_file.c, the *server-side* FETCH_FILE handler, is untouched.

tests/tap/specs/pg_walsender_standalone.pgaf's test_002_fetch_file drives
the server-side FETCH_FILE command via a plain psql -c "FETCH_FILE ..."
and does not use the removed CLI client; it needed no changes.

Verified: docker build (PG17) succeeds with no compiler warnings, monitor
installcheck's 19/19 tests pass, citus_indent --check and ci/banned.h.sh
pass clean, and pg_walsender_standalone.pgaf passes 5/5.
Documents the design decisions in this codebase for the next engineer:
what pg_walsender is and isn't (standalone, no monitor/archiver
integration yet), the wire protocol commands and the FETCH_FILE extension
(and why its client now lives in common/, not here), the replication-slot
design in depth (bookkeeping marker file, not a real slot; naming/safety
constraints; the still-unwired WAL-retention enforcement), the
repl_scanner.l/repl_gram.y grammar port (what's dropped, what's added, the
<fname> lexer state), hba.c's tokenizer design and the removed "monitor"
keyword, authentication (SCRAM, TLS, the auth-timeout deadline), the
routes.ini design, the vendored ustar writer's own formatting choice, and
how tests/tap/specs/pg_walsender_standalone.pgaf exercises all of this by
hand ahead of the archiver integration.
Adds docs/ref/pg_walsender.rst (Sphinx reference page: synopsis,
description, options for server mode and scram-secret -- no fetch-file,
which was removed from pg_walsender's own CLI), registers it in
docs/conf.py's man_pages list, and adds it to docs/ref/manual.rst's
toctree so the html build does not warn about an orphan document.

Verified: built the docs' own Sphinx toolchain (docs/requirements.txt)
in a throwaway Docker image and ran 'sphinx-build -b man docs
docs/_build/man'; it builds pg_walsender.1 along with every other man
page, with no warnings specific to this new page (the only remaining
warnings are the pre-existing tikz SVG image-not-readable ones, present
because this throwaway image does not run the docs/tikz build step).
Inspected the generated troff with 'man --local-file' and confirmed it
renders cleanly end to end (NAME/SYNOPSIS/DESCRIPTION/OPTIONS/
ENVIRONMENT/EXAMPLES/AUTHOR/COPYRIGHT).

The project's own Dockerfile.docs (used by 'make build-docs') could not
be used as-is: it COPYs Makefile.azure/Makefile.citus, which no longer
exist in this tree, and its base image (debian:bullseye-slim) has aged
out of the Debian package mirrors (404s on texlive-bin and others) --
both pre-existing issues, unrelated to this change. Verification instead
used a throwaway Dockerfile against docs/requirements.txt for the parts
relevant to a man-page build (no LaTeX/tikz needed for -b man).
…ack, not 0.0.0.0/0

Every connection this test makes to pg_walsender is node2 talking to its
own server over 127.0.0.1 -- there never was a reason for the hand-written
archiver-hba.conf to admit "0.0.0.0/0" (the entire internet, in principle).
127.0.0.1/32 is the actually correct, properly scoped rule for what this
test really does, written by hand exactly as an operator running
pg_walsender on its own (no archiver, no monitor, see this test's own
header comment) would maintain it themselves.

Verified: pg_walsender_standalone.pgaf still 5/5 green.
…wildcard route

Reframe the route key (routes.ini's section names, matched against a
connection's dbname, and archiver-hba.conf's ROUTE field) as what it
already is in the code: an opaque, operator-chosen string pg_walsender
never parses, splits, or gives filesystem meaning to. Nothing in this
codebase ever treats "<formation>/<group>" as anything but a plain string
compared with strcmp() -- that convention is pg_auto_failover's own,
useful because it reads well and is guaranteed unique, but not a
requirement pg_walsender imposes on anyone else driving it. Updated:
hba.c/hba.h, routes.c/routes.h, walsender.h, startup.c, cli_root.c's help
text, and the README's HBA/routes sections, all to describe this generic
mechanism first and pg_auto_failover's convention as one example of it,
plus an explicit callout that a key's "/" carries no filesystem meaning
(only a route's own explicit "path" property ever does).

Add routes.ini's own "*" wildcard route (WS_ROUTES_WILDCARD_KEY,
routes.c's routes_find()): a dbname matching no route of its own falls
through to it, exactly like PgBouncer's own [databases] "*" entry
(pgbouncer.org/config.html) -- same precedence (an exact match always
wins), same "*" syntax, chosen so anyone who has run a PgBouncer already
knows what to expect. Deliberately NOT PgBouncer's per-request name
substitution: PgBouncer's wildcard is safe substituting the requested
dbname into a *connection string* handed to a real PostgreSQL server,
which validates it; ours would substitute an unauthenticated client's
string into a *filesystem path*, and a route key is explicitly allowed to
contain "/" (pg_auto_failover's own convention does) -- a naive %r-style
substitution would turn that convention into path traversal. Every dbname
that falls through to "*" shares that one configured path verbatim.

This is what makes running pg_walsender useful outside pg_auto_failover
with zero multiplexing ceremony: a single-cluster deployment needs one
`[*]` section and never has to learn or type a special dbname value.
archiver-hba.conf's own ROUTE matching stays completely independent of
this (always against the literal dbname, never against which WsRoute the
wildcard resolved to).

docs/ref/pg_walsender.rst: a full worked standalone example (no
pg_auto_failover anywhere) -- create a replication role on a real primary,
seed an initial base backup and start continuous pg_receivewal capture
into the exact on-disk layout pg_walsender expects (verified against
cmd_base_backup.c's basebackups/.latest + backup_label handling and
wal_dir_scan.c's flat WAL layout), configure and start pg_walsender, then
build a fresh PGDATA and PITR-restore it with a restore_command driving
FETCH_FILE through a plain psql one-liner (the exact invocation shape
already proven by this test suite's own test_002_fetch_file). Verified the
new .rst builds clean with sphinx-build -b man (no warnings).

tests/tap/specs/pg_walsender_standalone.pgaf: new test_005_wildcard_route,
proving both the wildcard fallback and exact-match precedence against a
second route with a distinguishable systemid. Also fixed a stale README
line still describing the HBA rule as "0.0.0.0/0" after an earlier commit
scoped it to 127.0.0.1/32.

Verified: PG17 docker build clean (zero warnings), citus_indent/
banned.h.sh clean, pg_walsender_standalone.pgaf 6/6 green.
… a fabricated one

The previous Synopsis block was hand-assembled, not captured, and wrong on
two counts: it invented a "+" marker in front of "serve" that this
project's own CommandLine framework never prints for either of
pg_walsender's two sub-commands (both are leaves -- "+" means "has
sub-commands of its own", as pg_autoctl.rst's own tree shows for
create/drop/etc.; it was never a "this one is the default" marker), and it
merged the root command's usage line with serve's own detailed flag list
into a single block that matches neither `pg_walsender --help`'s actual
output nor `pg_walsender serve --help`'s.

Replaced with two verbatim captures, built and run against this branch's
own binary: `pg_walsender --help` (root help -- note neither "serve" nor
"scram-secret" gets "+", correctly) and `pg_walsender serve --help` (the
actual flag usage line), with a short explanation of why they differ:
--help/-h is the one case pg_walsender_default_argv()'s "no sub-command
means serve" splicing deliberately leaves alone (see its own comment,
cli_root.c), so `pg_walsender --help` shows the root tree, never serve's
flags.

Verified: PG17 docker build clean, sphinx-build -b man clean (no
warnings).
…dalone archiver

Design document only, nothing implemented -- see the file's own header.
Covers, in order:

- PostgreSQL's own continuous-archiving/PITR contract (archive_command's
  exit-code and overwrite-safety rules, .backup history files, restore_
  command's own expectations, timeline handling) and the gaps it exposes
  in the current design (no .backup in the FETCH_FILE allow-list, no
  write path into a route at all, WAL capture entirely external and
  unsupervised).

- Renaming pg_walsender to pg_walserver once it grows past being purely a
  *sender*: checked PostgreSQL's own glossary (avoiding a collision with
  its own "archiver process" term and this project's planned archiver
  feature) and external tools' vocabulary (pgBackRest's "repository"
  convention, none of which actually speak the native replication
  protocol the way this binary already does) before settling on it.

- routes.ini growing an `upstream` connection-string property per route,
  and new client-side sub-commands (fetch-systemid, basebackup, setup)
  built on it -- no new wire protocol needed for any of these.

- The archive_command push side (ARCHIVE_FILE, CopyIn) and, importantly,
  that this is not a green field: the pgaf-integrated design already has
  a working, monitor-backed, quorum-aware "already captured?" check
  (archiver_confirm.c's use of pgautofailover.wal_archived()) that should
  stay the primary mechanism there, gaining only a push *fallback*
  (a new common/ ws_archive_file_client(), called where archiver_confirm.c
  currently just retries forever) rather than being replaced. A new
  ARCHIVE_STATUS wire command (cheap CRC32C-based check, no transfer) is
  the standalone-only equivalent, for when there's no monitor to ask.

- An embedded, per-route supervised WAL capturer (the already-vendored
  pg_receivewal, relocated to common/ so both binaries link it) so a
  standalone pg_walserver needs no external, unsupervised pg_receivewal
  process at all.

- Fit as a pg_auto_failover building block, including the explicit
  caveat that archive_command-push (fires only on completed segments,
  asynchronous relative to commit) is a defense-in-depth addition for the
  archiver's failover/quorum-member role, not a replacement for the
  continuous streaming pull path there.

- Open questions and a 3-phase implementation order.
Half its job is no longer "sending": this PR already ships FETCH_FILE
(a read), and the design work in DESIGN-standalone-archiving.md adds an
actual write path (ARCHIVE_FILE push) and an embedded WAL capturer (pull).
"Sender" alone was already an inaccurate description before either of
those land, so the rename happens now, in this same PR, rather than being
deferred to a separate one purely to avoid touching an already-reviewed
diff again -- continuing to iterate on an open PR through review is the
normal shape of review.

pg_walserver: "sender" replaced by the more general "server" -- picked
over pg_archiver/pg_walarchive-shaped alternatives (collide with
PostgreSQL's own "archiver process" glossary term and this project's own
planned pg_auto_failover archiver feature) and over a pgbouncer-inspired
pg_walrepo/pg_walstore-style coinage (a new name over a borrowed one,
where pg_walserver needs neither). See DESIGN-standalone-archiving.md's
own "Naming" section for the full research trail.

Mechanical scope: directory move (src/bin/pg_walsender -> pg_walserver),
Makefile (PG_WALSENDER -> PG_WALSERVER, binary name), every pg_walsender/
PG_WALSENDER identifier and string across the moved directory plus every
outside reference (.gitattributes, Dockerfile, docs/conf.py, docs/ref/
manual.rst, the docs page itself renamed pg_walsender.rst -> pg_walserver.
rst, src/bin/Makefile, src/bin/pg_autoctl/Makefile, common/fetch_client.*,
common/ipaddr.h, common/scram.h), walsender.h renamed to walserver.h
(and its own include guard), and the test spec renamed
pg_walsender_standalone.pgaf -> pg_walserver_standalone.pgaf (including
its own step name mentioning the binary). No bare "walsender" text
existed anywhere referring to something other than this binary (verified
by grep before renaming), so this was unambiguous everywhere.

Verified: PG17 docker build clean (zero warnings), a real `pg_walserver
--help` run confirms the CLI itself now says "pg_walserver" throughout
and "pg_walsender" is gone from the built binary, citus_indent/
banned.h.sh clean, pg_walserver_standalone.pgaf still 6/6 green (including
the renamed test_003_start_replication_from_pg_walserver step), and
sphinx-build -b man clean on the renamed docs page.
….ini, real-standby test coverage

Implements phase 1 of DESIGN-standalone-archiving.md (the routes.ini
"upstream" property + the three client-side sub-commands), reusing
existing pg_autoctl code rather than reimplementing it:

- routes.h/routes.c: WsRoute gains an "upstream" property (a libpq
  connection string), parsed alongside "path".

- cli_upstream.c/.h: shared --route/--path/--upstream/--host/--port/
  --user resolution for the three new sub-commands, the same explicit-
  flag-wins-over-config layering restore_command_resolve() already uses.
  Parses a plain keyword=value "upstream" string directly with
  PQconninfoParse() (not this project's own parse_pguri_ssl_settings(),
  whose checkForCompleteURI path is built for postgres:// URIs, not this
  shape).

- cli_fetch_systemid.c/.h: `pg_walserver fetch-systemid` -- reuses
  pgctl_identify_system() (src/bin/common/pgctl.c) unchanged, a real
  replication-mode IDENTIFY_SYSTEM. Refuses to overwrite an already-
  recorded *different* system identifier unless --force.

- cli_basebackup.c/.h: `pg_walserver basebackup` -- reuses
  pg_basebackup_fetch(), which did not yet exist on this branch (it was
  only ever added on a separate, more advanced branch, splitting
  pg_autoctl's historical pg_basebackup() into a reusable fetch-only
  piece plus the standby-init-specific rmtree-and-move wrapper, and later
  gaining an HBA-readiness retry preflight). Ported both commits' changes
  forward to this branch's own src/bin/common/pgctl.c/pgctl.h/pgsql.h
  (a straight cherry-pick conflicted: the original commit also touched a
  service_archiver_basebackup.c that doesn't exist here) -- same logic,
  reused from where it already existed, not reinvented. Validates the
  result (backup_label/PG_VERSION present) before atomically swapping
  basebackups/.latest.

- cli_setup.c/.h: `pg_walserver setup` -- the wizard: write/validate the
  pg_walserver.ini section, then fetch-systemid (whose own
  pgctl_identify_system() connection doubles as the role-permission
  check: PostgreSQL refuses a replication-mode connection for a role
  lacking REPLICATION at the backend level, independent of HBA -- an
  earlier version ran a separate plain-SQL pg_roles.rolreplication query
  first, removed because that connection targets an ordinary database,
  which the replication role's own HBA rule usually does not admit at
  all), and, with --with-basebackup, basebackup -- synchronously, not
  returning until the first backup has actually succeeded.

- cli_root.c: wired all three into the CommandLine tree and the "no
  sub-command means serve" argv-splicing exemption list.

- routes.ini renamed to pg_walserver.ini throughout (code, docs, tests):
  what used to be colloquially "routes.ini" in prose is now the file's
  actual name, matching the binary's own.

- docs/ref/pg_walserver.rst's worked example now leads with `pg_walserver
  setup --with-basebackup` instead of the previous three hand-run steps
  (psql for the systemid, pg_basebackup for the backup, a hand-written
  ini section); added a new "real standby" example (primary_conninfo +
  standby.signal, no restore_command) alongside the existing PITR one.

- tests/tap/specs/pg_walserver_standalone.pgaf: the setup{} block now
  uses `pg_walserver setup --with-basebackup` instead of hand-writing
  pg_walserver.ini and archiver-systemid; new test_006_real_standby_
  with_core_tools proves a real pg_basebackup BASE_BACKUP from
  pg_walserver (exercising cmd_base_backup.c's actual wire serving, not
  just checking the file exists on disk) followed by a real,
  continuously-streaming standby -- primary_conninfo, standby.signal,
  stock PostgreSQL commands only -- that catches up via a genuine
  walreceiver (not pg_receivewal) and promotes.

  That last step surfaced a real PostgreSQL protocol fact, not a bug:
  libpqrcv_connect() (libpqwalreceiver.c) always sends the literal
  dbname "replication" for a physical replication connection, silently
  discarding whatever dbname a real primary_conninfo carries. A route
  reachable by a real physical standby by name therefore needs a second
  section literally keyed "replication" pointing at the same path (or,
  for a single-route deployment, the "*" wildcard) -- documented in both
  the README and docs/ref/pg_walserver.rst, and reflected in the test.

Verified: PG17 docker build clean (zero warnings), citus_indent/
banned.h.sh clean, sphinx-build -b man clean, and
pg_walserver_standalone.pgaf 7/7 green including the new real-standby
step.
…sical standbys

test_006_real_standby_with_core_tools (the previous commit) surfaced a
real PostgreSQL wire-protocol fact: libpqrcv_connect() always sends the
literal dbname "replication" for a physical replication connection,
silently discarding whatever dbname a real primary_conninfo carries
("the database name is ignored by the server in replication mode, but
specify 'replication' for .pgpass lookup", libpqwalreceiver.c's own
comment). dbname-based routing therefore cannot select a route by name
for a real physical standby -- only the "*" wildcard or a route literally
keyed "replication" is reachable by one.

Researched two protocol-native alternatives, both already shipped by real
PostgreSQL and unaffected by the dbname override (they happen at or
before the TLS handshake, below where that override lives):

- TLS SNI: libpq already sends it by default (sslsni, fe-secure-
  openssl.c); PostgreSQL's own backend already has a matching, documented
  server-side feature for exactly this (ssl_sni/hosts_file, sni_
  clienthello_cb() in be-secure-openssl.c, using SSL_client_hello_get0_
  ext() rather than the simpler SSL_get_servername() on OpenSSL's own
  advice against the latter's callback-ordering fragility).
- TLS client certificate CN: sslcert/sslkey are not touched by the
  dbname override, flowing through from the original conninfo unchanged
  even for a real standby -- the same mechanism PostgreSQL's own
  clientcert=verify-full HBA option already uses to map a certificate to
  a role, applied to route selection instead.

pg_walserver's own tls.c already links raw OpenSSL directly (no backend
or libpq-fe code), so both are architecturally straightforward reads, no
new dependency. Not designed in detail or scheduled into a phase yet.
A real physical standby's own walreceiver always sends the literal
dbname=replication on the wire (libpqrcv_connect() overrides whatever
primary_conninfo's dbname= says), so dbname-based routing alone cannot
disambiguate more than one route for it. This adds a second routing
signal: the TLS Server Name Indication (SNI) extension libpq already
sends (sslsni, on by default) from primary_conninfo's own host= value,
exactly the mechanism HTTPS virtual hosting uses.

- routes.ini gains a "hostname" route property (routes.h/routes.c),
  set via `pg_walserver setup --hostname`.
- tls.c reads the SNI hostname back post-handshake via the simple
  SSL_get_servername() (safe here because, unlike PostgreSQL's own
  ssl_sni backend feature, pg_walserver never switches certificates
  based on it).
- auth.c resolves a route in three steps: exact dbname match, then (TLS
  only) SNI hostname, then the "*" wildcard. HBA's own ROUTE matching
  stays independent, always against the literal dbname.
- `pg_walserver setup` auto-creates a self-signed certificate for
  --pgdata (reusing pg_create_self_signed_cert(), unchanged) the moment
  a second named route needs one, and warns if that route has no
  --hostname.
- `pg_walserver serve` now hard-refuses to start (log_fatal/exit(1))
  with more than one named route and no TLS configured; a single named
  route keeps working with no TLS at all.

New tap spec tests/tap/specs/pg_walserver_sni_routing.pgaf: two real,
independent routes disambiguated purely by SNI hostname while both
present the same useless dbname=replication a real standby forces,
plus the auto-cert-creation and no-TLS-refusal behaviors. 4/4 green,
alongside pg_walserver_standalone.pgaf's own 7/7.

Docs (README.md, docs/ref/pg_walserver.rst,
DESIGN-standalone-archiving.md) updated to describe the mechanism and a
worked two-route example.
…sh side

Task A: `pg_walserver create-cert --pgdata <path> --hostname <name>
[--force]` (cli_create_cert.c/.h), a thin CLI wrapper around
pg_create_self_signed_cert() (src/bin/common/pgctl.c). Refuses to
overwrite an existing server.crt/server.key unless --force, matching this
codebase's "never silently replace what's already there" principle
(cli_fetch_systemid.c's own systemid overwrite check). cli_setup.c's
ensure_tls_for_multiple_routes() is refactored to call the same shared
ws_create_cert_run() helper this sub-command uses, instead of duplicating
the pg_create_self_signed_cert() call-and-log sequence.

Task B: the push side of DESIGN-standalone-archiving.md's "The push side:
CHECK_FILE + ARCHIVE_FILE" section.

- repl_scanner.l/repl_gram.y: CHECK_FILE and ARCHIVE_FILE grammar
  productions, following the exact FETCH_FILE pattern (same <fname>
  lexer state for the filename argument). CHECK_FILE's own
  "crc32c:<hex>" argument is scanned as one token (a hex CRC can start
  with a digit, which the ordinary identifier rule can't match).
- cmd_check_file.c/.h: CHECK_FILE's wire shape is the lean, SHOW-like
  RowDescription/DataRow/CommandComplete row the design doc's own open
  question leaned towards, answering "missing"/"matches"/"differs".
- cmd_archive_file.c/.h: ARCHIVE_FILE's CopyIn receive path. The server
  never trusts a client's own CHECK_FILE checksum: once the whole CopyIn
  is received, it re-derives the overwrite-safety decision from the real
  bytes on disk vs. the real bytes just received (identical -> success/
  idempotent retry, matching PostgreSQL's own archive_command contract;
  different -> clean rejection; nothing there yet -> write it via a
  same-directory temp file plus atomic rename). Hard size cap at the
  route's own wal_segment_size plus a fixed slack.
- cmd_fetch_file.c/.h: ws_fetch_filename_is_servable() extended in place
  to also accept a base backup's own "<24hex>.<8hex>.backup" history file
  (Gap #1 in the design doc), shared by both the read and write paths.
- ws_util.c/.h: ws_file_crc32c(), a shared size+CRC32C helper for a whole
  file, built on the same INIT_CRC32C/COMP_CRC32C/FIN_CRC32C facility
  (port/pg_crc32c.h) real Postgres and pg_autoctl's own nodespec.c
  already use -- no new CRC32C implementation.
- framing.c/.h: ws_send_copy_in_response() ('G' CopyInResponse), the one
  new wire primitive ARCHIVE_FILE needs.
- cli_archive.c/.h + cli_root.c: `pg_walserver archive <path> <filename>
  --route <key> --host <host> [--port <port>] [--user <name>]
  [--sslmode <mode>]`, meant to run as (part of) a Postgres
  archive_command. Implements the design's own 4-step sequence: compute
  local size+CRC32C, CHECK_FILE, a short bounded intra-invocation
  recheck (two rechecks, one second apart -- the design doc's own other
  open question, resolved and documented in cli_archive.c; always run
  today since pg_walserver.ini has no "capture" property yet, revisit
  once it does), then ARCHIVE_FILE only if still needed. Exit 0 on
  success (including "already matches"), nonzero with a clean stderr
  message otherwise, exactly PostgreSQL's own archive_command contract.
  Deliberately does not reuse cli_upstream.c's cli_resolve_upstream() --
  archive connects to pg_walserver itself, not to a Postgres primary, a
  different shape of connection entirely; see cli_archive.h's own header
  comment for the full reasoning.

Verification: a clean `docker build --target run` (zero new
warnings/errors), `make docker-check`/banned.h.sh clean (one file needed
`make docker-indent`, applied), and a new tap spec,
tests/tap/specs/pg_walserver_archive_command.pgaf, covering CHECK_FILE
"missing"/"matches"/"differs", a real `pg_walserver archive` push and its
idempotent-retry/overwrite-safety-rejection behavior, and create-cert's
own fresh/refuse-without-force/--force behavior -- 4/4 green via
pgaftest against a freshly built pgaf:run-pg17 image.

Docs updated: src/bin/pg_walserver/README.md (new sections for the wire
commands, `create-cert`, and `pg_walserver archive`'s own 4-step logic),
docs/ref/pg_walserver.rst (create-cert and archive_command-based
archiving added to the worked example), and
DESIGN-standalone-archiving.md (the push side section and its two open
questions marked resolved/implemented, Phasing's phase 2 marked done --
following the same "Update: ... implemented" precedent already used
there for TLS SNI routing).

Out of scope, untouched: the embedded pull WAL capturer (phase 3,
"capture = pull") and any vendor relocation of pg_receivewal.
A route with `capture = pull` in pg_walserver.ini (set via `pg_walserver
setup --capture pull`) gets its own long-lived, supervised child capturing
real WAL straight into the route's own directory, via the vendored
pg_receivewal (src/bin/common/vendor/pg_receivewal/, relocated in the
previous commit) -- no external pg_receivewal process, no separate
supervisor unit to wire up by hand.

Supervision reuses this codebase's own existing pattern for long-lived,
restart-supervised services, rather than a bespoke fork/waitpid loop:

- src/bin/common/process_supervisor.c/h: a generic Service/RestartPolicy
  child supervisor extracted from pg_autoctl's own supervisor.c core
  (Erlang-style MaxR/MaxT restart backoff, PID-1-safe orphan reaping via
  one wildcard waitpid(-1, ...) call site). Not a literal move -- the real
  supervisor.c is entangled with pg_autoctl's keeper/monitor/nodespec
  state, so this is a fresh, symbol-distinct (process_supervisor_*/
  Process*) extraction of its generic core, living alongside it.
- src/bin/pg_walserver/cli_internal.c/h: a hidden `pg_walserver internal
  service pg-receivewal --route <key> --upstream <conninfo> --path <dir>`
  sub-command, mirroring pg_autoctl's own `internal service
  postgres|listener|node-active` pattern exactly -- each capturer child is
  started via fork()+execv() of the pg_walserver binary itself, re-entering
  as this hidden sub-command, which then calls the vendored
  pg_receivewal_main() in-process. This is what makes a restarted capturer
  safe under a container's PID 1: a live-upgraded binary on disk is picked
  up on the next restart without ever touching the supervising process,
  the same property pg_autoctl's own supervisor already relies on.
- capture.c's ws_capture_start_all() builds one ProcessService per active
  `capture = pull` route and hands them to process_supervisor_start();
  ws_capture_tick() is the only waitpid(-1, ...) call site in the whole
  process -- accept_loop.c's own per-connection child bookkeeping is
  threaded through as its otherChildExited callback instead of running a
  second, independent wildcard reaper (two independent wildcard waiters
  were found, during testing, to silently steal each other's zombies).
  This also gives pg_walserver itself PID-1 orphan-reaping for free.

docs/ref/pg_walserver.rst and README.md updated: the standalone worked
example's step 1 now shows `pg_walserver setup --capture pull`, with the
manual `nohup pg_receivewal ... &` step kept only as the push-only/
external-feed alternative.
Dimitri Fontaine added 29 commits October 14, 2026 16:47
…s, request standby replies

Three fixes from an external review (verified against the actual code
before acting; several of the review's other findings were checked and
refuted -- see PR discussion):

- framing.c's ws_read_startup_payload()/ws_read_message() rebuilt the
  4-byte big-endian length field via left shifts on a signed int32_t,
  which is undefined behavior in C when the result doesn't fit in
  int32_t (e.g. the high length byte has its MSB set). Now reconstructed
  as uint32_t and only cast to int32_t once assembled. Same treatment
  for startup.c's protocol-version code field (int32_t -> uint32_t),
  which was then right-shifted -- implementation-defined rather than
  UB, but the same fix removes the ambiguity either way.

- tls.c only called SSL_CTX_set_cipher_list(), which configures TLS 1.2
  and below; TLS 1.3 has its own suite list and its own setter
  (SSL_CTX_set_ciphersuites()), so a TLS 1.3 handshake was silently
  falling back to OpenSSL's compiled-in defaults instead of an explicit,
  reviewed policy. Added the missing call with OpenSSL's own documented
  TLS 1.3 defaults named explicitly.

- cmd_start_replication.c's send_keepalive() always sent
  replyRequested=false, so a client that only sends a StandbyStatusUpdate
  when asked (rather than on its own periodic timer, as pg_receivewal/
  pg_basebackup already do) would never advance this slot's restart_lsn.
  Since this project never blocks waiting for the reply either way,
  there's no cost to always requesting one.
…andLine

The prior refactor (ca69b69, 8e96093) only forward-declared each file's
getopt/run function pair before relocating its CommandLine definition
near the top of the file; every other helper function's full body (static
or not) was left wherever it already happened to sit, often still before
the CommandLine block.

This is pure code motion, no logic/behavior change: every remaining
function body (helpers included) is moved below its file's CommandLine
declaration, with a forward declaration added for each one ahead of the
CommandLine block, so every file's CommandLine is now the first
substantive thing after its header comment/includes/trivial static
state, matching pg_autoctl's own cli_create_node.c convention with zero
exceptions.

Files fixed: cli_archive.c, cli_archive_cleanup.c, cli_basebackup.c,
cli_create_cert.c, cli_fetch_systemid.c, cli_ls.c, cli_ps.c,
cli_restore_wal.c, cli_serve.c, cli_setup.c, cli_status.c.

cli_cluster.c, cli_internal.c, cli_list.c, cli_reload.c,
cli_scram_secret.c, and cli_stop.c were re-checked and already matched
the convention; no changes needed there.
segment filename math into src/bin/common/, matching pg_autoctl's own
layering

Three-part refactor, no user-visible behavior change:

1. archive-wal/restore-wal client -> src/bin/common/push_client.c/.h
   (new) and fetch_client.c/.h (existing, signature updated)
   - new: src/bin/common/push_client.c, src/bin/common/push_client.h
     (ws_push_file_client(), plus private statics push_client_file_
     crc32c()/check_file_status()/show_receivewal()/push_file())
   - fetch_client.h/.c: ws_fetch_file_client() gains an explicit
     applicationName parameter (no more hardcoded "fetch_client"
     fallback_application_name baked into common/)
   - renamed: cli_archive.c's ws_archive_run() -> ws_archive_push_file()
     (now a thin wrapper calling ws_push_file_client() with
     "pg_walserver_archive_wal"); cli_restore_wal.c's ws_restore_run()
     -> ws_restore_fetch_file() (wraps ws_fetch_file_client() with
     "pg_walserver_restore_wal")
   - cli_archive.c/cli_restore_wal.c dropped their own PQconnectdbParams()/
     CHECK_FILE/ARCHIVE_FILE/FETCH_FILE wire-protocol code entirely; both
     now get pgsql_init()/pgsql_open_connection()'s retry/backoff for free

2. WsRetentionAge/ws_parse_retention_age() -> src/bin/common/
   string_utils.h/.c
   - moved out of cli_archive_cleanup.h/.c (both cli_archive_cleanup.c
     and cli_basebackup.c already included string_utils.h)
   - cli_archive_cleanup.c's other 5 "/* ---...--- */" dividers flattened
     to plain comments in the same pass (the usage-comment divider on
     archive_cleanup_command itself, shared house style with ~15 other
     pg_walserver files, is left untouched)

3. WAL segment filename arithmetic/predicates -> new src/bin/common/
   wal_segment.c/.h, mirroring PostgreSQL's xlog_internal.h
   - new: src/bin/common/wal_segment.c, src/bin/common/wal_segment.h:
     wal_segments_per_xlogid() (XLogSegmentsPerXLogId),
     wal_segment_name_is_valid() (IsXLogFileName),
     wal_segment_name_is_partial() (IsPartialXLogFileName),
     wal_backup_history_name_is_valid() (IsBackupHistoryFileName),
     wal_segment_name_format() (XLogFileName),
     wal_segment_name_parse() (XLogFromFileName),
     wal_segment_name_extract_prefix() (project-specific: plain/.partial/
     .backup in one call), wal_lsn_to_segno() (XLByteToSeg composed with
     parseLSN()), wal_lsn_to_segment_name() (composition of the two)
   - deleted (replaced by the above): wal_dir_scan.c's private
     is_wal_segment_filename() and its exported wal_segment_filename()/
     ws_wal_segment_prefix_to_position()/ws_wal_lsn_to_segno(); cmd_base_
     backup.c's private is_wal_segment_filename(); cli_archive_cleanup.c's
     private is_hex_run()/is_wal_segment_name()/wal_prefix_from_name()/
     lsn_to_segno()/lsn_to_segment()/segment_name_to_tli_segno()
   - wal_dir_scan.h now #includes wal_segment.h; every call site in
     cmd_check_file.c, cmd_replication_slot.c, cmd_start_replication.c,
     cli_archive_cleanup.c, cmd_base_backup.c updated to the new names
   - wal_lsn_to_segno()/wal_lsn_to_segment_name() are reimplemented on
     src/bin/common/parsing.c's parseLSN() instead of a bare sscanf,
     matching upstream pg_parse_lsn's stricter validation
   - wal_segment_name_format() keeps this project's own pre-existing
     argument order for this function (timeline, segno, segSize, dest,
     destSize -- dest/destSize last), not the dest-first order the
     original plan sketch suggested, since that's what the function
     being replaced (wal_dir_scan.c's wal_segment_filename()) already
     used and every call site already expects

Rebuilt (--no-cache, PGVERSION=17) and exercised live: archive-wal push +
restore-wal fetch round trip is byte-identical; "basebackup --keep-age
30d" and "archive-cleanup --keep-age 72h" both parse and run their
cleanup pass correctly; "pg_walserver ls"/"list wal --segments" against a
route with a real segment, a .partial segment, and a .backup label file
all classify and size correctly through the new wal_segment_name_is_
valid()/_is_partial()/wal_backup_history_name_is_valid() primitives.

make docker-indent / docker-check both pass clean.
…/ into src/bin/pg_walserver/

src/bin/common/ holds genuinely generic utilities (pgsql.c's connect/
execute-SQL/retry facility has zero knowledge of any specific command).
fetch_client.c/push_client.c are different: both hard-code knowledge of
pg_walserver's own custom wire-protocol extensions (FETCH_FILE,
CHECK_FILE, ARCHIVE_FILE) -- that's pg_walserver business logic, not a
generic reusable utility, even though it's built on top of pgsql.c's
generic connection facility underneath.

fetch_client.c's previous common/ placement predated this change and was
justified by a speculative future need ("a future, separate 'archiving'
PR is expected to grow pg_autoctl's own restore command to call this same
function directly"). Checked: nothing in pg_autoctl references either
function today. Per pg_autoctl's own convention (domain-specific protocol
clients like monitor.c live in the binary that owns the protocol, not in
common/), both files move to src/bin/pg_walserver/ as plain domain files.
If/when that future pg_autoctl PR materializes, promoting a file back to
common/ is a trivial, low-risk move.

Also: push_client.c's private push_client_file_crc32c() existed only to
duplicate ws_util.c's ws_file_crc32c() byte for byte, specifically because
common/ had to stay independent of pg_walserver-specific headers. That
constraint is gone now that the file lives in pg_walserver/, so the
duplicate is removed; push_client.c calls ws_file_crc32c() directly.

Updated every stale "src/bin/common/fetch_client.c"/"push_client.c" path
reference in comments and README.md accordingly, including rewriting the
README's "FETCH_FILE's client" section (the removed common/-placement
rationale and the speculative pg_autoctl-sharing paragraph).

src/bin/pg_walserver/Makefile's LOCAL_SRC gains both files explicitly
(sources are listed there, not globbed); src/bin/common/Makefile.common
needs no change (globs *.c automatically, so removing them from there is
transparent).
ws_file_crc32c() (ws_util.c) was pure file I/O + CRC32C math -- open a
regular file, read it sequentially in chunks, compute size and CRC32C in
one pass -- with no pg_walserver-specific type anywhere in it. That makes
it a genuine common/ utility, unlike the FETCH_FILE/CHECK_FILE/ARCHIVE_FILE
protocol clients (fetch_client.c/push_client.c) that stayed in
pg_walserver/ for the opposite reason: those know about pg_walserver's own
wire-protocol commands, this doesn't know about anything beyond "a file".

Also found a second, independent implementation of the same thing:
pg_autoctl's nodespec.c had its own nodespec_file_crc(), computing a
CRC32C the same way but by reading the whole file into memory first
(read_file()) rather than streaming it. Consolidated: nodespec_file_crc()
is now a one-line wrapper around the new common file_crc32c(), gaining
pg_walserver's own bounded-memory streaming behavior for free.

New: src/bin/common/file_crc32c.c/.h, file_crc32c(path, sizeOut, crcOut).
Removed: ws_util.c's ws_file_crc32c() and its declaration in ws_util.h.
Updated callers: cmd_check_file.c, cmd_archive_file.c, push_client.c
(pg_walserver), nodespec.c (pg_autoctl). Updated README.md's reference to
the relocated function.
…ervalToString()

Both files had a byte-for-byte identical static format_uptime()
("<h>h<mm>m<ss>s"), a bespoke format invented during this same
not-yet-merged branch rather than a pre-existing contract. Replace both
copies with a thin per-file format_elapsed() that computes elapsed
seconds and calls common/string_utils.c's existing IntervalToString(),
with an explicit "-" sentinel for startedAt <= 0 (not running/unknown),
which IntervalToString itself has no notion of.

This changes the visible shape of the ps/status uptime column (e.g.
IntervalToString drops the smaller unit at larger magnitudes), which is
expected and accepted -- the old shape was never a real contract.
…irectory

Fixup: the previous commit staged a stale, pre-fix copy of
tar_stream.c/.h and cmd_base_backup.c (an earlier repo-wide 'git add -A'
ahead of docker-indent had already staged the version before this fix,
and a later path-scoped 'git add' for an unrelated commit didn't touch
these paths, so the commit picked up the stale staged blob instead of
the working tree's fixed content).

tar_stream_directory() now takes a caller-supplied excludeNames/
excludeCount list instead of a hardcoded 'backup_manifest' literal
(common/ stays free of pg_walserver-specific knowledge), and
cmd_base_backup.c's call site passes 'backup_manifest' explicitly.
Confirmed live: a real pg_basebackup run against pg_walserver now
correctly excludes backup_manifest from the tar itself (still served via
its own separate CopyOut manifest stream), where it was previously
being included in the tar by mistake because this wiring was missing.
WsRetentionAge/ws_parse_retention_age and sanitize_for_log kept their
pg_walserver-era names when they moved into src/bin/common/string_utils.c
in earlier commits, unlike every other symbol relocated into common/ this
session (wal_segment_name_format, file_crc32c, etc.), which all dropped
their project prefix.

Renamed to match this file's own pre-existing native convention
(stringToInt64, stringToDouble, IntervalToString, IntString -- bare
camelCase/PascalCase, no project prefix):

  WsRetentionAge        -> RetentionAge
  ws_parse_retention_age -> stringToRetentionAge
  sanitize_for_log       -> sanitizeForLog

Updated every call site: cli_archive_cleanup.c/.h, cli_basebackup.c,
auth.c, startup.h, cmd_archive_file.c, cmd_check_file.c, accept_loop.c,
cmd_fetch_file.c. cli_archive_cleanup.c's own static ws_retention_age_
cutoff() is untouched -- it's a genuinely pg_walserver-specific helper,
not one of the renamed symbols.
A previous session in this branch mistakenly introduced a documentation
convention that doesn't match this project's established style: a short
pointer comment in the .c file ("/* fn -- see some_header.h. */") with
the full prose documentation left in the .h file. The real convention,
confirmed by src/bin/common/pgsql.c/pgsql.h and
src/bin/pg_autoctl/monitor.c/monitor.h, is: full function documentation
lives in the .c file directly above the function body; .h files carry
bare prototypes (no comment) for functions, and keep documentation only
for types/macros/structs.

This commit relocates the full doc comments back to the .c files and
leaves bare declarations in the .h files, across:

  - src/bin/pg_walserver/receivewal.c/.h
  - src/bin/pg_walserver/cli_basebackup.c/.h
  - src/bin/pg_walserver/backup_bootstrap.c/.h
  - src/bin/pg_walserver/accept_loop.c/.h
  - src/bin/pg_walserver/cli_common.c/.h
  - src/bin/pg_walserver/wal_dir_scan.c/.h
  - src/bin/common/openssl_utils.c/.h
  - src/bin/common/tar_stream.c/.h
  - src/bin/common/string_utils.c/.h
  - src/bin/common/process_supervisor.c/.h
  - src/bin/common/wal_segment.c/.h

Beyond the short-pointer occurrences, a spot-check of these headers also
found function doc comments with no short-pointer counterpart at all
(cli_basebackup_run, ws_accept_loop's mutable-config paragraph,
wal_dir_find_latest, wal_dir_has_any_segment, wal_position_cache_read,
ws_receivewal_progress_write/_read, sanitizeForLog, and the
process_restart_counters_*() trio sharing one combined comment) -- these
were relocated the same way. ws_accept_loop's .c comment and its .h
mutable-config paragraph were merged rather than one replacing the
other, since both described genuine, non-overlapping behavior. The
process_restart_counters_*() shared comment was split into one doc per
function, each now living above its own definition.

No logic/behavior change -- pure comment relocation.
Catches the stragglers from the previous pass that used '-- see X.h's
own comment' phrasing instead of '-- see X.h.': the full prose doc
still lived in the .h while the .c only had a bare pointer back to it.
Pure relocation, no behavior change.

- src/bin/common/file_utils.c/.h: read_file_capped, open_regular_file,
  read_file_flags
- src/bin/common/system_utils.c/.h: monotonic_ms
- src/bin/pg_walserver/cli_ls.c/.h: cli_ls_run
- src/bin/pg_walserver/cli_ps.c/.h: cli_ps_run
- src/bin/pg_walserver/cli_setup.c/.h: cli_setup_run
- src/bin/pg_walserver/cli_status.c/.h: cli_status_run
- src/bin/pg_walserver/cli_upstream.c/.h: cli_resolve_upstream
- src/bin/pg_walserver/hba.c/.h: hba_ruleset_requires_client_cert
- src/bin/pg_walserver/cli_cluster.c/.h: ws_cluster_register_run
- src/bin/pg_walserver/routes.c/.h: routes_slot_name, config_load_global,
  config_set_global_property, config_file_path (WsGlobalConfig's own
  type doc stays in routes.h, untouched)
- src/bin/pg_walserver/tls.c/.h: ws_tls_server_load_ca,
  ws_tls_get_peer_cert_cn
Same anti-pattern as 18f445f/7075a8f, missed by both of those sweeps:
this one had no pointer comment at all in the .c (not even a short
"-- see X.h" stub), just a bare function body, with the full doc sitting
only in file_crc32c.h -- so neither sweep's text-pattern search (which
looked for a pointer phrase IN the .c file) ever matched it.
Found via a clang-19 AST-based sweep (-Xclang -ast-dump, matching every
function declared in a file's own .h against its definition in the .c,
per this project's file.c/file.h pairing convention) rather than the
text-pattern grepping the two prior rounds (18f445f, 7075a8f) used --
those could only catch cases with some pointer phrase left in the .c;
these 11 had none at all (a fully bare definition, full doc sitting only
in the .h), the exact same silent-miss shape caught in file_crc32c.c
(23d4e30) but never checked for tree-wide until now.

Fixed: scram_build_verifier/scram_server_first (scram.c/.h),
ws_authenticate (auth.c/.h), ws_replication_slot_exists/_read_restart_lsn/
_update_restart_lsn/_oldest_restart_lsn (cmd_replication_slot.c/.h),
ws_fetch_file_client (fetch_client.c/.h), hba_parse_file (hba.c/.h),
ws_ps_state_path/_write/_read (ps_state.c/.h).

Swept all 68 hand-written .c files in src/bin/common/ and
src/bin/pg_walserver/ (excluding bison/flex-generated and vendored
sources); re-ran after these fixes and confirmed zero remaining cases.
…code

routes.c/routes.h become clusters.c/clusters.h, and every identifier
built on "route" follows: WsRoute -> WsCluster, WS_ROUTES_WILDCARD_KEY
-> WS_CLUSTERS_WILDCARD_KEY, routes_load/routes_find/routes_find_exact/
routes_find_by_hostname/routes_slot_name/routes_persist_path/
routes_set_property/routes_drop_section -> the clusters_* equivalents,
and every local variable, parameter, and log/CLI-help string that said
"route" for this concept now says "cluster". hba.c/hba.h's HBA column
and matching field (generated/parsed pg_walserver_hba.conf format) move
from ROUTE/routeKey to CLUSTER/clusterKey; the test .pgaf specs under
tests/tap/specs/ that assert on HBA content or log wording are updated
to match.

The separate, unrelated IP-routing/CIDR-auto-discovery code (ipaddr.c/.h's
fetchLocalIPAddressForRouting(), hba.c's ws_setup_autodetect_cidr()/
ws_setup_local_ip_via_interfaces() and their own "kernel routing table"
comments) is untouched -- that's a genuine, different "route" meaning
(literal OS IP routing for the HBA CIDR setup wizard), not this concept.

"route"/"routing" referring to this concept is also gone from source
comments and user-visible strings (CLI help text, log messages): the
dbname/SNI dispatch mechanism is now described as "addressing" a
cluster, not "routing" to a route, to avoid the same ambiguity in
prose. The CLI's own "cluster" vocabulary (cluster register/drop/list/
...) was already in place and is unchanged.

pgaftest's test_spec_parse.c/.h are regenerated from test_spec_parse.y
via bison to match the two comment-only changes in the grammar source.
Sweeps docs/ref/*.rst and src/bin/pg_walserver/README.md to match the
code rename in the previous commit: every "route"/"Route"/"ROUTE"
referring to the pg_walserver.ini cluster concept becomes "cluster"/
"Cluster"/"CLUSTER", including section headings ("The routes file"
-> "The clusters file", "Route keys are opaque strings, not paths" ->
"Cluster keys...", "The wildcard route (*)" -> "The wildcard cluster
(*)") and the pg_walserver.rst/pg_walserver_serve.rst "Routing" /
"Routing more than one cluster by name: TLS SNI" sections, renamed to
"Addressing a cluster" / "Addressing more than one cluster by name:
TLS SNI" with every cross-reference updated to match. pg_walserver_
cluster.rst's "the clusters (routes) one pg_walserver..." drops the
now-redundant "(routes)" parenthetical.

Every literal captured log/HBA example shown in these docs is updated
to the wording the actually-renamed code now prints (verified against
the source, not guessed); none of this was re-captured live in this
pass, so it's a careful hand-edit rather than a fresh capture.

Left untouched: prose about the unrelated IP-routing/CIDR-auto-discovery
feature, and the literal test spec filename tests/tap/specs/pg_walserver_
sni_routing.pgaf referenced from a README.md heading.
Give pg_autoctl a genuinely new node kind that supervises `pg_walserver
serve` as a plain child process: no monitor registration, no keeper
FSM participation, just "run this binary" -- the same shape
service_postgres_ctl.c already uses to run Postgres, applied to
pg_walserver instead.

- src/bin/common/pgsetup.h: new PgInstanceKind value NODE_KIND_WALSERVER.
- src/bin/pg_autoctl/config.h, config.c: new PG_AUTOCTL_ROLE_WALSERVER /
  WALSERVER_ROLE = "walserver"; ProbeConfigurationFileRole() recognises
  a walserver node's own config file from its [pg_autoctl] role=walserver
  line, same convention as role=monitor/role=keeper.
- src/bin/pg_autoctl/service_walserver.h/.c (new): WalServerConfig (the
  minimal, non-Postgres equivalent of KeeperConfig: pgdata, port, name),
  walserver_config_write()/read(), service_walserver_ctl_start() (fork +
  execvp "pg_walserver serve --pgdata ... [--port ...]", modelled on
  service_postgres_ctl_start()), and start_walserver() (registers that
  one service with the supervisor, no keeper-init/activation dance).
- src/bin/pg_autoctl/cli_create_node.c: new "pg_autoctl create walserver
  --pgdata --port --name --run" command, writing the minimal config file
  above and, with --run, starting the service directly.
- src/bin/pg_autoctl/cli_service.c: "pg_autoctl run"'s role dispatch
  gets a PG_AUTOCTL_ROLE_WALSERVER case, calling the new cli_walserver_run().
- src/bin/pg_autoctl/cli_root.c, cli_common.h, supervisor.h: register the
  new create command and its SERVICE_NAME_WALSERVER.
- src/bin/pg_autoctl/nodespec.c: nodespec_read()/write() know the
  "walserver" node.ini kind string; nodespec_create_argv() builds the
  much simpler "pg_autoctl create walserver --pgdata ... [--port ...]
  [--run]" argv for it, skipping all the monitor/formation/SSL/auth
  machinery that doesn't apply. create/launch-deferred support comes for
  free from the existing, kind-agnostic gate in cli_node.c.
- src/bin/pg_autoctl/cli_show.c: "show file --contents" gets an honest
  "not supported for a walserver node" case (the switch over
  pgAutoCtlNodeRole is exhaustive, so this was a build error, not just a
  style choice -- every other ProbeConfigurationFileRole() call site
  already had a default: case that cleanly errors out, so those were
  left untouched).

Verified by an actual run inside the build-stage container: `pg_autoctl
create walserver --pgdata ... --port ... --run` and the two-step
`create walserver` + `run --pgdata ...` both really start and
supervise a listening pg_walserver serve child process.
…/launch deferred

`pg_walserver <name>` DSL sugar (test_spec_parse.y's pg_walserver_line)
used to exec "pg_walserver serve" directly as the container's own PID 1,
with zero pg_autoctl involvement. Default that through the new
`pg_autoctl create walserver` / `pg_autoctl node run` path instead,
giving it pg_autoctl's own create/launch-deferred support and process
supervision for free -- while preserving the one thing the old default
did that pg_autoctl itself has no reason to know about: seeding a real,
usable default pg_walserver_hba.conf into the node's pgdata before
"serve" ever starts (otherwise every connection is rejected until
something else writes one).

- test_spec_parse.y: pg_walserver_line now sets current_node->kind =
  NODE_KIND_WALSERVER (compose_gen_write_node_ini() then emits "kind =
  walserver" into the node's own node.ini), and its default
  commandOverride changes from "exec pg_walserver serve ..." to
  "mkdir -p <pgdata> && cp <bind-mounted default hba> <pgdata>/
  pg_walserver_hba.conf [|| true] && exec pg_autoctl node run <ini>" --
  same bind-mount-then-copy pattern already used for SSL certs, same
  reason (a bind-mounted file's auto-created parent directory would
  otherwise be root:root and un-writable by the container's own user for
  everything else pg_walserver needs to create under pgdata). The
  explicit "command \"...\"" override escape hatch is untouched --
  a spec that still wants the old "bypass pg_autoctl entirely" behavior
  keeps working exactly as before.
- test_spec_parse.y: aux_opt_list (shared by postgres_line and
  pg_walserver_line) now also accepts the same deferred/launch/create
  tokens node_opt_list already has ("deferred", "launch deferred",
  "create deferred", "create and launch deferred", "launch
  immediate"). Verified safe by rebuilding the generated parser with
  `bison -Wall`: zero new conflicts, zero new warnings, before and
  after this change (only pre-existing %empty style warnings, unrelated).
  test_spec_parse.c/test_spec_scan.c regenerated via `make generate`.
- compose_gen.c: compose_gen_write_node_ini()'s kind switch gets a
  "walserver" case; write_pg_walserver_default_hba()'s header comment
  updated to describe the new target path and command.

Verified via `pgaftest cluster prepare <spec> <dir>` (writes the real
generated docker-compose.yml/node.ini/hba files without starting
containers): a bare "pg_walserver ws1 deferred" node gets kind =
walserver, [launch] create/run = deferred, a generated default HBA file,
and a command that mkdir/cp's it into place before "pg_autoctl node run".
…o 6543

aux_opt_list (the small, closed modifier set pg_walserver_line/
postgres_line accept after the node name) had no way to set a custom
port for a `pg_walserver <name>` node: node->pgPort was only reachable
from ordinary node_opt_list's own "T_PORT T_INTEGER" rule, not from
aux_opt_list. compose_gen_write_node_ini()'s
"node->pgPort > 0 ? node->pgPort : 5432" then silently defaulted every
unconfigured walserver node to Postgres's own 5432, which makes no
sense for pg_walserver (this project's own convention elsewhere is
6543).

- test_spec_parse.y: add "port N" to aux_opt, reusing the exact same
  rule body ordinary node_opt's own T_PORT T_INTEGER rule already has.
  Verified with `bison -Wall -d`: identical warning set before/after
  (the pre-existing %empty-rule warnings only), zero new
  shift/reduce or reduce/reduce conflicts introduced -- T_PORT never
  starts a bare top-level cluster_item, so it is unambiguous here the
  same way T_DEFERRED/T_LAUNCH/T_CREATE/T_IMMEDIATE already are.
- compose_gen.c: compose_gen_write_node_ini()'s default now falls back
  to 6543 for NODE_KIND_WALSERVER nodes specifically (only when the
  spec didn't set an explicit "port N"); every other node kind keeps
  the 5432 default unchanged.
- test_spec_parse.c/.h regenerated via `make generate` (bison/flex),
  same convention as commit 920fade.
Convert the primary `pg_walserver <name>` node in each of the 4 specs
that still used the old "command \"tail -f /dev/null\", exec
`pg_walserver cluster register`/`serve` directly by hand" pattern over
to the new mechanism: `port <n>` (previous commit) + `launch
deferred`, with setup{}/test steps doing their cluster-register/
certificate/HBA provisioning against the standard NODE_PGDATA
(/var/lib/postgres/pgaf) and then releasing the node with
`exec <node> pg_autoctl node start` (the same precedent every other
launch-deferred node in this suite already uses, e.g.
launch_deferred_set_metadata.pgaf) instead of a bare
`nohup pg_walserver serve ... &`. Every spec's own assertions are
otherwise unchanged -- only how/where `pg_walserver serve` is started
and configured changes, not what each test actually verifies.

pg_walserver_list_ps_status.pgaf:
  - node "walserver": command override -> port 6543 + launch deferred.
  - setup{}: drop the hand-written pg_walserver_hba.conf (now the DSL's
    own auto-generated default); cluster register moved from /tmp/lps
    to /var/lib/postgres/pgaf.
  - test_002: "pg_autoctl node start" releases serve (was
    nohup+redirect-to-logfile); the "both clusters got their bootstrap
    backup" wait now polls `pg_walserver list backups` instead of
    grepping a serve log file, because serve's stdout/stderr is now
    inherited from pg_autoctl (the container's own stdout), not a file
    an exec step can read.

pg_walserver_pitr_and_secondary.pgaf:
  - node "server": command override -> port 6543 + launch deferred.
  - setup{}: drop the hand-written pg_walserver_hba.conf; cluster
    register and all later path references moved from /tmp/ws to
    /var/lib/postgres/pgaf; "pg_autoctl node start" releases serve
    (was nohup+redirect-to-logfile) at the same point in setup{} the
    old nohup was.

pg_walserver_clientcert.pgaf:
  - node "walserver": command override -> port 6546 + launch deferred.
  - setup{}: CA/cert generation, create-cert, cluster ini, scram
    secret and the spec's own custom clientcert=verify-full HBA all
    moved from /tmp/cc to /var/lib/postgres/pgaf (the custom HBA still
    overwrites the DSL's auto-generated default before "serve" ever
    starts, same timing as before).
  - test_000: "pg_autoctl node start" releases the primary instance
    (was nohup+redirect-to-logfile), AFTER the negative-test assertion
    above it. The negative-test instance itself (a separate, isolated
    /tmp/ccnoca scratch dir, port 6599, proving "clientcert=verify-full
    with no usable TLS CA fails closed at startup") is untouched: still
    a one-off, directly-exec'd `pg_walserver serve`, unrelated to and
    unaffected by the primary node's own conversion.

pg_walserver_sni_routing.pgaf:
  - node "walserver": command override -> port 6544 + launch deferred;
    "alias" kept as-is.
  - setup{}: drop the hand-written pg_walserver_hba.conf (the compose-
    subnet-wide default already admits every peer this spec needs);
    cluster register and all later path references moved from
    /tmp/sni to /var/lib/postgres/pgaf.
  - test_001: "pg_autoctl node start" releases serve (was
    nohup+redirect-to-logfile).
  - test_004 (negative test: "more than one named cluster without TLS
    refuses to start"): rewritten. The old version stopped the live
    instance with `pkill -x pg_walserver`, moved its certs away, tried
    a one-off `pg_walserver serve`, then restored the certs -- but the
    primary node's `pg_walserver serve` is now pg_autoctl-supervised
    (RP_PERMANENT, service_walserver.c's start_walserver()), so
    pkill-ing it would just make pg_autoctl respawn it immediately,
    defeating the test. Instead this now takes a throwaway copy of the
    same pgdata (same two already-registered named clusters,
    server.crt/key and any pid/state file removed from the copy only)
    and runs the one-off `pg_walserver serve` against that copy on its
    own port -- the same "bypass pg_autoctl, exec directly" shape, now
    without ever touching the still-running primary instance. (cli_
    serve.c's own startup validation rejects the missing-TLS case
    before ever writing/checking a pidfile, so no further cleanup of
    the copy is needed.) This is a deliberate adaptation, not a literal
    "leave the negative test untouched" -- see this task's final
    report for the full reasoning.
…ferred modifiers

pg_walserver <name> now defaults to going through pg_autoctl create
walserver / pg_autoctl node run (commit 920fade), not a direct PID-1 exec
of pg_walserver serve -- the doc still described the old behavior.

Also documented two pre-existing gaps in the node-modifiers table that
were never written up at all, caught while touching this section: the
bare "deferred"/"create deferred"/"create and launch deferred"/"launch
immediate" forms (only "launch deferred" had a row), and "port <N>" itself
(existed in the grammar for ordinary nodes well before this session, just
never documented, and now also meaningful for a pg_walserver <name> node
with its own correct default of 6543 instead of 5432).
…el/WAL-continuity helpers into common/, extract backup_list.c

Four related reorganizations to cli_archive_cleanup.c, zero behavior
change:

1. The boxed, dash-divider usage comment above archive_cleanup_command
   is flattened to a plain comment, matching this file's own
   convention everywhere else (and out of scope: every other cli_*.c
   file's dash-divider header deliberately stays as-is).

2. ws_retention_age_cutoff() moves next to stringToRetentionAge() in
   src/bin/common/string_utils.c/.h, renamed retentionAgeCutoff() to
   match that file's own camelCase convention (stringToInt64,
   IntervalToString, stringToRetentionAge).

3. parse_backup_label_time() -- fully generic, no pg_walserver types
   in its signature -- moves to src/bin/common/wal_segment.c/.h,
   renamed wal_backup_label_parse_time() to match that file's own
   wal_<noun>_<verb> convention.

4. WsBackupInfo/ws_backup_list_load()/backup_cmp() move out of
   cli_archive_cleanup.c entirely into a new, pg_walserver-specific
   src/bin/pg_walserver/backup_list.c/.h pair (added to the Makefile's
   LOCAL_SRC). cli_list.c and cli_ls.c, which both used WsBackupInfo/
   ws_backup_list_load() via cli_archive_cleanup.h purely as a side
   effect, now include backup_list.h directly instead -- the same
   "one cli_*.c depending on another cli_*.c's header for a shared
   utility" smell already fixed once this session for RetentionAge/
   cli_basebackup.c.

Also moves the WAL-continuity pre-flight check's generic core to
src/bin/common/wal_segment.c/.h:

- read_last_history_line() -> wal_history_read_last_line(): fully
  generic, no pg_walserver types.
- check_wal_range() -> wal_check_range(), and its WsContinuityProblem
  parameter type -> WalContinuityProblem (dropping the "Ws" prefix,
  same as every other common/ type, e.g. IntString, RetentionAge):
  fully generic already, just uses the renamed struct.
- ws_check_wal_continuity() is NOT generic as written: it takes
  WsCluster/WsBackupInfo (pg_walserver types) directly, purely to
  call wal_dir_find_latest() for the newest kept backup's own end
  boundary. Split it: a new generic wal_check_continuity() in
  wal_segment.c takes plain scalars (segSize, a WalContinuityEntry[]
  of already-resolved per-backup start tli/segno pairs, and the
  already-resolved "latest on-disk" tli/segno as plain parameters) and
  does the logging-free continuity-checking loop, returning one
  WalContinuityProblem per entry. ws_check_wal_continuity() in
  cli_archive_cleanup.c stays as a thin pg_walserver-side wrapper: it
  resolves WsCluster/WsBackupInfo into those plain entries (via
  wal_dir_find_latest(), wal_dir_scan.c), calls wal_check_continuity(),
  then does all the cluster/backup-specific log_error() reporting,
  reproducing the exact same two message templates the original
  function logged (the "no WAL at all to compare against" case is
  distinguished from a genuine wal_check_range() gap purely by
  position -- only the newest entry, when there was nothing on disk to
  anchor it, can hit that case -- so the wrapper picks the matching
  template without needing the common function to know about either
  one by name).

src/bin/common/wal_segment.c/.h and string_utils.c/.h end up with zero
WsCluster/WsBackupInfo/WsRoute references (verified via grep).

Every new/moved function has a full header comment in its own .c file,
bare declaration in its own .h, matching this session's established
convention.
…amed phase helpers

ws_archive_cleanup_run() did five distinct things in one function
(validate inputs + load the backup list + resolve ".latest"; decide
the kept/removed set per backup via --keep-count/--keep-age plus the
replication-slot floor; the WAL-continuity pre-flight check and
whether to proceed; actually remove non-kept backups; actually remove
expired WAL/.partial/.backup files) and, under its old name, read
almost identically to this file's own CLI dispatcher
cli_archive_cleanup_command_run() right below it -- the same "_run"
naming collision already fixed once this session for
ws_archive_run -> ws_archive_push_file.

Split into five static phase helpers, each named for what it does:

- resolve_latest_backup(): looks up and validates "basebackups/.latest"
- decide_kept_backups(): --keep-count/--keep-age math plus the
  replication-slot floor, logging which cutoff rule(s) applied
- run_continuity_preflight(): runs ws_check_wal_continuity() and
  decides whether the deletion phases may proceed
- remove_expired_backups(): the backup-directory removal loop
- remove_expired_wal_files(): the WAL/.partial/.backup removal loop

The renamed top-level function, ws_archive_cleanup_execute(), does
nothing but call these five in order and return the final result --
same computation, same log messages, same return value as before,
purely reorganized. Its one external call site
(cli_basebackup.c's cli_basebackup_command_run()) and its own
cli_archive_cleanup_command_run() call site are updated to match, as
is cli_archive_cleanup.h's declaration and cli_basebackup.h's comment
mentioning it by name.
Same stylistic fix already applied to cli_archive_cleanup.c this session:
replace the ASCII-art '/* ----...---- */' divider above each command's
CommandLine/usage synopsis with a plain '/* ... */' comment carrying the
same content, no box. Applies to cli_archive.c, cli_create_cert.c,
cli_fetch_systemid.c, cli_ps.c, cli_reload.c, cli_restore_wal.c,
cli_scram_secret.c, cli_serve.c, and cli_stop.c (each had exactly one).

cli_root.c had two: the one above cli_help_run/help_command, and the one
labeling the 'Root command table' section before root_subcommands[] --
both flattened the same way, for consistency across the file set.

cli_cluster.c's own stray divider, above ws_cluster_register_run, was
mislabeled as covering 'register | drop | list | set-upstream' even
though the file defines six commands (its label did not even mention
enable/prune) -- flattened into an accurate, register-specific plain
comment instead; each command's own run function already has its own
doc comment below it from this session's earlier doc-comment-convention
sweep, so nothing needed duplicating.

No behavior change.
…to ws_

cli_basebackup_run, cli_fetch_systemid_run, cli_setup_run, cli_status_run,
cli_ls_run and cli_ps_run each collided, by name, with their own file's
static <command>_command_run dispatcher, and (the first two) are also
called directly from cli_cluster.c as real cross-file APIs, not just
their own file's dispatcher -- exactly the case this project's
'cli_-prefix = thin CLI wrapper only, ws_-prefix = the real, reusable
implementation' convention exists for (already applied this session to
ws_archive_push_file, ws_restore_fetch_file, ws_archive_cleanup_execute).

Renamed:
  cli_basebackup_run      -> ws_basebackup_execute
  cli_fetch_systemid_run  -> ws_fetch_systemid_execute
  cli_setup_run           -> ws_setup_execute
  cli_status_run          -> ws_status_report
  cli_ls_run              -> ws_ls_report
  cli_ps_run              -> ws_ps_report

Updated every definition, its .h declaration, and every call site,
including cli_cluster.c's own calls to ws_basebackup_execute()/
ws_fetch_systemid_execute() and the matching mention in README.md.
cli_cluster.c's own six ws_cluster_*_run functions and
cli_create_cert.c's ws_create_cert_run are untouched: already
differentiated by the ws_/cli_ prefix split, not real collisions.

No behavior change.
…se helpers

Both functions had grown to do several, clearly separable things in one
body. Pure extraction, same logic, same order, same error handling --
each helper below is called exactly once, in the original order.

cli_serve.c: cli_serve_run (~283 lines) split into:
  cli_serve_check_pgdata_or_insecure   -- --pgdata/--insecure guard
  cli_serve_load_global_defaults       -- config paths + persisted
                                           port/auth-timeout/ssl-*-file
                                           defaults
  cli_serve_setup_tls                  -- cert/key + optional CA loading
  cli_serve_load_clusters_and_hba      -- HBA default-creation + parsing
                                           pg_walserver.ini and the HBA
                                           file
  cli_serve_check_sni_requirements     -- clientcert/named-cluster-vs-TLS
                                           sanity checks
  cli_serve_start_receivewal_children  -- start "receivewal=pull" children
  cli_serve_start_backup_bootstraps    -- kick off missing-backup bootstraps
  cli_serve_write_pidfile_or_die       -- write the pidfile
cli_serve_run itself now only orchestrates these in order, then seeds the
SCRAM mock secret, calls ws_accept_loop(), and cleans up the pidfile.

cli_setup.c: ws_setup_execute (~179 lines, after the earlier rename)
split into:
  cli_setup_validate_pgdata         -- pgdata validation/creation
  cli_setup_persist_global_config   -- persist port/ssl-*/auth-timeout
                                        global config properties
  cli_setup_create_cert_if_missing  -- auto-create a self-signed cert
  cli_setup_create_hba_if_missing   -- auto-create the default HBA file
                                        (with CIDR autodetection)
ws_setup_execute itself now only orchestrates these in order.

No behavior change.
…tils.c

cli_ps.c and cli_status.c each had a byte-for-byte identical static
format_elapsed() (only a one-clause doc-comment difference) wrapping
IntervalToString() (common/string_utils.c) with a "-" sentinel for an
unset/zero timestamp. Moved to src/bin/common/system_utils.c/.h as
format_elapsed_time(), a bare name matching that file's own existing
convention (monotonic_ms, pretty_print_bytes, get_system_info -- no ws_/
cli_ prefix). cli_ps.c and cli_status.c now both call it and no longer
keep their own static copy.

No behavior change.
…ader CLI sweep

- de-duplicate format_utc/directory_size (byte-identical between cli_ls.c
  and cli_list.c) into common/system_utils.c and common/file_utils.c,
  removing both now-redundant static copies
- fix cli_list.c's own 3 "_run" collisions with its "_command_run"
  dispatchers: cli_list_clusters_run/cli_list_backups_run/cli_list_wal_run
  -> ws_list_clusters_report/ws_list_backups_report/ws_list_wal_report
- flatten cli_list.c's remaining 6 dash-divider header comments to plain
  prose, matching the rest of this round's cli_*.c files
- flatten the 4 dash-dividers my own earlier task-scoping for the broader
  sweep agent missed: cli_status.c, cli_basebackup.c, cli_setup.c, cli_ls.c
- fix 2 stale ws_basebackup_execute() comment mentions in
  cli_archive_cleanup.c left over from the cli_basebackup_run rename

Verified: make docker-indent/docker-check clean, a --no-cache Docker
build with zero new warnings, and a live smoke test of pg_walserver
ls/list clusters/list backups/list wal against a synthetic pgdata
fixture confirming identical output after the relocation.
walserver_config_write() used a raw fopen(path, "w")/fclose() pair
(IGNORE-BANNED) instead of this project's own facilities, unlike every
other *_config_write_file() (keeper_config.c, monitor_config.c). Fixed
to match: fopen_with_umask() + a temp-file + rename, so a concurrent
reader always sees either the old complete file or the new complete
one, never a partial write.

Swept the rest of pg_walserver/pg_autoctl/common for the same pattern
and fixed every other instance found:

- cmd_base_backup.c (read_backup_label, send_manifest), cmd_start_
  replication.c (segment file open), tar_stream.c (emit_file_contents):
  raw fopen(path, "r"/"rb") -> fopen_read_only(), the existing
  designated wrapper (file_utils.c) for exactly this case.
- nodespec.c's nodespec_write_to_path(): raw fopen(path, "w")/fclose()
  -> fopen_with_umask()/fclose()-with-error-check, matching every other
  config writer in pg_autoctl.
- pghba.c's ident-map append: raw fopen(path, "a")/fformat()/fclose()
  (no error check on fclose at all) -> the existing append_to_file()
  facility (file_utils.c), which already does this exact thing.

Left alone (not instances of this pattern): cmd_replication_slot.c's
open()+flock() advisory lock (no existing facility provides flock
semantics; a genuinely distinct, lower-level need) and every other
open()/read()/write() use project-wide, which this project already
prefers over stdio for plain fd-based I/O -- only fopen()/fclose()
pairs that duplicate an existing file_utils.h wrapper were in scope.
Pre-existing fopen() use in pgaftest/parson (vendored) is untouched:
predates this PR, out of scope.

Verified: make docker-indent/docker-check clean, a --no-cache Docker
build with zero new warnings, and a live smoke test of
"pg_autoctl create walserver" confirming the new config write leaves
no stray .tmp file and produces identical ini content.
walserver_config_write() and nodespec_write_to_path() built their output
by issuing one fformat()/appendPQExpBuffer() call per line directly
against an open stream, then (after the previous commit's fix) closed
it and renamed a temp file into place by hand.

Switch both to the pattern pg_walserver's own clusters.c/hba.c already
use throughout: build the whole file in memory with a PQExpBuffer,
then hand it to write_file_atomic() (common/file_utils.c) as a single
write -- which already does the temp-file + fsync + rename + directory
fsync dance, strictly stronger than the hand-rolled temp+rename this
file had (no fsync at all) before this change.

nodespec.c: extracted nodespec_build(spec, PQExpBuffer) as the one
shared serializer behind both nodespec_write(spec, FILE *out) (still
takes a FILE* for its stdout-display caller, "pg_autoctl node show",
but now does one fwrite() of the finished buffer instead of ~15
separate fformat() calls against the stream) and
nodespec_write_to_path(), so the two can never drift against each
other.

Verified: make docker-indent/docker-check clean, a --no-cache Docker
build with zero new warnings, and live smoke tests of
"pg_autoctl create walserver" (walserver_config_write) and
"pg_autoctl node start" (nodespec_write_to_path, clearing the
[launch] deferred flag) -- both confirmed to write correct content
with no stray .tmp file left behind.
pghba_ensure_host_rule_exists()/pghba_ensure_host_rules_exist() already
built the new pg_hba.conf contents in memory with a PQExpBuffer, but
wrote them out with the plain write_file() (no fsync, no temp-file
rename -- a crash mid-write can leave a truncated pg_hba.conf behind).
Switch both to write_file_atomic(), matching the pattern this same
pass just applied to walserver_config_write()/nodespec_write_to_path(),
and that pg_walserver's own clusters.c/hba.c already use throughout.

Verified: make docker-indent/docker-check clean, a --no-cache Docker
build with zero new warnings, and a live "pg_autoctl create monitor"
run (exercises pghba_ensure_host_rules_exist() for real, granting the
autoctl_node replication HBA rule) confirming the expected
"# Auto-generated by pg_auto_failover" rule lands in pg_hba.conf with
no stray .tmp file left behind.
@dimitri
dimitri force-pushed the pg_walsender-server branch from 81f6f17 to fb117fd Compare October 14, 2026 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant