Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
a63dc6c
polkadot-test: the machine as a directory, applied with one command
mordamax Nov 13, 2026
8682589
bite: cut the relay's upgrade delay and cooldown on a shared-relay fork
mordamax Nov 13, 2026
1ae7b09
enact-upgrades: apply every parachain at once, then the relay
mordamax Nov 13, 2026
622ce2d
Docs: upgrades on a fork enact in minutes, and how to run a second one
mordamax Nov 13, 2026
3b434d1
Seed a Bulletin storage authorizer at bite time
csmnprschv Nov 13, 2026
fa93550
Seed both attestation allowances at bite time
csmnprschv Nov 13, 2026
f6a55be
Encode and decode overrides as SCALE bytes, not big-endian hex
csmnprschv Nov 13, 2026
d17063e
Give a bitten fork a DHT of its own (#26)
csmnprschv Nov 13, 2026
3334d3c
Point the TURN test at a deployed relay with TURN_HOST and DUB_BASE
mordamax Nov 13, 2026
0662c57
polkadot-test: serve TURN over TLS from a top-level nginx stream file
mordamax Nov 13, 2026
8644e5d
polkadot-test: keep TURN and JWT secrets in /etc/ppn/secrets.env
mordamax Nov 13, 2026
7d403df
Pick the DUB build each network needs
csmnprschv Nov 13, 2026
139152c
Stop a resumed fork from restoring its snapshots
csmnprschv Nov 13, 2026
646157d
Find a fork's dotNS registry through Asset Hub's dispatcher when the …
mordamax Nov 13, 2026
6ee2de1
Point the Polkadot fork's descriptor at PCF's
mordamax Nov 13, 2026
724cee5
polkadot-test: add PPN_PROFILE to the secrets file ppn start now checks
mordamax Nov 13, 2026
ea25251
Update the fork docs for the fork id and mainnet on v2.5.3
csmnprschv Nov 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,6 @@ server/nginx/ppn.conf
# The dashboard UI compiles into the CLI package (packages/dashboard-ui -> vite build).
# Built output ships in the npm tarball via prepack, never in git.
packages/cli/web/

# The VM branch keeps its one secret out of git
deploy/*/local.env
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,10 +89,11 @@ ppn start polkadot --fork --fresh-bite \

A `--upgrade` accepts any blob: a release asset, a PR's build artifact, a local build, and
`relay=` for the relay itself. The bite warp-syncs four chains, around 20 minutes, and a
parachain's upgrade goes live an hour after the spawn, because Polkadot's upgrade delay is 600
relay blocks. The dashboard shows the spec version flip; the log of the process applying the
upgrades is under its logs tab. If that process gave up, or you want to redo one chain, submit
the apply step yourself with no blob, and the one the bite authorized is used:
parachain's upgrade goes live a few minutes after the spawn: the bite cuts the relay's upgrade
delay to 30 blocks (live Polkadot: 600, an hour). The dashboard shows the spec version flip;
the log of the process applying the upgrades is under its logs tab. If that process gave up,
or you want to redo one chain, submit the apply step yourself with no blob, and the one the
bite authorized is used:

```bash
ppn upgrade people --enact-timeout 70 # apply what the bite authorized for People
Expand Down
16 changes: 11 additions & 5 deletions config/versions.env
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ ZOMBIENET_VERSION="v0.4.15"
# Pinned per network as bite.doppelganger, since the build has to be able to
# execute the runtimes of the network being bitten. Restoring a published bundle
# (`make start FORK=1`) needs none of it: the forked network runs on the regular
# node binaries.
# node binaries, except collators, which need the fork-id fix (docs/FORK.md).

# =============================================================================
# Kubo IPFS
Expand Down Expand Up @@ -83,9 +83,16 @@ KUBO_VERSION="v0.39.0"
# public `docker.io/paritytech/device-uniqueness-backend` images tagged 0.1-0.3, which were
# built from older source. v0.5.0 refreshes the vendored People metadata to 3000000 (the
# invite pool minted nothing against it before) and logs metadata drift at boot, which is
# the diagnostic a fork running a runtime dub has never seen needs. Both kept the asset naming (dub-<version>-<triple>.tar.gz), the roles
# PPN spawns (all-in-one plus the three single-instance workers) and every config key in
# dub.ts — checked by booting each role against this env, not read off the release notes.
# the diagnostic a fork running a runtime dub has never seen needs.
#
# v0.6.0 ships every asset twice, one build per People runtime, and names them
# dub-<version>-<build>-<triple>.tar.gz. The runtime is fixed when the binary is built: it
# selects the vendored metadata and whether the invite-tickets roles exist at all. A network
# says which it needs through `dubBuild` in its descriptor, defaulting to testnet; only the
# Polkadot fork wants `polkadot`, whose People runtime carries neither Game nor ProofOfInk.
# Naming a role the binary does not have is a boot failure, so the same field also decides
# which services and databases are generated.
#
# `INVITE_INVITER_*` in upstream's .env.example is compose's host-side naming; the key the
# binary reads is still INVITER_ADDRESS. v0.7.0 defaults turn-api to Cloudflare and refuses a
# TURN_SECRET without TURN_PROVIDER, so dub.ts names coturn, which needs a non-empty ICE_SERVERS.
Expand All @@ -94,7 +101,6 @@ KUBO_VERSION="v0.39.0"
# runtime without it (paritytech/individuality-community#196).
DUB_REPO="paritytech/device-uniqueness-backend-community"
DUB_TAG="${DUB_TAG:-v0.8.0}"
DUB_NETWORK="${DUB_NETWORK:-testnet}"

# Portable PostgreSQL for the backend services. Fetched straight from upstream
# (public repo, no auth) because it is not a PPN build product. This is the
Expand Down
40 changes: 40 additions & 0 deletions deploy/polkadot-test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# polkadot-test.substrate.dev

The one machine that runs the Polkadot fork, from its own branch: main plus this directory and the
fork's engine changes, rebased onto main. The machine picks them up with one command. When
preview-net-v1 grows a fork target this directory moves there and the branch dies.

```bash
deploy/polkadot-test/apply.sh # pull, build, fetch, ports.env, nginx, unit, restart
deploy/polkadot-test/apply.sh --no-restart # everything except touching the running network
```

The fetch rewrites `polkadot-parachain` in place, so it puts the stock build back, and it fails
(ETXTBSY) while the collators are running the patched one. The collators need the build with the
fork-id fix (`docs/FORK.md`), so stop the network around an apply:

```bash
sudo systemctl stop ppn-polkadot
deploy/polkadot-test/apply.sh --no-restart
# reinstall the build with the fork-id fix
sudo systemctl start ppn-polkadot
```

What is where:

| File | Role |
| --- | --- |
| `vm.env` | Domain, public IP, run user, start flags. Committed. |
| `local.env` | The dashboard actions token. Gitignored; created on first apply. |
| `apply.sh` | The command. Idempotent; installs missing packages on a fresh box. |
| `start.sh` | What the unit runs: `ppn start polkadot --fork $START_FLAGS`. |
| `ppn-polkadot.service` | systemd unit template; `__ROOT__`/`__USER__` filled by apply. |
| `nginx/` | v1's template and websocket snippet, ours to patch here. |
| `/etc/ppn/secrets.env` | Created by apply on a fresh box with `TURN_SECRET`, `JWT_ED25519_SECRET` and `PPN_PROFILE=local`, and never rotated. Optional keys from `config/secrets.env.example`, such as push notifications, are added by hand. `ports.env` names it as `PPN_SECRETS_FILE`. |

Not done by apply, ever: the bite. That is a deliberate `make bite NETWORK=polkadot` (plus
`UPGRADES=...` for a runtime under test; see `docs/POLKADOT-FORK.md`), followed by `apply.sh` or
`systemctl start ppn-polkadot`.

And the certificate, once: `sudo certbot certonly --standalone -d <domain>` while nginx is
stopped; apply skips nginx until it exists and prints the command.
189 changes: 189 additions & 0 deletions deploy/polkadot-test/apply.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
#!/usr/bin/env bash
# The one command on the VM. Idempotent: run it after every push to this branch.
#
# deploy/polkadot-test/apply.sh pull, build, fetch, nginx, unit, restart the network
# deploy/polkadot-test/apply.sh --no-restart same, but leave the running network alone
# deploy/polkadot-test/apply.sh --no-pull work from the checkout as it is
#
# A bite is never done here: `make bite NETWORK=polkadot` is a deliberate act.
set -euo pipefail

HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$HERE/../.." && pwd)"
cd "$ROOT"
source "$HERE/vm.env"

RESTART=1
PULL=1
for arg in "$@"; do
case "$arg" in
--no-restart) RESTART=0 ;;
--no-pull) PULL=0 ;;
*) echo "apply.sh: unknown argument '$arg'" >&2; exit 2 ;;
esac
done

say() { echo "apply: $*"; }
changed() { ! cmp -s "$1" "$2"; }

# ---- 1. dependencies ------------------------------------------------------------------------
# What a fresh Ubuntu is missing for this to run at all. Node comes from NodeSource when it is
# absent or too old; pnpm through corepack; everything else from apt.
NODE_MAJOR_MIN=22
APT_WANTED=(nginx libnginx-mod-stream certbot gettext-base jq lsof unzip curl tmux git build-essential)
APT_MISSING=()
for pkg in "${APT_WANTED[@]}"; do dpkg -s "$pkg" >/dev/null 2>&1 || APT_MISSING+=("$pkg"); done
NEED_NODE=0
if ! command -v node >/dev/null 2>&1; then
NEED_NODE=1
elif [[ "$(node --version | sed 's/^v\([0-9]*\).*/\1/')" -lt "$NODE_MAJOR_MIN" ]]; then
NEED_NODE=1
fi
if [[ ${#APT_MISSING[@]} -gt 0 || "$NEED_NODE" == 1 ]]; then
say "installing: ${APT_MISSING[*]}${NEED_NODE:+ node}"
sudo apt-get update -qq
if [[ "$NEED_NODE" == 1 ]]; then
curl -fsSL "https://deb.nodesource.com/setup_${NODE_MAJOR_MIN}.x" | sudo -E bash -
sudo apt-get install -y nodejs
fi
[[ ${#APT_MISSING[@]} -gt 0 ]] && sudo apt-get install -y "${APT_MISSING[@]}"
fi
if ! command -v pnpm >/dev/null 2>&1; then
sudo corepack enable >/dev/null 2>&1 || corepack enable >/dev/null 2>&1 || sudo npm install -g pnpm
fi
if ! command -v gh >/dev/null 2>&1; then
say "WARNING: gh is not installed; \`make fetch\` needs GITHUB_TOKEN or gh auth for private release assets"
fi
sudo install -d -o "$RUN_USER" -g "$RUN_USER" "$LOG_DIR"

# ---- 2. code ----------------------------------------------------------------------------
# ports.env is patched below on every run, so the local copy is disposable: drop it before the
# pull, or a pull that touches the file fails.
if [[ "$PULL" == 1 ]]; then
git checkout -q -- config/ports.env 2>/dev/null || true
git pull -q --ff-only
fi
say "at $(git rev-parse --short HEAD) on $(git branch --show-current)"
pnpm install --frozen-lockfile >/dev/null
BUILD_LOG="$(mktemp)"
if ! pnpm -r build > "$BUILD_LOG" 2>&1; then cat "$BUILD_LOG"; rm -f "$BUILD_LOG"; exit 1; fi
rm -f "$BUILD_LOG"

# ---- 3. artifacts -----------------------------------------------------------------------
# Picks up a changed node pin or dub tag; a no-op when provenance says everything is current.
FETCH_LOG="$(mktemp)"
if ! make fetch NETWORK="$NETWORK" > "$FETCH_LOG" 2>&1; then cat "$FETCH_LOG"; rm -f "$FETCH_LOG"; exit 1; fi
grep -vE "^\s*✓|^$" "$FETCH_LOG" | grep -iE "fetch|download|missing|✗" || true
rm -f "$FETCH_LOG"

# ---- 4. the machine's values into ports.env --------------------------------------------
# zombienet hands custom processes no environment, so the dashboard and bootnode patching read
# these from the file. The actions token is the one secret; it lives in local.env.
LOCAL_ENV="$HERE/local.env"
if [[ ! -f "$LOCAL_ENV" ]]; then
printf 'DASHBOARD_ACTIONS_TOKEN=%s\n' "$(openssl rand -hex 24)" > "$LOCAL_ENV"
say "created $LOCAL_ENV with a new dashboard actions token"
fi
source "$LOCAL_ENV"
# TURN_SECRET and JWT_ED25519_SECRET replace public dev defaults; DUB refuses to start on a
# secrets file without the JWT seed. Created once, never rotated here.
if ! sudo test -f "$SECRETS_FILE"; then
sudo install -d -m 0750 -o root -g "$RUN_USER" "$(dirname "$SECRETS_FILE")"
printf 'TURN_SECRET=%s\nJWT_ED25519_SECRET=0x%s\n' \
"$(openssl rand -hex 24 | tr -d '\n' | base64)" "$(openssl rand -hex 32)" \
| sudo tee "$SECRETS_FILE" >/dev/null
sudo chown "root:$RUN_USER" "$SECRETS_FILE" && sudo chmod 0640 "$SECRETS_FILE"
say "created $SECRETS_FILE"
fi
# `ppn start` rejects a secrets file without a profile; the fork runs with the dev accounts.
if ! sudo grep -q '^PPN_PROFILE=' "$SECRETS_FILE"; then
echo 'PPN_PROFILE=local' | sudo tee -a "$SECRETS_FILE" >/dev/null
say "$SECRETS_FILE: added PPN_PROFILE=local"
fi
sed -i \
-e "s|^BOOTNODE_HOSTNAME=.*|BOOTNODE_HOSTNAME=$DOMAIN|" \
-e "s|^PPN_PUBLIC_URL=.*|PPN_PUBLIC_URL=https://$DOMAIN|" \
-e "s|^DASHBOARD_HOST=.*|DASHBOARD_HOST=127.0.0.1|" \
-e "s|^P2P_LISTEN_IP=.*|P2P_LISTEN_IP=$PUBLIC_IP|" \
-e "s|^DASHBOARD_ACTIONS_TOKEN=.*|DASHBOARD_ACTIONS_TOKEN=${DASHBOARD_ACTIONS_TOKEN:-}|" \
-e "s|^PPN_SECRETS_FILE=.*|PPN_SECRETS_FILE=$SECRETS_FILE|" \
config/ports.env
say "ports.env: domain $DOMAIN, listen ip $PUBLIC_IP, secrets $SECRETS_FILE"

# ---- 5. nginx ---------------------------------------------------------------------------
TLS_DIR="/etc/letsencrypt/live/$DOMAIN"
if [[ ! -f "$TLS_DIR/fullchain.pem" ]]; then
say "WARNING: no certificate at $TLS_DIR — nginx not touched. Issue one, then re-run:"
say " sudo systemctl stop nginx && sudo certbot certonly --standalone -d $DOMAIN --deploy-hook 'systemctl reload nginx'"
else
export PPN_NETWORK="$NETWORK" PPN_DOMAIN="$DOMAIN" PPN_TLS_DIR="$TLS_DIR" LOG_DIR DATA_DIR="$ROOT/data-fork-$NETWORK"
set -a; source config/ports.env; set +a
# Re-exported after ports.env, which must not be allowed to blank them.
export LOG_DIR DATA_DIR
ROUTED="$(mktemp)"; RENDERED="$(mktemp)"
node bin/ppn.mjs nginx-conf "$HERE/nginx/ppn.conf.template" "$ROUTED" >/dev/null
# Named explicitly so nginx's own $host and friends survive.
envsubst '$PPN_DOMAIN $PPN_TLS_DIR $DATA_DIR $LOG_DIR $RELAY_ALICE_PORT $RELAY_BOB_PORT $RELAY_CHARLIE_PORT $RELAY_DAVE_PORT $RELAY_EVE_PORT $RELAY_FERDIE_PORT $PEOPLE_PORT $ASSET_HUB_PORT $BULLETIN_PORT $WEB3_STORAGE_PORT $ETH_RPC_PORT $WEB3_STORAGE_PROVIDER_PORT $IPFS_GATEWAY_PORT $DUB_PORT $DASHBOARD_PORT $RELAY_ALICE_P2P_PORT $ASSET_HUB_P2P_PORT $PEOPLE_P2P_PORT $BULLETIN_P2P_PORT $WEB3_STORAGE_P2P_PORT $RELAY_ALICE_P2P_WSS_PORT $ASSET_HUB_P2P_WSS_PORT $PEOPLE_P2P_WSS_PORT $BULLETIN_P2P_WSS_PORT $WEB3_STORAGE_P2P_WSS_PORT' \
< "$ROUTED" > "$RENDERED"
if LEFT=$(grep -oE '\$\{[A-Za-z_][A-Za-z0-9_]*\}' "$RENDERED" | sort -u | tr '\n' ' ') && [[ -n "$LEFT" ]]; then
echo "apply: nginx config has unsubstituted variables: $LEFT" >&2; exit 1
fi
NGINX_CHANGED=0
if changed "$HERE/nginx/websocket-proxy.conf" /etc/nginx/snippets/websocket-proxy.conf; then
sudo install -m 0644 "$HERE/nginx/websocket-proxy.conf" /etc/nginx/snippets/websocket-proxy.conf; NGINX_CHANGED=1
fi
# stream{} cannot live in sites-enabled (that is inside http{}), so the template's tail
# after the marker line goes to its own file, included from nginx.conf's top level.
STREAM="$(mktemp)"
sed -n '/^# ---- stream ----$/,$p' "$RENDERED" > "$STREAM"
sed -i '/^# ---- stream ----$/,$d' "$RENDERED"
if changed "$RENDERED" /etc/nginx/sites-available/ppn.conf; then
sudo install -m 0644 "$RENDERED" /etc/nginx/sites-available/ppn.conf; NGINX_CHANGED=1
fi
sudo install -d /etc/nginx/streams-enabled
if changed "$STREAM" /etc/nginx/streams-enabled/ppn.conf; then
sudo install -m 0644 "$STREAM" /etc/nginx/streams-enabled/ppn.conf; NGINX_CHANGED=1
fi
if ! grep -q '^include /etc/nginx/streams-enabled/' /etc/nginx/nginx.conf; then
echo 'include /etc/nginx/streams-enabled/*.conf;' | sudo tee -a /etc/nginx/nginx.conf >/dev/null; NGINX_CHANGED=1
fi
rm -f "$ROUTED" "$RENDERED" "$STREAM"
sudo ln -sf /etc/nginx/sites-available/ppn.conf /etc/nginx/sites-enabled/ppn.conf
sudo rm -f /etc/nginx/sites-enabled/default
if [[ "$NGINX_CHANGED" == 1 ]]; then
sudo nginx -t && sudo systemctl enable --now nginx >/dev/null && sudo systemctl reload nginx
say "nginx: config updated and reloaded"
else
sudo systemctl is-active --quiet nginx || sudo systemctl enable --now nginx
say "nginx: unchanged"
fi
fi

# ---- 6. systemd unit --------------------------------------------------------------------
UNIT=/etc/systemd/system/ppn-polkadot.service
RENDERED_UNIT="$(mktemp)"
sed -e "s|__ROOT__|$ROOT|g" -e "s|__USER__|$RUN_USER|g" "$HERE/ppn-polkadot.service" > "$RENDERED_UNIT"
if changed "$RENDERED_UNIT" "$UNIT"; then
sudo install -m 0644 "$RENDERED_UNIT" "$UNIT"
sudo systemctl daemon-reload
say "unit: installed"
fi
rm -f "$RENDERED_UNIT"
sudo systemctl enable ppn-polkadot.service >/dev/null 2>&1 || true

# ---- 7. the network ---------------------------------------------------------------------
if [[ ! -f "fork-bundle-$NETWORK/manifest.json" ]]; then
say "no bundle at fork-bundle-$NETWORK — bite first: make bite NETWORK=$NETWORK"
elif [[ "$RESTART" == 1 ]]; then
# Anything running outside systemd (a `make start` in tmux) goes too: two networks on one
# port set is worse than a restart.
sudo systemctl stop ppn-polkadot.service 2>/dev/null || true
node bin/ppn.mjs kill >/dev/null 2>&1 || true
sudo systemctl start ppn-polkadot.service
say "network: restarting under systemd (journalctl -u ppn-polkadot -f)"
else
say "network: left alone (--no-restart)"
fi

echo
say "done. dashboard https://$DOMAIN token in $LOCAL_ENV"
2 changes: 2 additions & 0 deletions deploy/polkadot-test/local.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Copy to local.env (gitignored). apply.sh writes one with a random token if none exists.
DASHBOARD_ACTIONS_TOKEN=
Loading
Loading