Repository navigation
Publish the Polkadot fork's Trinity stack descriptor - #65
Merged
Merged
Conversation
Adds bite-time `--core`s and `--collators` options for running forked parachains with a custom topology: ```bash make start FORK=1 FRESH_BITE=1 CORES=people=3 COLLATORS=people=5 ``` This will support individuality-community’s future stress test gate, which will test People with three cores, and multiple collators. ### What changes - ppn bite writes the requested collator authorities and core assignments into the fork state. - The relay validator set grows as needed to ensure every core has a backing group, up to 10 validators. - manifest.json records the topology, and the generated fork TOML starts the matching validators and collators. - ppn start rejects bundles whose topology does not match the requested flags and prints the required re-bite command. - Validator keys are now derived from node names rather than hard-coded. The existing six remain unchanged except for Charlie and Dave’s BEEFY keys, which now match their actual zombienet keystores. ### Validation Tested a live previewnet fork with CORES=people=3 COLLATORS=people=5: - 8 relay cores and 8 validators - People assigned to three cores - Five collators each authored 12 of 60 sampled blocks - 2.03-second average People block time - ppn fork verify passed Unit coverage was added for topology parsing, core planning, key derivation, fork TOML generation, and bundle compatibility checks. All tests pass except dashboard API -> streams a log over SSE, which also fails on main. ### Follow-ups - Add cores/collators inputs and FRESH_BITE=1 to the [polkadot-pop-e2e](https://github.com/paritytech/polkadot-pop-e2e) release gate.
DUB's turn-api minted TURN credentials with nothing to relay them and an
empty server list. This runs eturnal next to DUB and wires the two
together, so p2p clients get working STUN/TURN servers from `POST
/api/v1/turn/issue`.
- New `turn` process: eturnal from eturnal.net on Linux, from the
processone Homebrew tap on macOS
- DUB gets `TURN_SECRET` and `ICE_SERVERS` from the same code that
configures eturnal. A deployment must set its own `TURN_SECRET` (base64
of a printable string)
- `/api/network` and the dashboard list the ICE servers, with a STUN
health probe
- `ppn nginx-conf` emits a TLS stream for `turns:` at a new
`{{GENERATED_STREAMS}}` marker, so the deploy template needs that marker
in its `stream {}` block
- `11-turn.zndsl` checks STUN, credential issuance and an authenticated
Allocate against the relay
Details in `docs/TURN.md`
stems from the following issue: https://github.com/paritytech/polkadot-pop-e2e/actions/runs/36420479843/job/108921710796?pr=37 I wanted to do a fresh bite but `Bulletin` & `Asset Hub` were not progressing due to: `HRMP head mismatch` error. **Why it happens**: the bite snapshots each parachain first and the relay about two minutes later. In this run Asset Hub was taken at #464508 and the relay at #156007. HRMP messages delivered to Asset Hub and Bulletin in that window leave the relay's channel head past what each parachain has recorded. --------- Signed-off-by: Cosmin Paraschiv <cosmin@parity.io> Co-authored-by: Cosmin Paraschiv <cosmin@parity.io>
Two things. `ppn --version` answered nothing before, and a bare number is misleading in a checkout, where package.json holds the placeholder the release rewrites at publish. And the README showed one fork example, so the no-sudo half of forking was invisible. - `ppn version` reports the version plus how the copy was installed: a dist reads its release manifest, an npm install the stamped package version, a checkout falls back to git HEAD. `--json` for consumers. - `ppn --version` / `-V` print the bare number. - The spawn stamp now shares the dist-manifest reader instead of keeping its own. - README: forking a sudo testnet vs a mainnet without sudo, with the runtime authorized at bite time. - README: `ppn bite` on its own, fork resume and `--clean`, `--pin-products`, remote upgrades, the duplicate security block removed. --------- Signed-off-by: Cosmin Paraschiv <cosmin@parity.io> Co-authored-by: Cosmin Paraschiv <cosmin@parity.io>
Bumps `DUB_TAG` to v0.6.0, whose People extension set includes `CheckMetadataHash`, so the chain-writer can sign against runtimes carrying it. v0.6.0 ships one asset per People runtime, so `ppn fetch` names it with a new `DUB_NETWORK` (default `testnet`). Unblocks the previewnet gate for paritytech/individuality-community#61 and every PR there since #101.
Runs dub-postgres with `dynamic_shared_memory_type=mmap`, so its dynamic shared memory segments live in `$PGDATA/pg_dynshmem` instead of `/dev/shm`. - Fixed dub-postgres becoming unusable when systemd-logind's `RemoveIPC` deletes its `/dev/shm` segments after the user's last session ends, which made every new connection fail with `could not open shared memory segment` (58P01) and took down dub-api and the services behind it.
…#42) ## Summary With `PPN_HOME` set and a data dir outside that workspace, `ppn start` runs the nodes from the workspace, but the custom processes (IPFS, the DUB services, Postgres, TURN, ...) run from a different one: the checkout, or `~<owner>/.ppn`. They then read another workspace's `config/ports.local.env`, `bin/` and `data/`. This PR passes each custom process the run's workspace, network and data directory explicitly. ## What goes wrong - zombienet-sdk 0.4.15 starts every node and custom process with `env_clear()`, then adds only `TZ`, `LANG`, `PATH` and the block's own `env` (`crates/provider/src/native/node.rs`, `initialize_process`). So none of the `PPN_HOME`, `DATA_DIR` or other variables `ppn start` gives zombie-cli reach a custom process. - Each launcher therefore finds its workspace again (`scripts/lib/workspace.sh`, `workspaceRoot()`), without `$PPN_HOME`: from its cwd, zombienet's per-process directory under the data dir, then a checkout with `bin/`, then the script owner's `~/.ppn`. With the default data dir (`<workspace>/data-fork-<network>`) the cwd finds the right workspace. With `--data-dir` elsewhere, or `--ephemeral` (zombienet's temp dir), it does not. - Consequences: - `PPN_HOME=/somewhere ppn start --data-dir /elsewhere` from a checkout: the nodes run from `/somewhere`, the custom processes from the checkout, with its binaries, data directory and Postgres cluster. - Installed from npm with `PPN_HOME` set: the custom processes fall back to `~<owner>/.ppn`. - Even in the right workspace, `ports.local.env` is one shared file, describing whichever `ppn start` wrote it last. Reproduced live (below, "Tests"). ## The fix - `customProcessEnv()` (in `lib/spawn-env.ts`, beside the spawn contract) returns `PPN_HOME`, `PPN_NETWORK` and `PPN_DATA_DIR` for this run. - `withCustomProcessEnv()` adds them to the `env` of every `[[custom_processes]]` block, before the block's own entries. It handles a block with no `env`, a single-line `env = [...]` and the generators' multi-line arrays. An entry the block already sets wins; nodes are untouched; applying it twice changes nothing. - `ppn start` spawns `<config>.spawn.toml`, written beside the config it was given or generated, so zombienet's relative paths resolve the same way. `*.spawn.toml` is gitignored and dropped in `prepack`. ## What does not change - Deployments that spawn zombie-cli themselves: the generated configs are unchanged. - The checked-in `local-dev.toml`, so the TOML drift check is unaffected. - Runs whose data dir is inside the workspace (the default) resolve the same workspace as before, now passed explicitly. ## Tests `packages/cli/tests/custom-process-env.test.ts` (6 tests): - all 18 custom processes of `zombienet-configs/local-dev.toml` get the three entries and keep their own entries, command and args; the nodes' env is unchanged; - the `dubCustomProcesses()` blocks get them, including `turn` (the last block, with no `env`); - a block's own `PPN_NETWORK` wins, and `env = []` is handled; - applying it twice changes nothing, and a config without custom processes is unchanged; - the spawn file's location. Results on `main` `d298d260` (v0.8.0), macOS arm64, Node 25.4.0, pnpm 10.33.0: - `tsc -b`, depcruise and `network-config` 212 of 212 pass. - Every CLI test file except `dashboard.test.ts` (it binds fixed ports): 231 of 231, `custom-process-env` 6 of 6. - Stacked with the two other PPN pull requests from the same author, every CLI test file passes: 256 of 256, plus `dashboard-ui`, `scripts/validate-ports.sh`, `pnpm build` and the TOML drift check. - Skipping blocks without `env` fails 2 of the 6 tests. Run by hand: `PPN_HOME=<A> ppn start paseo-next-v2 --fork --clean` from a checkout, with every process of the run inspected (command line, cwd, open files) a minute after "network is up": | Run | Custom processes | | --- | --- | | `main`, default data dir (inside A) | all from A: the cwd walk finds it | | `main`, `--data-dir` outside A, checkout without binaries | from the checkout, and failing: `checkout/bin/ipfs: No such file or directory`, "postgres not found at checkout/bin/postgres-dist/bin/postgres", "dub not found at checkout/bin/dub", no `eth-rpc`. The nodes run from A, and `ppn` reports "network is up" | | `main`, the same, checkout with fetched binaries | from the checkout: Postgres initialises a new cluster in `checkout/data/identity-pgdata` (not the run's data dir, which `ppn start` wrote to A's `ports.local.env`), the four DUB roles run `checkout/bin/dub`, IPFS looks for its repo in `checkout/bin/.ipfs`, `eth-rpc` is not found (`bin/<network>/` needs `PPN_NETWORK`) | | `main`, `--ephemeral`, checkout with fetched binaries | from the checkout: Postgres initialises a new cluster in `checkout/data/identity-pgdata` instead of the temp dir `ppn start` chose (`$TMPDIR/zombie-identity-<pid>`), the DUB roles run `checkout/bin/dub`, IPFS asks for `ipfs init` in `checkout/bin/.ipfs`, `eth-rpc` is not found | | this PR, `--ephemeral`, the same | all from A: Postgres on `$TMPDIR/zombie-identity-<pid>/identity-pgdata`, `A/bin/dub`, `A/bin/ipfs`, `A/bin/<network>/eth-rpc`; nothing in the checkout. `eth-rpc`'s base path is `<workspace>/data-fork-<network>/eth-rpc-db`, the persistent default data dir (first open question) | | this PR, the same | all from A: Postgres on `<data dir>/identity-pgdata`, `A/bin/dub`, `A/bin/ipfs` with `A/bin/.ipfs`, `A/bin/<network>/eth-rpc` on `<data dir>/eth-rpc-db`; nothing in the checkout | Not run: TURN, Linux. ## Open questions - Is `PPN_DATA_DIR` right for an ephemeral run? It mirrors what `ports.local.env` already says (`localEnvContent`), which for `--ephemeral` is the persistent default data dir. Services read `DATA_DIR || ports.PPN_DATA_DIR`, and `eth-rpc.sh` reads `PPN_DATA_DIR` from the environment first, so an ephemeral run keeps `eth-rpc`'s database in `<workspace>/data-fork-<network>/eth-rpc-db` (seen live; the same value `main` writes to `ports.local.env`). Should an ephemeral run leave `PPN_DATA_DIR` out, or point it at a temp dir? - Alternative: emit `{{PPN_HOME}}` tokens in the generated configs and let zombie-cli fill them. Not chosen: a deployment that does not set the variable would pass the literal `{{PPN_HOME}}`, and `PPN_NETWORK` is a waived key of the spawn contract. - `spawn-env.ts`'s header said zombie-cli's environment is "inherited by node-level processes"; the zombienet-sdk source says it is not, beyond the TOML templating. The patch rewords that sentence. Worth a maintainer's confirmation. Closes #45
…tead of killing their holders (#41) ## Summary To free its service ports, PPN killed whatever was listening on them (`lsof -i :port` then SIGKILL), and `ppn kill` ran `killall -9` by binary name. Both can kill processes PPN never started: another network on the same machine, a developer's own Postgres or IPFS, or a client connected to those ports. This PR makes an occupied port an error that names its holder, and makes `ppn start` run the network as one process group that `ppn kill` stops exactly. ## What goes wrong - **`ppn start` kills port holders.** Before starting, `scripts/kill-port.sh` SIGKILLs every process `lsof` lists for each service port, whoever owns it, including processes that are only connected to the port. - **`ppn kill` kills by name.** `killall -9 polkadot`, `ipfs`, `postgres` and port sweeps reach every matching process on the machine, not only this workspace's. - **The service launchers respawn forever.** When a port is taken, `ipfs-daemon.sh`, `dub/postgres.sh`, `dub/service.sh` and `turn.sh` loop on `Address already in use` instead of stopping. - **Nothing records what a network is.** When `ppn start` is killed with SIGKILL, zombie-cli and everything it started keep running, and only the name and port sweeps above find them again. ## The fix - **`ppn start` runs the network as one process group.** zombie-cli is spawned detached, so it leads a new group, and everything zombienet starts joins it. The group (its id and the leader's start time, so a reused pid is not mistaken for it) is recorded beside the network's data (`ppn-process-group.json`) and in the workspace's `run/` index. Ctrl-C, SIGTERM and SIGHUP are forwarded to the group as SIGINT, zombie-cli's own teardown (SIGKILL after 15 s, or at once on a second signal). Once zombie-cli has exited, the rest of the group (the custom processes zombienet does not stop) gets SIGTERM, then SIGKILL after 15 s. If `ppn` exits any other way, the group gets SIGTERM. - **`ppn kill` stops exactly the recorded groups,** SIGTERM first, SIGKILL after the grace period. A group whose id now belongs to something else is left alone. The path-anchored sweep (`sweepPatterns`, every `pkill -f` pattern anchored to this workspace's `bin/` or this package's launchers and `ppn service` processes) is kept as the fallback: for a network started without a record (an older `ppn`, or a supervisor that spawns zombie-cli itself), and for processes that left their group. `ppn kill` says when it runs, and names what it found. - **`ppn start` first stops a group a previous run of this workspace left alive,** which its record proves is ours, and only then checks the ports. - **`ppn start` refuses an occupied port** (`refuseOccupiedPorts`, `lsof -iTCP:<port> -sTCP:LISTEN`). The error lists each port, its `ports.env` key and the holder as `command(pid)`, and says nothing was stopped. The command is argv[0]'s basename (`ps -o args=`): `lsof`'s own name for a Node process is `MainThread` on Linux. The TURN relay's ports are checked only when the network runs the relay. - **`scripts/kill-port.sh` becomes `scripts/require-free-ports.sh`,** which checks listeners and never kills. The launchers exit with its message instead of respawning. ## What does not change - Starting and stopping a network that owns its ports works as before; Ctrl-C still stops it, through zombie-cli's own teardown. - `ppn kill` still works on a network started by an older `ppn`, through the fallback sweep. - Nothing outside this workspace is stopped any more. A port held by something else is an error to resolve, where it used to be killed. ## Tests New files `packages/cli/tests/process-group.test.ts` (14 tests) and `packages/cli/tests/port-ownership.test.ts` (10): - **The recorded group:** gone when it has no member; ours while its leader has the recorded start time; not ours when that pid has another start time (a reused pid); once the leader has exited, ours only if a member runs this workspace's or package's files. The record is written beside the data and in the workspace index, removed only for its own group id, and an invalid index entry is ignored. - **Stopping a group:** SIGTERM to the whole group, done when it empties; SIGKILL to what is left after the grace period; nothing signalled for a group that is gone. Two tests use real detached groups: stopping the leader also stops its child and leaves a process outside the group running, and a member that ignores SIGTERM is killed after the grace period. - **Ports and the fallback sweep:** the refusal message and that every port is checked; TURN ports only for a network that runs the relay; the sweep patterns match this workspace's nodes and services and not another workspace, a path that differs only in an escaped `.`, a system `polkadot` or a Homebrew Postgres; `require-free-ports.sh` passes on a free port and refuses a held one while the holder keeps listening. The holder is a child started with its own argv[0], and both `ppn start`'s check and the script must name it by that and its pid, on any platform. Each of these, undone, fails tests: the start-time check (2), signalling the group rather than the pid (2), SIGKILL after the grace period (2), skipping zombies (1), the sweep's path escaping (1), naming the holder by argv[0] (2 on Linux, where `lsof` says `MainThread`). Results on `main` `d298d260` (v0.8.0), macOS arm64, Node 25.4.0, pnpm 10.33.0: - `tsc -b`, depcruise, `network-config` 212 of 212 and `pnpm build` pass. - Every CLI test file except `dashboard.test.ts` (it binds fixed ports): 249 of 249. - Stacked with the two other PPN pull requests from the same author (#42, #43), every CLI test file passes: 271 of 271, plus `dashboard-ui`, `scripts/validate-ports.sh`, `pnpm build` and the TOML drift check. - Linux (`node:24-bookworm`, as `make test-unit` runs it): every CLI test file 261 of 261, `network-config` 212 of 212. Run by hand with `ppn start paseo-next-v2 --fork` (stacked with #42 and #43, two checkouts with their ports moved apart): | Case | Result | | --- | --- | | Something else listens on a service port | `ppn start` exits 1 naming the port, its key and `node(<pid>)`; the listener keeps running and nothing is started. | | Ctrl-C | "stopped process group N (13 process(es))"; within 2 s nothing of the workspace and no listener is left. | | `kill -9` of `ppn` mid-run | The network keeps running (47 processes). The next `ppn start` says a previous run is still running, stops its group (55 processes) and comes up; `ppn kill` then stops the new group (59) without the fallback sweep. | | Two networks on one machine | `ppn kill` in one stops its group (49 processes); the other keeps its 16 listeners and its People node answers RPC. | After every case no `postgres:` process was left. Zombienet runs the nodes and custom processes in its own group; only Postgres' own children (checkpointer, writers, backends) make groups of their own, and they stop with the postmaster. Not run: a closed terminal (SIGHUP) and SIGTERM to `ppn`, a network with TURN on, and a live network on Linux (this PR's CI runs the zombienet integration and fork-e2e jobs there). ## Open questions - Removing `kill-port.sh` changes the published `scripts/`. If a private deployment calls it, should a deprecated `kill-port.sh` stay and exec `require-free-ports.sh`? - `ppn kill` still deletes every `/tmp/zombie-*` directory, including other networks'. Scoping that needs zombie-cli's base directory; not in this PR. A Ctrl-C of `ppn start` no longer runs that deletion (it stops its own group only), so its `/tmp/zombie-*` directory stays until `ppn kill`. Is that acceptable? - eturnal 1.12.3 fixes its Erlang node name (`-name eturnal@127.0.0.1`), so two relays on one machine probably collide even on different ports (not verified). Whether eturnal stays in the group was not checked either: it was not installed on the test machine. Closes #44
## Summary
zombie-cli exits, and takes the whole network down, the first time any
node's metrics endpoint is slow to answer.
`ppn start` gives no way to turn that check off: zombie-cli's
`--node-verifier` option cannot be passed through. This
PR adds `ppn start --node-verifier <metric|none>` (and `NODE_VERIFIER=`
for `make start`). Without the option nothing
changes.
## What goes wrong
`ppn start` always runs `zombie-cli spawn -p native [-d <data>] <toml>`.
zombie-cli 0.4.15 (the pinned
`ZOMBIENET_VERSION`) defaults to `--node-verifier metric`, and then
(`crates/cli/src/main.rs`):
- waits until every node answers Prometheus (`wait_until_is_up(20)`);
- every 15 s afterwards checks again (`wait_until_is_up(5)`), and
**exits** with "One of the nodes crashed" the first
time one does not answer. The exit tears down every node.
That is the right default for a quick local network, and wrong when the
caller watches the network itself:
- a long fork restore or warp, during which a node's metrics endpoint is
slow;
- debugging one failing node without losing the other twenty;
- a harness with its own readiness and liveness checks that wants to
report which node failed and why.
Today the only way to get `none` is to bypass `ppn start` and spawn
zombie-cli by hand.
## The fix
- `ppn start --node-verifier <metric|none>` (commander `choices`: any
other value is refused before anything starts),
and `NODE_VERIFIER=` for `make start`.
- The option is passed to zombie-cli only when given.
- `zombieSpawnArgs()` builds zombie-cli's arguments in one place, so
they can be tested.
## What does not change
- Without the option, zombie-cli's command line is exactly as before,
and zombie-cli keeps its own default,
`metric`.
## Tests
`packages/cli/tests/node-verifier.test.ts` (4 tests):
- omitted means no flag, with and without `--ephemeral`;
- a given value is passed before `-d`;
- the option offers exactly `metric` and `none`;
- `--node-verifier bogus` is rejected with commander's `invalidArgument`
("Allowed choices are metric, none").
Results on `main` `d298d260` (v0.8.0), macOS arm64, Node 25.4.0, pnpm
10.33.0:
- `tsc -b`, depcruise and `network-config` 212 of 212 pass.
- Every CLI test file except `dashboard.test.ts` (it binds fixed ports):
229 of 229, `node-verifier` 4 of 4.
- Stacked with the two other PPN pull requests from the same author,
every CLI test file passes: 256 of 256, plus
`dashboard-ui`, `scripts/validate-ports.sh`, `pnpm build` and the TOML
drift check.
- `ppn start --help` lists the option; `make -n start
NODE_VERIFIER=none` contains `--node-verifier none`, and plain
`make -n start` does not.
- With the pass-through removed from `zombieSpawnArgs`, 1 of the 4 tests
fails.
Run by hand: `ppn start paseo-next-v2 --fork --clean --node-verifier
none`. zombie-cli's command line was
`zombie-cli spawn -p native --node-verifier none -d <data dir> …`
(without the option: `zombie-cli spawn -p native -d
…`), "network is up, in 20.50s", and a minute later the relay chain and
all three parachains were still
importing blocks.
Not run: a node crash under `none`, Linux.
## Open questions
- Should `ppn start` warn that, with `none`, nothing notices a crashed
node any more?
- Docs: add a line next to `EPHEMERAL=1` in the README's options list?
Only the `make help` line is in this PR.
Closes #46
Moves the device-uniqueness-backend pin to v0.7.0 - Changed the DUB pin from v0.6.0 to v0.7.0, whose vendored People and Asset Hub metadata is spec 3_004_000, the runtime previewnet now starts from - Changed dub-api to name TURN_PROVIDER=coturn, since v0.7.0 defaults turn-api to Cloudflare and refuses to boot with TURN_SECRET set and no provider named - Fixed the no-relay case passing an empty ICE_SERVERS, which v0.7.0 rejects at boot, by handing it a loopback placeholder instead - Added a check to the registration test that the new lite username is finalized in People Chain's Resources.Consumers for the registered account, with no full username - Added a check that search reports the registered username as ASSIGNED
A server's secrets file is now checked against one list, so a deployment can no longer drop a key without noticing. Prod lost DUB because `JWT_ED25519_SECRET` lived only in a hand-edited file that a render rewrote - `config/secrets.env.example` lists every key in three sections (every server, deployable servers, optional), each with one line on how to make it - `ppn secrets check [file]` fails on a missing or empty required key, a key not in the example, or a deployable key in a local file - `ppn start` refuses a named secrets file that fails the check - `@parity/ppn/secrets` exports the reader, so a deployment can test its render against the same list --------- Signed-off-by: Cosmin Paraschiv <cosmin@parity.io> Co-authored-by: Cosmin Paraschiv <cosmin@parity.io>
Unless PPN_PACKAGE_ROOT is set, repoRoot() finds the root by looking for config/ports.env, starting from network-config's directory. When network-config is installed from npm or being packed, it has its own config/ports.env, so repoRoot() returns network-config's directory, which doesn't have secrets.env.example. Signed-off-by: Cosmin Paraschiv <cosmin@parity.io>
Follow-up to #53 for how the CLI finds `config/secrets.env.example` - Changed the lookup to search up for `config/secrets.env.example` itself instead of the first directory holding a `config/ports.env` - Fixed `PPN_PACKAGE_ROOT` being ignored for the secrets example while every other root lookup follows it - Changed the CLI test script to set `PPN_PACKAGE_ROOT`, so a pack running in parallel can no longer make the tests read another package's `config/` - Added a publish-surface test that installs both tarballs and reads the secrets example through `@parity/ppn/secrets` with no `PPN_PACKAGE_ROOT` set
## Summary
`ppn fork toml` gives every network's collators the workspace's binary
folder, except previewnet's, which gets none.
When the workspace isn't the checkout (`PPN_HOME` set, or Engine
installed from npm), a previewnet fork's collators
then look for `polkadot-omni-node` outside the workspace (here, Engine's
own `bin/`) and exit at once:
```
…/engine/scripts/omni-node.sh: line 108: …/engine/bin/polkadot-omni-node: No such file or directory
```
This PR passes previewnet's `bin/` from the workspace, as every other
network already gets `bin/<network>`.
One line in `packages/cli/src/commands/fork.ts`.
## What goes wrong
- `fork.ts` (`toml`): `binDir: bundleNet === 'previewnet' ? undefined :
path.join(WS, 'bin', bundleNet)`.
- `generateForkToml` still falls back to `${repoDir}/bin` for the
relay's `default_command`, so the relay validators
start. But each parachain gets `options.binDir` as is, so with
`undefined` no `PPN_BIN_DIR` is written into the
collators' `env`.
- `omni-node.sh` takes `PPN_BIN_DIR`, then `BIN`, then the `bin/` of the
workspace `scripts/lib/workspace.sh` finds on
its own. zombienet starts each node with a cleared environment plus only
its block's `env` (the cause behind
#45/#42), so `BIN` and `PPN_HOME` from `ppn start` never reach the
collator, and `workspace.sh` finds another
workspace: the Engine checkout in our run, with no `polkadot-omni-node`
in its `bin/` (from npm it would be
`~/.ppn`, per #45; not run here).
- In a checkout the workspace *is* the checkout (`REPO === WS`), so both
folders are the same and nothing shows. That
is why CI's `fork-e2e (previewnet)` passes.
## The fix
```diff
- binDir: bundleNet === 'previewnet' ? undefined : path.join(WS, 'bin', bundleNet),
+ binDir: bundleNet === 'previewnet' ? path.join(WS, 'bin') : path.join(WS, 'bin', bundleNet),
```
previewnet's binaries live in `bin/`, every other network's in
`bin/<network>`, as the comment above the line says and
as `binDirFor()` in `start.ts` already does.
## Why setting an env var doesn't work around it
The caller already sets `PPN_HOME` to the workspace, and `ppn start`
passes `PPN_HOME` and `BIN` to zombie-cli. They
don't reach the collator, because zombienet gives a node only its own
block's `env`. The folder has to be written into
the generated config, which is what this does.
## What doesn't change
- Every network other than previewnet: same expression as before.
- previewnet in a checkout: `PPN_BIN_DIR` now names the same `bin/` the
launcher already fell back to.
- The relay: still `${WS}/bin` through `default_command`.
## Tests
- `pnpm install --frozen-lockfile`, `tsc -b`, `packages/cli` tests
(278/278), `packages/network-config` tests
(212/212) and `lint:boundaries` pass on this branch.
- Live, on `main` at `0004226`, with `PPN_HOME` set to a workspace
outside the checkout and a previewnet fork built
fresh from its published bundle each time (no saved fork reused):
- without this change: `Collator-1500` exits at start with the error
above;
- with it: the fork starts, the collators get
`PPN_BIN_DIR=<workspace>/bin`, and a product's end-to-end scenario
on the fork (sign a remark on People, read it back finalized and shown
in the app) passes 4/4.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Grants the identity backend's attester a DotnsGateway allowance on Asset Hub, so attested people get their dotNS name on genesis networks and forks. - Added `increase-dotns-attestation-allowance`, which sudo-grants the attester (`PPN_ALLOWANCE_SS58`, Alice by default) 1000 `DotnsGateway.increase_attestation_allowance` on Asset Hub and reads the allowance back to confirm it. - Added the process to genesis networks and to forks of networks with sudo, wherever both People and Asset Hub are present. - Added a skip for an Asset Hub without DotnsGateway, which looks again for two minutes so a fork whose upgrade adds the pallet still gets the grant. - Added CI steps that check the grant fails without `PPN_ALLOWANCE_SS58` in the deployable profile and never goes to Alice. - Changed `set-dispatcher-address` to read `DispatcherAddress` back and retry, since in genesis mode it now shares the sudo account on Asset Hub with the new grant. - Changed the People allowance grant to share the attester account lookup with the new grant. - Changed `docs/DEVICE-UNIQUENESS-BACKEND.md`, `docs/PROFILES.md` and `CLAUDE.md` to list the new grant and to say what lets the writer pick up a late grant. - Fixed every dotNS reservation failing with `DotnsGateway::NoAttestationAllowance` while the People registration still landed, which [polkadot-pop-e2e#60](paritytech/polkadot-pop-e2e#60) hits once it waits for the dotNS name the way a host does.
Writes the running network's Trinity stack descriptor and light-client specs, and serves them from the dashboard. Closes #54 The issue asked for an unsigned descriptor from the engine's API, with signing left to preview-net-v1. A signature covers exact bytes, so the engine writes a file and signs it itself when given a key, and preview-net-v1 only has to supply `PPN_STACK_SECRET`. - Added `ppn service trinity-stack`, a custom process in genesis and fork spawns that writes `trinity-stack.json` and a `<spec>.smol.json` light spec per chain into the data directory, and rewrites them every 6 hours. - Added signing of the descriptor when the secrets file sets `PPN_STACK_SECRET`, except on a fork, which is always published unsigned. - Added the relay's sync checkpoint to its light spec, retried every 30 seconds until the relay has finalized a block. - Added light spec bootnodes that point at the network's own nodes, over their P2P WS port and, behind https, also nginx's WSS port. - Added a check that refuses a chain spec naming a node on any host other than `BOOTNODE_HOSTNAME`, so a fork never publishes its source's nodes. - Added `fork` and each chain's `forkedAt` to a fork's descriptor when the spawn stamp records the bite and the network file names the source's pointer. - Added `--fork-manifest` to `ppn stamp-spawn`, so a deployment that spawns a fork itself records its bite. - Added `biteHashes` to the fork bundle manifest: the source's block hash at each bite block, read during the bite and named as each chain's `forkedAt`. - Added `/.well-known/trinity-stack.json`, `/api/stack` and `/chainspecs/<spec>.smol.json` to the dashboard. - Added a Trinity stack panel to the dashboard overview with the stack's link and QR code. - Added `stack.tld` and `stack.pointer` to network files, set for previewnet's pointer and Polkadot's TLD. - Added `PPN_STACK_FAUCET_URL`, `PPN_STACK_EXPIRES_AT`, `PPN_STACK_VALID_FOR` and `PPN_STACK_REFRESH_SECONDS` to `config/ports.env`. - Added `PPN_EPHEMERAL` to `ports.local.env`, and the service writes nothing for an ephemeral run. - Changed every service to read the data directory and public URL from the service context, resolved once.
Makes the dashboard say which instance of which network it is, so production, staging, forks and laptops can be told apart at a glance. - Changed the dashboard heading and tab title to show the network's name, prefixed with "Fork of" on a fork, and the host it runs on. - Added `PPN_DISPLAY_NAME` to `config/ports.env`, which replaces the network's display name in the dashboard and the Trinity stack descriptor, such as "Previewnet Staging".
Lets hosts and tools take a network's dotNS contracts from its Trinity stack descriptor, such as `dotns.storeFactory` on previewnet, instead of a copy per network or reads from Asset Hub. - The descriptor now carries `dotns`: each dotNS contract by its `DotnsConstants` key, lowercase, plus `protocolRegistry` for the registry itself. - A genesis previewnet takes the addresses from the `bin/dotns-addresses.json` that `ppn fetch` takes from the dotns release, leaving out contracts without a key, such as the store beacons and the flat pricing contract. - A fork takes the keys from its source's descriptor at `stack.pointer` and each address from its own registry's `get(key)`, so a key rewired since the bite shows what the fork actually runs. - A key the fork's registry does not hold, such as a release's `multicall3`, keeps the source's address. - While the source descriptor or eth-rpc does not answer, a fork publishes no `dotns` and the writer tries again in 30 seconds. - paseo-next-v2 and devnet forks now publish a descriptor, with `stack.tld` and `stack.pointer` pointing at https://stack-paseo-next.parity-testnet.parity.io/.well-known/trinity-stack.json and https://stack.api.polkadotcommunity.foundation/.well-known/trinity-stack.json. - `@parity/trinity-stack` moves to 0.2.0, which defines the field: paritytech/trinity-user-agents#1803 Part of paritytech/trinity-user-agents#1312
…scriptor (#61) Previewnet's Trinity stack descriptor now states its payment asset and Coinage instance, plus its dotNS web gateway on deployments that set one. - Previewnet's descriptor lists pUSD (asset 50000413 on Asset Hub) as the payment asset and Coinage instance 0 on People. - A network file can set `stack.assets` and `stack.coinage` for any network whose runtime has them. - Setting `PPN_STACK_WEB_GATEWAY_URL` adds a `web-gateway` service. Only the instance the gateway resolves names against should set it. - A gateway URL whose path does not end in `/` stops the descriptor from being written, since `@parity/trinity-stack` is now 0.2.0, which checks web gateway URLs.
Adds `--authoring=slot-based` to the Bulletin collator. Why: the next Bulletin runtime (`spec_version` 1_000_028, polkadot-bulletin-chain #816) enforces `RelayParentOffset = 1`, which the lookahead collator can't satisfy. A lookahead collator stops proposing once the upgrade is enacted.
Lets clients call a local network's services with query parameters through the dashboard, such as `GET /dub/api/v1/usernames/search?prefix=alice` - HTTP requests through the dashboard proxy now reach the service with their query string - WebSocket upgrades through the proxy now keep their query string too
## Summary Bumps the dotNS genesis pin from v0.8.0 to v1.1.0, so a previewnet built by the engine boots the same dotNS release as Paseo. The contract addresses are unchanged between the two releases (`deployments/expected.json`), so `networks/previewnet.json` and its pinned resolver stay as they are. v1.1.0 publishes the same `dotns-genesis-testnet.json` asset the fetch expects.
# Conflicts: # CLAUDE.md # config/versions.env # docs/DEVICE-UNIQUENESS-BACKEND.md # networks/README.md # networks/polkadot.json # packages/cli/src/commands/fetch.ts # packages/cli/src/commands/fork.ts # packages/cli/src/commands/turn.ts # packages/cli/src/fork/overrides.ts # packages/cli/src/fork/validators.ts # packages/cli/tests/fork-validators.test.ts # packages/dashboard-ui/src/lib/Overview.svelte # packages/network-config/src/dub.ts # packages/network-config/src/fork-toml.ts # packages/network-config/src/networks.ts # packages/network-config/src/toml-generator.ts # packages/network-config/tests/dub.test.ts # scripts/dub/postgres.sh # scripts/turn.sh # tests/scripts/test-dub-registration.ts # tests/scripts/test-turn.ts # zombienet-configs/local-dev.toml
…source's is absent
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
csmnprschv
approved these changes
Nov 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
polkadot-test.substrate.dev can publish a Trinity stack descriptor, so a host imports the Polkadot fork from https://polkadot-test.substrate.dev/.well-known/trinity-stack.json
dubBuildin its network file, not main's globalDUB_NETWORKDotnsGateway::DispatcherAddress. PCF deployed dotNS after the bite, so the descriptor lists the dotNS the fork actually runs and leaves out keys with no contract hereppn startnow rejects a secrets file withoutPPN_PROFILE, soapply.shaddsPPN_PROFILE=localto the existing oneMerging this fast-forwards the branch of #17