Skip to content

Publish the Polkadot fork's Trinity stack descriptor - #65

Merged
csmnprschv merged 28 commits into
cosmin-polkadot-forkfrom
polkadot-test-trinity
Nov 12, 2026
Merged

csmnprschv merged 28 commits into
cosmin-polkadot-forkfrom
polkadot-test-trinity

Conversation

@mordamax

Copy link
Copy Markdown
Collaborator

polkadot-test.substrate.dev can publish a Trinity stack descriptor, so a host imports the Polkadot fork from https://polkadot-test.substrate.dev/.well-known/trinity-stack.json

  • Brings main into the fork branch: the descriptor and light specs, DUB v0.8.0 and the TURN relay as merged there. The polkadot fork still picks its DUB build from dubBuild in its network file, not main's global DUB_NETWORK
  • The descriptor names the fork as a copy of PCF's Polkadot descriptor (https://stack.api.pcf.network/.well-known/trinity-stack.json), with each chain's source block at the bite
  • A bundle bitten before bite hashes were recorded gets them from the source chains, so the bundle running now needs no re-bite
  • When the source's dotNS registry has no code on the fork, the registry comes from Asset Hub's DotnsGateway::DispatcherAddress. PCF deployed dotNS after the bite, so the descriptor lists the dotNS the fork actually runs and leaves out keys with no contract here
  • ppn start now rejects a secrets file without PPN_PROFILE, so apply.sh adds PPN_PROFILE=local to the existing one

Merging this fast-forwards the branch of #17

andrzejSulkowski and others added 28 commits September 28, 2026 13:33
Adds bite-time `--core`s and `--collators` options for running forked
parachains with a custom topology:

 ```bash
   make start FORK=1 FRESH_BITE=1 CORES=people=3 COLLATORS=people=5
 ```

This will support individuality-community’s future stress test gate,
which will test People with three cores, and multiple collators.

 ### What changes

- ppn bite writes the requested collator authorities and core
assignments into the fork state.
- The relay validator set grows as needed to ensure every core has a
backing group, up to 10 validators.
- manifest.json records the topology, and the generated fork TOML starts
the matching validators and collators.
- ppn start rejects bundles whose topology does not match the requested
flags and prints the required re-bite command.
- Validator keys are now derived from node names rather than hard-coded.
The existing six remain unchanged except for Charlie and Dave’s BEEFY
keys, which now match their actual
   zombienet keystores.

 ### Validation

 Tested a live previewnet fork with CORES=people=3 COLLATORS=people=5:

 - 8 relay cores and 8 validators
 - People assigned to three cores
 - Five collators each authored 12 of 60 sampled blocks
 - 2.03-second average People block time
 - ppn fork verify passed

Unit coverage was added for topology parsing, core planning, key
derivation, fork TOML generation, and bundle compatibility checks.

All tests pass except dashboard API -> streams a log over SSE, which
also fails on main.

 ### Follow-ups

- Add cores/collators inputs and FRESH_BITE=1 to the
[polkadot-pop-e2e](https://github.com/paritytech/polkadot-pop-e2e)
release gate.
DUB's turn-api minted TURN credentials with nothing to relay them and an
empty server list. This runs eturnal next to DUB and wires the two
together, so p2p clients get working STUN/TURN servers from `POST
/api/v1/turn/issue`.

- New `turn` process: eturnal from eturnal.net on Linux, from the
processone Homebrew tap on macOS
- DUB gets `TURN_SECRET` and `ICE_SERVERS` from the same code that
configures eturnal. A deployment must set its own `TURN_SECRET` (base64
of a printable string)
- `/api/network` and the dashboard list the ICE servers, with a STUN
health probe
- `ppn nginx-conf` emits a TLS stream for `turns:` at a new
`{{GENERATED_STREAMS}}` marker, so the deploy template needs that marker
in its `stream {}` block
- `11-turn.zndsl` checks STUN, credential issuance and an authenticated
Allocate against the relay

Details in `docs/TURN.md`
stems from the following issue:
https://github.com/paritytech/polkadot-pop-e2e/actions/runs/36420479843/job/108921710796?pr=37

I wanted to do a fresh bite but `Bulletin` & `Asset Hub` were not
progressing due to: `HRMP head mismatch` error.

**Why it happens**: the bite snapshots each parachain first and the
relay about two minutes later. In this run Asset Hub was taken at
#464508 and the relay at #156007. HRMP messages delivered to Asset Hub
and Bulletin in that window leave the relay's channel head past what
each parachain has recorded.

---------

Signed-off-by: Cosmin Paraschiv <cosmin@parity.io>
Co-authored-by: Cosmin Paraschiv <cosmin@parity.io>
Two things. `ppn --version` answered nothing before, and a bare number
is misleading in a checkout, where package.json holds the placeholder
the release rewrites at publish. And the README showed one fork example,
so the no-sudo half of forking was invisible.

- `ppn version` reports the version plus how the copy was installed: a
dist reads its release manifest, an npm install the stamped package
version, a checkout falls back to git HEAD. `--json` for consumers.
- `ppn --version` / `-V` print the bare number.
- The spawn stamp now shares the dist-manifest reader instead of keeping
its own.
- README: forking a sudo testnet vs a mainnet without sudo, with the
runtime authorized at bite time.
- README: `ppn bite` on its own, fork resume and `--clean`,
`--pin-products`, remote upgrades, the duplicate security block removed.

---------

Signed-off-by: Cosmin Paraschiv <cosmin@parity.io>
Co-authored-by: Cosmin Paraschiv <cosmin@parity.io>
Bumps `DUB_TAG` to v0.6.0, whose People extension set includes
`CheckMetadataHash`, so the chain-writer can sign against runtimes
carrying it. v0.6.0 ships one asset per People runtime, so `ppn fetch`
names it with a new `DUB_NETWORK` (default `testnet`).

Unblocks the previewnet gate for paritytech/individuality-community#61
and every PR there since #101.
Runs dub-postgres with `dynamic_shared_memory_type=mmap`, so its dynamic
shared memory segments live in `$PGDATA/pg_dynshmem` instead of
`/dev/shm`.

- Fixed dub-postgres becoming unusable when systemd-logind's `RemoveIPC`
deletes its `/dev/shm` segments after the user's last session ends,
which made every new connection fail with `could not open shared memory
segment` (58P01) and took down dub-api and the services behind it.
…#42)

## Summary

With `PPN_HOME` set and a data dir outside that workspace, `ppn start`
runs the nodes from the workspace, but the
custom processes (IPFS, the DUB services, Postgres, TURN, ...) run from
a different one: the checkout, or
`~<owner>/.ppn`. They then read
another workspace's `config/ports.local.env`, `bin/` and `data/`. This
PR passes each custom process the run's
workspace, network and data directory explicitly.

## What goes wrong

- zombienet-sdk 0.4.15 starts every node and custom process with
`env_clear()`, then adds only `TZ`, `LANG`, `PATH` and
the block's own `env` (`crates/provider/src/native/node.rs`,
`initialize_process`). So none of the `PPN_HOME`,
`DATA_DIR` or other variables `ppn start` gives zombie-cli reach a
custom process.
- Each launcher therefore finds its workspace again
(`scripts/lib/workspace.sh`, `workspaceRoot()`), without
`$PPN_HOME`: from its cwd, zombienet's per-process directory under the
data dir, then a checkout with `bin/`, then
the script owner's `~/.ppn`. With the default data dir
(`<workspace>/data-fork-<network>`) the cwd finds the right
workspace. With `--data-dir` elsewhere, or `--ephemeral` (zombienet's
temp dir), it does not.
- Consequences:
- `PPN_HOME=/somewhere ppn start --data-dir /elsewhere` from a checkout:
the nodes run from `/somewhere`, the
custom processes from the checkout, with its binaries, data directory
and Postgres cluster.
- Installed from npm with `PPN_HOME` set: the custom processes fall back
to `~<owner>/.ppn`.
- Even in the right workspace, `ports.local.env` is one shared file,
describing whichever `ppn start` wrote it
    last.

Reproduced live (below, "Tests").

## The fix

- `customProcessEnv()` (in `lib/spawn-env.ts`, beside the spawn
contract) returns `PPN_HOME`, `PPN_NETWORK` and
  `PPN_DATA_DIR` for this run.
- `withCustomProcessEnv()` adds them to the `env` of every
`[[custom_processes]]` block, before the block's own entries.
It handles a block with no `env`, a single-line `env = [...]` and the
generators' multi-line arrays. An entry the
block already sets wins; nodes are untouched; applying it twice changes
nothing.
- `ppn start` spawns `<config>.spawn.toml`, written beside the config it
was given or generated, so zombienet's
relative paths resolve the same way. `*.spawn.toml` is gitignored and
dropped in `prepack`.

## What does not change

- Deployments that spawn zombie-cli themselves: the generated configs
are unchanged.
- The checked-in `local-dev.toml`, so the TOML drift check is
unaffected.
- Runs whose data dir is inside the workspace (the default) resolve the
same workspace as before, now passed
  explicitly.

## Tests

`packages/cli/tests/custom-process-env.test.ts` (6 tests):

- all 18 custom processes of `zombienet-configs/local-dev.toml` get the
three entries and keep their own entries,
  command and args; the nodes' env is unchanged;
- the `dubCustomProcesses()` blocks get them, including `turn` (the last
block, with no `env`);
- a block's own `PPN_NETWORK` wins, and `env = []` is handled;
- applying it twice changes nothing, and a config without custom
processes is unchanged;
- the spawn file's location.

Results on `main` `d298d260` (v0.8.0), macOS arm64, Node 25.4.0, pnpm
10.33.0:

- `tsc -b`, depcruise and `network-config` 212 of 212 pass.
- Every CLI test file except `dashboard.test.ts` (it binds fixed ports):
231 of 231, `custom-process-env` 6 of 6.
- Stacked with the two other PPN pull requests from the same author,
every CLI test file passes: 256 of 256, plus
`dashboard-ui`, `scripts/validate-ports.sh`, `pnpm build` and the TOML
drift check.
- Skipping blocks without `env` fails 2 of the 6 tests.

Run by hand: `PPN_HOME=<A> ppn start paseo-next-v2 --fork --clean` from
a checkout, with every process of the run
inspected (command line, cwd, open files) a minute after "network is
up":

| Run | Custom processes |
| --- | --- |
| `main`, default data dir (inside A) | all from A: the cwd walk finds
it |
| `main`, `--data-dir` outside A, checkout without binaries | from the
checkout, and failing: `checkout/bin/ipfs: No such file or directory`,
"postgres not found at checkout/bin/postgres-dist/bin/postgres", "dub
not found at checkout/bin/dub", no `eth-rpc`. The nodes run from A, and
`ppn` reports "network is up" |
| `main`, the same, checkout with fetched binaries | from the checkout:
Postgres initialises a new cluster in `checkout/data/identity-pgdata`
(not the run's data dir, which `ppn start` wrote to A's
`ports.local.env`), the four DUB roles run `checkout/bin/dub`, IPFS
looks for its repo in `checkout/bin/.ipfs`, `eth-rpc` is not found
(`bin/<network>/` needs `PPN_NETWORK`) |
| `main`, `--ephemeral`, checkout with fetched binaries | from the
checkout: Postgres initialises a new cluster in
`checkout/data/identity-pgdata` instead of the temp dir `ppn start`
chose (`$TMPDIR/zombie-identity-<pid>`), the DUB roles run
`checkout/bin/dub`, IPFS asks for `ipfs init` in `checkout/bin/.ipfs`,
`eth-rpc` is not found |
| this PR, `--ephemeral`, the same | all from A: Postgres on
`$TMPDIR/zombie-identity-<pid>/identity-pgdata`, `A/bin/dub`,
`A/bin/ipfs`, `A/bin/<network>/eth-rpc`; nothing in the checkout.
`eth-rpc`'s base path is `<workspace>/data-fork-<network>/eth-rpc-db`,
the persistent default data dir (first open question) |
| this PR, the same | all from A: Postgres on `<data
dir>/identity-pgdata`, `A/bin/dub`, `A/bin/ipfs` with `A/bin/.ipfs`,
`A/bin/<network>/eth-rpc` on `<data dir>/eth-rpc-db`; nothing in the
checkout |

Not run: TURN, Linux.

## Open questions

- Is `PPN_DATA_DIR` right for an ephemeral run? It mirrors what
`ports.local.env` already says (`localEnvContent`),
which for `--ephemeral` is the persistent default data dir. Services
read `DATA_DIR || ports.PPN_DATA_DIR`, and
`eth-rpc.sh` reads `PPN_DATA_DIR` from the environment first, so an
ephemeral run keeps `eth-rpc`'s database in
`<workspace>/data-fork-<network>/eth-rpc-db` (seen live; the same value
`main` writes to `ports.local.env`).
Should an ephemeral run leave `PPN_DATA_DIR` out, or point it at a temp
dir?
- Alternative: emit `{{PPN_HOME}}` tokens in the generated configs and
let zombie-cli fill them. Not chosen: a
deployment that does not set the variable would pass the literal
`{{PPN_HOME}}`, and `PPN_NETWORK` is a waived key
  of the spawn contract.
- `spawn-env.ts`'s header said zombie-cli's environment is "inherited by
node-level processes"; the zombienet-sdk
source says it is not, beyond the TOML templating. The patch rewords
that sentence. Worth a maintainer's
  confirmation.

Closes #45
…tead of killing their holders (#41)

## Summary

To free its service ports, PPN killed whatever was listening on them
(`lsof -i :port` then SIGKILL), and `ppn kill`
ran `killall -9` by binary name. Both can kill processes PPN never
started: another network on the same machine, a
developer's own Postgres or IPFS, or a client connected to those ports.
This PR makes an occupied port an error that
names its holder, and makes `ppn start` run the network as one process
group that `ppn kill` stops exactly.

## What goes wrong

- **`ppn start` kills port holders.** Before starting,
`scripts/kill-port.sh` SIGKILLs every process `lsof` lists for
each service port, whoever owns it, including processes that are only
connected to the port.
- **`ppn kill` kills by name.** `killall -9 polkadot`, `ipfs`,
`postgres` and port sweeps reach every matching
  process on the machine, not only this workspace's.
- **The service launchers respawn forever.** When a port is taken,
`ipfs-daemon.sh`, `dub/postgres.sh`,
`dub/service.sh` and `turn.sh` loop on `Address already in use` instead
of stopping.
- **Nothing records what a network is.** When `ppn start` is killed with
SIGKILL, zombie-cli and everything it started
  keep running, and only the name and port sweeps above find them again.

## The fix

- **`ppn start` runs the network as one process group.** zombie-cli is
spawned detached, so it leads a new group,
and everything zombienet starts joins it. The group (its id and the
leader's start time, so a reused pid is not
mistaken for it) is recorded beside the network's data
(`ppn-process-group.json`) and in the workspace's `run/`
index. Ctrl-C, SIGTERM and SIGHUP are forwarded to the group as SIGINT,
zombie-cli's own teardown (SIGKILL after 15
s, or at once on a second signal). Once zombie-cli has exited, the rest
of the group (the custom processes
zombienet does not stop) gets SIGTERM, then SIGKILL after 15 s. If `ppn`
exits any other way, the group gets
  SIGTERM.
- **`ppn kill` stops exactly the recorded groups,** SIGTERM first,
SIGKILL after the grace period. A group whose id
now belongs to something else is left alone. The path-anchored sweep
(`sweepPatterns`, every `pkill -f` pattern
anchored to this workspace's `bin/` or this package's launchers and `ppn
service` processes) is kept as the
fallback: for a network started without a record (an older `ppn`, or a
supervisor that spawns zombie-cli itself),
and for processes that left their group. `ppn kill` says when it runs,
and names what it found.
- **`ppn start` first stops a group a previous run of this workspace
left alive,** which its record proves is ours,
  and only then checks the ports.
- **`ppn start` refuses an occupied port** (`refuseOccupiedPorts`, `lsof
-iTCP:<port> -sTCP:LISTEN`). The error lists
each port, its `ports.env` key and the holder as `command(pid)`, and
says nothing was stopped. The command is
argv[0]'s basename (`ps -o args=`): `lsof`'s own name for a Node process
is `MainThread` on Linux. The TURN relay's
  ports are checked only when the network runs the relay.
- **`scripts/kill-port.sh` becomes `scripts/require-free-ports.sh`,**
which checks listeners and never kills. The
  launchers exit with its message instead of respawning.

## What does not change

- Starting and stopping a network that owns its ports works as before;
Ctrl-C still stops it, through zombie-cli's
  own teardown.
- `ppn kill` still works on a network started by an older `ppn`, through
the fallback sweep.
- Nothing outside this workspace is stopped any more. A port held by
something else is an error to resolve, where it
  used to be killed.

## Tests

New files `packages/cli/tests/process-group.test.ts` (14 tests) and
`packages/cli/tests/port-ownership.test.ts` (10):

- **The recorded group:** gone when it has no member; ours while its
leader has the recorded start time; not ours
when that pid has another start time (a reused pid); once the leader has
exited, ours only if a member runs this
workspace's or package's files. The record is written beside the data
and in the workspace index, removed only for
  its own group id, and an invalid index entry is ignored.
- **Stopping a group:** SIGTERM to the whole group, done when it
empties; SIGKILL to what is left after the grace
period; nothing signalled for a group that is gone. Two tests use real
detached groups: stopping the leader also
stops its child and leaves a process outside the group running, and a
member that ignores SIGTERM is killed after
  the grace period.
- **Ports and the fallback sweep:** the refusal message and that every
port is checked; TURN ports only for a network
that runs the relay; the sweep patterns match this workspace's nodes and
services and not another workspace, a path
that differs only in an escaped `.`, a system `polkadot` or a Homebrew
Postgres; `require-free-ports.sh` passes on
a free port and refuses a held one while the holder keeps listening. The
holder is a child started with its own
argv[0], and both `ppn start`'s check and the script must name it by
that and its pid, on any platform.

Each of these, undone, fails tests: the start-time check (2), signalling
the group rather than the pid (2), SIGKILL
after the grace period (2), skipping zombies (1), the sweep's path
escaping (1), naming the holder by argv[0] (2 on
Linux, where `lsof` says `MainThread`).

Results on `main` `d298d260` (v0.8.0), macOS arm64, Node 25.4.0, pnpm
10.33.0:

- `tsc -b`, depcruise, `network-config` 212 of 212 and `pnpm build`
pass.
- Every CLI test file except `dashboard.test.ts` (it binds fixed ports):
249 of 249.
- Stacked with the two other PPN pull requests from the same author
(#42, #43), every CLI test file passes: 271 of
271, plus `dashboard-ui`, `scripts/validate-ports.sh`, `pnpm build` and
the TOML drift check.
- Linux (`node:24-bookworm`, as `make test-unit` runs it): every CLI
test file 261 of 261, `network-config` 212 of 212.

Run by hand with `ppn start paseo-next-v2 --fork` (stacked with #42 and
#43, two checkouts with their ports moved
apart):

| Case | Result |
| --- | --- |
| Something else listens on a service port | `ppn start` exits 1 naming
the port, its key and `node(<pid>)`; the listener keeps running and
nothing is started. |
| Ctrl-C | "stopped process group N (13 process(es))"; within 2 s
nothing of the workspace and no listener is left. |
| `kill -9` of `ppn` mid-run | The network keeps running (47 processes).
The next `ppn start` says a previous run is still running, stops its
group (55 processes) and comes up; `ppn kill` then stops the new group
(59) without the fallback sweep. |
| Two networks on one machine | `ppn kill` in one stops its group (49
processes); the other keeps its 16 listeners and its People node answers
RPC. |

After every case no `postgres:` process was left. Zombienet runs the
nodes and custom processes in its own group;
only Postgres' own children (checkpointer, writers, backends) make
groups of their own, and they stop with the
postmaster.

Not run: a closed terminal (SIGHUP) and SIGTERM to `ppn`, a network with
TURN on, and a live network on Linux (this
PR's CI runs the zombienet integration and fork-e2e jobs there).

## Open questions

- Removing `kill-port.sh` changes the published `scripts/`. If a private
deployment calls it, should a deprecated
  `kill-port.sh` stay and exec `require-free-ports.sh`?
- `ppn kill` still deletes every `/tmp/zombie-*` directory, including
other networks'. Scoping that needs zombie-cli's
base directory; not in this PR. A Ctrl-C of `ppn start` no longer runs
that deletion (it stops its own group only),
so its `/tmp/zombie-*` directory stays until `ppn kill`. Is that
acceptable?
- eturnal 1.12.3 fixes its Erlang node name (`-name eturnal@127.0.0.1`),
so two relays on one machine probably
collide even on different ports (not verified). Whether eturnal stays in
the group was not checked either: it was
  not installed on the test machine.

Closes #44
## Summary

zombie-cli exits, and takes the whole network down, the first time any
node's metrics endpoint is slow to answer.
`ppn start` gives no way to turn that check off: zombie-cli's
`--node-verifier` option cannot be passed through. This
PR adds `ppn start --node-verifier <metric|none>` (and `NODE_VERIFIER=`
for `make start`). Without the option nothing
changes.

## What goes wrong

`ppn start` always runs `zombie-cli spawn -p native [-d <data>] <toml>`.
zombie-cli 0.4.15 (the pinned
`ZOMBIENET_VERSION`) defaults to `--node-verifier metric`, and then
(`crates/cli/src/main.rs`):

- waits until every node answers Prometheus (`wait_until_is_up(20)`);
- every 15 s afterwards checks again (`wait_until_is_up(5)`), and
**exits** with "One of the nodes crashed" the first
  time one does not answer. The exit tears down every node.

That is the right default for a quick local network, and wrong when the
caller watches the network itself:

- a long fork restore or warp, during which a node's metrics endpoint is
slow;
- debugging one failing node without losing the other twenty;
- a harness with its own readiness and liveness checks that wants to
report which node failed and why.

Today the only way to get `none` is to bypass `ppn start` and spawn
zombie-cli by hand.

## The fix

- `ppn start --node-verifier <metric|none>` (commander `choices`: any
other value is refused before anything starts),
  and `NODE_VERIFIER=` for `make start`.
- The option is passed to zombie-cli only when given.
- `zombieSpawnArgs()` builds zombie-cli's arguments in one place, so
they can be tested.

## What does not change

- Without the option, zombie-cli's command line is exactly as before,
and zombie-cli keeps its own default,
  `metric`.

## Tests

`packages/cli/tests/node-verifier.test.ts` (4 tests):

- omitted means no flag, with and without `--ephemeral`;
- a given value is passed before `-d`;
- the option offers exactly `metric` and `none`;
- `--node-verifier bogus` is rejected with commander's `invalidArgument`
("Allowed choices are metric, none").

Results on `main` `d298d260` (v0.8.0), macOS arm64, Node 25.4.0, pnpm
10.33.0:

- `tsc -b`, depcruise and `network-config` 212 of 212 pass.
- Every CLI test file except `dashboard.test.ts` (it binds fixed ports):
229 of 229, `node-verifier` 4 of 4.
- Stacked with the two other PPN pull requests from the same author,
every CLI test file passes: 256 of 256, plus
`dashboard-ui`, `scripts/validate-ports.sh`, `pnpm build` and the TOML
drift check.
- `ppn start --help` lists the option; `make -n start
NODE_VERIFIER=none` contains `--node-verifier none`, and plain
  `make -n start` does not.
- With the pass-through removed from `zombieSpawnArgs`, 1 of the 4 tests
fails.

Run by hand: `ppn start paseo-next-v2 --fork --clean --node-verifier
none`. zombie-cli's command line was
`zombie-cli spawn -p native --node-verifier none -d <data dir> …`
(without the option: `zombie-cli spawn -p native -d
…`), "network is up, in 20.50s", and a minute later the relay chain and
all three parachains were still
importing blocks.

Not run: a node crash under `none`, Linux.

## Open questions

- Should `ppn start` warn that, with `none`, nothing notices a crashed
node any more?
- Docs: add a line next to `EPHEMERAL=1` in the README's options list?
Only the `make help` line is in this PR.

Closes #46
Moves the device-uniqueness-backend pin to v0.7.0

- Changed the DUB pin from v0.6.0 to v0.7.0, whose vendored People and
Asset Hub metadata is spec 3_004_000, the runtime previewnet now starts
from
- Changed dub-api to name TURN_PROVIDER=coturn, since v0.7.0 defaults
turn-api to Cloudflare and refuses to boot with TURN_SECRET set and no
provider named
- Fixed the no-relay case passing an empty ICE_SERVERS, which v0.7.0
rejects at boot, by handing it a loopback placeholder instead
- Added a check to the registration test that the new lite username is
finalized in People Chain's Resources.Consumers for the registered
account, with no full username
- Added a check that search reports the registered username as ASSIGNED
A server's secrets file is now checked against one list, so a deployment
can no longer drop a key without noticing. Prod lost DUB because
`JWT_ED25519_SECRET` lived only in a hand-edited file that a render
rewrote

- `config/secrets.env.example` lists every key in three sections (every
server, deployable servers, optional), each with one line on how to make
it
- `ppn secrets check [file]` fails on a missing or empty required key, a
key not in the example, or a deployable key in a local file
- `ppn start` refuses a named secrets file that fails the check
- `@parity/ppn/secrets` exports the reader, so a deployment can test its
render against the same list

---------

Signed-off-by: Cosmin Paraschiv <cosmin@parity.io>
Co-authored-by: Cosmin Paraschiv <cosmin@parity.io>
Unless PPN_PACKAGE_ROOT is set, repoRoot() finds the root by looking
for config/ports.env, starting from network-config's directory. When
network-config is installed from npm or being packed, it has its own
config/ports.env, so repoRoot() returns network-config's directory,
which doesn't have secrets.env.example.

Signed-off-by: Cosmin Paraschiv <cosmin@parity.io>
Follow-up to #53 for how the CLI finds `config/secrets.env.example`

- Changed the lookup to search up for `config/secrets.env.example`
itself instead of the first directory holding a `config/ports.env`
- Fixed `PPN_PACKAGE_ROOT` being ignored for the secrets example while
every other root lookup follows it
- Changed the CLI test script to set `PPN_PACKAGE_ROOT`, so a pack
running in parallel can no longer make the tests read another package's
`config/`
- Added a publish-surface test that installs both tarballs and reads the
secrets example through `@parity/ppn/secrets` with no `PPN_PACKAGE_ROOT`
set
## Summary

`ppn fork toml` gives every network's collators the workspace's binary
folder, except previewnet's, which gets none.
When the workspace isn't the checkout (`PPN_HOME` set, or Engine
installed from npm), a previewnet fork's collators
then look for `polkadot-omni-node` outside the workspace (here, Engine's
own `bin/`) and exit at once:

```
…/engine/scripts/omni-node.sh: line 108: …/engine/bin/polkadot-omni-node: No such file or directory
```

This PR passes previewnet's `bin/` from the workspace, as every other
network already gets `bin/<network>`.
One line in `packages/cli/src/commands/fork.ts`.

## What goes wrong

- `fork.ts` (`toml`): `binDir: bundleNet === 'previewnet' ? undefined :
path.join(WS, 'bin', bundleNet)`.
- `generateForkToml` still falls back to `${repoDir}/bin` for the
relay's `default_command`, so the relay validators
start. But each parachain gets `options.binDir` as is, so with
`undefined` no `PPN_BIN_DIR` is written into the
  collators' `env`.
- `omni-node.sh` takes `PPN_BIN_DIR`, then `BIN`, then the `bin/` of the
workspace `scripts/lib/workspace.sh` finds on
its own. zombienet starts each node with a cleared environment plus only
its block's `env` (the cause behind
#45/#42), so `BIN` and `PPN_HOME` from `ppn start` never reach the
collator, and `workspace.sh` finds another
workspace: the Engine checkout in our run, with no `polkadot-omni-node`
in its `bin/` (from npm it would be
  `~/.ppn`, per #45; not run here).
- In a checkout the workspace *is* the checkout (`REPO === WS`), so both
folders are the same and nothing shows. That
  is why CI's `fork-e2e (previewnet)` passes.

## The fix

```diff
- binDir: bundleNet === 'previewnet' ? undefined : path.join(WS, 'bin', bundleNet),
+ binDir: bundleNet === 'previewnet' ? path.join(WS, 'bin') : path.join(WS, 'bin', bundleNet),
```

previewnet's binaries live in `bin/`, every other network's in
`bin/<network>`, as the comment above the line says and
as `binDirFor()` in `start.ts` already does.

## Why setting an env var doesn't work around it

The caller already sets `PPN_HOME` to the workspace, and `ppn start`
passes `PPN_HOME` and `BIN` to zombie-cli. They
don't reach the collator, because zombienet gives a node only its own
block's `env`. The folder has to be written into
the generated config, which is what this does.

## What doesn't change

- Every network other than previewnet: same expression as before.
- previewnet in a checkout: `PPN_BIN_DIR` now names the same `bin/` the
launcher already fell back to.
- The relay: still `${WS}/bin` through `default_command`.

## Tests

- `pnpm install --frozen-lockfile`, `tsc -b`, `packages/cli` tests
(278/278), `packages/network-config` tests
  (212/212) and `lint:boundaries` pass on this branch.
- Live, on `main` at `0004226`, with `PPN_HOME` set to a workspace
outside the checkout and a previewnet fork built
  fresh from its published bundle each time (no saved fork reused):
- without this change: `Collator-1500` exits at start with the error
above;
- with it: the fork starts, the collators get
`PPN_BIN_DIR=<workspace>/bin`, and a product's end-to-end scenario
on the fork (sign a remark on People, read it back finalized and shown
in the app) passes 4/4.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Grants the identity backend's attester a DotnsGateway allowance on Asset
Hub, so attested people get their dotNS name on genesis networks and
forks.

- Added `increase-dotns-attestation-allowance`, which sudo-grants the
attester (`PPN_ALLOWANCE_SS58`, Alice by default) 1000
`DotnsGateway.increase_attestation_allowance` on Asset Hub and reads the
allowance back to confirm it.
- Added the process to genesis networks and to forks of networks with
sudo, wherever both People and Asset Hub are present.
- Added a skip for an Asset Hub without DotnsGateway, which looks again
for two minutes so a fork whose upgrade adds the pallet still gets the
grant.
- Added CI steps that check the grant fails without `PPN_ALLOWANCE_SS58`
in the deployable profile and never goes to Alice.
- Changed `set-dispatcher-address` to read `DispatcherAddress` back and
retry, since in genesis mode it now shares the sudo account on Asset Hub
with the new grant.
- Changed the People allowance grant to share the attester account
lookup with the new grant.
- Changed `docs/DEVICE-UNIQUENESS-BACKEND.md`, `docs/PROFILES.md` and
`CLAUDE.md` to list the new grant and to say what lets the writer pick
up a late grant.
- Fixed every dotNS reservation failing with
`DotnsGateway::NoAttestationAllowance` while the People registration
still landed, which
[polkadot-pop-e2e#60](paritytech/polkadot-pop-e2e#60)
hits once it waits for the dotNS name the way a host does.
Writes the running network's Trinity stack descriptor and light-client
specs, and serves them from the dashboard. Closes #54

The issue asked for an unsigned descriptor from the engine's API, with
signing left to preview-net-v1. A signature covers exact bytes, so the
engine writes a file and signs it itself when given a key, and
preview-net-v1 only has to supply `PPN_STACK_SECRET`.

- Added `ppn service trinity-stack`, a custom process in genesis and
fork spawns that writes `trinity-stack.json` and a `<spec>.smol.json`
light spec per chain into the data directory, and rewrites them every 6
hours.
- Added signing of the descriptor when the secrets file sets
`PPN_STACK_SECRET`, except on a fork, which is always published
unsigned.
- Added the relay's sync checkpoint to its light spec, retried every 30
seconds until the relay has finalized a block.
- Added light spec bootnodes that point at the network's own nodes, over
their P2P WS port and, behind https, also nginx's WSS port.
- Added a check that refuses a chain spec naming a node on any host
other than `BOOTNODE_HOSTNAME`, so a fork never publishes its source's
nodes.
- Added `fork` and each chain's `forkedAt` to a fork's descriptor when
the spawn stamp records the bite and the network file names the source's
pointer.
- Added `--fork-manifest` to `ppn stamp-spawn`, so a deployment that
spawns a fork itself records its bite.
- Added `biteHashes` to the fork bundle manifest: the source's block
hash at each bite block, read during the bite and named as each chain's
`forkedAt`.
- Added `/.well-known/trinity-stack.json`, `/api/stack` and
`/chainspecs/<spec>.smol.json` to the dashboard.
- Added a Trinity stack panel to the dashboard overview with the stack's
link and QR code.
- Added `stack.tld` and `stack.pointer` to network files, set for
previewnet's pointer and Polkadot's TLD.
- Added `PPN_STACK_FAUCET_URL`, `PPN_STACK_EXPIRES_AT`,
`PPN_STACK_VALID_FOR` and `PPN_STACK_REFRESH_SECONDS` to
`config/ports.env`.
- Added `PPN_EPHEMERAL` to `ports.local.env`, and the service writes
nothing for an ephemeral run.
- Changed every service to read the data directory and public URL from
the service context, resolved once.
Makes the dashboard say which instance of which network it is, so
production, staging, forks and laptops can be told apart at a glance.

- Changed the dashboard heading and tab title to show the network's
name, prefixed with "Fork of" on a fork, and the host it runs on.
- Added `PPN_DISPLAY_NAME` to `config/ports.env`, which replaces the
network's display name in the dashboard and the Trinity stack
descriptor, such as "Previewnet Staging".
Lets hosts and tools take a network's dotNS contracts from its Trinity
stack descriptor, such as `dotns.storeFactory` on previewnet, instead of
a copy per network or reads from Asset Hub.

- The descriptor now carries `dotns`: each dotNS contract by its
`DotnsConstants` key, lowercase, plus `protocolRegistry` for the
registry itself.
- A genesis previewnet takes the addresses from the
`bin/dotns-addresses.json` that `ppn fetch` takes from the dotns
release, leaving out contracts without a key, such as the store beacons
and the flat pricing contract.
- A fork takes the keys from its source's descriptor at `stack.pointer`
and each address from its own registry's `get(key)`, so a key rewired
since the bite shows what the fork actually runs.
- A key the fork's registry does not hold, such as a release's
`multicall3`, keeps the source's address.
- While the source descriptor or eth-rpc does not answer, a fork
publishes no `dotns` and the writer tries again in 30 seconds.
- paseo-next-v2 and devnet forks now publish a descriptor, with
`stack.tld` and `stack.pointer` pointing at
https://stack-paseo-next.parity-testnet.parity.io/.well-known/trinity-stack.json
and
https://stack.api.polkadotcommunity.foundation/.well-known/trinity-stack.json.
- `@parity/trinity-stack` moves to 0.2.0, which defines the field:
paritytech/trinity-user-agents#1803

Part of paritytech/trinity-user-agents#1312
…scriptor (#61)

Previewnet's Trinity stack descriptor now states its payment asset and
Coinage instance, plus its dotNS web gateway on deployments that set
one.

- Previewnet's descriptor lists pUSD (asset 50000413 on Asset Hub) as
the payment asset and Coinage instance 0 on People.
- A network file can set `stack.assets` and `stack.coinage` for any
network whose runtime has them.
- Setting `PPN_STACK_WEB_GATEWAY_URL` adds a `web-gateway` service. Only
the instance the gateway resolves names against should set it.
- A gateway URL whose path does not end in `/` stops the descriptor from
being written, since `@parity/trinity-stack` is now 0.2.0, which checks
web gateway URLs.
Adds `--authoring=slot-based` to the Bulletin collator.

Why: the next Bulletin runtime (`spec_version` 1_000_028,
polkadot-bulletin-chain #816) enforces `RelayParentOffset = 1`, which
the lookahead collator can't satisfy. A lookahead collator stops
proposing once the upgrade is enacted.
Lets clients call a local network's services with query parameters
through the dashboard, such as `GET
/dub/api/v1/usernames/search?prefix=alice`

- HTTP requests through the dashboard proxy now reach the service with
their query string
- WebSocket upgrades through the proxy now keep their query string too
## Summary

Bumps the dotNS genesis pin from v0.8.0 to v1.1.0, so a previewnet built
by the engine boots the same dotNS release as Paseo.

The contract addresses are unchanged between the two releases
(`deployments/expected.json`), so `networks/previewnet.json` and its
pinned resolver stay as they are. v1.1.0 publishes the same
`dotns-genesis-testnet.json` asset the fetch expects.
# Conflicts:
#	CLAUDE.md
#	config/versions.env
#	docs/DEVICE-UNIQUENESS-BACKEND.md
#	networks/README.md
#	networks/polkadot.json
#	packages/cli/src/commands/fetch.ts
#	packages/cli/src/commands/fork.ts
#	packages/cli/src/commands/turn.ts
#	packages/cli/src/fork/overrides.ts
#	packages/cli/src/fork/validators.ts
#	packages/cli/tests/fork-validators.test.ts
#	packages/dashboard-ui/src/lib/Overview.svelte
#	packages/network-config/src/dub.ts
#	packages/network-config/src/fork-toml.ts
#	packages/network-config/src/networks.ts
#	packages/network-config/src/toml-generator.ts
#	packages/network-config/tests/dub.test.ts
#	scripts/dub/postgres.sh
#	scripts/turn.sh
#	tests/scripts/test-dub-registration.ts
#	tests/scripts/test-turn.ts
#	zombienet-configs/local-dev.toml
@mordamax
mordamax requested a review from csmnprschv as a code owner November 10, 2026 21:46
@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​parity/​trinity-stack@​0.2.07710010094100
Addeduqr@​0.1.310010010082100

View full report

@csmnprschv
csmnprschv merged commit a876249 into cosmin-polkadot-fork Nov 12, 2026
15 checks passed
@csmnprschv
csmnprschv deleted the polkadot-test-trinity branch November 12, 2026 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants