From a45ac63bb2ec8bb605524d5c7e08516ffcaa9d43 Mon Sep 17 00:00:00 2001 From: ewowi Date: Thu, 19 Nov 2026 14:29:33 +0100 Subject: [PATCH 1/4] Add the fleet clock, timed group changes and a window on every driver Every MoonLight device on the network now keeps one fleet clock, so effects show the same moment side by side and a group's changes apply together. Every output driver, HDMI, NDI, HLS and RTSP included, shows a start and count window of the lights, so two screens can each show half of one canvas. MoonBase is matched by its handoff version, release lists come from a feed on Pages, and a device group counts a member in MoonBase as missing. Skipped: commit/improv, the Improv smoke test needs a board's port; the change touches no Improv path. Skipped: commit-code/kpi, it needs the code gate, which fails by the two rises below. Ratchet: check_code mutable static state 44 -> 45 and blocking call on the render path 202 -> 203, both accepted by the product owner: FleetTime's one process-wide publication, which every task reads fleet time from (moving it beside platform::uptimeUs is in Plan-20261109's backlog), and FleetClock::poll in Discovery's tick20ms, the socket read on boards without tasks (Teensy), as HttpServerModule::pollUplinks reads its uplinks there. **Core** - ClockSync.h: NTP's four-timestamp exchange, its clock filter (the least round trip of eight), a discipline that slews under 128 ms and steps past it, and FleetTime, two published slots of 32-bit halves read without a lock. - FleetClock: the reference is the lowest live device heard beaconing on 239.255.77.80; the exchange runs on UDP 65511 on a task of its own, answered to the asking port; a device that boots listens one beacon interval and inherits the fleet's time, so a reboot of the reference never steps the fleet back; held by the Devices container, its task started in prepare(). - Hlc's physical part and effects' elapsed() run on fleet time; the fade keeps the local clock. - DeviceGroupSync: a shared control or look changed here goes out as an OSC bundle applying 100 ms ahead; a member on the fleet clock holds it until then, at most a second ahead; OscPacket reads and writes one-message bundles. - FirmwareImage.h: kHandoffVersion, the contract between the app and MoonBase, read from a record after MoonBase's descriptor; the app reports MoonBase behind only on another number; unit_MoonBaseContract pins the contract. - DevicesModule::present: a member heard running MoonBase is missing; a group member's device field refuses a quoted value, a MAC goes by mac. - FirmwareUpdateModule: a MoonBase or radio chip mismatch shows on its own tab only. - platform: uptimeUs() on every port; reboot()'s doc says how a host restarts. **Light domain** - DriverBase: start and count on every driver that lights something; LayoutPicture crops to a window, and HDMI, NDI, HLS and RTSP place through placeWindowed; HUB75 offers none until its panels open at the window's extent. **UI** - install-picker.js: the release list from install/releases.json on Pages, asked at most once per its ten-minute max-age unless a view asks past it; the map separates group boxes that share no member; the group update popup counts down and closes when done. **Scripts/MoonDeck** - release.yml writes the release feed; diag/measure_fleet_clock.py measures how closely the devices' fleet clocks agree. **Tests** - unit_ClockSync, unit_FleetClock (reference choice, exchange, stray replies, inherited time), unit_SharedControls (timed changes held and capped), unit_OscPacket (bundles), unit_DriverBase_window, unit_LayoutPicture and unit_HdmiDriver (windows), unit_DeviceGroup (MoonBase missing, the device field), unit_FirmwareUpdateModule, unit_FirmwareImage, unit_MoonBaseContract; test_main resets FleetTime per case. Host suite 3503, JS and Python pass. **Docs/CI** - MIGRATING: the fleet clock and its port, timed group changes, the member's mac field, MoonBase by handoff version. - Plan-20261109: split into the steps toward supersync and the work beside them; steps 3 and 6 to 8 done, step 9 built with its 3x3 bench test named, the TSF note; the platform clock adjustment in the backlog. - Plan-20261117: the fourth round's review and the plan against the textbook, with its five refinements. **Reviews** - Reviewer, a reboot of the lowest device stepping every device back to its uptime: done, a booting device listens and inherits the fleet's time. - Reviewer, a held change waiting however far ahead: done, held at most ten leads. - Reviewer, a new reference asked only at the old interval: done, asked at once. - Reviewer, a beacon slot read with the old id and the new time: done, the id emptied first. - Reviewer, the status bar asking the feed every second: done, once per max-age. - Reviewer, docs against code (the originating device's lead, FleetTime::nowUs, the Teensy wrap, the header lead): done. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/release.yml | 11 + docs/design/code-hosting.md | 2 +- docs/explanation/architecture/moonbase.md | 3 +- docs/help/help.json | 87 ++- docs/reference/MIGRATING.md | 24 + docs/reference/metrics/code.json | 8 +- docs/reference/metrics/docgen.json | 6 +- docs/reference/metrics/prose.json | 6 +- docs/reference/openapi.yaml | 2 +- ...9 - Devices, groups and the network map.md | 525 ++++++++---------- ...0261117 - Faster commit and merge gates.md | 29 + moonbase/main/CMakeLists.txt | 2 + moonbase/main/moonbase_main.cpp | 27 +- moondeck/MoonDeck.md | 13 +- moondeck/diag/measure_fleet_clock.py | 138 +++++ moondeck/moondeck_config.json | 9 + moondeck/moondeck_ui/map.js | 4 +- moondeck/run/preview_installer.py | 3 +- moonwalk/07-maintain.yaml | 2 +- src/core/services/OscPacket.h | 35 +- src/core/sources.cmake | 1 + src/core/system/DeviceGroup.cpp | 20 +- src/core/system/DeviceGroup.h | 14 +- src/core/system/DeviceGroupSync.cpp | 54 +- src/core/system/DeviceGroupSync.h | 24 +- src/core/system/Devices.h | 19 + src/core/system/DevicesModule.cpp | 4 +- src/core/system/DevicesModule.h | 45 +- src/core/system/FirmwareUpdateModule.h | 22 +- src/core/system/FleetClock.cpp | 216 +++++++ src/core/system/FleetClock.h | 154 +++++ src/core/util/ClockSync.h | 205 +++++++ src/core/util/FirmwareImage.h | 66 ++- src/core/util/Hlc.h | 6 +- src/light/drivers/DriverBase.h | 21 +- src/light/drivers/HdmiDriver.h | 10 +- src/light/drivers/HlsDriver.h | 10 +- src/light/drivers/Hub75Driver.h | 3 + src/light/drivers/HueDriver.h | 2 - src/light/drivers/LayoutPicture.h | 59 +- src/light/drivers/NdiDriver.h | 12 +- src/light/drivers/NetworkSendDriver.h | 2 - src/light/drivers/PanelCardDriver.h | 2 - src/light/drivers/ParallelLedDriver.h | 2 - src/light/drivers/PwmLightDriver.h | 2 - src/light/drivers/RmtLedDriver.h | 2 - src/light/drivers/RtspDriver.h | 8 +- src/light/layers/Layer.h | 10 +- src/platform/esp32/platform_esp32.cpp | 2 + src/platform/esp32/platform_esp32_ota.cpp | 22 +- src/platform/host/platform_host.cpp | 10 +- src/platform/platform.h | 10 +- src/platform/teensy/platform_teensy.cpp | 9 + src/platform/teensy/platform_teensy_ota.cpp | 1 + src/ui/app.js | 28 +- src/ui/devicegroup.js | 26 +- src/ui/devicemap.js | 42 +- src/ui/devicepopup.js | 6 +- src/ui/install-picker.js | 125 ++--- src/ui/releases.js | 2 - test/CMakeLists.txt | 3 + test/js/device-group.test.mjs | 8 +- test/js/install-picker-releases.test.mjs | 79 +-- test/js/network-map.test.mjs | 32 +- test/unit/core/group_bench.h | 6 +- test/unit/core/unit_ClockSync.cpp | 69 +++ test/unit/core/unit_DeviceGroup.cpp | 23 +- test/unit/core/unit_DevicesModule_flows.cpp | 6 +- test/unit/core/unit_FirmwareImage.cpp | 35 +- test/unit/core/unit_FirmwareUpdateModule.cpp | 16 +- test/unit/core/unit_FleetClock.cpp | 147 +++++ .../unit/core/unit_HttpServerModule_apply.cpp | 6 +- test/unit/core/unit_MoonBaseContract.cpp | 53 +- test/unit/core/unit_OscPacket.cpp | 47 +- test/unit/core/unit_SharedControls.cpp | 63 +++ test/unit/light/unit_DriverBase_window.cpp | 30 + test/unit/light/unit_HdmiDriver.cpp | 18 + test/unit/light/unit_LayoutPicture.cpp | 38 ++ test/unit/test_main.cpp | 4 +- 79 files changed, 2249 insertions(+), 648 deletions(-) create mode 100644 moondeck/diag/measure_fleet_clock.py create mode 100644 src/core/system/FleetClock.cpp create mode 100644 src/core/system/FleetClock.h create mode 100644 src/core/util/ClockSync.h create mode 100644 test/unit/core/unit_ClockSync.cpp create mode 100644 test/unit/core/unit_FleetClock.cpp create mode 100644 test/unit/light/unit_DriverBase_window.cpp diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index dd379b16c..5c70884d2 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -875,6 +875,17 @@ jobs: echo "Staged binaries + relative manifests into releases/$T" done + # The release feed the device UI, MoonDeck and the installer read (install-picker.js): the releases kept above, newest first, in + # GitHub's release shape trimmed to the fields they use. Pages serves it to any origin with a max-age, so the browsers' HTTP caches + # pace the checks and no page spends GitHub's 60 anonymous API requests an hour. + mkdir -p pages/install + for T in $KEEP; do gh api "repos/$GITHUB_REPOSITORY/releases/tags/$T"; done \ + | jq -s 'sort_by(.published_at) | reverse + | map({tag_name, name, prerelease, draft, published_at, created_at, html_url, + assets: [.assets[] | {name, browser_download_url, digest}]})' \ + > pages/install/releases.json + echo "Staged the release feed: $(jq length pages/install/releases.json) releases" + # The install page and the picker modules at the root; each release's binaries and manifests live under releases//. - name: Stage GitHub Pages site run: uv run moondeck/ci/release_stage.py pages-site diff --git a/docs/design/code-hosting.md b/docs/design/code-hosting.md index 9d4e957c3..d5ae0eca1 100644 --- a/docs/design/code-hosting.md +++ b/docs/design/code-hosting.md @@ -45,7 +45,7 @@ Plain git moves unchanged: the repo-health and KPI scripts, MoonCloud on Cloudfl |---|---|---|---| | Actions, 4 workflows | build, test, release, docs deploy | Forgejo Actions reads GitHub-style workflow files; most `actions/*` steps run | `.gitlab-ci.yml`, a rewrite of the four workflows | | Hosted runners: Linux x86 and arm64, macOS, Windows | the host builds: `.deb` on both architectures, `.dmg`, `setup.exe` | Codeberg's Woodpecker is `linux/amd64` only and *"provided as-is"*; the others need self-hosted runners | macOS and Windows in beta, outside the service-level agreement; arm64 hosted on every tier, the small size on the free one | -| Releases and `api.github.com` | the installer, the UI's update badge and OTA picker, MoonBase's recovery page and the Dockerfile read releases, the release `name` and the `prerelease` flag | the Gitea-shaped releases API: similar fields, different URLs | the Releases API, with assets as links to files in a package registry | +| Releases and `api.github.com` | the release workflow writes the release feed the installer, the UI's update badge and OTA picker and MoonBase's recovery page read from Pages; the Dockerfile reads releases; all use the release `name` and the `prerelease` flag | the Gitea-shaped releases API: similar fields, different URLs | the Releases API, with assets as links to files in a package registry | | `raw.githubusercontent.com` | MoonLive script downloads, the Gallery | `/raw/` URLs, CORS to verify | `/-/raw/` URLs, CORS to verify | | Pages, custom domain | moonmodules.org/MoonLight and the installer at `/install/` | Codeberg Pages from a `pages` branch or an Actions build; with self-hosting, any static host | GitLab Pages | | ghcr.io | the Docker image | the package registry | the container registry | diff --git a/docs/explanation/architecture/moonbase.md b/docs/explanation/architecture/moonbase.md index 05f909427..60ccae9e2 100644 --- a/docs/explanation/architecture/moonbase.md +++ b/docs/explanation/architecture/moonbase.md @@ -49,7 +49,8 @@ The chip id matters because there is one MoonBase per chip, one paste apart, and ## Telling the two images apart -Each image reports its version from the app descriptor IDF puts in every binary, with `PROJECT_VER` set to the same computed version for both. The app can therefore read the factory partition's version without booting it, and say when the two were built apart. A device that cannot name its own recovery image cannot be diagnosed: two boards that looked identical, one of which could not install firmware, took a bisect of the git log to tell apart. +Each image reports its version from the app descriptor IDF puts in every binary, with `PROJECT_VER` set to the same computed version for both. The app can therefore read the factory partition's version while the app runs. +Whether the two work together is a separate number: the handoff version, a record MoonBase carries right after its descriptor, which numbers the contract between the two images (the staged URL, the config MoonBase reads, the access point and the routes the app's pages call on it). The app warns only when MoonBase's number differs from its own, so a MoonBase from an older release, or under a local build, stays quiet until the contract changes ([FirmwareImage.h](../../../src/core/util/FirmwareImage.h)). A device that cannot name its own recovery image cannot be diagnosed: two boards that looked identical, one of which could not install firmware, took a bisect of the git log to tell apart. MoonBase is a standalone ESP-IDF project (`moonbase/`, ~750 KB against an 896 KB slot) sharing no source files with the app. That is the deliberate trade for an image that must stay small and, once working, hardly change. diff --git a/docs/help/help.json b/docs/help/help.json index 2c8d46279..030720b5c 100644 --- a/docs/help/help.json +++ b/docs/help/help.json @@ -3453,11 +3453,11 @@ "image": "https://moonmodules.org/MoonLight/assets/light/drivers/HueDriver.png", "controls": { "start": { - "line": "The first light of the shared buffer this driver reads." + "line": "The first light shown." }, "count": { - "line": "How many lights it reads from `start`; 0 reads them all.", - "more": "So drivers share one buffer: a status LED takes `start 0, count 1` and the strip `start 1`." + "line": "How many, from `start`; 0 for the rest.", + "more": "A status LED takes `start 0, count 1`, the strip `start 1`." }, "bridgeIp": { "line": "The bridge's LAN IP, entered in the UI (4 octets)." @@ -3510,11 +3510,11 @@ "line": "The profile name an older config saved, read once and then emptied." }, "start": { - "line": "The first light of the shared buffer this driver reads." + "line": "The first light shown." }, "count": { - "line": "How many lights it reads from `start`; 0 reads them all.", - "more": "So drivers share one buffer: a status LED takes `start 0, count 1` and the strip `start 1`." + "line": "How many, from `start`; 0 for the rest.", + "more": "A status LED takes `start 0, count 1`, the strip `start 1`." }, "protocol": { "line": "The wire protocol: Art-Net, E1.31 or DDP; the port follows.", @@ -3595,6 +3595,13 @@ ], "image": "https://moonmodules.org/MoonLight/assets/light/drivers/PreviewDriver.png", "controls": { + "start": { + "line": "The first light shown." + }, + "count": { + "line": "How many, from `start`; 0 for the rest.", + "more": "A status LED takes `start 0, count 1`, the strip `start 1`." + }, "targetFps": { "line": "The rate the preview aims for: lower for detail, raise for smoothness.", "tests": [ @@ -3618,11 +3625,11 @@ "guide": [ { "title": "The NDI runtime", - "text": "You install it yourself; without it the driver says so and nothing else changes. The steps: [installing the NDI runtime](https://moonmodules.org/MoonLight/reference/watching-the-output.html#installing-the-ndi-runtime)." + "text": "You install it yourself; until then the driver says so and runs on. The steps: [installing the NDI runtime](https://moonmodules.org/MoonLight/reference/watching-the-output.html#installing-the-ndi-runtime)." }, { "title": "Receivers", - "text": "To see the output you need a receiver. NDI Video Monitor, part of NDI Tools, is the simplest; OBS gains an NDI Source through the DistroAV plugin. MoonLight appears under the name in `sourceName`, or the device's own name when that is blank." + "text": "NDI Video Monitor, part of NDI Tools, is the simplest receiver; OBS gains an NDI Source through the DistroAV plugin. MoonLight appears under the name in `sourceName`, or the device's own name when that is blank." }, { "title": "Status line", @@ -3647,6 +3654,13 @@ "fixtureRef": { "line": "The profile name an older config saved, read once and then emptied." }, + "start": { + "line": "The first light shown." + }, + "count": { + "line": "How many, from `start`; 0 for the rest.", + "more": "A status LED takes `start 0, count 1`, the strip `start 1`." + }, "sourceName": { "line": "The name a receiver lists; blank uses the device's own." }, @@ -3663,7 +3677,7 @@ }, "HlsDriver": { "line": "Streams the layer as H.264 over HLS from the device's own HTTP server, which a TV, VLC or a browser can play.", - "summary": "Open the `url` the card shows. Where [Ndi](https://moonmodules.org/MoonLight/moonmodules/light/drivers.html#ndi) feeds production tools, this feeds anything that plays video.\n\nEach light a `scale` by `scale` block, placed where the layout puts it and corrected as the wall is. Latency is HLS's own, 2 to 5 seconds, so this is for watching rather than live control. The status line carries the state, the drops, or why the encoder stopped.", + "summary": "Open the `url` the card shows.\n\nEach light a `scale` by `scale` block, placed where the layout puts it and corrected as the wall is. Latency is HLS's own, 2 to 5 seconds, so this is for watching rather than live control. The status line carries the state, the drops, or why the encoder stopped.", "attribution": "[Delphium](https://moonmodules.org/MoonLight/reference/attributions.html#delphium), [HLS](https://moonmodules.org/MoonLight/reference/attributions.html#hls), [FFmpeg](https://moonmodules.org/MoonLight/reference/attributions.html#ffmpeg)", "guide": [ { @@ -3693,6 +3707,13 @@ "fixtureRef": { "line": "The profile name an older config saved, read once and then emptied." }, + "start": { + "line": "The first light shown." + }, + "count": { + "line": "How many, from `start`; 0 for the rest.", + "more": "A status LED takes `start 0, count 1`, the strip `start 1`." + }, "targetFps": { "line": "Encode-rate ceiling, and the bandwidth knob: bitrate follows.", "more": "The bitrate follows from grid size and rate, near 800 kbit for 512x512 at 30 fps. Lower the rate to save bandwidth: fewer frames beat a blockier picture." @@ -3752,6 +3773,13 @@ "fixtureRef": { "line": "The profile name an older config saved, read once and then emptied." }, + "start": { + "line": "The first light shown." + }, + "count": { + "line": "How many, from `start`; 0 for the rest.", + "more": "A status LED takes `start 0, count 1`, the strip `start 1`." + }, "targetFps": { "line": "Frame-rate ceiling, which is also the keyframe interval.", "more": "The rate is a ceiling: a heavy effect that renders at 8 fps streams at 8 fps. Each module's tick time in the UI says which effect spends it." @@ -3801,6 +3829,13 @@ "fixtureRef": { "line": "The profile name an older config saved, read once and then emptied." }, + "start": { + "line": "The first light shown." + }, + "count": { + "line": "How many, from `start`; 0 for the rest.", + "more": "A status LED takes `start 0, count 1`, the strip `start 1`." + }, "mode": { "line": "The screen mode; 1280x720 suits a 1080p monitor, which scales it by exactly 1.5.", "tests": [ @@ -3866,11 +3901,11 @@ "line": "The profile name an older config saved, read once and then emptied." }, "start": { - "line": "The first light of the shared buffer this driver reads." + "line": "The first light shown." }, "count": { - "line": "How many lights it reads from `start`; 0 reads them all.", - "more": "So drivers share one buffer: a status LED takes `start 0, count 1` and the strip `start 1`." + "line": "How many, from `start`; 0 for the rest.", + "more": "A status LED takes `start 0, count 1`, the strip `start 1`." }, "format": { "line": "The card's wire format." @@ -3972,11 +4007,11 @@ "line": "The profile name an older config saved, read once and then emptied." }, "start": { - "line": "The first light of the shared buffer this driver reads." + "line": "The first light shown." }, "count": { - "line": "How many lights it reads from `start`; 0 reads them all.", - "more": "So drivers share one buffer: a status LED takes `start 0, count 1` and the strip `start 1`." + "line": "How many, from `start`; 0 for the rest.", + "more": "A status LED takes `start 0, count 1`, the strip `start 1`." }, "pins": { "line": "Data GPIO list: `18,17,16` or ranges like `20-23`. Empty idles." @@ -4036,11 +4071,11 @@ "line": "The profile name an older config saved, read once and then emptied." }, "start": { - "line": "The first light of the shared buffer this driver reads." + "line": "The first light shown." }, "count": { - "line": "How many lights it reads from `start`; 0 reads them all.", - "more": "So drivers share one buffer: a status LED takes `start 0, count 1` and the strip `start 1`." + "line": "How many, from `start`; 0 for the rest.", + "more": "A status LED takes `start 0, count 1`, the strip `start 1`." }, "pins": { "line": "One GPIO per output channel, in the fixture's channel order." @@ -4088,11 +4123,11 @@ "line": "The profile name an older config saved, read once and then emptied." }, "start": { - "line": "The first light of the shared buffer this driver reads." + "line": "The first light shown." }, "count": { - "line": "How many lights it reads from `start`; 0 reads them all.", - "more": "So drivers share one buffer: a status LED takes `start 0, count 1` and the strip `start 1`." + "line": "How many, from `start`; 0 for the rest.", + "more": "A status LED takes `start 0, count 1`, the strip `start 1`." }, "pins": { "line": "Data GPIO list: `18,17,16` or ranges like `20-23`. Empty idles." @@ -5058,7 +5093,7 @@ "guide": [ { "title": "The images", - "text": "Each image has its own tab: the app, MoonBase where present, the WiFi coproc on a P4 with WiFi. A Restart in MoonBase button sits on the card.\n\nWhen MoonBase's version differs from the app's, the card warns and the top bar shows `⬆ MoonBase`; installing the matching one from its tab clears it." + "text": "Each image has its own tab: the app, MoonBase where present, the WiFi coproc on a P4 with WiFi. A Restart in MoonBase button sits on the card.\n\nA MoonBase that does not match the app gets a dot on its tab, a warning there and `⬆ MoonBase` in the top bar; installing the matching one there clears them." }, { "title": "The WiFi coproc", @@ -5359,11 +5394,11 @@ "guide": [ { "title": "Members", - "text": "Add a row and pick a device; a device is offered once Discovery knows its MAC. A member unheard for three of its announcements is marked missing, here and on the map." + "text": "Add a row and pick a device; a device is offered once Discovery knows its MAC. A member unheard for three of its announcements, or in MoonBase, is marked missing, here and on the map." }, { "title": "Shared", - "text": "The members hold the group too, and an edit on any of them reaches the others within seconds. Deleting the group on one deletes it on all; a device removed from it drops it.\n\nA device remembers the last eight deleted groups; a member off through more than eight deletions brings its copy back, to delete again." + "text": "The members hold the group too, and an edit on any of them reaches the others within seconds. Deleting the group on one deletes it on all; a device removed from it drops it." }, { "title": "Shared controls", @@ -5371,7 +5406,7 @@ }, { "title": "Updating the members", - "text": "Click the group's box on the map and press Update group twice: the first member behind installs alone, then three at a time, this device last. The first failure stops the waves." + "text": "Click the group's box on the map and press Update group twice: the first member behind installs alone, then three at a time, this device last. The popup counts down and closes when all are done; the first failure stops the waves and stays with its reason." }, { "title": "Reaching the members", @@ -5399,7 +5434,7 @@ "line": "Its identity." }, "status": { - "line": "Present when it was heard within three of its announcements, missing otherwise." + "line": "Present when it was heard running MoonLight within three of its announcements, missing otherwise, a board in MoonBase among them." } } } diff --git a/docs/reference/MIGRATING.md b/docs/reference/MIGRATING.md index 4364e68e8..0e78f5690 100644 --- a/docs/reference/MIGRATING.md +++ b/docs/reference/MIGRATING.md @@ -31,6 +31,30 @@ For a device still on old firmware (no Backup button yet), the [installer page]( ## v7.1.0 +### The devices keep one fleet clock, on UDP 65511 + +**Action: allow UDP 65511 and the multicast group 239.255.77.80 where a firewall sits between devices.** + +Every MoonLight device keeps a fleet clock, asking the lowest device that runs it for its time on UDP port 65511 and saying it runs the clock to 239.255.77.80 every sixteen seconds. Effects animate on it, so the same effect on two devices shows the same moment; when a device first syncs, its animations jump once to the fleet's time. A group's status names the device it follows (`clock: MM-C3, ±4.7 ms`). Only devices on this release or later run it, and only they are asked. + +### A device group's change applies at a time + +**Action: *update* the members of a group to this release together; until then a member on older firmware follows a change within seconds rather than at once.** + +A shared control or look changed on a member goes out as an OSC bundle stamped with the fleet time it applies at, 100 ms ahead, so the members switch together. A member on older firmware ignores the bundle and takes the change from the group's next repeat. + +### A group member is set by `mac` from the API + +**Action: *update* a script that sets a member by MAC through the `device` field: send it as `mac`.** + +`PATCH /api/list//members/` with `field: "device"` takes the picker's index, a number, and refuses a quoted value, where a MAC sent there was read as an index from its leading digits and added another device. A MAC goes in `field: "mac"`. + +### MoonBase matches the app by handoff version + +**Action: *nothing*, or install MoonBase once.** + +The app reports MoonBase behind only when it speaks another handoff version, the number of the contract between the two images, rather than whenever their versions differ. A MoonBase installed before this release carries no handoff version, so the warning and the map's ⬆ stay until the MoonBase from this release or a later one is installed from the Firmware card's MoonBase tab; after that, a new release leaves it quiet unless the contract changed. + ### A `receivers` field takes a device group's name, and holds up to 32 **Action: *nothing*.** diff --git a/docs/reference/metrics/code.json b/docs/reference/metrics/code.json index b2a29d987..7a7dedf9e 100644 --- a/docs/reference/metrics/code.json +++ b/docs/reference/metrics/code.json @@ -1,8 +1,8 @@ { "#define used as a constant": 1, - "(duplicated lines)": 3605, - "(total)": 1729, - "blocking call on the render path": 202, + "(duplicated lines)": 3602, + "(total)": 1731, + "blocking call on the render path": 203, "complex function": 283, "deeply nested": 102, "duplicated block": 822, @@ -11,7 +11,7 @@ "long function": 44, "long parameter list": 33, "module type in the UI": 90, - "mutable static state": 44, + "mutable static state": 45, "name spread": 23, "raw allocation in light": 28 } diff --git a/docs/reference/metrics/docgen.json b/docs/reference/metrics/docgen.json index 2c6a00c8d..78cb7c2ff 100644 --- a/docs/reference/metrics/docgen.json +++ b/docs/reference/metrics/docgen.json @@ -1,9 +1,9 @@ { - "(total)": 1671, - "appendix sections nothing refers to": 135, + "(total)": 1662, + "appendix sections nothing refers to": 133, "member deep dives": 35, "multi-line comment blocks": 271, "over-wide comment lines": 598, "over-wide doc lines": 28, - "split paragraphs": 604 + "split paragraphs": 597 } diff --git a/docs/reference/metrics/prose.json b/docs/reference/metrics/prose.json index f5b169d99..e0797d258 100644 --- a/docs/reference/metrics/prose.json +++ b/docs/reference/metrics/prose.json @@ -1,11 +1,11 @@ { - "(total)": 6533, - "MoonLight.EmDash": 843, + "(total)": 6529, + "MoonLight.EmDash": 840, "MoonLight.Negation": 4902, "MoonLight.SectionLength": 31, "MoonLight.SelfReference": 14, "MoonLight.Spelling": 36, - "MoonLight.Temporal": 155, + "MoonLight.Temporal": 154, "MoonLight.VagueClose": 23, "MoonLight.Weasel": 529 } diff --git a/docs/reference/openapi.yaml b/docs/reference/openapi.yaml index 71d721b29..59d288e1a 100644 --- a/docs/reference/openapi.yaml +++ b/docs/reference/openapi.yaml @@ -818,7 +818,7 @@ components: type: string enum: [self, container, manual] description: "How it takes a new version: it installs one itself (a board, or a host whose program sits in a folder it can write), its container is replaced from outside, or someone installs it by hand (a program in a folder it cannot write)." - moonbaseMismatch: { type: string, description: "MoonBase's version, present only while it differs from the app's, so a map offers the matching one." } + moonbaseMismatch: { type: string, description: "MoonBase's version, present only while it speaks another handoff version than the app, so a map offers the matching one." } coprocessorMismatch: { type: string, description: "The WiFi coprocessor's firmware line, present only while it differs from what the app needs." } mac: { type: string, description: "The MAC address, the device identity." } name: { type: string, description: "The device name, which every network name derives from." } diff --git a/docs/work/Plan-20261109 - Devices, groups and the network map.md b/docs/work/Plan-20261109 - Devices, groups and the network map.md index 0ef371381..0d2092078 100644 --- a/docs/work/Plan-20261109 - Devices, groups and the network map.md +++ b/docs/work/Plan-20261109 - Devices, groups and the network map.md @@ -1,203 +1,168 @@ # Plan: the network map, groups and supersync -Devices becomes a top-level module that shows what is on the network, lets devices form groups, and lets a group act as one: the same controls and the same clock, up to one effect spread over many boards (supersync), and finally groups across the internet (MoonOrbit), which also lets a helper look at another user's device. Each step ships on its own and the map comes first, so every later step is visible the moment it lands. Ideas that come up while building go to the Backlog and are promoted to a step when picked. +Devices shows what is on the network, lets devices form groups, and lets a group act as one: the same controls and the same clock, up to one effect spread over many boards (supersync), and finally groups across the internet (MoonOrbit). Each step ships on its own, and the map came first, so every later step is visible the moment it lands. -## The phases +The plan has two parts. **The steps** lead to supersync, numbered 1 to 16. **Beside the steps** are the fleet's other work the map made possible: updating every kind of device, moving other firmwares to MoonLight and back, and the map's own reach. The design the steps rest on follows them, as reference. -| Phase | Steps | What it gives | Its demo | -|--|--|--|--| -| **See the network** | 1 to 5 | a map of every device, what each offers and sends, its health, and a blink to find it | the bench drawn as one map | -| **Act as a group** | 6 to 9 | groups, shared controls and presets without typing addresses, group updates, one clock | a line sweeping straight across boards | -| **One effect across the group** | 10 to 14 | supersync, from streaming to every board computing its own part | a ball crossing screens; Pong with two gamepads | -| **Beyond the LAN** | 15 and 16 | the same groups across the internet, and help for another user by looking at their device | a Pong rally between two homes; a user's setup replicated on a helper's desktop | +## Where it stands -## The design basis +| Phase | Steps | Status | What it gives | Its demo | +|--|--|--|--|--| +| **See the network** | 1 to 5 | ✅ | a map of every device, what each offers and sends, its health, and a blink to find it | the bench drawn as one map | +| **Act as a group** | 6 to 9 | 6 to 8 ✅, 9 🧪 | groups, shared controls and presets, group updates, one clock | a line sweeping straight across boards | +| **One effect across the group** | 10 to 14 | ⏳ | supersync, from streaming to every board computing its own part | a ball crossing screens; Pong with two gamepads | +| **Beyond the LAN** | 15 and 16 | ⏳ | the same groups across the internet, and help for another user by looking at their device | a Pong rally between two homes; a user's setup replicated on a helper's desktop | -- **One building block.** Devices is a top-level container like Services. Its children are MoonModules: the discovery list, and one module per group, added and removed the way an effect is. -- **The UI stays generic.** The map is a list drawn as a graph, the way a list can already be drawn as pads: a list source declares `listAsGraph()`, each row a node and an `edges` array on a row its adjacency list, the textbook graph representation. `app.js` draws it as SVG with a force-directed layout (Fruchterman-Reingold, hand-written, no library). The rows keep the list's streaming and live updates, the UI learns nothing about Devices, and any list can be drawn this way. -- **Listen always, ask on demand** (product owner, 2026-11-09). Listening costs only the packets that arrive anyway and stays on, since group state and a missing member depend on it. Asking costs traffic and work: ArtPoll and SSDP probes, the REST reads of peers for the map and the summaries. Those run only while something reads them, a Devices card open in a browser or a module that needs them, the pull model the preview and [Demand](../../src/core/module/Demand.h) follow. No scan button. -- **Each device reports only itself.** A board on a switched network sees broadcast, the multicast groups it joins and its own unicast, never traffic between two others. So every MoonLight device reports its own flows, and the page merges what its peers report into one map. A flow between two foreign devices stays unseen. -- **No upper limit**, the architecture's [dynamic over fixed-size](../explanation/architecture/moonmodule.md#lifecycle-propagation-to-children) rule. The device list, the flow table and a group's members grow on demand. They grow at a new sighting, a rare event and not a frame, by doubling, so N devices cost about log₂ N allocations, in PSRAM where the board has it. The hot path only iterates. -- **Graceful degradation when memory is short**, the [allocate-and-degrade rule](../explanation/architecture/moonlight.md#degradation-cascade) the light pipeline already follows. Growth is checked against `platform::HEAP_RESERVE` before allocating, so the fleet never takes memory the lights, WiFi or HTTP need. When growth is refused, it steps down, best to worst: - 1. **Keep what matters, drop the rest.** This device, group members and configured peers are pinned. Among the others, the device heard longest ago is replaced first (least recently used), foreign and unknown devices before MoonLight ones. - 2. **Drop the flows memory refuses.** A flow or service that does not fit is left out of the map that second, and the status says memory full. - 3. **Read, don't hold.** A device reading its group streams the members' summaries through without storing them. - 4. **Say so.** The card's status says "memory full", and the map draws what is held. A count of the devices heard but not held would itself take memory per device, so the status names the state rather than a number. +✅ done · 🧪 built, its test open · 🚧 in progress · ⏳ not started - When memory returns, the lists grow again by themselves. Never a crash, never the lights' memory. -- **Swarm scale.** Hundreds of devices on one network, so nothing costs O(N²). A device hears N presence packets per period; a change to a group goes out once by multicast, not as N unicasts; the page reads N peers with a bounded number of requests in flight; the map draws a large group as one node with its count, opened on a click. +## The steps toward supersync -## What exists +### See the network -- **Discovery** ([DevicesModule.h](../../src/core/system/DevicesModule.h)): passive, from the 44-byte presence packet on multicast and broadcast ([WledPacket.h](../../src/core/system/WledPacket.h)); MoonLight, WLED and Hue bridges, typed by plugins ([DevicePlugin.h](../../src/core/system/DevicePlugin.h)); a host setting naming a board resolves from it ([NameResolver.h](../../src/core/util/NameResolver.h)). A code-wired child of Network today ([main.cpp](../../src/main.cpp), [module_types.cpp](../../src/module_types.cpp)). -- **Shared controls** over OSC ([OscService.h](../../src/core/services/OscService.h)): the controls half of supersync, live ([mooncore.md](../explanation/architecture/mooncore.md#multi-device-runtime)). -- **Audio sync**, following one sender and counting frames per sender ([AudioService.h](../../src/core/services/AudioService.h)). -- **Light between devices** over Art-Net, E1.31 and DDP: NetworkSendDriver and the Network Receive effect. -- **Firmware by URL or upload** on every device: `POST /api/firmware/url`, and `/api/firmware/upload` where the device has no TLS ([HttpServerModule.cpp](../../src/core/system/HttpServerModule.cpp)). -- **Presets** across the whole device ([ControlModule](../../src/core/system/ControlModule.h)). -- **Peer lists in four places**: `hosts` on OscService, AudioService and NetworkSendDriver, and `host` on MidiService. -- **Two caps that break the dynamic rule**: the device list holds 32 (`kMaxDevices` in [DevicesModule.h](../../src/core/system/DevicesModule.h)), and the list seam counts rows in a `uint8_t` (`ListSource` in [Control.h](../../src/core/module/Control.h)), so no list can pass 255. OscService feeds back to at most 8 hosts. Step 1 removes the first two, step 7 the third; all three are in [What it removes](#what-it-removes). +#### 1. Devices on top, drawn as a map ✅ -## What the network carries +Landed in: the [Devices page](../moonmodules/core/system.md) (its card, the map, growth and what memory refuses, from `DevicesModule.h`); the allocation rule and `platform::fitsWithReserve` in [moonmodule.md](../explanation/architecture/moonmodule.md); the list drawn as a graph, `ListSource::listAsGraph()` in [Control.h](../../src/core/module/Control.h); the ArtPoll reply named after the device in [NetworkReceiveEffect.h](../../src/light/effects/NetworkReceiveEffect.h); the move in [MIGRATING](../reference/MIGRATING.md) ("Discovery moves from Network to Devices"). -Looked at before built (product owner, 2026-11-09): a throwaway listener on the Mac, not committed, reported what the bench network carries on the well-known ports, so the data model follows the real traffic rather than a guess. Three minutes, one ArtPoll and one SSDP search every 30 s, presence read from the desktop's own list: +#### 2. This device's flows and services ✅ (OSC, MQTT and Hue on the P4, PanelCard's ColorLight service on the NanoPi, 2026-11-17) -- **Presence:** 11 MoonLight boards and a WLED, the WLED heard every ~30 s. -- **SSDP:** the Hue bridge (.143), its `SERVER` header naming it and its `USN` carrying a stable uuid, found with no HueDriver configured. -- **Art-Net:** two MoonLight boards answered ArtPoll (.156, .204, both running Network Receive), each as `MoonLight` / `MoonLight NetworkReceive`, hard-coded in [NetworkReceiveEffect.h](../../src/light/effects/NetworkReceiveEffect.h), so a poll cannot tell them apart. Their reply carries a MAC. -- **Audio sync:** .221 sending both formats to their multicast groups, 42 packets a second, about 5.5 KB/s. -- **mDNS:** six senders that are no MoonLight board (phones, computers, media devices), unnamed until their records are parsed. -- **Nothing** on sACN, DDP, OSC, RTP-MIDI or WLED's notifier: idle, or unicast between others, which the Mac cannot see. The desktop's DDP stream to .156 was running all along, unseen by the listener and reported by both ends. +Landed in: what each module reports, and why the socket counts nothing, in [NetworkUse.h](../../src/core/module/NetworkUse.h); the services, rows and edges the map draws, and the sighting rule, in [DevicesModule.h](../../src/core/system/DevicesModule.h) ("The map's edges come from the modules"); each reporter in its own module's header; Home Assistant and every other client of the WLED API as a named inbound flow in [HttpServerModule.h](../../src/core/system/HttpServerModule.h); pinned in `unit_DevicesModule_flows.cpp` and `unit_HttpServerModule_apply.cpp`. -What it means for the data model: +- **Verified by:** NetworkSendDriver to a bench board and audio sync from a WLED, both drawn; .101's RTP-MIDI session shown on its node. -- **One device, several sightings.** A node's kind comes from several sources, a MoonLight board by presence and by ArtPoll, so nodes merge by address. -- **Identity.** A device is its MAC. MoonLight's and MoonBase's presence packet appends the base MAC, the same on every interface; WLED's 44 bytes carry none, but WLED, MoonLight and ESPHome name it in their mDNS records; ArtPollReply has one and SSDP a uuid. Discovery keys a row by the MAC once a source names it (built 2026-11-10: the packet, the ESPHome and WLED records; Tasmota's status page and Hue's SSDP id to come). -- **A multicast stream goes to a group, not a device.** The map has a node for a multicast group, its listeners joined to it. -- **Services beside flows.** A device offers things while nothing flows: .101 shares MIDI as an RTP-MIDI session on port 5004, waiting for a desk and a peer. So a node carries its services (an RTP-MIDI session, Art-Net or DDP reception and its universes, OSC in, an audio sync send, its page) as well as its flows, and a session's participants become flows once they join. Foreign RTP-MIDI hosts announce themselves as `_apple-midi._udp` over mDNS. -- **Both ends of a flow report it.** The desktop's DDP to .156 shows in the sender's status (1505 packets/s) and the receiver's (1500, 0 lost, from MM-Desktop), as free text today. Matching the two finds a broken link: a sender reporting a flow its receiver does not. -- **A defect:** the ArtPollReply names become the device's own name (step 1). +#### 3. Peers' flows: the whole MoonLight map ✅ (bench 2026-11-19: a flow between two other devices on a third board's map, drawn broken when its receiver stopped) -## The message design +Landed in: the peers' maps merged, a broken link and an unreachable peer in [DevicesModule.h](../../src/core/system/DevicesModule.h) ("The peers' maps merge into this one") and [devicemap.js](../../src/ui/devicemap.js); the page's port and the MAC in the presence packet in [WledPacket.h](../../src/core/system/WledPacket.h); a link-local address refused and the list never saved in DevicesModule.h ("Ports and addresses", "Aging out"); older firmware no longer listing a board on port 8080 in [MIGRATING](../reference/MIGRATING.md) ("Discovery moves from Network to Devices"). -No new protocol (product owner, 2026-11-09): each kind of message already has its textbook one, and what is designed is the rules on top. The only new formats are documents: the summary and flows JSON, and the OSC addresses of group state. +- **Verified by:** the desktop's DDP to .156 drawn on a third board's map; the receiver stopped and the link drawn broken. +- **Tested (2026-11-19), by the product owner and the agent.** The desktop's map showed MM-NanoPi (.156:8080) with its flows and MM-Pi5-1 (.204:8080) with its lights, MM-P4-hdmi once, every label inside its cell. On .221's map the desktop's DDP to .156 was drawn, neither end being .221; .156's NetworkReceive disabled over the API, .221's map drew the link red and dashed within seconds; enabled again after. -| Kind | Loss | Protocol | -|--|--|--| -| **Streams**: light, audio, video | acceptable, latest wins | Art-Net, DDP, E1.31, the MoonLight audio sync format, each its own | -| **Group state**: preset, controls, clock, roles | repaired by the next send | OSC on the groups' one well-known multicast address: the definitions (step 6), the shared controls and looks (step 7) | -| **Requests**: summary, flows, identify, firmware | must arrive | REST over TCP, every peer answering cross-origin (`Access-Control-Allow-Origin: *`) | -| **Discovery** | acceptable | presence, ArtPoll, SSDP, mDNS | +#### 4. A peer's summary, and identify ✅ -- **Audio sync stays its own packet.** It is a stream at ~42 packets a second, WLED-compatible in one format; carrying must-arrive state in it would tie that state to a lossy stream. -- **State, not commands.** A group's state is published as absolute values ("preset X, apply at T"), on a change at once and then repeated, so a lost packet is repaired by the next repeat and a member that joins or reboots catches up. Applying a value twice changes nothing, so no acknowledgements are needed and a swarm costs one multicast send. Presence already works this way. REST stays for what is not state: an update, identify, reading a peer. -- **Apply at a time.** OSC bundles carry an NTP timetag, OSC 1.0's own "execute at T". [OscPacket.h](../../src/core/services/OscPacket.h) recognizes bundles and rejects them today, "a timed batch we have no use for yet"; step 9 gives them their use. -- **No races, no loops:** - - **Leaderless: every member may write** (product owner, 2026-11-09). Each value is a last-writer-wins register (Shapiro et al., *Conflict-free Replicated Data Types*, 2011): it carries a timestamp and its writer's device id, a device applies only a newer one, and a tie goes to the higher id. Every member converges on the same value whatever order packets arrive in, so concurrent edits do not race: two people on one slider at once, one edit wins and all agree which. The timestamp is a hybrid logical clock (Kulkarni et al., 2014), which orders edits before step 9 syncs the clocks and by real time after. Duplicates, reordering and stale values are ignored for the same reason. Today's shared controls are peer-to-peer with an echo check ([OscService.h](../../src/core/services/OscService.h)); whether two boards changing one control at once can end apart is a hypothesis, which the registers settle either way. - - **No forwarding.** A device publishes only what it wrote itself and never re-sends what it received, so no loop can form. - - **Coalescing.** A slider drag publishes at most ~20 times a second, the latest value winning. - - **Repeats by Trickle** (RFC 6206, the IETF's for low-power networks, used by RPL): a member stays quiet when it has lately heard others send the same version, its interval doubling while the group agrees and snapping back to fast on any difference. Traffic stays near flat as a group grows, where every member repeating every second would cost each device N packets a second. - - **Roles, derived, not elected.** A job needing one doer, the clock reference, the member that renders in step 10 and sends keyframes in step 13, the one running a group update, goes to the lowest live device id unless the owner pins one. Every member computes the same answer from the same membership, so no messages are exchanged to agree. -- **MoonOrbit carries the same messages.** One WebSocket per device to the relay carries the same OSC packets and request/response pairs, so a group behaves alike on the LAN and over the internet. +Landed in: the [Devices page](../moonmodules/core/system.md) ("The summary is `/api/system`", a node's popup, its firmware and identify); the summary's fields in [the REST API](../reference/openapi.yaml) (`/api/system`); keyboard-reachable nodes and the duplicate `Content-Length` refusal (CodeRabbit, PR #242) in `buildListGraph` and [HttpHead.h](../../src/core/util/HttpHead.h), each pinned by a test. -## One effect across the group: the complexity levels +#### 5. What drives each device's lights ✅ -An effect spreads over a group as easily as its output can be computed on each device from what every device shares. The levels, easiest first, with the 83 effects' shape surveyed on 2026-11-09 (18 draw from a `Random8` stream, 14 from `hashInt`; all animate off `elapsed()` in [EffectBase.h](../../src/light/effects/EffectBase.h), so one clock swap reaches all): +Landed in: the [Devices page](../moonmodules/core/system.md) (the emoji a node's lights show); the `outputs` field in [the REST API](../reference/openapi.yaml) (`/api/system`), from `DriverBase::drivesLights` and `windowLength`. -| Level | The effect's output depends on | Examples | Each device needs | -|--|--|--|--| -| 1. Position and time | where a light sits and when | the shaders (Truchet, Raymarch), Plasma, Noise, Rainbow, Metaballs | the synced clock, a phase anchored to it, and its region of the group canvas | -| 2. Shared input | level 1, plus an input | AudioSpectrum (a band per device), the audio effects | level 1, plus the same input frames: audio sync from one sender, which exists | -| 3. Randomness, no memory | level 1, plus a random draw per frame | sparkles, twinkles | a draw from (seed, group tick, light), not from a stream | -| 4. Memory of its own past | its previous frames | Fire, Particles, Bouncing Balls, Water Ripple | the same simulation steps at the same times: lockstep | -| 5. Chaos, coupled across the canvas | its previous frames, every cell on its neighbors | Game of Life, ripples crossing devices | lockstep, plus detecting and repairing a divergence | -| 6. Live input on one device | a gamepad or a touch | MoonInvaders, Pong | the input as group state, applied at a time | +### Act as a group -What breaks each level, which synced clocks and a shared seed alone leave open: +#### 6. Device groups, shared by their members ✅ (bench 2026-11-19: a five-member group edited from a board, a member missing in MoonBase) -- **Level 1: a phase starts at the device's first frame.** 25 effects animate through `BeatPhase` ([math16.h](../../src/core/util/math16.h)), shaders and Plasma among them. It accumulates `dt × rate` in exact 64-bit integers, so the frame rate cannot make it drift, but it counts from the first frame each device rendered, and a speed change lands at each device's next frame boundary. Two devices are then offset by whatever separated their starts and their frames, for good. The fix is one class: the phase anchored to the effect's start on the synced clock, and a rate change applied at its "apply at T" with the integral split at T exactly, which makes the phase an exact function of synced time on every device. That one change moves all 25 to level 1. -- **Level 1: shaders are level 1 by construction.** A pixel's color is a function of where it is and when, with no state ([power-functions.md](../moonmodules/light/power-functions.md), [shader.h](../../src/light/powerfunctions/shader.h)); Truchet already turns its tiles by `hashInt` on the cell index "so two devices agree on tile (7, 3)". Shaders also gain the most, the heaviest effects per pixel, each device computing only its own share. Three things to watch: - - **Coordinates from the group canvas.** `uv` centers and scales by the panel's own short side; across a group it takes the canvas's size and the light's position in it, or every device draws the whole pattern on itself. - - **Floats are harmless here.** A last-bit difference changes one pixel's color by at most a step and never accumulates, since nothing carries over to the next frame; it matters only where state does. - - **Not every member can run every shader.** Raymarch compiles only on chips with a hardware FPU, so a group checks from the members' summaries that each one has the effect, and streams it to a member that does not. -- **Level 3: a random stream drifts with the frame rate.** A `Random8` stream advances per draw, so a board at 60 fps and one at 45 fps are at different points in it within a second. The textbook answer is a counter-based generator (Salmon et al., *Parallel Random Numbers: As Easy as 1, 2, 3*, 2011): a draw is a hash of (seed, tick, light), which `hashInt` already is, with the tick derived from the synced clock (`tick = synced time / group period`), never the device's frame count. -- **Level 4: a simulation steps per frame.** [FireEffect.h](../../src/light/effects/FireEffect.h) advances once per rendered frame with no time gate, so its state drifts with the frame rate even with the same seed. The textbook answer is a fixed timestep (Fiedler, *Fix Your Timestep*): step n happens at synced time t₀ + n·dt on every device, a late device running the missed steps, and rendering shows the latest step. -- **Level 4: memory that forgets heals itself.** A fade, a blur trail, Fire's cooling and WLED Soap's smoothing each shrink a difference every step (a contraction), so two devices that start apart converge within seconds once their steps and random draws agree. Memory that keeps, a particle's position or a ball's speed, carries a difference forever. So level 4 is two cases: forgetting needs lockstep and level 3 randomness but no keyframe; keeping needs a keyframe too. -- **Levels 4 and 5: floats can differ across chips.** A fused multiply-add on one compiler and not another changes a last bit, and a simulation carries that bit forward, a chaotic one growing it into a different picture. Lockstep effects use integer math, which the coding standards already prefer; Bouncing Balls uses `float` and is checked when it is converted. -- **Level 5: chaos amplifies any difference.** In Game of Life one wrong cell grows into a different board. The textbook answer, from the lockstep netcode of real-time strategy games, is desync detection: each device publishes a checksum of its state every N steps as group state, and a mismatch is repaired by a keyframe, the state copied from the render role holder. Game of Life already fingerprints its board with crc16 ([GameOfLifeEffect.h](../../src/light/effects/GameOfLifeEffect.h)), and its board is bits, about 2 KB at 128×128, a cheap keyframe. -- **Levels 4 and 5: a late joiner has no past.** An effect whose memory keeps, or a level 5 one, joins by keyframe, or by replaying from the effect's start where that is short; one whose memory forgets joins by waiting a moment. -- **Levels 4 and 5: simulating the whole canvas costs memory.** In lockstep each device simulates the whole group canvas and shows its region, the simplest correct shape; halo exchange (each simulating its region and trading border cells per step, the HPC answer) needs a reliable message per step and stalls on WiFi, so it is not used. A device that cannot hold the whole canvas degrades to being streamed. +Landed in: [DeviceGroup.h](../../src/core/system/DeviceGroup.h) (its members by MAC, its id, missing members, the box on the map, the replication and what it costs), [DeviceGroupSync.h](../../src/core/system/DeviceGroupSync.h) and [Hlc.h](../../src/core/util/Hlc.h); a member's identity in [mooncore.md](../explanation/architecture/mooncore.md#device-name-one-identity-every-network-name-derives-from-it). -**Each effect declares its level**, a virtual on EffectBase, and the group picks the strategy from it. The default is the most conservative, streamed, so an effect nobody has looked at is always correct and opts into a cheaper level once checked. +- **Roles.** A role is pinned with the role's first user (step 8's waves, step 9's clock reference); until then each derives from the lowest live device id ([the message design](#the-messages)). +- **Tested (2026-11-18), by the product owner and the agent.** rtpDGroup made on .101 with .221 as a member: .221 created it within seconds, and the Mac's map, no member, drew its box. After the review fixes (781cba15): a rename on .221 reached .101 within 3 s and one on .101 came back, each carrying its writer's stamp; the group deleted on .221 was gone from .101 within 3 s, both saved its marker, and after both rebooted it stayed gone. +- **Tested (2026-11-19), by the agent.** DesktopGroup grown to five members (the desktop app, .221 P4, .122 classic, .115 C3, .203 Teensy 4.1) by two rows added on .115 over the API: the desktop, .221 and .122 held the same members within seconds, and the Teensy took the group as soon as step 8's update gave it the module. .122 sent into MoonBase showed missing on the desktop at once (5 members, 1 missing) and present again 24 s after its app booted. Two defects met and fixed: a MAC sent to the row's `device` field read as an index from its leading digits and added the wrong boards (the field now refuses a quoted value; a MAC goes to `mac`), and a member in MoonBase counted as present, since MoonBase sends the same presence packet (present now means heard running MoonLight). +- **Verified by:** two groups on the bench, made on one board and edited on another, both maps drawing the same boxes; a member unplugged and shown missing; a unit test feeding the element set every arrival order. -**Pinned by one test for every effect: split equals whole.** On the desktop the effect renders once at full canvas and once as two half-canvas instances at different frame rates on one synced clock; at each checkpoint the halves equal the whole. A declared level that fails it is a defect. +#### 7. A group acts as one ✅ (bench 2026-11-19: five members, a change everywhere within 3 s, simultaneous changes converging, a restarted member catching up) -**Prior art: WLED-MM SuperSync** (October 2023, in the local clone `MoonModules/WLED-MM`, commits `37e162472` to `71abd117b`; upstream WLED took the clock half as "Use strip.now instead of millis() in effects AKA supersync", `58eb151ba`): +Landed in: a group's name in `receivers`, resolved each second, and the shared bound `kMaxReceivers` in [NameResolver.h](../../src/core/util/NameResolver.h) and [ReceiverList.h](../../src/core/util/ReceiverList.h); membership as the one setting, what travels, last writer wins per parameter, a look by its name and the catch-up in [DeviceGroupSync.h](../../src/core/system/DeviceGroupSync.h) and [SharedControls.h](../../src/core/system/SharedControls.h); unicast to each member in [DeviceGroup.h](../../src/core/system/DeviceGroup.h) ("Reaching the members"); MidiService's one participant in [control surfaces](../reference/control-surface.md); the move from OSC leader and followers to membership in [MIGRATING](../reference/MIGRATING.md). -- **The clock:** `strip.now`, WLED's time with its UDP-synced offset, in place of `millis()`. -- **Levels 1 and 2, done by hand per effect.** Soap, Octopus, Crazy Bees and Polar Lights replaced per-frame accumulation (`noise += mov` each call, `step++`) with a function of `strip.now`: frame count out, time in. MoonLight's effects already animate off `elapsed()`. -- **Level 3, begun:** a random seed set once at the effect's start, fixed (`random16_set_seed(535)`) or from the clock (`random16_set_seed(strip.now)`). It gives the same start where devices start together, and a stream that drifts with the frame rate after it, the gap step 12 closes. -- **The group canvas by configuration:** every node's matrix set to the whole canvas, its panel 0 placed at its offset; a SuperSync checkbox per effect made a node compute its panel only. -- **Fleet consistency:** the Nodes popup fetched each node's info and config, compared version, build, brightness scale, fps, matrix size and panel layout, marked "update needed", and a SuperSync button pushed the LED and light config to each node and rebooted it. -- **Two code paths per effect**, deterministic only while sync was on (`notifyDirect`), kept to compare both modes. +- **Tested (2026-11-18), by the product owner and the agent.** DesktopGroup: the desktop app, .221 (P4), .122 (classic) and .115 (C3), each with OSC listening and feedback to `DesktopGroup`. An encoder changed on any of the four reached the other three, the first device read within 3 to 20 ms; brightness changed on .122 reached every member once each mapped `fader1` to `Drivers.brightness`. A member restarted 40 s before still resolved its group to nobody, and a change made on it went out once its Discovery list had heard the others. A group member was sent to on the feedback port, where no member listens; it is sent to on the OSC port since. +- **Tested (2026-11-19), by the agent**, on the five members above with their OscService modules removed, on dev.153 and later: `encoder1` (every member's `Drivers.palette`) set on the Teensy over the API reached all five within 3 s. Five races, two members setting `encoder1` to different values in the same instant (P4 and C3, classic and Teensy, desktop and P4, C3 and classic, Teensy and desktop), each ended on one value on all five. The C3 restarted while the desktop changed the group's value came back on the newer value, not its saved one. +- **Verified by:** DesktopGroup, the desktop app, .221, .122 and .115 with their OscService modules removed: a fader, a switch and an encoder on any member followed by the others, members with different slots on one parameter converging; an effects preset pressed on one applied on the others, a driver preset staying; a member restarted taking the group's newer values; two members moving one control at once ending on one value. -What it teaches this plan: one deterministic path for every effect, never a sync mode beside a normal one; the region from the group canvas, not a per-node matrix configured by hand; the consistency check as group state rather than a config push and a reboot; and level 4's split into memory that forgets and memory that keeps, which Soap's smoothing showed. +#### 8. Update a group ✅ (bench 2026-11-19: three members behind, updated in waves in two minutes) -## Steps +Landed in: the waves, the popup and its skipped members in [DeviceGroup.h](../../src/core/system/DeviceGroup.h) ("Updating the members") and [devicegroup.js](../../src/ui/devicegroup.js), pinned in `device-group.test.mjs`; one peer from its popup in [DevicesModule.h](../../src/core/system/DevicesModule.h) ("Behind, and defaults") and the shared install flow in [deviceinstall.js](../../src/ui/deviceinstall.js). -### See the network +- **Verified by:** a bench group updated from one board's page, a Teensy among them. +- **Tested (2026-11-19), by the product owner and the agent.** DesktopGroup from .115's page, three of five members behind latest (dev.153): .122 (dev.150, through MoonBase) alone at 09:47, about 50 s; then the Teensy (dev.146, no TLS, so download and upload) and .221 (dev.150) together; all three on dev.153 by 09:48:56. The popup kept saying "3 of 5 members are behind" throughout; it now counts down as each member ends well and closes once every member is updated, staying open with the reason on a failure. -#### 1. Devices on top, drawn as a map ✅ +#### 9. Clock sync 🧪 (built 2026-11-19: the fleet clock, apply at T, effects on fleet time; test: the spread under 1 ms and the line sweep across boards) -Landed in: the [Devices page](../moonmodules/core/system.md) (its card, the map, growth and what memory refuses, from `DevicesModule.h`); the allocation rule and `platform::fitsWithReserve` in [moonmodule.md](../explanation/architecture/moonmodule.md); the list drawn as a graph, `ListSource::listAsGraph()` in [Control.h](../../src/core/module/Control.h); the ArtPoll reply named after the device in [NetworkReceiveEffect.h](../../src/light/effects/NetworkReceiveEffect.h); the move in [MIGRATING](../reference/MIGRATING.md) ("Discovery moves from Network to Devices"). +- Absorbs the backlog card, with two changes: the reference is the clock reference role, not a configured leader; and a member measures its offset by the NTP exchange (four timestamps, request and reply), not a one-way send, since the round trip is what measures the delay. The same exchange then works over any transport, the LAN and step 15's relay alike. Aim: under a millisecond on the LAN. +- **No NTP module** (product owner, 2026-11-19): NTP proper is wall time from an internet server, about 10 ms; a group needs its members to agree with each other, whatever the internet does. Only NTP's on-wire exchange is taken (RFC 5905 § 8). +- **One fleet clock** (product owner, 2026-11-19): every MoonLight device on the network shares one time base, the reference the lowest live device id Discovery hears, so a device in two groups has one clock. Each device asks the reference once a second until synced, then every 16 s, as NTP's poll interval backs off. +- **Where it lives.** The math in `core/util/ClockSync.h`, pure and host-tested: offset and delay from t1 to t4, NTP's clock filter (the last eight samples, the one with the least delay trusted), a small correction slewed and a large one stepped, so the clock never runs backwards under an effect. The exchange on its own socket and a small receive task, as `/moonlight/clock` by unicast to the reference: a timestamp taken at a 20 ms read would be up to 20 ms late, so t2 and t4 are taken as the task wakes on the datagram, and t3 just before the reply goes out. One reading, `FleetTime::nowUs()`, for the hybrid clock's physical part, step 11's `elapsed()` and "apply at T". The group's status names the reference and the spread. +- **Apply at T, a fixed lead first** (product owner, 2026-11-19): a shared control or look goes out in an OSC bundle whose timetag is 100 ms ahead; a member holds it until then, and one not synced applies it on arrival. The lead is derived from the measured round trips once the bench shows WiFi's. +- **A change applies at a time, not on arrival**: a preset or control change is stamped "apply at T", a little in the future, in an OSC bundle's timetag, which OscPacket now accepts. Members that receive it at different moments switch together, which is what lets latency, and step 15's internet, stay invisible. +- With step 7's shared controls, a group now shares both its controls and its time: the base every level of the next phase builds on. +- **If the exchange is not precise enough over WiFi: the access point's own clock.** Every 802.11 access point broadcasts its 64-bit TSF timer (Timing Synchronization Function) in its beacons, and every station on it keeps its copy aligned to within microseconds; the ESP32 reads it with `esp_wifi_get_tsf_time()`. It covers the WiFi devices on one access point, not Ethernet boards, desktops, or devices on other access points or mesh nodes, which keep the exchange. The textbook combination keeps the fleet clock as the one time base and adds TSF as a sharper source in `ClockSync.h` where it applies: the exchange measures the offset between TSF time and the reference, and TSF then holds the WiFi members together far more tightly than WiFi round trips allow. Anjin's "Wifi comes with gifts" (Discord, 2026-11-19) may point here; his weekend work on time sync is to ask about before building it. +- **Measured by:** the offset spread across a bench group; stop rule named before the first run. +- **Built (2026-11-19).** The arithmetic in [ClockSync.h](../../src/core/util/ClockSync.h) and fleet time for every task (`FleetTime`, two published slots, so a reader preempting the writer never waits); the service in [FleetClock.h](../../src/core/system/FleetClock.h), held by the Devices container beside the hybrid clock, its beacon on 239.255.77.80 and its exchange on port 65511, answered to the asking port; the hybrid clock's physical part and effects' `elapsed()` on fleet time, so `LinesEffect` is the line sweep; shared controls and looks sent as OSC bundles applying 100 ms ahead, held by a member on the fleet clock (`DeviceGroupSync`). The measurement is `moondeck/diag/measure_fleet_clock.py`. One open point: the device where a change is made applies it at once, so it leads the members by the lead time. +- **Tested (2026-11-19), by the agent.** The desktop, .122 (classic) and .115 (C3) on one reference, MM-dn2-2, within seconds of booting. First measurement over WiFi, 60 exchanges each: a spread of 5.1 ms with ±4.1 ms measurement error, the boards' best round trips 4 to 8 ms, so neither the sync nor the measurement resolves below a few milliseconds there. Next: the boards' WiFi power save, then TSF. +- **Verified by:** a vertical line sweeping across boards side by side, the textbook display-sync test (genlock checks on video walls): straight across a seam when the clocks agree, a step when they do not, the step's width over the line's speed being the clock error. +- **The bench for it (product owner, 2026-11-19):** nine ESP32 boards on WiFi, each driving one 16x16 panel, laid out 3x3. LinesEffect, axis x, on every board: each sweeps its own panel on fleet time, so in step the nine lines move together and a board behind shows its line trailing. One line across the whole 3x3 is step 11's group canvas. -#### 2. This device's flows and services ✅ (OSC, MQTT and Hue on the P4, PanelCard's ColorLight service on the NanoPi, 2026-11-17) +### One effect across the group -- **Modules report, the socket counts nothing** (product owner, 2026-11-09). Each network module reports its services and its flows, protocol, direction, peer and rate, from the counters it already keeps for its status line, the way every module reports its memory and tick time. Counting at the UDP socket would have caught every module, but a per-peer table there runs on every packet from several tasks, needing a lock or an allocation in the send path; reporting adds nothing per packet. A module that sends without reporting stays off the map. -- The services the device offers, from the modules that offer them: an RTP-MIDI session, Art-Net or DDP reception, the audio sync port, the web page. -- The map draws this device's flows as edges labeled with protocol and rate, a multicast stream to a group node, and its services in its tooltip and detail. A remote address not in the list gets a row of its own, so a sender nobody configured shows up. -- **Verified by:** NetworkSendDriver to a bench board and audio sync from a WLED, both drawn; .101's RTP-MIDI session shown on its node. -- **Built (2026-11-10).** `NetworkUse` ([NetworkUse.h](../../src/core/module/NetworkUse.h)) and `MoonModule::reportNetworkUse`, reported by NetworkSendDriver (frames a second to each receiver, the group or the broadcast address), NetworkReceiveEffect (its ports, the sender it receives), AudioService through AudioSync (its port and followed sender, or each format's group or each host), MidiService (the RTP-MIDI session and its peer) and HttpServerModule (the page's port). Discovery collects them once a second through `Scheduler::instance()`, gives each peer a row, `group` for a multicast group or the broadcast address (`DevType::Group`), and writes each row's edges, this device's outbound flows from its own row and a peer's inbound flow from the peer's row, and this device's services. A flow from a peer, or through a group, is a sighting; one sent to a silent receiver is not, so a dead receiver still ages out. Tests ([unit_DevicesModule_flows.cpp](../../test/unit/core/unit_DevicesModule_flows.cpp)): a peer's row, a group's, the outbound and inbound edges, the services, a disabled module reporting nothing. On the desktop: its row's edge to .156, "DDP 46 fps", and its service "HTTP 8080". Then OscService (its port, the client, where feedback goes), PanelCardDriver (a "ColorLight" service, raw Ethernet having no IP peer), HueDriver (the bridge) and MqttModule (the broker, when set as an address, since the connection resolves a name inside and keeps no address). -- **Built (2026-11-10), at the product owner's ask: Home Assistant on the map.** HttpServerModule reports each client of the WLED API (`/json`, `/win`, `/presets.json`) as an inbound flow, "WLED API 6 req/min" over the last whole minute, from a table that grows as a client first asks and drops one quiet for five minutes. A flow may name its peer (`NetworkUse::flow`'s `peerName`), and the client's User-Agent product token does: `HomeAssistant/` is Home Assistant (RFC 9110 § 10.1.5). The name labels a generic row only, never a MoonLight board's or a WLED's. Tests: the clients, the name, the rate, a quiet client dropped; a named peer listed by its name. +#### 10. One effect streamed: correct for every effect ⏳ -#### 3. Peers' flows: the whole MoonLight map 🧪 (test: .156 and .204 on port 8080 merged, MM-P4-hdmi once, the bench's nodes readable) +- The member holding the render role renders the whole group canvas and sends each member its region over DDP; the group sets up NetworkSendDriver and the members' Network Receive. Every level is correct this way, so it ships first and stays the fallback. +- Bounded by bandwidth and the LAN: a large canvas or MoonOrbit needs the levels of the next steps. +- **Verified by:** one effect running across three bench boards as one picture. -- Each device's graph is in its own state; the page reads each MoonLight peer's graph over REST and merges them, so flows between two other MoonLight devices appear. -- Both ends of a flow report it, so a flow its receiver does not report is drawn as a broken link. -- A peer that does not answer keeps its node, marked unreachable. A few requests run at a time, so a swarm loads progressively rather than all at once. -- **Verified by:** the desktop's DDP to .156 drawn on a third board's map; the receiver stopped and the link drawn broken. -- **Built (2026-11-10).** In [app.js](../../src/ui/app.js): while the map is on screen, every node of the same kind as this device's own reads the same list at its `url`, through `GET /api/modules/`, four at a time with a three-second timeout, again five seconds after the round ends. Devices a peer sees and this one does not join the map. In each graph a device's own row holds what it sends and another row what it receives from that row, so a link is drawn once, the sender's label winning, and drawn broken (red, dashed) when its receiver, a peer that answered, reports nothing arriving. A peer that does not answer is drawn unreachable; one answering without the list, older firmware, is drawn as it was. No firmware change: the module route and its list detail serve it. -- **Bench (2026-11-10), seven boards on this branch** (.221, .101, .132, .158, .203, .122, .207) and the desktop: the desktop's "E1.31 44 fps" to MM-NanoPi and .221's "audio sync 23 fps" to 239.0.0.1 and 239.0.0.2 drawn, the other boards as nodes without flows, which is what they run. Three faults seen. MM-NanoPi (.156, port 8080) and MM-Pi5-1 (.204, port 8080) are drawn unreachable, the port gap below. MM-P4-hdmi appears twice, once at a link-local 169.254.x.x address, which no peer can reach (RFC 3927). Unlinked nodes are pushed to the box's edges, where their labels overlap and are clipped, and a node shows its name only. -- **Built (2026-11-10), after the bench.** *The port:* a MoonLight device whose page is not on port 80 appends the port after the 44-byte presence header ([WledPacket.h](../../src/core/system/WledPacket.h)), read from the web server's own HTTP service report; WLED reads 40 bytes or more and ignores the rest, and absent means 80, as in an http URL. mDNS's SRV record was the other carrier, but Discovery never queries mDNS (it starved the ESP32's own advertising) and the desktop and Linux builds advertise none. A row keeps the port and its `url` carries it. *Link-local:* Discovery gives no row to 169.254.0.0/16 (RFC 3927). *Not saved:* the list is runtime state, rebuilt within seconds of boot, so disabling Discovery frees it and saves nothing (pre-merge review). *The layout:* the linked nodes, this device among them, are placed by force; the others sit in a grid beneath, by name, the picture growing with them, each label kept inside. *On each node:* its address, its version, and what it offers, from each MoonLight peer's own row, which now carries its version and firmware. Older firmware does not read a 46-byte presence, so it no longer lists a device on port 8080 ([MIGRATING](../reference/MIGRATING.md)). +#### 11. Group canvas: position, time and shared input (levels 1 and 2) ⏳ -#### 4. A peer's summary, and identify ✅ +- The group has one virtual size and each member holds its region in it; `elapsed()` reads the synced clock; `BeatPhase` is anchored to the effect's synced start, its rate changes applied at their synced time; `uv` takes the canvas. Every member renders its own region, and no pixels travel. +- A member lacking the effect (Raymarch on a chip without an FPU) gets it streamed, as step 10 does. +- The level declaration on EffectBase, the group's choice from it, and the split-equals-whole test; level 1 and 2 effects declared, each passing it. +- **The demo effect: a ball crossing the screens.** An elastic ball in closed form, x a triangle wave over the canvas width, y a parabola repeating each bounce, both functions of synced time: level 1, no state, each board drawing it while it is in its region. Regions placed at the boards' real positions, gaps included, so the ball takes the right time to fly between screens. BouncingBalls loses energy and relaunches, which carries memory; this is its elastic, sideways twin. +- **Verified by:** the ball crossing three bench boards with no jump at a seam; Plasma, Truchet and AudioSpectrum across them as one picture, the spectrum's bands split over the boards; a speed change mid-run keeping the boards in phase; a board started a minute late in phase from its first frame. -Landed in: the [Devices page](../moonmodules/core/system.md) ("The summary is `/api/system`", a node's popup, its firmware and identify); the summary's fields in [the REST API](../reference/api.md) (`/api/system`); keyboard-reachable nodes and the duplicate `Content-Length` refusal (CodeRabbit, PR #242) in `buildListGraph` and [HttpHead.h](../../src/core/util/HttpHead.h), each pinned by a test. +#### 12. Randomness from the group tick (level 3) ⏳ -#### 5. What drives each device's lights ✅ +- A group tick from the synced clock; effects drawing per-frame randomness move from a `Random8` stream to `hashInt(seed, tick, light)`. +- **Verified by:** the split-equals-whole test for each converted effect; a sparkle effect on boards at different frame rates showing the same sparkles. + +#### 13. Lockstep simulation (levels 4 and 5) ⏳ + +- A fixed timestep from the synced clock, in core, which a simulating effect steps by; integer math; the whole canvas simulated on each device. +- An effect whose memory forgets (Fire's cooling, a fade trail) converges by itself and needs nothing more. One whose memory keeps, and every level 5 one, publishes a state checksum every N steps as group state; a mismatch is repaired by a keyframe from the render role holder, and a late joiner joins by keyframe. +- Fire (forgets), Particles (keeps) and Game of Life (chaos) first, one of each case, the rest by the test. +- **Measured by:** how often a bench group diverges over an hour, and the keyframe's cost; stop rule named before the first run. +- **Verified by:** Game of Life across three boards as one board for an hour, a board rebooted and rejoining by keyframe. -Landed in: the [Devices page](../moonmodules/core/system.md) (the emoji a node's lights show); the `outputs` field in [the REST API](../reference/api.md) (`/api/system`), from `DriverBase::drivesLights` and `windowLength`. +#### 14. Live input across the group (level 6) ⏳ -### Before groups +- A gamepad or touch on one member becomes group state, stamped "apply at T" a little ahead, so every member applies it at the same step: the delay-based lockstep of networked games. Where the delay shows, the effect stays streamed. +- **Pong across two boards**, a paddle on each and the ball crossing between them. Its attract mode re-rolls each paddle's aiming error per exchange; drawn as a hash of the exchange number, the rally is deterministic (levels 1 and 3), and a gamepad per board makes it level 6. +- **Verified by:** Pong played with a gamepad on each of two boards, the bench's Xbox Series pad and 8BitDo Zero 2, each paired with the computer showing its board's page ([the gamepad plan](Plan-20261001%20-%20Gamepads%20and%20MIDI%20desks%20(2%20of%207).md) owns how a pad reaches a board); MoonInvaders from one board's gamepad across three. -Planned by the product owner (2026-11-10) after step 5 and before step 6, each shipping on its own. +### Beyond the LAN -#### Every Hue bridge on the map ⏳ +#### 15. MoonOrbit: groups over the internet, through MoonCloud (stretch) ⏳ -- Not only a bridge a HueDriver points at: Hue's own discovery, mDNS `_hue._tcp`, through a passive listen to mDNS announcements, or SSDP on port 1900. With the rest of the Backlog's More discovery, since they share the mechanism. -- It needs a hardware test: mDNS queries were ended by a pool exhaustion on the ESP32, so the passive listen is a hypothesis until a board runs it. -- **Verified by:** a Hue bridge nobody configured on the bench map. +Devices in different places, owned by different people, in one group: a show running at several sites, or friends' installations running as one. MoonOrbit is a working name. + +- **A relay, not a stream.** Only the clock and the changes travel: the controls, presets and "apply at T" stamps. Each device renders locally, so internet bandwidth stays small and latency hides behind step 9's scheduled changes. Pixels over the internet stay out of scope, so a group across sites runs effects at levels 1 to 6 and step 10's streaming stays within each site. +- **Outbound only.** A device behind a home router cannot accept a connection, so each member opens one outbound WebSocket to MoonCloud, and MoonCloud relays within the group. On Cloudflare that is one Durable Object per group, beside the existing Worker ([MoonCloud](../explanation/architecture/mooncloud.md)). +- **Opt-in and private**, as MoonCloud is today: a group is joined by a secret code the owner shares, the relay keeps no history, and the [privacy policy](../legal/privacy-policy.md) gains what the relay sees. Messages could be signed or encrypted with the group secret so the relay cannot read them; decided in the step. +- **Reach:** ESP32 boards and the desktop, which have TLS; a Teensy joins through an open page until it has TLS. +- **Measured by:** offset spread and change-to-light delay between two sites; stop rule named before the first run. +- **Verified by:** a bench board and a board at another address in lockstep, judged on a shared video call; a Pong rally between the two homes, one pad in each, paired with that home's own computer or phone. + +#### 16. Remote support over MoonOrbit ⏳ + +A user with a problem shares their device, and a helper sees it and replicates the setup on their own desktop: "share your device in MoonOrbit and I will reproduce it". A support session is a group of two over step 15's relay, the helper reading the user's device as a page reads a peer on the LAN. + +- **The user starts it.** A "share for support" button on the user's device shows a short code, valid for a limited time and ended by the user at any moment. The helper enters it in their own MoonLight, and the user's device appears on the helper's map. +- **Read-only by default.** The helper reads the summary, the state, the log and the flows through the relay's request/response. A change on the user's device needs the user to allow writing for that session. +- **Replicate on the desktop.** "Copy setup" loads the user's state into the helper's desktop instance, desktop first as every change is; a module needing hardware the desktop lacks (pins, a driver's peripheral) loads and idles, as it would on a board without that hardware. The copy is the same one the Backlog's "copy a device's settings" needs, so both share one export. +- **Secrets never travel.** Password controls (WiFi, MQTT, API keys) are left out of what is shared and copied; the [privacy policy](../legal/privacy-policy.md) names what a session shows. +- **Verified by:** a bench board at another address shared by code, its setup copied onto the desktop and showing the same effect, no password in the copy, and a write refused until the sharer allows it. + +## Beside the steps + +Each ships on its own, in any order. + +### Updating every kind of device #### Every desktop updates itself through the Firmware module ✅ -Landed in: [the service](../reference/installing-on-linux.md#the-service) (the program it owns, updating as a board, rolling back); the `update` field and the `sha256` of `/api/firmware/url` in [the REST API](../reference/openapi.yaml); the [MIGRATING](../reference/MIGRATING.md) entry (the service, the program's and the settings' new paths). - -Asked by the product owner (2026-11-12): one way to update any MoonLight device, in two steps, this one and the map using it. A desktop prepares the new version beside the running one and a restart activates it, as a board writes its other slot and reboots into it, through the same routes, the same Firmware card and the same asset naming. The line runs between a native install, which updates itself on every OS, and a container, which its host replaces. Settled with the product owner: the Linux package runs a service enabled on install, rollback by systemd, settings moved by hand, the digest from the browser. - -- **The image is the bare program.** The release publishes each desktop's program as `firmware-host---v.bin`, the firmware registry's names (`host-linux-arm64`), beside the archives and installers it ships. A desktop build carries that name as its firmware, where it said `unknown`, so every path that builds `firmware--v.bin` reaches a desktop: the Firmware card and its picker, the map, MoonDeck, Home Assistant's update over MQTT. -- **`otaBegin`, `otaWrite`, `otaEnd` on a desktop.** The image goes to `MoonLight.next` beside the running program. `otaEnd` vets it, an executable of this OS and architecture by its header (ELF, Mach-O, PE), as a board vets an image's magic and chip, and checks the SHA-256 where the request names one; it then keeps the running program as `MoonLight.prev` and renames the new one into place (rename(2), atomic; Windows renames the running `.exe` aside, which it allows). The running process keeps its old image. `http_fetch_to_ota` and `/api/firmware/upload` drive it unchanged, libcurl or WinHTTP fetching for the first. The `hasOta` gates become the update mode. -- **The restart activates it.** `platform::reboot()` starts the program again: on macOS and Linux the new program takes over the process in place (`execv`), same terminal window, same process to systemd or a container runtime, so a reboot is never a start their restart counting sees, with every socket marked close-on-exec where it is made; on Windows the new `.exe` starts in the same console and the old one exits. The reboot button shows on every device. -- **The update mode.** `/api/system`'s `update`, and the Firmware card's: `self` for a native install whose program sits in a folder it can write, `host` for a container, `manual` for a program in a folder it cannot write, such as the package's seed under `/usr/lib`. -- **The Linux package runs a service, and the service owns its program.** Bespoke, with its reason: the textbook Linux update is an APT repository, which needs root to update from the map, and the network-facing service never holds root; a self-owned copy beside a package-tracked seed is how Syncthing, a network daemon, replaces its own binary. New for MoonLight and uncommon on Linux, where the package manager is the norm. The `.deb` ships the program as a seed at `/usr/lib/moonlight/MoonLight`, a `moonlight` system user in the `video` group, and `moonlight.service`, which its post-install script enables and starts the way debhelper's snippets do, skipping where no systemd runs. The unit runs as that user with `StateDirectory=moonlight`, settings in `/var/lib/moonlight` (`XDG_DATA_HOME`), and `Restart=always`. The post-install script copies the seed to `/var/lib/moonlight/bin/MoonLight`, owned by the service user, on every install: an `apt install` is the administrator's explicit choice, so it wins over a self-update, older or newer, and drops `.prev`. The menu entry opens `http://localhost:8080/` instead of starting a second copy on the same port. No sudo rule and no Docker socket, so the network-facing service never holds root. -- **Rollback is systemd's, for the service, by boot assessment.** The swap leaves `MoonLight.pending` beside the program, and the new program deletes it once it has stayed up `kBootStableMs`, the minute a board's boot record uses: systemd's Automatic Boot Assessment, for one program. `ExecStopPost=` runs the rollback after every stop, in the service's sandbox as its user: a pending program whose start failed, a crash before `main` included, is replaced by `.prev` and `Restart=` starts the service again; a clean stop and a confirmed program are left alone, so restarts of a good program never undo it. `StartLimitBurst=3` within `StartLimitIntervalSec=120` stops a program that keeps failing after that. Elsewhere `.prev` stays beside the program, to rename back by hand. First built as `OnFailure=` starting a second, root-run unit; belrik's review (Discord, 2026-11-13) named the stop hook, systemd's own construct for cleanup after a failed start, which took the second unit and root out. -- **The Firmware card on a desktop** installs as a board's does, by URL from its picker or by upload, when the update mode is `self`; otherwise it offers the download. The shared install picker tells a desktop's own image (`firmware-desktop-…bin`) from the installer's download of the same name. -- **Built (2026-11-13).** The image writer, the swap and the restart in [platform_host_update.cpp](../../src/platform/host/platform_host_update.cpp), the header vetting in [ProgramImage.h](../../src/platform/host/ProgramImage.h), the download through the one HTTP client (`httpDownload`), the firmware name from CMake, the sockets closed on exec, `hasOta` gone, the bare programs staged by `package_host.py`, the Firmware card installing on `self` and offering the download otherwise, the picker telling a desktop's program from its download, the live scenario leaving a desktop to restart itself. Tests: `unit_platform_update.cpp`, `unit_ProgramImage.cpp`, `ui-update-asset.test.mjs`, `test_package_deb.py`. CodeRabbit's 15 findings and a CodeQL alert fixed in the PR's second commit, among them the Linux restart, which read `/proc/self/exe` after the swap and found the old program "(deleted)". -- **Removes:** the archive path, the reboot button hidden on desktops, `unknown` as a desktop's firmware name, the Firmware card's desktop download in place of an install, MoonWalk's hand-written unit steps. **Fixed on the way:** the reboot button hidden for a container (`^docker$` never matched `docker-arm64`). From belrik's review: the desktop's audio context picked its backend once at start, so a PulseAudio or PipeWire server started after MoonLight was never found without a restart; the context is now created anew whenever the device list is read while no device is open, and the Linux page names the drop-in that points the service at the server. -- **Costs** a MIGRATING entry: the hand-written unit goes, the program moves from `/usr/bin` to `/usr/lib/moonlight`, and settings move from the old user's home to `/var/lib/moonlight` with one copy command. The Dockerfile copies the program from the new seed path. Unblocks step 8 for a group that mixes boards and desktops. -- **Tested (2026-11-13), by the product owner and the agent.** This Mac: an upload installs and restarts in place, same PID, `.pending` cleared after a minute; a board image refused as no program; a URL with a wrong digest refused, with the right one installed and restarted. .204, the Raspberry Pi 5, from the PR's CI `.deb`: the service runs as `moonlight` holding `CAP_NET_RAW` alone, `update: self`; an upload restarts it in place, same PID, no start counted; `/bin/false` installed as an update rolls back to `.prev` within a second; an `apt install` puts the package's program back and drops `.prev`; HDMI shows the effect under the service user. .156, the NanoPi: the PR's program in its container image says `update: host`, refuses both install routes with 501 and restarts in place. The old hand-written unit and a source-build drop-in had to go first, which the MIGRATING entry covers for the unit. CI on the PR: every desktop compiles, Linux and Windows included. -- **Tested (2026-11-15), the hook rollback on .204** from PR #248's CI `.deb` (7.1.0-dev.106): `/bin/false` installed as the pending program, the service started, the hook ran 37 ms after the failed start ("the new program failed to start, the previous one is back"), `.pending` and `.prev` gone, and `Restart=` brought the previous program back five seconds later; the unit carries `ExecStopPost` and no second unit. On the same install: the service user in `audio`, the Yeti listed and picked by name with ALSA's null device and the duplicate gone, and the Network card "Host: 192.168.1.204 (1000 Mbit)" on the Pi's cable. +Landed in: [the service](../reference/installing-on-linux.md#the-service) (the program it owns and why, updating as a board, rolling back by boot assessment); the image writer, the swap and the restart in [platform_host_update.cpp](../../src/platform/host/platform_host_update.cpp) and the header vetting in [ProgramImage.h](../../src/platform/host/ProgramImage.h); the `update` mode (`self`, `container`, `manual`) and the `sha256` of `/api/firmware/url` in [the REST API](../reference/openapi.yaml); the [MIGRATING](../reference/MIGRATING.md) entries (the firmware id `host--`, the service and the program's and settings' new paths, `/api/reboot` restarting a host in place). + +Asked by the product owner (2026-11-12): one way to update any MoonLight device. A desktop prepares the new version beside the running one and a restart activates it, as a board writes its other slot and reboots into it, through the same routes, the same Firmware card and the same asset naming; a container is its host's to replace. + +- **Tested (2026-11-13 and 2026-11-15), by the product owner and the agent.** This Mac: an upload and a URL with its digest installed and restarted in place, same PID, `.pending` cleared after a minute; a board image and a wrong digest refused. .204, the Raspberry Pi 5, from the CI `.deb`: the service runs as `moonlight` holding `CAP_NET_RAW` alone; an upload restarts it in place, no start counted; `/bin/false` installed as the pending program rolled back by the stop hook 37 ms after the failed start; an `apt install` puts the package's program back and drops `.prev`. .156's container refuses both install routes with 501. CI: every desktop compiles. - **Verified by:** .204's package updated from its own Firmware card and rolled back from a program made to fail; this Mac updated from its own Firmware card, by URL and by upload; Windows by CI's `smoke-update`, which installs a program through its own upload route and checks it restarts with `.prev` beside it, on each runner that builds a desktop; .156's container saying its host updates it. -- **In the field (Discord, 2026-11-13):** robwomp tests the package on their r76, the Pi 5 being their day-to-day test machine and the r3 their production one; Anjin tests too. belrik runs MoonLight under Podman and Kubernetes and offered to review the unit: [moonlight.service](../../moondeck/ci/linux/moonlight.service) and its [rollback](../../moondeck/ci/linux/rollback), the links sent; the second unit has since folded into the first on belrik's review. #### The map updates a desktop as it updates a board ✅ -Landed in: [installing on Linux](../reference/installing-on-linux.md) (a package updates from the map, a container by its host); the `update` field of `/api/system` in [the REST API](../reference/openapi.yaml). +Landed in: [installing on Linux](../reference/installing-on-linux.md) (a package updates from the map, a container by its host); a node's update and the device's `update` read from its summary in [DevicesModule.h](../../src/core/system/DevicesModule.h) ("Behind, and defaults"); the `update` field of `/api/system` in [the REST API](../reference/openapi.yaml); the `.dmg` app in a folder its user writes in [MIGRATING](../reference/MIGRATING.md). -- **The same call as for a board or a Teensy:** the node's popup posts `firmware--v.bin` to `/api/firmware/url`, with GitHub's digest, and watches `/api/system` until the device answers on the new version. -- **What only the map adds:** the device's `update` read from `/api/system`; a container's node says its host updates it, a program in a read-only folder that it is updated by hand. -- **Removes** the desktop special cases the map and the update badge carry: the badge opening the release page for every desktop, `desktopAssetPrefix`, the desktop branch of the asset choice. **Fixed on the way:** the update badge read the browser's OS, and always x64 on Linux. -- **Built (2026-11-12), on the archive naming:** the popup offering an update to a `self` device and telling a container its host updates it; the asset choice by the device's own model (`ui-update-asset.test.mjs`). The uniform name reduces it to the board's path. -- **A hypothesis to test first:** the app from the release's `.dmg` updates itself too. An app dragged into Applications sits in a folder its user writes, so it reports `update: self` and replaces the program inside its signed bundle; macOS may then refuse to launch the bundle. Step 1 was tested on the bare program run from Terminal. - **Verified by:** .204, this Mac and a Windows PC each updated from another device's map; the `.dmg` app on this Mac updated from its own Firmware card and opened again from Finder; .156's container node saying its host updates it. #### A container updated from the map, through Watchtower ⏳ @@ -220,6 +185,21 @@ Asked by the product owner (2026-11-12), after the desktops. - **What the map offers.** A node's popup shows "Update MoonBase" and "Update WiFi coprocessor" beside the app's update, only when due, each pressed twice like the app's, posting to its route and watching `/api/system` until the peer reports it current. A board without TLS is sent to its own Firmware card, as for the app. - **Verified by:** a bench board with an older MoonBase and the P4 with an older coprocessor image, each brought current from another device's map. +#### The map marks a device behind its channel ✅ (a bench board behind, on the host's map, 2026-11-17) + +Landed in: the rule, `channelUpdate` in [releases.js](../../src/ui/releases.js), read by the status bar's badge, a node's popup and its mark, pinned in `ui-update-asset.test.mjs`; the channel for a user in the [update firmware](../moonwalk.md#update-firmware) process; a node's mark, and a MoonBase or WiFi coprocessor behind its app, in [DevicesModule.h](../../src/core/system/DevicesModule.h) ("Behind, and defaults"); `mode`, `moonbaseMismatch` and `coprocessorMismatch` in [the REST API](../reference/openapi.yaml). + +- **Verified by:** a bench board one release behind and one current, side by side on the map; the behind one updated from its popup, its mark gone. + +#### MoonBase on every device ⏳ (after WLED to MoonLight from the map) + +- Every layout carries MoonBase in a factory slot: recovery without a cable on every board, one update path, and on an 8 or 16 MB board the second app slot's flash going to the filesystem. Today the two-slot variants are the larger S3s (N8R8, N16R8) and the P4. +- Waits for the migration step: its power-cut rounds on each chip family are what MoonBase on that chip is trusted on. +- A device already on a two-slot layout cannot change its table through a network update, so it moves with the same migration image, uploaded through MoonLight's own update page: the program that migrates a WLED device migrates a two-slot MoonLight one. A MIGRATING entry. +- **Verified by:** a bench S3 N16R8 and the P4 moved from two slots to MoonBase over the network, then updated through MoonBase and recovered from a cut mid-update. + +### Other firmwares, to MoonLight and back + #### WLED to MoonLight from the map ✅ Landed in: [Migrating over the air](../reference/migrating-over-the-air.md) (what can move, by chip and firmware, and what can stop it); the [Move a WLED device](../moonwalk.md#migrate-wled) process; the migration image's steps, the bootloader and table per flash size and type, in [migrate_main.cpp](https://github.com/MoonModules/MoonLight/blob/main/moonbase/migrate/main/migrate_main.cpp); one image per family built with its first firmware in [build_esp32.py](https://github.com/MoonModules/MoonLight/blob/main/moondeck/build/build_esp32.py) (`migration_family`, `migration_tables`); a board in MoonBase on the map in [moonbase.md](../explanation/architecture/moonbase.md); the power-cut rounds in [wled_migration_test.py](https://github.com/MoonModules/MoonLight/blob/main/moondeck/build/wled_migration_test.py). Verified from the map: WLED 0.15 on the classic, the C3 and the 16 MB S3, WLED-MM on the P4 (2026-11-11); the 4 and 8 MB S3 are left to users' reports. @@ -234,47 +214,24 @@ Landed in: [Migrating over the air](../reference/migrating-over-the-air.md) (Tas Landed in: [moonbase.md](../explanation/architecture/moonbase.md) (its page installs through the picker loaded from the installer's site); the [retry step](../moonwalk.md#migrate-wled) of the migration process. Verified (2026-11-11): the P4 in MoonBase installing MoonLight from its own page. -#### Every neighbor's MAC, by ARP ✅ (Windows run 2026-11-17: MM-Windows at .245 gave the Hue bridge at .143, an added address, its MAC 00:17:88:41:67:f0) - -- **What ARP is.** On a local network a packet travels by MAC address, the IP address being the logical name on top. A device about to send to an address asks the network who holds it, the holder answers with its MAC, and the sender keeps the pair in its address table for minutes: the Address Resolution Protocol, [RFC 826](https://www.rfc-editor.org/rfc/rfc826). Every operating system keeps that table, and lwIP does on the ESP32. -- **What it gives the map.** After any exchange with a device on this subnet, its MAC is in this device's own table, so a device known only by its address, Tasmota or anything added by address, gets the identity a MoonLight board sends in its packet, without a firmware-specific API. Tasmota 16 refuses its status page to a request without its own page as Referer, a guard against cross-site requests, so ARP is the textbook source that leaves that guard alone. -- **Built (2026-11-10).** `platform::arpLookup` reads the stack's table: lwIP's under its core lock on the ESP32, `GetIpNetTable2` on Windows, `/proc/net/arp` on Linux, the routing table's link-layer entries on macOS, none on a Teensy, whose QNEthernet keeps its table private. macOS 27 answers that sysctl, the one arp(8) makes, with an empty table unless the binary is signed with an identifier, a behavior network scanners report ([NetScanner](https://github.com/Criseda/NetScanner/pull/51), [NetGuard](https://github.com/Lyosis/NetGuard/pull/46)); so the build signs the desktop binary as `org.moonmodules.moonlight`, the bundle's identifier, and the release signs every macOS binary with it. Local Network permission is not it. The address probe looks the MAC up after its connection and keys the row by it. Tested with a faked entry on the desktop; verified 2026-11-10 on the macOS desktop and the Olimex (ESP32): Tasmota's row shows c8:f0:9e:73:88:4c on both; verified 2026-11-13 on moonpi1 (Linux): the router added by address shows 94:83:c4:c3:ed:c8, the MAC this Mac's own table holds. -- **More uses**, each its own decision: - - *A fallback for every row without a MAC*, not only added addresses: any device this one has talked to. Receiving a device's broadcasts alone does not always put it in the table. - - *The vendor*: a MAC's first three bytes name its maker in the IEEE OUI registry, Espressif, Signify (Hue), Raspberry Pi, Apple, so a small table tells an ESP32 from a Pi or a bridge without asking it. - - *Wake-on-LAN*, which needs the MAC: the map waking a sleeping PC or Pi that runs MoonLight. - - *An active scan*: ask for every address of the subnet and list what answers, silent devices included, as `arp-scan` does; a burst of broadcast, so on demand. - - *Changes spotted*: one MAC at a new address is a device that moved; a new MAC at a known address, another device that took it. -- **Limits.** Only the own subnet, a router answering for everything beyond it; entries expire in minutes; a MAC says who and who made it, never what it runs. - -#### A security and privacy check of the map ✅ - -Landed in: [security](../explanation/security.md) (the trust boundary and the `Origin` and `Host` rule, the Devices map, secrets, firmware, what leaves the device); the rule's code in `http::fromBeyondNetwork` ([HttpHead.h](../../src/core/util/HttpHead.h)), used by the app and MoonBase; [MIGRATING](../reference/MIGRATING.md) ("A page from outside the local network is refused"). Verified (2026-11-11) on the S3 bench board, the access point's UI included. Steps 15 and 17 take the map beyond the network and get their own review there. - -#### MoonDeck and the web installer reviewed against the map 🚧 (MoonDeck's half built 2026-11-18, points 1 to 5 below; next: the web installer's half, and the verify line's replug test) - -The map now finds devices, reads their summary, installs firmware, moves other firmwares to MoonLight and applies a device model's defaults, which MoonDeck and the web installer each do in their own way. A review of both against it, looking first for what two of them do twice (the product owner, 2026-11-11). Candidates, each kept, merged or removed with its reason: - -- **Finding devices.** MoonDeck's network bar and its registry in `moondeck.json` against Discovery's list, which any device serves through `GET /api/modules/Devices`, each row its name, address, type and MAC, with `/api/system` per MoonLight row for its firmware and board; MoonDeck could read the list from one device instead of keeping its own. -- **Device models.** MoonDeck's deviceModel push after each discovery (`PATCH /api/state`), the installer's `APPLY_OP` over serial and the map's Apply device defaults: one document, three transports, and whether each still earns its place. -- **Installing.** MoonDeck's network and USB flashing, the Firmware card, the installer and the map's Install all reach a release's firmware; the install picker is shared already, and the rest is checked for a second copy of release lookup, variant choice or progress reporting. -- **Board details.** The map's popup (`/api/system`), MoonDeck's device panels and the installer's device report read the same facts; one reader, or each its own reason. -- **Other firmwares.** The test script's WLED restore, the links to WLED's, Tasmota's and ESPHome's installers, and the map's moves: one place that knows each firmware's layout and update page. -- **Decided for MoonDeck (product owner, 2026-11-18).** The map runs fleet operations, MoonDeck runs what needs the source tree (build, flash a local build over USB, monitor, test, check, gate), and MoonDeck shows the map rather than a device list of its own: - 1. **One map, two hosts.** The map moves out of `src/ui/app.js` into `src/ui/devicemap.js` and its styles into `devicemap.css`, which the firmware bundles into its `app.js` and `style.css` and MoonDeck's server serves from the repo at `/ui/`. MoonDeck's Live tab opens on its Map pane and draws it from a MoonLight's Discovery list (`GET /api/modules/Discovery`), the desktop app on `localhost:8080` by default; a click opens the device's own popup, from the same code. ✅ map and popup (2026-11-18) - 2. **Minimal discovery of its own, for a new developer with no MoonLight running.** MoonDeck listens on UDP 65506 for as long as it runs (`moondeck/_presence.py`), the packet every MoonLight board and desktop sends every 10 s with its web port and MAC (`WledPacket.h`), sending nothing. It shares the port with a desktop app on the same computer: both bind it shared (`SO_REUSEPORT`, `UdpSocket::bindShared` on the desktop), so each runs whether the other does or not. The map reads, in order, the Discovery list of the MoonLight its bar names, then of the first MoonLight MoonDeck heard, which brings that board's peers too; with none heard, it draws what MoonDeck heard and the boards on its USB ports. It replaces the subnet scan's 254 HTTP probes. ✅ (2026-11-18: 13 bench devices heard beside the running desktop app) - 3. **The USB layer is MoonDeck's, added to the map's node popup** through a hook: this computer's port for the device, Flash local build, Monitor. ✅ (2026-11-18: the popup's `actions` hook, `moondeck_ui/popup.js`; Identify, Flash local build with `flash_esp32.py --mac`, Monitor, by MAC through `moondeck/_usb.py`) - 4. **A port is found when it is used, never stored**, on the device or in `moondeck.json`. A port path is the computer's name for a connection, which the device cannot see, which two computers would each write differently, and which a replug makes stale, so a stored one is checked before every use anyway. The key is the MAC: a native USB board (S3, C3, C6, the P4's USB-JTAG) reports it as its USB serial number, read without a reset; a board behind a UART bridge (CP210x, CH343) is known by the adapter's serial, the board's own chip and stable across ports, mapped to the MAC by one esptool probe and cached on this computer; a bridge with no serial (a cheap CH340) shows as unknown until Identify probes it, which resets it. Replugged, the board is found on its new port; on another desk or network, that computer builds its own cache and the map matches the device by MAC. The flash keeps its guard: esptool reads the MAC before writing and stops on a stranger. ✅ (2026-11-18: `usbMacs` in moondeck.json, written by Identify; `last_port`, `usbSerial` and `_heal_last_ports` gone) - 5. **What leaves MoonDeck:** the subnet scan and probes (`discover_devices`, `refresh_devices`, `_probe_device`), the device list in `moondeck.json` (address, name, firmware, model, online), `last_port` and `_heal_last_ports`, the deviceModel push (the map's Apply device defaults), and the Live tab's device table. `moondeck.json` keeps the WiFi credentials per network for provisioning over USB, MoonDeck's preferences, the USB adapter to MAC cache and the live tests' selection by MAC. MoonDeck's network update serves the local build at a URL and hands it to the device's `/api/firmware/url`, so a network install has one path. ✅ (2026-11-18), the network update too: `/api/firmware/url` refuses an `http://` address (`readImageUrl`, MIGRATING), so the popup's Install local build hands the shared install flow (`src/ui/deviceinstall.js`) the local build as a `{file}` source, uploaded with its bar, through MoonBase where the board writes its app from it; MoonDeck's `/api/ota` is gone. - 6. **Later, its own decision:** the map installing over USB through Web Serial, as the web installer does; browsers allow it on https and localhost only, so from the desktop app's page and not from a board's. -- **Verified by:** the list of overlaps found, each with its decision, and the code and docs that one of each pair leaves behind removed. For MoonDeck: its Live tab draws the same map as a board's page; with no MoonLight running it lists a board that announces itself and one on USB; a board replugged to another port is flashed by MAC without a manual pick. +#### WLED, WLED-MM, Tasmota and ESPHome installed on a device 🚧 (built: the installer's section and the map's links, 2026-11-17; next: the way back through MoonBase for WLED, the MoonDeck action) -#### MoonBase on every device ⏳ (after WLED to MoonLight from the map) +Asked by the product owner (2026-11-17): the way back from MoonLight, and a bench that can set up any firmware the map moves. -- Every layout carries MoonBase in a factory slot: recovery without a cable on every board, one update path, and on an 8 or 16 MB board the second app slot's flash going to the filesystem. Today the two-slot variants are the larger S3s (N8R8, N16R8) and the P4. -- Waits for the migration step: its power-cut rounds on each chip family are what MoonBase on that chip is trusted on. -- A device already on a two-slot layout cannot change its table through a network update, so it moves with the same migration image, uploaded through MoonLight's own update page: the program that migrates a WLED device migrates a two-slot MoonLight one. A MIGRATING entry. -- **Verified by:** a bench S3 N16R8 and the P4 moved from two slots to MoonBase over the network, then updated through MoonBase and recovered from a cut mid-update. +- **Over USB, or through MoonBase.** A device's own update route refuses an image that is not MoonLight's (`isAppImage`), and a board MoonLight moved carries MoonBase's partition table, which none of the four boots from, so no plain upload installs them. +- **Through MoonBase, asked by the product owner (2026-11-17): the migration image run backwards.** The migration image (`moonbase/migrate/`, 387 lines) already moves a board from WLED's, Tasmota's or ESPHome's layout to MoonBase's, power-cut safe: it copies itself out of the slot it overwrites, writes the bootloader and table it carries, and carries the networks over. The way back is the same steps toward the target's layout: the target's app downloaded from its own release into a slot MoonBase is not running from, the target's table (WLED's 4 and 16 MB tables are in `esp32/partitions/` already, built for the migration image) and its bootloader, then the choice of app. + - *What it costs per firmware:* its table and bootloader per flash size, its release asset's name, and its networks written its own way (WLED's `wsec.json`, Tasmota's settings, ESPHome's preferences), then a power-cut round on each chip family, as the way in had. + - *What it loses:* MoonBase itself, since none of the four has a factory slot, so the way back to MoonLight is the migration image again, through the old firmware's update page. + - *Order:* WLED first, whose tables exist and whose settings the way in already reads; WLED-MM shares its layout; Tasmota's safeboot layout and ESPHome's later. +- **The textbook source is each project's own installer.** WLED, WLED-MM, Tasmota and ESPHome each publish an ESP Web Tools installer that knows their builds, versions and partition layouts: [WLED](https://install.wled.me/), [WLED-MM](https://moonmodules.org/WLED-WebInstaller/), [Tasmota](https://tasmota.github.io/install/), [ESPHome](https://web.esphome.io/). The WLED installers' sources: [lost-hope's WLED-WebInstaller](https://lost-hope.github.io/WLED-WebInstaller/) and [MoonModules' WLED-WebInstaller](https://github.com/MoonModules/WLED-WebInstaller), WLED-MM's. Copying their images or manifests into MoonLight's installer would mean following four release schedules. +- **Where it shows, by candidate:** + - *The web installer:* a short "Other firmware" section linking the four installers, the one place a user with a cable already is. The links move there from [migrating over the air](../reference/migrating-over-the-air.md), which then points to the section. + - *The map:* a move's confirm, which already says going back needs a cable, links that section, so the way back is one click from where the way in was chosen. + - *MoonDeck:* a bench action, a firmware per port, that writes WLED or Tasmota over esptool as `wled_migration_test.py`'s `as_wled` does, so a migration path is set up again in one click; for development, not a user path. +- **A generic installer from releases, investigated (2026-11-17, asked by the product owner).** One mechanism reading a project's GitHub releases works where a release says where each part goes, which is what an [ESP Web Tools manifest](https://esphome.github.io/esp-web-tools/) does: a `*.manifest.json` asset is used as it is (ESPHome publishes one), a `*.factory.bin` is flashed at 0 with its chip read from the image header (Tasmota), and only the WLED family needs a per-project table, since its releases ship app images alone. That table names the boot files WLED's own installers keep (`bin/boot/` on [install.wled.me](https://install.wled.me/) and [MoonModules' installer](https://moonmodules.org/WLED-WebInstaller/)) rather than copying them, until WLED's CI attaches a factory image as Tasmota's and ESPHome's do. A browser fetches through a mirror that sends CORS, since GitHub's downloads send none. It lives in the install picker, so the web installer, the map's way back and MoonDeck's bench action read one mechanism. +- **Recommended:** the installer section and the map's link first, a few lines; the way back through MoonBase for WLED next, as its own step with its own power-cut rounds; the MoonDeck action for the bench. +- **Removes:** the four links' second home on the migration page. +- **Verified by:** each installer reached from the section; a bench board set to WLED from MoonDeck and moved back to MoonLight from the map. #### WLED's LED outputs and 2D layout carried into MoonLight ⏳ @@ -306,140 +263,122 @@ WLED is the obvious partner; ESPHome and Tasmota run on far more devices in a ho - **What stays theirs:** their own guards (Tasmota's Referer check, ESPHome's native API key), never bypassed; MoonLight talks to them through what each publishes, as the map already does. - **Verified by:** each first candidate on the bench, a WLED, an ESPHome and a Tasmota board each doing its part beside a MoonLight board on the map. -#### The map marks a device behind its channel ✅ (a bench board behind, on the host's map, 2026-11-17) - -Asked by the product owner (2026-11-17). +### The map's reach -- **One rule, the status bar's.** A device on an official release is behind when a newer official release ships; a device on a -dev build, or one in developer mode (System's `mode`, which `/api/system` and its own map row report), is behind when a newer official release or a newer `latest` ships, the official one first. A -dev device that installs an official release follows official releases from then on, unless it is in developer mode, its version and mode being what decide the channel. -- **Built (2026-11-17).** `channelUpdate` in app.js is the rule, read by the status bar's badge, a node's popup and its mark, from the install picker's one release list; tests in `ui-update-asset.test.mjs`. -- **On the node.** Each MoonLight row carries its version and firmware already, so the map applies the rule per node without a request: a node behind gets a mark of its own, an amber ring and a ⬆ beside its name, distinct from the red dashed ring that marks a peer not answering (`.graph-unreachable`). A MoonBase or a WiFi coprocessor behind its app, which the device's own row names (`moonbaseMismatch`, `coprocessorMismatch`), gets the same mark, its tooltip saying which. -- **Verified by:** a bench board one release behind and one current, side by side on the map; the behind one updated from its popup, its mark gone. +#### Every Hue bridge on the map ⏳ -#### WLED, WLED-MM, Tasmota and ESPHome installed on a device 🚧 (built: the installer's section and the map's links, 2026-11-17; next: the way back through MoonBase for WLED, the MoonDeck action) +- Not only a bridge a HueDriver points at: Hue's own discovery, mDNS `_hue._tcp`, through a passive listen to mDNS announcements, or SSDP on port 1900. With the rest of the Backlog's More discovery, since they share the mechanism. +- It needs a hardware test: mDNS queries were ended by a pool exhaustion on the ESP32, so the passive listen is a hypothesis until a board runs it. +- **Verified by:** a Hue bridge nobody configured on the bench map. -Asked by the product owner (2026-11-17): the way back from MoonLight, and a bench that can set up any firmware the map moves. +#### Every neighbor's MAC, by ARP ✅ (Windows run 2026-11-17: MM-Windows at .245 gave the Hue bridge at .143, an added address, its MAC 00:17:88:41:67:f0) -- **Over USB, or through MoonBase.** A device's own update route refuses an image that is not MoonLight's (`isAppImage`), and a board MoonLight moved carries MoonBase's partition table, which none of the four boots from, so no plain upload installs them. -- **Through MoonBase, asked by the product owner (2026-11-17): the migration image run backwards.** The migration image (`moonbase/migrate/`, 387 lines) already moves a board from WLED's, Tasmota's or ESPHome's layout to MoonBase's, power-cut safe: it copies itself out of the slot it overwrites, writes the bootloader and table it carries, and carries the networks over. The way back is the same steps toward the target's layout: the target's app downloaded from its own release into a slot MoonBase is not running from, the target's table (WLED's 4 and 16 MB tables are in `esp32/partitions/` already, built for the migration image) and its bootloader, then the choice of app. - - *What it costs per firmware:* its table and bootloader per flash size, its release asset's name, and its networks written its own way (WLED's `wsec.json`, Tasmota's settings, ESPHome's preferences), then a power-cut round on each chip family, as the way in had. - - *What it loses:* MoonBase itself, since none of the four has a factory slot, so the way back to MoonLight is the migration image again, through the old firmware's update page. - - *Order:* WLED first, whose tables exist and whose settings the way in already reads; WLED-MM shares its layout; Tasmota's safeboot layout and ESPHome's later. -- **The textbook source is each project's own installer.** WLED, WLED-MM, Tasmota and ESPHome each publish an ESP Web Tools installer that knows their builds, versions and partition layouts: [WLED](https://install.wled.me/), [WLED-MM](https://moonmodules.org/WLED-WebInstaller/), [Tasmota](https://tasmota.github.io/install/), [ESPHome](https://web.esphome.io/). The WLED installers' sources: [lost-hope's WLED-WebInstaller](https://lost-hope.github.io/WLED-WebInstaller/) and [MoonModules' WLED-WebInstaller](https://github.com/MoonModules/WLED-WebInstaller), WLED-MM's. Copying their images or manifests into MoonLight's installer would mean following four release schedules. -- **Where it shows, by candidate:** - - *The web installer:* a short "Other firmware" section linking the four installers, the one place a user with a cable already is. The links move there from [migrating over the air](../reference/migrating-over-the-air.md), which then points to the section. - - *The map:* a move's confirm, which already says going back needs a cable, links that section, so the way back is one click from where the way in was chosen. - - *MoonDeck:* a bench action, a firmware per port, that writes WLED or Tasmota over esptool as `wled_migration_test.py`'s `as_wled` does, so a migration path is set up again in one click; for development, not a user path. -- **A generic installer from releases, investigated (2026-11-17, asked by the product owner).** One mechanism reading a project's GitHub releases and their assets works when a release says where each part goes, and the ESP Web Tools manifest (`builds[].chipFamily`, `parts[]{path, offset}`, [the standard](https://esphome.github.io/esp-web-tools/)) is that description. Three rules, in order: - 1. A `*.manifest.json` asset: use it. [ESPHome](https://firmware.esphome.io/esphome-web/manifest.json) publishes one, 19 builds, a merged `*.factory.bin` at offset 0 per ESP32 chip; [Tasmota's installer](https://tasmota.github.io/install/manifests.json) publishes static ones over `tasmota32.factory.bin`. - 2. A `*.factory.bin` asset: flash it at 0, its chip read from the image header (byte 12's chip id, esptool's `IMAGE_CHIP_ID`) and its kind from the partition table's magic at 0x8000. Tasmota's own releases. - 3. Otherwise a per-project table, data rather than code: the releases URL, a download mirror that sends CORS, an asset pattern with named groups (chip, flash size, variant), the app offset 0x10000, and the bootloader and partition table per chip and flash size. Only the WLED family needs it: [WLED](https://github.com/wled/WLED/releases), [WLED-MM](https://github.com/MoonModules/WLED-MM/releases) ship app images alone, and their installers keep the boot parts in their own `bin/boot/` (WLED-MM's `releases.js`, WLED's `app.js` `CHIP_CONFIG`). - - *The asset-to-partition mapping is the gap*, as the product owner suspected: neither a WLED release nor its image header names the bootloader, the table or the offsets. Their installers fill it, at stable addresses that send CORS: [install.wled.me](https://install.wled.me/)'s `bin/boot/` holds a bootloader per chip and a partition table per chip and flash size, its [README](https://github.com/wled/WLED-WebInstaller/blob/main/bin/boot/README.md) mapping each to its offset, and one bootloader serving several flash sizes with its flash-size header byte rewritten, as esptool's `--flash_size` does; [MoonModules' installer](https://moonmodules.org/WLED-WebInstaller/)'s `bin/boot/` holds one set per chip, and its `bin/Release/release_0_15_3/manifest.json` is a static ESP Web Tools manifest, bootloader and app with their offsets. So rule 3's table names the installer's boot files rather than carrying copies. The textbook fix is upstream: WLED's and WLED-MM's CI attaching a `*.factory.bin` or a manifest with the boot parts, as Tasmota and ESPHome do, after which rule 3 goes. - - *A browser needs a mirror*: GitHub's release downloads send no CORS header (the releases API does), so a page fetches through [download.wled.me](https://download.wled.me) (WLED and WLED-MM paths), tasmota.github.io or firmware.esphome.io; MoonDeck and a device fetch directly. - - *troyhacks' WLED* publishes no releases: its binaries are Actions artifacts that need a GitHub login, and its nightly release workflow fails ([its releases](https://api.github.com/repos/troyhacks/WLED/releases)), so it is out of reach until it publishes a release. - - *Where it lives*: the install picker, beside MoonLight's own releases, so the web installer, the map's way back through MoonBase and MoonDeck's bench action read one mechanism; ESP8266 images carry no chip id, so they stay out. -- **Recommended:** the installer section and the map's link first, a few lines; the way back through MoonBase for WLED next, as its own step with its own power-cut rounds; the MoonDeck action for the bench. -- **Removes:** the four links' second home on the migration page. -- **Verified by:** each installer reached from the section; a bench board set to WLED from MoonDeck and moved back to MoonLight from the map. +Landed in: `platform::arpLookup`, per operating system, in [platform.h](../../src/platform/platform.h), and the macOS build's signing identifier, without which macOS 27 answers with an empty table, at its introduction in [CMakeLists.txt](../../CMakeLists.txt); an added address keyed by the MAC in [DevicesModule.h](../../src/core/system/DevicesModule.h) ("One device, one row"); ARP as the source that leaves Tasmota's Referer guard alone in [security](../explanation/security.md#the-devices-map). Verified: Tasmota's MAC on the macOS desktop and the Olimex (2026-11-10), the router added by address on moonpi1 (2026-11-13), the Hue bridge from MM-Windows (2026-11-17). -### Act as a group +- **More uses**, each its own decision: + - *A fallback for every row without a MAC*, not only added addresses: any device this one has talked to. Receiving a device's broadcasts alone does not always put it in the table. + - *The vendor*: a MAC's first three bytes name its maker in the IEEE OUI registry, Espressif, Signify (Hue), Raspberry Pi, Apple, so a small table tells an ESP32 from a Pi or a bridge without asking it. + - *Wake-on-LAN*, which needs the MAC: the map waking a sleeping PC or Pi that runs MoonLight. + - *An active scan*: ask for every address of the subnet and list what answers, silent devices included, as `arp-scan` does; a burst of broadcast, so on demand. + - *Changes spotted*: one MAC at a new address is a device that moved; a new MAC at a known address, another device that took it. -#### 6. Device groups, shared by their members 🧪 (built 2026-11-18: the DeviceGroup module, the map's boxes, the replication in [DeviceGroupSync.h](../../src/core/system/DeviceGroupSync.h); test: two devices editing one group on the bench) +#### A security and privacy check of the map ✅ -- **The module.** A DeviceGroup under Devices, added and removed as an effect is ([DeviceGroup.h](../../src/core/system/DeviceGroup.h)): its members picked from Discovery's devices and kept by their MAC-derived identity from step 4, so a rename or a new lease keeps them ([device identity](../explanation/architecture/mooncore.md#device-name-one-identity-every-network-name-derives-from-it)); its id the creating device's MAC and the fleet clock's millisecond at its making, which grows across a restart, an id a sibling or a deletion marker holds skipped. A member unheard for three of its announcements is missing. A role is pinned with the role's first user (step 8's waves, step 9's clock reference); until then each derives from the lowest live device id ([the message design](#the-message-design)). -- **On the map.** Each row names the groups holding its device; the map draws a labeled box around a group's members, members pulled together by the layout, a missing member faded and marked inside its box; a group a peer defines merges by its id. -- **Replicated.** Every member holds the definition and any member may edit it: the name a last-writer-wins register, the members a last-writer-wins element set (each member its timestamp and an added or removed flag), so two members added at once on two devices both stay; a deleted group leaves a deleted mark. Published over OSC to the definitions' one address by every member, never forwarded, repeated by Trickle, ordered by a hybrid logical clock that keeps each part's writer: a copy received or renamed keeps the stamp its writer gave it ([the message design](#the-message-design)). Only the members hold the module; every other device sees the group on its map, which merges the peers' rows. -- **What it costs.** Zero work on the render path; a member 48 bytes; the definitions on one well-known address (239.255.77.79:65510), every device listening, so a device hears a group it was added to; their packets near flat as a group grows (Trickle). A group deleted anywhere leaves a saved marker, the newest eight kept. -- **Tested (2026-11-18), by the product owner and the agent.** rtpDGroup made on .101 with .221 as a member: .221 created it within seconds, and the Mac's map, no member, drew its box. That test showed a received copy stamping the name as its own; fixed, with a unit test. With the review fixes on both (781cba15, 2026-11-18): a rename on .221 reached .101 within 3 s and one on .101 came back, each carrying its writer's stamp; the group deleted on .221 was gone from .101 within 3 s, both saved its marker, and after both rebooted it stayed gone. -- **Verified by:** two groups on the bench, made on one board and edited on another, both maps drawing the same boxes; a member unplugged and shown missing; a unit test feeding the element set every arrival order. +Landed in: [security](../explanation/security.md) (the trust boundary and the `Origin` and `Host` rule, the Devices map, secrets, firmware, what leaves the device); the rule's code in `http::fromBeyondNetwork` ([HttpHead.h](../../src/core/util/HttpHead.h)), used by the app and MoonBase; [MIGRATING](../reference/MIGRATING.md) ("A page from outside the local network is refused"). Verified (2026-11-11) on the S3 bench board, the access point's UI included. Steps 15 and 17 take the map beyond the network and get their own review there. -#### 7. A group acts as one 🧪 (built 2026-11-18: a group's name in a `receivers` field; the group carries its shared controls and looks, membership the one setting; test: DesktopGroup with its OscService modules removed) - -- **A group in `receivers`** (product owner, 2026-11-18). OscService, AudioService and NetworkSendDriver keep their `receivers` text field, and a token equal to a device group's shared name expands to its members, found by MAC in Discovery's rows, each at its current address and this device left out: `resolveReceivers` in [NameResolver.h](../../src/core/util/NameResolver.h), the one home all three call. A typed address or name works beside it. It runs on an edit and once a second, so a member's new lease or a membership edit follows within a second. An unheard member keeps its place with no address, so Network Send's split of the window over the members holds while one is missing; a change in the member count re-prepares the driver. The three share one bound, `kMaxReceivers` (32) in [ReceiverList.h](../../src/core/util/ReceiverList.h), where OSC and audio sync had 8; the receivers past it are left out and the card says how many ([MIGRATING](../reference/MIGRATING.md)). No new control type. -- **Membership is the one setting** (product owner, 2026-11-18): a member follows the group's shared controls with nothing else set up, so join and the device follows, leave and it stops, in the one edit DeviceGroupSync already replicates. [DeviceGroupSync.h](../../src/core/system/DeviceGroupSync.h) is a surface of the Control module, attached with no seed, and [SharedControls.h](../../src/core/system/SharedControls.h) holds what the members share. -- **What travels: the sender's assignments, applied by name.** A switch, encoder or fader that changes on a member, from the UI, the API or a desk, goes out as its target parameter, its absolute value after the change and a hybrid-clock stamp (`Drivers.brightness = 77`), as `/moonlight/devicegroup/control` on the groups' address (239.255.77.79:65510), OSC encoded as the definitions are. A receiver applies the parameter by name through the control path, whatever its own slots drive. A parameter no slot drives on the sender never travels, so a board's pins and network stay its own. Each slot's parameter is seen when the sync attaches and when the slot is pointed elsewhere, so neither counts as a change. -- **Last writer wins per parameter.** A value applies when its stamp is newer than the one held for that parameter in that group; two changes at once end on one value everywhere, the higher writer winning a tie. A value a group set is not sent on, so no loop forms. The table holds 48 values across the groups (`kMaxValues`); past it a value still applies with no stamp, and the group's status says `shared controls full`. -- **A look travels as its name** (product owner, 2026-11-18). An effects preset applied on a member, the rule that keeps Home Assistant from rewiring pins, goes out as `/moonlight/devicegroup/look`: the group, the preset's name, its stamp, and the address and port its maker serves it at. Each member fetches the document from the maker's File Manager route (`/api/file?path=/.config/presets/.json`) through [JsonFetch](../../src/core/util/JsonFetch.h)'s worker, off the render path, applies it live between frames, and keeps nothing; a repeat of the same stamp is applied once. A driver or layout preset stays on its board. A receiver's pads stay as they are. -- **Catch-up is the group's anti-entropy.** The stamped values and the latest recall ride the definition's Trickle repeats, each message carrying the sender's digest of the group's values; a member whose digest differs resets its interval (RFC 6206), so a member back from a restart hears the group within seconds and takes its newer values, its own saved ones never sent with a fresh stamp. Messages are read every 20 ms (Discovery's `tick20ms`), so a change reaches the members within a frame or two. -- **OscService keeps groups as receivers.** A group's name in its `receivers` still sends OSC to each member on the OSC port, for a cue list or a show controller feeding every board in a room. Its catch-up and pad relay are gone: one job, one mechanism. -- **Unicast to each member** (product owner, 2026-11-18), for OSC, audio sync and Network Send, as the services' unicast mode already sends. -- **MidiService's `participant` stays one peer.** Its session holds one participant at a time, the sharing board refusing a second while the first holds it ([control surfaces](../reference/control-surface.md)), and a desk is passed on by the one board it is plugged into; the group carries that board's surface on. -- **The verify line, mapped.** Shared controls across a group, a member added and receiving without touching OSC: the group's shared controls. Audio sync across a group: the group's name in audio sync's `receivers`. Two boards changing one control at the same moment end on the same value: last writer wins per parameter. A preset change followed by every member: the look's recall. A member rebooted and caught up, packets dropped and the group still converging: the values and the recall on the Trickle repeats. Reading the members' summaries over REST stays with the page that needs it. -- **Tested (2026-11-18), by the product owner and the agent.** DesktopGroup: the desktop app, .221 (P4), .122 (classic) and .115 (C3), each with OSC listening and feedback to `DesktopGroup`. An encoder changed on any of the four reached the other three, the first device read within 3 to 20 ms; brightness changed on .122 reached every member once each mapped `fader1` to `Drivers.brightness`. A member restarted 40 s before still resolved its group to nobody, and a change made on it went out once its Discovery list had heard the others. A group member was sent to on the feedback port, where no member listens; it is sent to on the OSC port since. -- **Verified by:** DesktopGroup, the desktop app, .221, .122 and .115 with their OscService modules removed: a fader, a switch and an encoder on any member followed by the others, members with different slots on one parameter converging; an effects preset pressed on one applied on the others, a driver preset staying; a member restarted taking the group's newer values; two members moving one control at once ending on one value. +#### MoonDeck and the web installer reviewed against the map 🚧 (MoonDeck's half done 2026-11-18; next: the web installer's half, and the replug test) -#### 8. Update a group 🚧 (a group's box opens a popup that updates its members in waves; awaiting the bench group) +Where MoonDeck, the web installer and the map each do one job, finding devices, device models, installing, board details and other firmwares, one of each pair stays and the other goes. -- From a device's page: update one peer, or the whole group, through each peer's `/api/firmware/url`, or by download and upload for a peer without TLS, the Firmware card's two paths reused. -- In waves: one device first, then a few at a time, stopping at the first failure, the device doing it last. A swarm finishes in minutes while a bad release stops at one device. Done means the peer is heard again with the new version. -- **Verified by:** a bench group updated from one board's page, a Teensy among them. -- **In the field (robwomp, Discord 2026-11-13):** a Matrix Portal updated over the air while driving a HUB75 panel direct showed a few jumps and otherwise ran on, "a loose cable, not a whole OTA update": what a wave looks like to the audience. -- **Built (2026-11-10), at the product owner's ask: one peer from its popup.** An "Update to …" button, pressed twice, posts the release image's address to the peer's `/api/firmware/url`, as its own Firmware card does, and watches `/api/system` until the peer answers on a new version; a peer without TLS (501) is sent to its own Firmware card, a desktop to the release page. A click on a group's box opens a group popup (`devicegroup.js`): each member with its version and the update its channel offers, and one "Update group" button, pressed twice. The first member behind installs alone, the rest three at a time, the page's own device last; the first failure stops the waves, names the member and why, and leaves the rest untouched. A container, a hand-updated or too-old member is listed as skipped with its reason, a missing one as "not heard"; each line shows its own progress through the shared install flow. +- **MoonDeck's half, decided by the product owner (2026-11-18), landed:** the map runs fleet operations and MoonDeck what needs the source tree. MoonDeck's Live tab draws the same map and popup (`src/ui/devicemap.js`), hears the devices itself on UDP 65506 (`moondeck/_presence.py`), adds its USB actions to the popup (`moondeck_ui/popup.js`), and finds a board's port by its MAC when used, never storing one (`moondeck/_usb.py`); its subnet scan, device list, deviceModel push and own update route are gone. How it works: [MoonDeck.md](../../moondeck/MoonDeck.md). +- **The web installer's half, open:** its device models applied over serial (`APPLY_OP`) against the map's Apply device defaults, and its device report against the popup. +- **Later, its own decision:** the map installing over USB through Web Serial, as the web installer does; browsers allow it on https and localhost only, so from the desktop app's page and not from a board's. +- **Verified by:** the list of overlaps found, each with its decision, and the code and docs that one of each pair leaves behind removed. For MoonDeck: its Live tab draws the same map as a board's page; with no MoonLight running it lists a board that announces itself and one on USB; a board replugged to another port is flashed by MAC without a manual pick. -#### 9. Clock sync ⏳ +## The design -- Absorbs the backlog card, with two changes: the reference is the clock reference role, not a configured leader; and a member measures its offset by the NTP exchange (four timestamps, request and reply), not a one-way send, since the round trip is what measures the delay. The same exchange then works over any transport, the LAN and step 15's relay alike. Aim: under a millisecond on the LAN. -- **A change applies at a time, not on arrival**: a preset or control change is stamped "apply at T", a little in the future, in an OSC bundle's timetag, which OscPacket now accepts. Members that receive it at different moments switch together, which is what lets latency, and step 15's internet, stay invisible. -- With step 7's shared controls, a group now shares both its controls and its time: the base every level of the next phase builds on. -- **Measured by:** the offset spread across a bench group; stop rule named before the first run. -- **Verified by:** a vertical line sweeping across boards side by side, the textbook display-sync test (genlock checks on video walls): straight across a seam when the clocks agree, a step when they do not, the step's width over the line's speed being the clock error. +The principles every step follows, the messages they send, and how an effect spreads over a group. -### One effect across the group +### The principles -#### 10. One effect streamed: correct for every effect ⏳ +- **One building block.** Devices is a top-level container like Services. Its children are MoonModules: the discovery list, and one module per group, added and removed the way an effect is. +- **The UI stays generic.** The map is a list drawn as a graph, the way a list can already be drawn as pads: a list source declares `listAsGraph()`, each row a node and an `edges` array on a row its adjacency list, the textbook graph representation. `app.js` draws it as SVG with a force-directed layout (Fruchterman-Reingold, hand-written, no library). The rows keep the list's streaming and live updates, the UI learns nothing about Devices, and any list can be drawn this way. +- **Listen always, ask on demand** (product owner, 2026-11-09). Listening costs only the packets that arrive anyway and stays on, since group state and a missing member depend on it. Asking costs traffic and work: ArtPoll and SSDP probes, the REST reads of peers for the map and the summaries. Those run only while something reads them, a Devices card open in a browser or a module that needs them, the pull model the preview and [Demand](../../src/core/module/Demand.h) follow. No scan button. +- **Each device reports only itself.** A board on a switched network sees broadcast, the multicast groups it joins and its own unicast, never traffic between two others. So every MoonLight device reports its own flows, and the page merges what its peers report into one map. A flow between two foreign devices stays unseen. +- **No upper limit**, the architecture's [dynamic over fixed-size](../explanation/architecture/moonmodule.md#lifecycle-propagation-to-children) rule. The device list, the flow table and a group's members grow on demand. They grow at a new sighting, a rare event and not a frame, by doubling, so N devices cost about log₂ N allocations, in PSRAM where the board has it. The hot path only iterates. +- **Graceful degradation when memory is short**, the [allocate-and-degrade rule](../explanation/architecture/moonlight.md#degradation-cascade) the light pipeline already follows. Growth is checked against `platform::HEAP_RESERVE` before allocating, so the fleet never takes memory the lights, WiFi or HTTP need. When growth is refused, it steps down, best to worst: + 1. **Keep what matters, drop the rest.** This device, group members and configured peers are pinned. Among the others, the device heard longest ago is replaced first (least recently used), foreign and unknown devices before MoonLight ones. + 2. **Drop the flows memory refuses.** A flow or service that does not fit is left out of the map that second, and the status says memory full. + 3. **Read, don't hold.** A device reading its group streams the members' summaries through without storing them. + 4. **Say so.** The card's status says "memory full", and the map draws what is held. A count of the devices heard but not held would itself take memory per device, so the status names the state rather than a number. -- The member holding the render role renders the whole group canvas and sends each member its region over DDP; the group sets up NetworkSendDriver and the members' Network Receive. Every level is correct this way, so it ships first and stays the fallback. -- Bounded by bandwidth and the LAN: a large canvas or MoonOrbit needs the levels of the next steps. -- **Verified by:** one effect running across three bench boards as one picture. + When memory returns, the lists grow again by themselves. Never a crash, never the lights' memory. +- **Swarm scale.** Hundreds of devices on one network, so nothing costs O(N²). A device hears N presence packets per period; a change to a group goes out once by multicast, not as N unicasts; the page reads N peers with a bounded number of requests in flight; the map draws a large group as one node with its count, opened on a click. -#### 11. Group canvas: position, time and shared input (levels 1 and 2) ⏳ +### The messages -- The group has one virtual size and each member holds its region in it; `elapsed()` reads the synced clock; `BeatPhase` is anchored to the effect's synced start, its rate changes applied at their synced time; `uv` takes the canvas. Every member renders its own region, and no pixels travel. -- A member lacking the effect (Raymarch on a chip without an FPU) gets it streamed, as step 10 does. -- The level declaration on EffectBase, the group's choice from it, and the split-equals-whole test; level 1 and 2 effects declared, each passing it. -- **The demo effect: a ball crossing the screens.** An elastic ball in closed form, x a triangle wave over the canvas width, y a parabola repeating each bounce, both functions of synced time: level 1, no state, each board drawing it while it is in its region. Regions placed at the boards' real positions, gaps included, so the ball takes the right time to fly between screens. BouncingBalls loses energy and relaunches, which carries memory; this is its elastic, sideways twin. -- **Verified by:** the ball crossing three bench boards with no jump at a seam; Plasma, Truchet and AudioSpectrum across them as one picture, the spectrum's bands split over the boards; a speed change mid-run keeping the boards in phase; a board started a minute late in phase from its first frame. - -#### 12. Randomness from the group tick (level 3) ⏳ +No new protocol (product owner, 2026-11-09): each kind of message already has its textbook one, and what is designed is the rules on top. The only new formats are documents: the summary and flows JSON, and the OSC addresses of group state. -- A group tick from the synced clock; effects drawing per-frame randomness move from a `Random8` stream to `hashInt(seed, tick, light)`. -- **Verified by:** the split-equals-whole test for each converted effect; a sparkle effect on boards at different frame rates showing the same sparkles. +| Kind | Loss | Protocol | +|--|--|--| +| **Streams**: light, audio, video | acceptable, latest wins | Art-Net, DDP, E1.31, the MoonLight audio sync format, each its own | +| **Group state**: preset, controls, clock, roles | repaired by the next send | OSC on the groups' one well-known multicast address: the definitions (step 6), the shared controls and looks (step 7) | +| **Requests**: summary, flows, identify, firmware | must arrive | REST over TCP, every peer answering cross-origin (`Access-Control-Allow-Origin: *`) | +| **Discovery** | acceptable | presence, ArtPoll, SSDP, mDNS | -#### 13. Lockstep simulation (levels 4 and 5) ⏳ +- **Audio sync stays its own packet.** It is a stream at ~42 packets a second, WLED-compatible in one format; carrying must-arrive state in it would tie that state to a lossy stream. +- **State, not commands.** A group's state is published as absolute values ("preset X, apply at T"), on a change at once and then repeated, so a lost packet is repaired by the next repeat and a member that joins or reboots catches up. Applying a value twice changes nothing, so no acknowledgements are needed and a swarm costs one multicast send. Presence already works this way. REST stays for what is not state: an update, identify, reading a peer. +- **Apply at a time.** OSC bundles carry an NTP timetag, OSC 1.0's own "execute at T". [OscPacket.h](../../src/core/services/OscPacket.h) recognizes bundles and rejects them today, "a timed batch we have no use for yet"; step 9 gives them their use. +- **No races, no loops:** + - **Leaderless: every member may write** (product owner, 2026-11-09). Each value is a last-writer-wins register (Shapiro et al., *Conflict-free Replicated Data Types*, 2011): it carries a timestamp and its writer's device id, a device applies only a newer one, and a tie goes to the higher id. Every member converges on the same value whatever order packets arrive in, so concurrent edits do not race: two people on one slider at once, one edit wins and all agree which. The timestamp is a hybrid logical clock (Kulkarni et al., 2014), which orders edits before step 9 syncs the clocks and by real time after. Duplicates, reordering and stale values are ignored for the same reason. Today's shared controls are peer-to-peer with an echo check ([OscService.h](../../src/core/services/OscService.h)); whether two boards changing one control at once can end apart is a hypothesis, which the registers settle either way. + - **No forwarding.** A device publishes only what it wrote itself and never re-sends what it received, so no loop can form. + - **Coalescing.** A slider drag publishes at most ~20 times a second, the latest value winning. + - **Repeats by Trickle** (RFC 6206, the IETF's for low-power networks, used by RPL): a member stays quiet when it has lately heard others send the same version, its interval doubling while the group agrees and snapping back to fast on any difference. Traffic stays near flat as a group grows, where every member repeating every second would cost each device N packets a second. + - **Roles, derived, not elected.** A job needing one doer, the clock reference, the member that renders in step 10 and sends keyframes in step 13, the one running a group update, goes to the lowest live device id unless the owner pins one. Every member computes the same answer from the same membership, so no messages are exchanged to agree. +- **MoonOrbit carries the same messages.** One WebSocket per device to the relay carries the same OSC packets and request/response pairs, so a group behaves alike on the LAN and over the internet. -- A fixed timestep from the synced clock, in core, which a simulating effect steps by; integer math; the whole canvas simulated on each device. -- An effect whose memory forgets (Fire's cooling, a fade trail) converges by itself and needs nothing more. One whose memory keeps, and every level 5 one, publishes a state checksum every N steps as group state; a mismatch is repaired by a keyframe from the render role holder, and a late joiner joins by keyframe. -- Fire (forgets), Particles (keeps) and Game of Life (chaos) first, one of each case, the rest by the test. -- **Measured by:** how often a bench group diverges over an hour, and the keyframe's cost; stop rule named before the first run. -- **Verified by:** Game of Life across three boards as one board for an hour, a board rebooted and rejoining by keyframe. +### One effect across the group: the complexity levels -#### 14. Live input across the group (level 6) ⏳ +An effect spreads over a group as easily as its output can be computed on each device from what every device shares. The levels, easiest first, with the 83 effects' shape surveyed on 2026-11-09 (18 draw from a `Random8` stream, 14 from `hashInt`; all animate off `elapsed()` in [EffectBase.h](../../src/light/effects/EffectBase.h), so one clock swap reaches all): -- A gamepad or touch on one member becomes group state, stamped "apply at T" a little ahead, so every member applies it at the same step: the delay-based lockstep of networked games. Where the delay shows, the effect stays streamed. -- **Pong across two boards**, a paddle on each and the ball crossing between them. Its attract mode re-rolls each paddle's aiming error per exchange; drawn as a hash of the exchange number, the rally is deterministic (levels 1 and 3), and a gamepad per board makes it level 6. -- **Verified by:** Pong played with a gamepad on each of two boards, the bench's Xbox Series pad and 8BitDo Zero 2, each paired with the computer showing its board's page ([the gamepad plan](Plan-20261001%20-%20Gamepads%20and%20MIDI%20desks%20(2%20of%207).md) owns how a pad reaches a board); MoonInvaders from one board's gamepad across three. +| Level | The effect's output depends on | Examples | Each device needs | +|--|--|--|--| +| 1. Position and time | where a light sits and when | the shaders (Truchet, Raymarch), Plasma, Noise, Rainbow, Metaballs | the synced clock, a phase anchored to it, and its region of the group canvas | +| 2. Shared input | level 1, plus an input | AudioSpectrum (a band per device), the audio effects | level 1, plus the same input frames: audio sync from one sender, which exists | +| 3. Randomness, no memory | level 1, plus a random draw per frame | sparkles, twinkles | a draw from (seed, group tick, light), not from a stream | +| 4. Memory of its own past | its previous frames | Fire, Particles, Bouncing Balls, Water Ripple | the same simulation steps at the same times: lockstep | +| 5. Chaos, coupled across the canvas | its previous frames, every cell on its neighbors | Game of Life, ripples crossing devices | lockstep, plus detecting and repairing a divergence | +| 6. Live input on one device | a gamepad or a touch | MoonInvaders, Pong | the input as group state, applied at a time | -### Beyond the LAN +What breaks each level, which synced clocks and a shared seed alone leave open: -#### 15. MoonOrbit: groups over the internet, through MoonCloud (stretch) ⏳ +- **Level 1: a phase starts at the device's first frame.** 25 effects animate through `BeatPhase` ([math16.h](../../src/core/util/math16.h)), shaders and Plasma among them. It accumulates `dt × rate` in exact 64-bit integers, so the frame rate cannot make it drift, but it counts from the first frame each device rendered, and a speed change lands at each device's next frame boundary. Two devices are then offset by whatever separated their starts and their frames, for good. The fix is one class: the phase anchored to the effect's start on the synced clock, and a rate change applied at its "apply at T" with the integral split at T exactly, which makes the phase an exact function of synced time on every device. That one change moves all 25 to level 1. +- **Level 1: shaders are level 1 by construction.** A pixel's color is a function of where it is and when, with no state ([power-functions.md](../moonmodules/light/power-functions.md), [shader.h](../../src/light/powerfunctions/shader.h)); Truchet already turns its tiles by `hashInt` on the cell index "so two devices agree on tile (7, 3)". Shaders also gain the most, the heaviest effects per pixel, each device computing only its own share. Three things to watch: + - **Coordinates from the group canvas.** `uv` centers and scales by the panel's own short side; across a group it takes the canvas's size and the light's position in it, or every device draws the whole pattern on itself. + - **Floats are harmless here.** A last-bit difference changes one pixel's color by at most a step and never accumulates, since nothing carries over to the next frame; it matters only where state does. + - **Not every member can run every shader.** Raymarch compiles only on chips with a hardware FPU, so a group checks from the members' summaries that each one has the effect, and streams it to a member that does not. +- **Level 3: a random stream drifts with the frame rate.** A `Random8` stream advances per draw, so a board at 60 fps and one at 45 fps are at different points in it within a second. The textbook answer is a counter-based generator (Salmon et al., *Parallel Random Numbers: As Easy as 1, 2, 3*, 2011): a draw is a hash of (seed, tick, light), which `hashInt` already is, with the tick derived from the synced clock (`tick = synced time / group period`), never the device's frame count. +- **Level 4: a simulation steps per frame.** [FireEffect.h](../../src/light/effects/FireEffect.h) advances once per rendered frame with no time gate, so its state drifts with the frame rate even with the same seed. The textbook answer is a fixed timestep (Fiedler, *Fix Your Timestep*): step n happens at synced time t₀ + n·dt on every device, a late device running the missed steps, and rendering shows the latest step. +- **Level 4: memory that forgets heals itself.** A fade, a blur trail, Fire's cooling and WLED Soap's smoothing each shrink a difference every step (a contraction), so two devices that start apart converge within seconds once their steps and random draws agree. Memory that keeps, a particle's position or a ball's speed, carries a difference forever. So level 4 is two cases: forgetting needs lockstep and level 3 randomness but no keyframe; keeping needs a keyframe too. +- **Levels 4 and 5: floats can differ across chips.** A fused multiply-add on one compiler and not another changes a last bit, and a simulation carries that bit forward, a chaotic one growing it into a different picture. Lockstep effects use integer math, which the coding standards already prefer; Bouncing Balls uses `float` and is checked when it is converted. +- **Level 5: chaos amplifies any difference.** In Game of Life one wrong cell grows into a different board. The textbook answer, from the lockstep netcode of real-time strategy games, is desync detection: each device publishes a checksum of its state every N steps as group state, and a mismatch is repaired by a keyframe, the state copied from the render role holder. Game of Life already fingerprints its board with crc16 ([GameOfLifeEffect.h](../../src/light/effects/GameOfLifeEffect.h)), and its board is bits, about 2 KB at 128×128, a cheap keyframe. +- **Levels 4 and 5: a late joiner has no past.** An effect whose memory keeps, or a level 5 one, joins by keyframe, or by replaying from the effect's start where that is short; one whose memory forgets joins by waiting a moment. +- **Levels 4 and 5: simulating the whole canvas costs memory.** In lockstep each device simulates the whole group canvas and shows its region, the simplest correct shape; halo exchange (each simulating its region and trading border cells per step, the HPC answer) needs a reliable message per step and stalls on WiFi, so it is not used. A device that cannot hold the whole canvas degrades to being streamed. -Devices in different places, owned by different people, in one group: a show running at several sites, or friends' installations running as one. MoonOrbit is a working name. +**Each effect declares its level**, a virtual on EffectBase, and the group picks the strategy from it. The default is the most conservative, streamed, so an effect nobody has looked at is always correct and opts into a cheaper level once checked. -- **A relay, not a stream.** Only the clock and the changes travel: the controls, presets and "apply at T" stamps. Each device renders locally, so internet bandwidth stays small and latency hides behind step 9's scheduled changes. Pixels over the internet stay out of scope, so a group across sites runs effects at levels 1 to 6 and step 10's streaming stays within each site. -- **Outbound only.** A device behind a home router cannot accept a connection, so each member opens one outbound WebSocket to MoonCloud, and MoonCloud relays within the group. On Cloudflare that is one Durable Object per group, beside the existing Worker ([MoonCloud](../explanation/architecture/mooncloud.md)). -- **Opt-in and private**, as MoonCloud is today: a group is joined by a secret code the owner shares, the relay keeps no history, and the [privacy policy](../legal/privacy-policy.md) gains what the relay sees. Messages could be signed or encrypted with the group secret so the relay cannot read them; decided in the step. -- **Reach:** ESP32 boards and the desktop, which have TLS; a Teensy joins through an open page until it has TLS. -- **Measured by:** offset spread and change-to-light delay between two sites; stop rule named before the first run. -- **Verified by:** a bench board and a board at another address in lockstep, judged on a shared video call; a Pong rally between the two homes, one pad in each, paired with that home's own computer or phone. +**Pinned by one test for every effect: split equals whole.** On the desktop the effect renders once at full canvas and once as two half-canvas instances at different frame rates on one synced clock; at each checkpoint the halves equal the whole. A declared level that fails it is a defect. -#### 16. Remote support over MoonOrbit ⏳ +**Prior art: WLED-MM SuperSync** (October 2023, in the local clone `MoonModules/WLED-MM`, commits `37e162472` to `71abd117b`; upstream WLED took the clock half as "Use strip.now instead of millis() in effects AKA supersync", `58eb151ba`): -A user with a problem shares their device, and a helper sees it and replicates the setup on their own desktop: "share your device in MoonOrbit and I will reproduce it". A support session is a group of two over step 15's relay, the helper reading the user's device as a page reads a peer on the LAN. +- **The clock:** `strip.now`, WLED's time with its UDP-synced offset, in place of `millis()`. +- **Levels 1 and 2, done by hand per effect.** Soap, Octopus, Crazy Bees and Polar Lights replaced per-frame accumulation (`noise += mov` each call, `step++`) with a function of `strip.now`: frame count out, time in. MoonLight's effects already animate off `elapsed()`. +- **Level 3, begun:** a random seed set once at the effect's start, fixed (`random16_set_seed(535)`) or from the clock (`random16_set_seed(strip.now)`). It gives the same start where devices start together, and a stream that drifts with the frame rate after it, the gap step 12 closes. +- **The group canvas by configuration:** every node's matrix set to the whole canvas, its panel 0 placed at its offset; a SuperSync checkbox per effect made a node compute its panel only. +- **Fleet consistency:** the Nodes popup fetched each node's info and config, compared version, build, brightness scale, fps, matrix size and panel layout, marked "update needed", and a SuperSync button pushed the LED and light config to each node and rebooted it. +- **Two code paths per effect**, deterministic only while sync was on (`notifyDirect`), kept to compare both modes. -- **The user starts it.** A "share for support" button on the user's device shows a short code, valid for a limited time and ended by the user at any moment. The helper enters it in their own MoonLight, and the user's device appears on the helper's map. -- **Read-only by default.** The helper reads the summary, the state, the log and the flows through the relay's request/response. A change on the user's device needs the user to allow writing for that session. -- **Replicate on the desktop.** "Copy setup" loads the user's state into the helper's desktop instance, desktop first as every change is; a module needing hardware the desktop lacks (pins, a driver's peripheral) loads and idles, as it would on a board without that hardware. The copy is the same one the Backlog's "copy a device's settings" needs, so both share one export. -- **Secrets never travel.** Password controls (WiFi, MQTT, API keys) are left out of what is shared and copied; the [privacy policy](../legal/privacy-policy.md) names what a session shows. -- **Verified by:** a bench board at another address shared by code, its setup copied onto the desktop and showing the same effect, no password in the copy, and a write refused until the sharer allows it. +What it teaches this plan: one deterministic path for every effect, never a sync mode beside a normal one; the region from the group canvas, not a per-node matrix configured by hand; the consistency check as group state rather than a config push and a reboot; and level 4's split into memory that forgets and memory that keeps, which Soap's smoothing showed. ## Backlog @@ -454,6 +393,8 @@ Seeing the network: Acting on devices and groups: +- **The fleet clock's adjustment in the platform layer** (product owner, 2026-11-19): the published fleet time (`FleetTime`'s one static in [ClockSync.h](../../src/core/util/ClockSync.h)) moves beside `platform::uptimeUs()`, as an operating system's kernel holds the clock adjustment (`adjtime`) its NTP daemon steers; core keeps the arithmetic and the service, and `src/core` holds no mutable static for it (check_code's rule). + - **Group id from a random clock sequence** (CodeRabbit on #256, 2026-11-18): a group's id is the creating MAC and the moment it was made, so one board making a group at the same uptime on two boots repeats an id a peer's deletion marker may still refuse. RFC 4122 version 1 adds a random clock sequence for this, which needs a platform random source first. - **A retained leave record per device group** (CodeRabbit on #256, 2026-11-18): a device that leaves a group, or is removed from it, drops the module and with it the stamp of its removal, so an older copy from a third member adopts the group again, and a self-removal a peer's copy settles before it went out never reaches the others. The fix is a saved record of the leave (group id and its stamp) beside the deletion markers: re-adopted only on a newer addition, and the self-removal kept sendable. - **List options carry a value, not a position** (CodeRabbit on #256, 2026-11-18): a list picker sends the index of the option the page showed, which the device resolves against its list at submit time, so a Discovery row aging out between the two adds the wrong device to a group. A core change for every picker: option sets whose entries carry a stable key, the MAC for a group member, resolved by key. diff --git a/docs/work/Plan-20261117 - Faster commit and merge gates.md b/docs/work/Plan-20261117 - Faster commit and merge gates.md index 3b30ac276..160bc294b 100644 --- a/docs/work/Plan-20261117 - Faster commit and merge gates.md +++ b/docs/work/Plan-20261117 - Faster commit and merge gates.md @@ -104,6 +104,35 @@ With step 15's scheduling the critical path becomes the test build, then the lon **Actions**: step 15 extended (the cheap, often-failing checks first; the scenarios gate isolated as the measurement it now is); step 3's next part unchanged (docgen and complexity per file in the hook); step 25 (check_code names each new clone pair against the committed list). +## After-action review: the fourth commit round (2026-11-19) + +A day's work, about 70 files and 2,200 insertions, took five gate runs from 12:33 to 14:22. Run 1 stopped when the Reviewer, beside the gates, found two fleet-clock defects; run 2 failed three ratchets written during the day (code +24, prose +39, docgen 3 errors); the fixes in rounds 3 to 5 moved code inputs each time, and one refactor surfaced an order-dependent test (a process-wide `FleetTime` no case reset). The write-time hook had not run since 08:35 (`build/hook/checked.json` untouched), most likely because the session continued from a summary and its watcher was not registered again (a hypothesis); nothing reported its silence, and the agent read the silence as a clean tree. The last run passed every gate but the code ratchet at +2, both named in the commit. + +## The plan against the textbook (2026-11-19) + +The textbook shape is layered, each layer catching its class of defect at its cheapest moment: **shift left and fail fast** (Humble and Farley's commit stage: [under five minutes, never ten](https://www.oreilly.com/library/view/continuous-delivery-reliable/9780321670250/ch07.xhtml)), an **inner loop** of seconds on the file written and an **outer loop** of CI on the whole tree, every stage checking **only what changed** ([pre-commit](https://pre-commit.com/) on the staged files, [Nx affected](https://nx.dev/docs/features/ci-features/affected)), fed by **small batches** ([Google: about 100 lines a change](https://google.github.io/eng-practices/review/developer/small-cls.html); [DORA](https://dora.dev/guides/dora-metrics-four-keys/)). Agentic development adds that the agent needs a verification signal at every step ([Anthropic's best practices](https://code.claude.com/docs/en/best-practices)), that a guardrail must prove it is alive, since an agent cannot notice a missing check, and that the agent's memory and rule changes stay visible to the human. + +| Layer | When | What runs | Catches | +|--|--|--|--| +| Inner loop | every edit, seconds | the per-file checks on the file written (docgen, code, prose, compile) | budgets, complexity, wording, as written | +| Step loop | per finished plan step, minutes | the affected tests, and the Reviewer on that step's diff | design and logic defects while the step is fresh | +| Commit stage | once, under ten minutes | the per-file checks over the whole diff first, then the affected builds and tests | what slipped, in seconds rather than minutes | +| Outer loop | merge, on CI | everything, whole tree | cross-cutting and platform breakage | + +| Practice | Textbook | MoonLight has | Gap | Refinement, and what it removes | +|--|--|--|--|--| +| Per-file checks at commit | pre-commit and [lefthook](https://lefthook.dev/usage/commands/run) run their hooks on the staged files | `hook_written.py` at write time; the gate runs whole-tree checks | the gate trusts the hook ran; on 2026-11-19 it had not | `run_gates.py` runs the hook's `check()` over every changed file as its first gate, before any build; removes step 20's bypass proposal and the by-hand hook runs | +| A guardrail that proves it is alive | a [dead man's switch](https://healthchecks.io/docs/monitoring_cron_jobs/): alert when the heartbeat stops | the hook writes `build/hook/checked.json` | nothing reads it, so a silent watcher reads as a clean tree | the runner's first line: "hook last ran at