Repository navigation
M4 P0: Three simultaneous SCL-configured 4I4V SV streams with shared PTP grid and per-stream Studio pages #333
Copy link
Copy link
Open
Description
Outcome
From the owner's 01_SV_Stream_4I+4V_(9-2LE).scd, one powered ESP32-P4 must publish three independent 4I+4V IEC 61850 Sampled Values streams simultaneously, with all three tied to one common board/PTP timebase. A single ARStack Studio GUI provides stream pagination for Signal Table, Waveform and Phasor. Each virtual MU has dedicated setpoints, quality, phase state, publisher identity, SV counter, packet template, statistics and control readback. Do not introduce three independent real-time timers, three competing Ethernet send tasks or an XML parser in firmware.
Owner-provided SCD contract
- 3 IEDs:
MU01_4I_4V_1,MU01_4I_4V_2,MU01_4I_4V_3; each LLN0/MSVCB01+PhsMeas1, 16 leaves (4I+4V INT32/Quality pairs), 64 bytes payload. svID=MU01_SV1,MU01_SV2,MU01_SV3; same APPID0x4000, distinct SV multicast MAC suffix01/02/03; VLAN ID 0/priority 4.- Each
smpRate=4000/SmpPerSec/nofASDU=1; aggregate canonical 12,000 SV frames/s. The same APPID across different MACs is an explicit SCD fact, not a reason to alter or collapse streams. - Uploaded fixture SHA-256:
a4aac361dc1a857793400cf5b9d67d4d6a9d9f04ccfa143802c9bd4b7dad4a4e. - Filename says 9-2LE; require explicit profile-family selection or a documented accepted template rule (do not infer blindly from file name).
Implementation packages
- M4.0-A: SCD parser/host compiler resolves 3 stream identities by IED + AP + LD + LLN0/CB; uses existing SCL parser, existing single-stream compiler and binary codec; explicit 9-2LE and validated counter wrap.
- M4.0-B: Shared
MultiSvConfigV2envelope + bounded transactionalMULTISVupload/readback; oldPROFILE V1remains supported. - M4.0-C: 3 preallocated firmware
StreamContextcontexts, per-streamSvLiveSignalBank, DDS phase generator, independent mutable setpoints, per-stream packet buffer/counters/faults. - M4.0-D: One PTP-anchored sample epoch + one GPTimer ISR notification + one deterministic TX publisher task; one 250us grid for all three 4000 Hz streams. No per-stream timer, FreeRTOS task or per-packet heap allocations; preserve fail-closed Ethernet link policy and Studio-close autonomy.
- M4.0-E: Native Qt/QML single GUI with MU1/MU2/MU3 pages synchronized across existing SignalMatrix, SignalPreview phasor/waveform, and shared global PTP/connection strip; no changes of setpoints across pages.
- M4.0-F: CI reference SCD fixture, parse-compile-roundtrip, 3-way monotonic counters and identical sample epochs, streaming setpoint isolation, network byte identity, overload rejection, 100M Ethernet budget, reconnect adoption and GUI pagination.
- M4.1: Lab firmware + Studio paired build, inspect 12,000 distinct canonical frames/s in one Wireshark/Npcap test LAN and per-stream matching APPID, SV multicast MAC and smpCnt.
Rules
- First implementation capacity is exactly 3 active SV; preserve bounds/versioned design for future N.
- A single clock/PTP authority, with identical sample tick/epoch and synchronization decision for same-rate streams. Packet egress on one Ethernet MAC is serial, so same sampling epoch, not physically simultaneous three-wire delivery.
- Internal PTP SOURCE is enough for normal simulated test traffic; external GPS/PTP hardware, relay purchases and expensive conformance tools are not release gates.
- No mixing of identities or setpoints. Stream-specific stop/reconfiguration must not silently reset a sibling. Atomic three-stream deployment must either pass completely or leave the prior profile intact.
- Keep M3.H.3 stable firmware and assets immutable. New implementation is gated behind an explicit V2 capability and later versioned release.
- This issue is engineering implementation work, not a request for more PTP grandmaster acceptance documentation.
Detailed authority: docs/ARD_M4_THREE_SV_SCL_SHARED_PTP_STUDIO.md on main (to be created in the same planning task).
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
masarray commentedon Oct 25, 2026
M4 coding checkpoint — P0A underway (25 Oct 2026)
Code delivered on branch:
feat/m4-three-sv-scl-roster/ PR #334.M4ThreeSvRosterCompilerwith exactly 3 streams, explicit 9-2LE family, validated 4000smpCntwrap, 4000 Hz shared sample-rate contract, 16-leaf/64-byte layout check, whole-roster rejection on one invalid entry.0x4000with different multicast MACs; reject an identical complete wire identity.SmvOpts sampleSynchronized=true; CTest performs three V1 device-profile binary encode/decode round trips and checks exact 12,000-frame/s intended aggregate.No changes to M3.H.3 official stable binaries. A compiler test proving 3 profiles is not proof of three live ESP32-P4 Ethernet streams.
masarray commentedon Oct 25, 2026
M4-P0B code checkpoint — binary roster codec (25 Oct 2026)
PR #335 is stacked on P0A #334.
M4ThreeSvBinaryCodec: versioned 28-byte V2 envelope (ARM3) with three nested legacyARSVV1 device profiles.std::span, no runtime heap, bounded total/record lengths, big-endian wire fields, nested CRC and aggregate CRC32.Not yet active on firmware: V2 binary is an on-host/on-repo contract only until the ESP32 firmware explicitly advertises and handles it. M3.H.3 stays untouched.
masarray commentedon Oct 25, 2026
M4 checkpoint updated — P0A + P0B MERGED TO MAIN
Main commits:
e56560bdf32bbe5ec138fcac900eb21efe77b91bvia PR #334bcc497a914f0bec8702551f9caa0e18cbd2962devia PR #335Cross-platform CI: P0A C++ GCC, Clang, MSVC passed, and all nine associated PR workflows passed. P0B tested as a stacked PR with GCC, Clang and MSVC passed before rebasing to main (code contents unchanged); both PRs are merged and stable release M3.H.3 remains untouched.
Next prioritized coding task: P0C1 implement firmware-side
MULTISVV2 staging / immutable roster commit / readback / protocol capability while preserving old V1. Then P0C2 one PTP sample grid -> three canonical packet templates and one deterministic TX publisher task. Then P0D per-stream isolated setpoints, P0E native Studio three-page UI, P0F matched firmware/Studio traffic test release.Not done: three concurrent SV outputs, PTP phase association of all 3, GUI pagination, firmware protocol V2 support, and release M4. Never infer these from passing host compiler tests.
masarray commentedon Oct 25, 2026
M4-P0C1 coding progress — firmware multi-SV binary control plane
PR #336: #336
Already coded (feature branch, CI running):
MULTISV BEGIN/CHUNK/VALIDATE/COMMIT/ABORT/STATUS/SHOWin existing ESP32-P4 UART control task.Critical distinction: firmware's
MULTISV COMMITTEDmeans control-plane configured but TX INACTIVE. Capability isMULTISV-CONFIG-STAGING-V2, notSV-MULTI3-V1. The firmware does NOT emit three streams until P0C2 scheduler + three data paths are completed.Next: resolve CI, merge P0C1 if all required gates pass, then implement P0C2 one hardware/PTP 4000 Hz sample grid with 3 preallocated stream contexts and ordered EMAC TX, followed by P0D isolated setpoints and P0E one Studio/three GUI pages.
Protocol and safe operation: docs/M4_P0C1_MULTISV_FIRMWARE_PROTOCOL.md.
masarray commentedon Oct 25, 2026
M4 update 26 Oct 2026:
P0C1 COMPLETE/MERGED — PR #336 -> main commit
e35c09fe3263d31d1d6d4804f9795417c3b3460e. Firmware now receives 3 V2 SCD-derived SV profiles via boundedMULTISV BEGIN/CHUNK/VALIDATE/COMMITwith atomic control-plane roster and readback, legacy V1 preserved. Nine CI workflows PASSED. Deliberately no triple SV TX claim yet.P0C2A CODED/OPEN — PR #338: deterministic three-slot 4000Hz shared-grid planner with identical same-tick
smpCnt/smpSynch, per-slot enable, skipped-tick accounting and group fail-closed on PTP lease discontinuity. New CTest checks 8000 ticks and 24000 intended frame descriptors. CI running.Next P0C2B MUST integrate with real ESP32 code: one existing GPTimer ISR, one publisher task, one EMAC/PTP authority, three stream-specific prebuilt SV packet templates and three dedicated waveform banks. Validate actual packet throughput/latency on physical board; prior compiler/scheduler tests are not wire proof. Continue P0D per-slot atomic live edits and P0E Studio MU pagination afterward.
Keep 3-MU uploaded SCD golden L2 identities and official M3.H.3 stable untouched.
masarray commentedon Oct 25, 2026
M4 execution checkpoint — 26 Oct 2026, next-phase coding
Merged to main: P0A #334 (3 SCD profile compiler), P0B #335 (ARM3 V2 binary), P0C1 #336 (firmware atomic MULTISV staging/readback), P0C2A #338 (deterministic 3-slot shared GPTimer/PTP sample-grid planner; all 8 relevant PR workflows PASS; merge
45d5b3b7967c34ec651c84d0827e3260b89dc05b).In active PR review/CI:
MULTISV SELECTafter committed V2 roster; existing verified START triggers 3 distinct canonical SV Ethernet frame templates per single sample tick, sharedsmpCnt/smpSynch/PTP epoch, per-MU DDS and memory, no additional RTOS tasks/timers. Own PTP timer and Ethernet failure handling preserved, diagnostic mirrors disabled in 3SV mode. Adds 3-frame EMAC submit duration/max/over-250us telemetry. Optimized sine LUT from three separate 16KiB runtime-generated tables to ONE shared ~4KiB flash quarter-wave LUT; 4096-position parity test (<=1 Q30 LSB).SvLiveSignalBankcontainer; per-slot complete 8-channel setpoint, phase/quality, independent bank generation; CTest asserts changes to MU2 don't affect MU1/MU3 and GUI page switching retains independent values.CI status: ESP32-P4 and PTP SOURCE/RECEIVER firmware compile PASSED on current #339; GCC/Clang C++ passed; MSVC and native preview still completing. PR #340 CI running. Do not merge until checks are green.
Not done yet: physical capture of nominal 12,000 SV frames/s, per-MU live
SVSTREAMcontrol/telemetry and independent running edits (P0D2), Studio three-page Table/Waveform/Phasor and SCL deploy UX (P0E), matched M4 release / hands-on bench test (P0F). The three extra streams initially inherit the current live bank as distinct immutable snapshots on START; do not call them independently editable yet.Next actions: merge #339/#340 after fully green → P0D2 wire independent MU banks and control-plane atomic transactions/readback to realtime publisher → P0E single Studio GUI navigation and deploy → P0F CI & practical packet capture. Keep M3.H.3 stable binaries unchanged; do not demand purchased PTP/IED hardware as a gating condition.
masarray commentedon Oct 25, 2026
M4 latest execution checkpoint — 26 Oct 2026:
c37f277ec96894929f9d270b4c45e5f5e9a1fdaf: opt-in 3 canonical Ethernet SV publishers on the same existing GPTimer/PTP sample grid, one publisher task, 3 SCD-derived packet identities, optimized quarter-wave shared Q30 LUT, measured TX-loop batch duration instrumentation. All 10 PR workflow checks PASSED. No final physical-board 12k-fps capture yet.7ee7ad39d76f8072b7940f438143049549b21bb6: independent 3× atomic waveform banks with per-MU generation and isolation CTests. All 8 PR workflow checks PASSED.SVSTREAM <1..3> SET/FREQ/SHOWwith per-MU generation/readback. This must pass ESP32 firmware CI prior to merge.Remaining: P0D3 transactional multi-channel scenario application and per-stream START/STOP, P0E unified Studio 3-page Table/Waveform/Phasor pagination + SCD upload UX, P0F matched binaries and 12k-fps real-traffic lab acceptance. Legacy M3.H.3 release stays unchanged. Do not claim physical PTP/throughput measurements without captured evidence.
masarray commentedon Oct 26, 2026
M4 latest verified handoff is now in main: docs/M4_HANDOFF_TO_NEXT_THREAD_20261026.md, commit
de5d04fd7e1f6c03eca15b9b373f1d88c5e67127.Merged: P0A #334, P0B #335, P0C1 #336, P0C2A #338, opt-in 3-SV real TX code P0C2B #339, isolated signal banks P0D1 #340.
In progress: P0D2 PR #341 (3-bank firmware setpoint/readback integration), CI passes on observed workflows; confirm current state before merge.
Not done: multi-command atomic MU scenarios/P0D3, per-MU START/STOP, native Studio MU pagination + SCL deploy UX/P0E, matched M4 artifacts and measured 3x4000fps wire capture/PTP batch timing/P0F.
Next thread should read the new handoff and execute code+CI, avoid duplicate branches or outdated P0C1 roadmap. M3.H.3 official stable assets unchanged.
masarray commentedon Oct 26, 2026
M4 continuation checkpoint — 2026-10-26
DONE (merged): P0D2 PR #341 squash-merged to
mainasc57e154c0e30d40df4a5fc8c97ee039556b9d828. Before merge all seven associated CI workflows were SUCCESS. Independent MU1/MU2/MU3 signal-bank edits feed triple-SV TX; single-SV legacy path preserved.IN PROGRESS: P0D3 PR #342, head
be01dfb245a2437a68ac885d3e1c0a3bd3a5aca0, branchfeat/m4-p0d3-per-mu-scenario-transactions. Implements 3 complete bank scenario transactions with canonical hash, stale-generation guard, timeout, commit/readback, and per-MU START/STOP transmission gate captured once per shared 250us tick. All DDS engines advance on same grid even while individual stream TX is disabled. Portable regression tests and protocol note included. CI running; not merged yet.PENDING: P0E single native Studio pagination and three-profile SCD→ARM3 deploy/readback/reconnect adoption; P0F paired firmware/Studio builds and physical one-board 3×4000fps capture, PTP sample alignment and TX deadline/skew measurement. Stable M3.H.3 assets unchanged. Never equate CI with physical on-wire acceptance.
masarray commentedon Oct 26, 2026
M4 P0E1 checkpoint — 2026-10-26
Code is now in PR #343 (
feat/m4-p0e-studio-mu-pagination-model, headbde59a4c2e25bf2a1ae518eb491bb313f8321926). QtM4StreamScenarioModelowns three separate IED-qualified MU draft states. Existing Studio SignalMatrix, waveform and phasor surface are reused as one GUI with MU1/2/3 pagination. SclProfileModel supplies exactly-three qualified SCD profile rows. In this interim GUI milestone, START and all legacy single-SV mutation requests are explicitly disabled whenever M4 draft paging is active, avoiding accidental one-MU-only deployment. Added nativearstack_m4_pages_harnesswired to Linux/Windows Qt and native preview CI.CI: PR #343 workflows currently in progress; not merged. The UI is explicitly draft-only. No
MULTISVbinary upload, firmware readback or host-operated M4 START is claimed. These remain P0E2/P0E3, followed by measured on-wire P0F. PR #342 P0D3 has all observed core CI successful; its Windows native matched Studio preview packaging remains in progress. No M3.H.3 stable changes.masarray commentedon Oct 26, 2026
P0D3 official code merge checkpoint
PR #342 merged to
mainasc79bbfbc17c78641dea5e48a9b8bd981b185c31cafter all 10 GitHub Actions workflows on final PR head6183623404888160298071d0cb737e4e0c9f9434completed successfully, including native matched Studio preview, C++ GCC/Clang/MSVC, Clang ASan/UBSan, ESP32-P4 injector, PTP and Studio Qt.Implemented: per-MU 8-channel atomic CFG transactions with canonical hash, timeout, generation guard, confirmed readback; isolated per-stream TX enable bits sampled on one shared publisher tick. No real-hardware 3×4000fps/PTP performance claim. PR #343 P0E1 native Studio paginated MU drafts under CI, explicitly draft-only (legacy START blocked).
masarray commentedon Oct 26, 2026
Final live checkpoint of this implementation turn — 2026-10-26
DONE / MERGED into main: PR #341 P0D2 (
c57e154c...) and PR #342 P0D3 (c79bbfbc17c78641dea5e48a9b8bd981b185c31c); #342 full 10/10 required observed workflow suite success before squash merge. Updated this thread's authoritative M4 handoff on main as526466400ac0d51f8079cb4537ccafef473b3740.IN PROGRESS / OPEN PR #343: P0E1 single native Studio MU1/MU2/MU3 pagination, branch
feat/m4-p0e-studio-mu-pagination-model, HEAD419fc4326f7abe1b22a274322fc14f299ef709d6. Model stores independent waveform, phase, quality, eight channels and frequency per MU and SCD identity; the single SignalMatrix/Waveform/Phasor surface reuses model on page switch. Only classifier-ready 4kHz 9-2LE profiles activate M4 drafts. Hardened all legacy single-SV mutation paths against M4 draft mode; START disabled. Native harness covers identity, out-of-range tabs, frequency, phase, quality, waveform isolation. Latest GitHub CI: Linux AppImage, RC2 Acceptance, CI Routing, Docs and Product Naming succeeded; Qt, Windows Lifecycle and Native Preview still running at inspection. DO NOT MERGE UNTIL ALL GREEN.PENDING: P0E2 host ARM3 encoder and MULTISV serialized upload/verified identity+readback; P0E3 atomic scenario/live per-MU Studio control, robust reconnect adoption; P0F exact-head paired firmware/Studio and physical 3×4000fps+PTP evidence. No physical throughput/phase claim.
masarray commentedon Oct 26, 2026
M4 continuation checkpoint — 2026-10-26 (P0E1 review repairs)
feat/m4-p0e-studio-mu-pagination-model: legacy scenario persistence must not store M4 MU drafts, and pending frequency edits must commit before MU navigation (479ed050).c1c0ce0) and Windows native-preview CI gate (bf7f965). Exact PR headbf7f965206a92619ba1996e8c313be836126cd0c: CI queued/in progress at checkpoint; NOT merged, not claimed passing.masarray commentedon Oct 26, 2026
M4 P0E milestone checkpoint — 2026-10-26
DONE / merged: #343 P0E1 one native Studio MU1/MU2/MU3 GUI with independent Table/Waveform/Phasor draft states, plus fixes for legacy scenario persistence and frequency-navigation commit. All 17 executed checks passed on exact head (1 release publish step intentionally skipped). Squash merge 6de01f3.
IN PR: #344 P0E2A real Studio host compiler. Production
SclProfileModelcaches one ARM3 V2 encoded envelope and full-envelope 64-bit hash from three exact validated SCD stream profiles, including isolated per-MU VLAN overrides, repeated genuine APPID 0x4000 and SCD MAC/svID identities; fail-closed on ambiguous/invalid source. Native C++ SCD compiler harness exercises codec roundtrip/CRC, 3 identities, isolation, deterministic re-encoding, and cache invalidation on source clear. Windows/Linux Qt and matched preview CI gates added. HEADaa0a16889d3b1a1565da85b4672cb980a31e4857; CI ongoing, not yet merged.PENDING: P0E2B ack-serialized firmware MULTISV BEGIN/CHUNK/VALIDATE/COMMIT/SHOW/SELECT and boot/generation/hash readback; P0E3 per-MU scenario readback/reconnect; P0F paired hardware on-wire 3 × 4000fps and PTP phase/batch timing evidence. No physical performance claims. M3.H.3 official release assets unchanged.
masarray commentedon Oct 26, 2026
M4 continuation — P0E2A landed; P0E2B underway (2026-10-26)
6de01f3ccf6bde396c0c48a85ceec6d465cc6c9f.310be156e5e71a496ae65639d8374d075235de06. Linux Qt, Windows Qt, matched Windows preview/ESP32 firmware CI and other executed checks PASS; release artifact publishing intentionally skipped. Test covers 3 exact MAC/svID/APPID/VLAN identities, MU2 override isolation, deterministic encode and source invalidation.feat/m4-p0e2b-ack-gated-uploader,c49cfe54a3f2602e150cb39bc2a6633f2850f9ab) adds compiled host ACK-driven MULTISV STATUS/ABORT/BEGIN/48B CHUNK/VALIDATE/COMMIT/SHOW/SELECT state machine. Verifies exact transaction/range/CRC, committed full hash+size+generation, three independent firmware readback rows and boot changes. Native Linux/Windows harness + matched preview CI running; do not merge prematurely.masarray commentedon Oct 26, 2026
P0E2B review/CI checkpoint (2026-10-26)
maindirectly; exact head2d42a09beade084ed704101638d6eee052ff395d. No replay of merged feat(M4-P0E2A): compile exact Studio 3-MU SCD roster to atomic ARM3 V2 binary #344.MULTISV TXMODEinformational SHOW output was treated as an ACK. Fixed ina71d3794; precise stage/rejected-line diagnostics added in2d42a09b, matching Codex P1 review finding. New-head Linux Qt build/harness PASS; Windows Qt harness PASS; firmware build PASS.DRAFT ONLYuntil productionDeviceControllerintegration and real firmware readback/session/timeout gates. No hardware 12kfps claim.main(6de01f3,310be156). M3.H.3 Stable untouched.masarray commentedon Oct 26, 2026
M4 milestone completion — 2026-10-26, GitHub verified
DONE MERGED: P0E1 #343
6de01f3ccf6bde396c0c48a85ceec6d465cc6c9f(single Studio, MU1/MU2/MU3 isolated draft pagination); P0E2A #344310be156e5e71a496ae65639d8374d075235de06(real SCD → 3 exact device profiles → CRC-checked ARM3 V2 + full-envelope firmware hash, per-MU VLAN isolation, Qt Linux/Windows regression); P0E2B #345be54592af7be086f1d883d4d0db9fe1af3807224(bounded ACK-driven host MULTISV transfer engine, exact 3-MU firmware SHOW/readback and SELECT gate, Linux/Windows Qt + matched firmware/native Windows preview/installer CI PASS, release upload intentionally skipped). Codex P1 harness review fixed and resolved.MAIN: latest post-handoff docs checkpoint commit
f5e3575c19fa749a2d4c0510a061866374f1fb68;docs/M4_HANDOFF_TO_NEXT_THREAD_20261026.mdnow begins with authoritative current checkpoint to prevent other threads repeating merged work.PENDING P0E2C: transport-neutral transaction code in #345 is compiled but NOT hooked into the actual single
DeviceController/DeviceIoWorker; no production UART upload or M4 START button yet. Must add ACK/session/boot timeout and STOPPED safety, wired deploy/readback/SELECT and verified START without deselecting M4. PENDING P0E3: per-MU live full-scenario UI readback/reconnect adoption. PENDING P0F: real ESP32-P4 3×4000fps/one PTP grid Wireshark capture and overrun/skew evidence. CI does not prove physical performance. Official M3.H.3 release assets and V1 protocol untouched.masarray commentedon Oct 26, 2026
M4 P0E2C1 exact-head continuation checkpoint — 2026-10-26 (CI-conserving)
782b02fa40135d3a1244924252013bde4a71ec33. Branch:feat/m4-p0e2c1-single-owner-studio-deploy. OPEN, mergeable, NOT MERGED as of latest check.M4MultiSvUploadTransactionto one DeviceIoWorker withcommandTransmitted-gated ACKs, 5s per-step and 120s monotonic overall deadline, session/boot/disconnect/rejected-transport cancellation, STOPPED and staging+TX capability checks, actual 3-row/hash/generation firmware SHOW/SELECT verification. No extra worker, no firmware hot-path modifications.782b02f: ModernRibbon Stop, Main F6, WorkflowBar STOP dispatch now allow synchronous cancellation. Review thread resolved.qt-build-smokeincluding deterministic testsM4 roster never authorizes V1 STARTandM4 upload STOP revokes transaction proof, S8B Windows ownership, ESP32-P4 firmware build, Linux AppImage, Windows launcher parser, public-navigation and naming guards. PENDING at checkpoint: Windows native matched preview and Windows close/process exit/reopen lifecycle. No failures; no extra CI jobs/matrices, and no rerun of passed jobs.mainonly once all executed gates pass. Don't merge based on in-progress runs. Then implement P0E3 independentSVSTREAM CFGtransaction/readback and physically measure 3×4000fps/PTP in P0F.docs/M4_P0E2C1_STUDIO_SERIAL_DEPLOYMENT.mdis in PR feat(M4-P0E2C1): Studio three-MU deploy with single-owner ACK+readback gating #346.