[Japabu]

Idle CPUs now wait in the deep states their firmware's _CST declares, chosen per idle period by the pass that puts them to sleep. The wake-latency bound has explicit authority (Rights::WAKE_LATENCY, SYS_WAKE_LATENCY_HOLD = 147). This is stage 5 of the scheduler/audio/idle plan. It takes the T14's one-thread clock from ratio 38 to 42 at 5–10 of 17 osbench group starts (Linux: 13) and runs crunch.single 1.10 times faster. issues/idle-cpus-enter-the-c-states-cst-names.md stays open, because #1066's exit is unmet.

Problem

ToyOS idled in sti; hlt (C1). On the T14 (i5-1135G7), turbo counts a C1 core as active, so MSR_TURBO_RATIO_LIMIT held a one-thread load to ratio 38 (3789 MHz) instead of 42 (#1066).

What changes

The states come from the firmware, interpreted in userland.

• acpiserver evaluates each processor's _CST and _CSD (system/acpiserver/src/cst.rs), and again on that processor's Notify(…, 0x81).
• It hands the kernel, through op::IDLE on the acpi claim, only the states that need nothing done around their entry.
• Withheld by name:
  • type 1, because C1 is hlt;
  • Intel FFH asking bus-master avoidance;
  • any other FFH encoding;
  • SystemIO on Intel;
  • type-3 SystemIO, whose caches OSPM must keep coherent (ACPI 6.5 §8.1.4);
  • states that _CSD coordinates in software.
• An Intel FFH state of type 3 is handed over. 302223-007 Table 4 encodes bus-master avoidance as its only OSPM duty, and the processor flushes what a C-state powers off as it enters it (SDM Vol. 3B §20.19.6.4).
• Residency is 2 × latency, a policy (_CST declares none).

The processor _OSC commits ToyOS's claim (Intel bits 4 and 9) with no query. This is the orchestrator's ruling of 2026-10-27, under the owner's direction.

The kernel's boundary (lib/userbound/src/idle.rs) refuses:

• an MWAIT hint that leaf 5 does not enumerate;
• a residency shorter than the state's exit;
• a CPU without ARAT or an invariant TSC;
• a port the acpi holder could not read itself (kernel-kept, protected or lent).

The port comes from the firmware's AML, which is trusted platform code, so that fence needs no list of parts (ruled 2026-10-28). A declared port that is not a trapped C-state request returns at once: a short wake, never a hang.

The pass chooses the idle state, and its timer plan arms the choice's wake: one timer write per idle cycle, as on main (ruled 2026-10-28).

• Machine::idle_plan is asked in the pass, against the timer the CPU would sleep under.
• The plan's wake joins TimerPlan::no_later_than. It is either the early wake before a deadline (by the state's exit) or the revisit.
• The SleepToken carries the plan to the wait, which touches no timer.
• The simulator plans C1.

The selection rule (kernel/pure/sched/idle.rs) picks the deepest state that satisfies all three:

• its exit is within the held wake bound;
• its residency fits before the timer;
• fewer than 4 of the CPU's last 8 unforeseen wakes came sooner than its residency.

When the history alone holds a CPU shallow, the CPU revisits after the next deeper state's residency. A revisit that nothing interrupted clears the history it covered, so a CPU left idle climbs one state per residency.

Waking: a wake forward prefers, among equally near free cores, the one whose state is quickest to leave. Spawn placement is unaffected; a host test holds it.

The wake-latency bound has explicit authority.

• SysCap::hold_wake_latency takes the WAKE_LATENCY right.
• A hold applies from the moment it is made. The idle CPU publishes its exit, fences, and reads the bound again. The hold stores the bound, fences, and kicks every CPU whose exit exceeds it (CpuHandles::wake_slower_than, host-tested).
• soundserver holds half a period while a stream plays, a policy that stage 4 replaces.

Instruments:

• The counters row gains the deep-idle count and nanoseconds. It also gains core C6/C7 residency on each model whose SDM Vol. 4 (325462-093) section names Table 2-20, the table that defines 3FDH and 3FEH. Of the tables that supersede it, only Table 2-57 names either address, and it defines both (quotes).
• Around every crunch and malloc_churn row, and across tail, osbench notes the CPUs the row's threads ran on. For each CPU, over the readings' own window, it also notes:
  • the APERF/MPERF ratio. SDM Vol. 3B §17.2 and AMD's APM Vol. 2 §17.3 define only this ratio, so no clock or busy fraction is read off it; the clock stays cpu.clock's;
  • core C6/C7 residency, which counts at the TSC's rate (SDM Vol. 4 Table 2-20);
  • the deep periods that ended in the window.
• The window rule (osbench_rules::window, host-tested). The window comes from the counter stamps the two readings carry. It stands only between the row less its two reads' cost and the row plus the kernel's 10 ms join; outside that, or where a CPU was read stale, the CPU is refused by name. window_us, row_us and reads_us are printed. The kernel's join is recorded in issues/a-counters-read-within-10-ms-of-another-returns-the-older-reading-as-new.md.
• The kernel leaves Sierra Forest (06_AFH) out of the core residency counters: Table 2-57 counts its residency at the P1 clock, not at the TSC's rate that Counter::CoreC6 promises.
• Around each group's cpu.clock, osbench names the processes that ran beside it, from the roster. A counters read kicks every CPU, which would disturb a 5 ms reading.
• Defect, fixed: this instrument opened the counters' Diag, which took the process's one SysCap endowment first. osbench identity and osbench fair then found none and exited 1 in every B3 and B4 boot. The metal judge reports it (r4 judge4-*), but it was never run over those readbacks. sys::syscap() is now the one owner of the endowment.
• osbench_capability_groups_exit_0 (QEMU, x86-64) boots benchcase.toml with those two groups and holds each to exit 0. The image's endowments are not host-buildable, and the benches run osbench only nightly.

AArch64 keeps wfi (issues/aarch64-cpus-idle-only-in-wfi.md).

Measured

• Third boots: at f6510defd, alternating A*, B3, A*, B3 on each bench. A* has the deep states off, all else equal.
• B4 to B7: one boot per bench each, at d286dddf3, d60d727aa, 55269e97f and 18fe4e150.
• Evidence: r8, r7, r6, r5, r4, r3, r2.
• In the table, the B column lists B3, B3, B4, B5, B6 and then B7.

Reading (p50)	T14 A*	T14 B3	AMD A*	AMD B3
crunch.single.deflate	60.37 / 60.29 ms	54.58 / 54.58 / 54.63 / 54.80 / 54.88 / 54.60 ms	66.59 / 63.52 ms	63.37 / 63.59 / 66.66 / 63.70 / 65.72 / 63.09 ms
malloc_churn.1048576.t1	29.6k / 29.5k	32.6k / 32.6k	38.5k / 37.7k	39.4k / 39.5k
cpu.clock at 4189, of 17 group starts	0 / 0	8 / 10 / 10 / 5 / 10 / 10	—	—
ipc.reqrep.8 (ns)	7670 / 7669	7231 / 7132 / 6992 / 6969	8005 / 8326	6492 / 6593 / 6562 / 6642

• T14 crunch and churn: the clock beside deflate and malloc_churn.1048576.t1 (each row's mhz, cpu.clock's reading) is 4187–4189 MHz in all eight boots with the deep states, against 3788–3789 in the four A* boots. Across deflate, every core but the thread's is 99.5–99.9 % of the row in core C7. Linux reaches 4190 at 13 group starts.
• T14 group starts at ratio 42: across the eight boots with the deep states, 8, 8, 8, 10, 10, 5, 10 and 10 groups reach it. Only sched, files, ipc and work reach it in all eight, so the count varies from boot to boot rather than with a fixed set of groups.
• T14 groups still at 3789:
  • Boot, identity, energy and system run while acpiserver is still loading its tables (r4 t14-2-B3-boot-groups-timed.txt). acpiserver ran for each whole clock window: a measured busy core.
  • In net, procs, mem, tail and nvme, nothing beside osbench ran more than 0.9 ms of the 5.5 ms window.
  • Mem is unexplained.
  • Net, procs, tail and nvme reached 4189 under r1's always-deepest control. The open hypothesis is that the history keeps idle CPUs shallow after the group before.
• AMD deflate:
  • Measured: across twelve boots, three read 66.6–66.8 ms (on CPUs 4, 4 and 10), B6 read 65.7 ms (CPU 0), and eight read 62.9–63.7 ms. The thread's APERF/MPERF ratio differed by under 1 % across them, and the slow mode came back on A*, with every idle CPU in C1.
  • Inferred: the spread is the bench's, not the idle states'. Its cause is open (issues/the-amd-benchs-deflate-runs-6-percent-slow-in-some-boots-at-the-same-clock.md).
  • An SMT-sibling rule built on the opposite reading was measured against this control and is not part of the change.
• Accepted exit cost (ruled 2026-10-28; numbers with Linux's in the track issue). Each is B3 against A*, at p50:
  • T14 time.sleep_late: 1 µs 1.5×, 10 µs 1.4×, 10 ms 1.6×; 1 ms 1.25× and 0.82×.
  • T14 wake.storm.drain: w4 3.3–3.4×, w16 1.9–2.2×, w64 1.5–1.6×.
  • AMD time.sleep_late: 10 µs 1.4×, 100 µs 1.1×. The AMD's 1 ms row improves 5.8×.
  • Everything else in sched, time, ipc, tail and work is at A* or better, or moves in one run of two only.
  • The single-run outliers are AMD pingpong.pipe.same_cpu (7605 against 5.6k ns) and wake.storm.call.w64 (15100 against 9.6–10.1k). Their other B3 runs read at A*.
• Testcases boots (r2 and r4 with the deep states, r5 A* without them):
  • hda_tone passes with no underruns, and acpi_tables_loaded passes with every CPU's declaration kept.
  • Soundserver's worst IRQ lateness, over every window line, is unchanged by the deep states. T14: 0–564 µs with them (r2, r4), 181–572 µs with every idle CPU in C1 (r5 A*). AMD: 0–2661 µs both ways, with the 2661 µs window on A* too. No window underran.
  • test_rs_wake_onto_busy fails on every revision; it is a standing defect.
  • test_rs_pci_reclaim asks the AMD for an Intel NIC it lacks.

Checks

Check	Revision	Result
cargo run -- --ci host	18fe4e1	exit 0, 104 steps
QEMU osbench_capability_groups_exit_0	18fe4e1	exit 0
cargo run -- --build-only	145e659	exit 0
Instrument rows out of range at d60d727	run4, run5	87 lines with deep_permille above 1000; the reason for the fix (r6)
r6 mutation: the window rule unchecked	55269e9	exits 101
Harness judge, B6 benchcase	55269e9	exit 0 on both benches. No instrument line refused; every window within 0.1 % of its row; deep_ended_us exceeded its window on 44 lines and was removed (r7)
Harness judge, B7 benchcase	18fe4e1	exit 0 on both benches. 96 and 192 instrument lines, none refused, every window inside its bounds; aperf_mperf_permille 188–1749 (T14) and 776–2358 (AMD); C7 0–999 ‰ (r8)
r7 mutations: join unbounded, reads unbounded, stale ignored, 06_AFH admitted	862f68c	each exits 101
Harness judge, B5 benchcase and A* testcases	d60d727	exit 0 on all four boots: PASS osbench on both benches; hda_tone, hda_tone_resampled and acpi_tables_loaded pass on both A* boots (r5 logs)
QEMU osbench_capability_groups_exit_0	49a98e3	exit 0; with the fix reverted, exit 1 (identity and fair exit=1)
Harness judge over the B4 readbacks	d286ddd	exit 1 on both benches: osbench_identity exited 1, the defect fixed here. hda_tone, hda_tone_resampled and acpi_tables_loaded pass
8 r1 mutations	b392ba8	each exits 101 (r1)
7 r2 mutations	7b78def onward	each exits 101 (r2)
4 r3 mutations: revisit learning; the hold kick at the bound, of the holder, and past the ring	d286ddd	each exits 101 (r3)
r4 mutation: Plan::learn's arms swapped	49a98e3	exits 101 (r4)
QEMU hda_plays_to_the_controller_that_reaches_an_output, acpi_press_is_one_sci	59dcd1f	exit 0. Every QEMU plan is C1, so later changes leave this standing

Risks

• The T14's one-thread clock is held to its three-active-core turbo bin by hlt idle; deep idle lifts it 10 % #1066's exit is unmet: 5–10 of 17 group starts reach ratio 42 (8, 8, 8, 10, 10, 5, 10, 10 over eight boots), against Linux's 13.
  • Boot, identity, energy and system overlap acpiserver's start-up, a measured busy core.
  • The rest are open on the track, with two diagnostic boots staged as its follow-up.
• On the AMD, the sti; in entry relies on the Family 16h BKDG for wake on interrupt.
• AMD's 0x415 is withheld.
• Residency at 2 × latency and the 4-of-8 history are policies.

After d60d727 the branch changes only issues/, a test registration comment and an assertion message, then merges main twice: SYS_VM_REMAP (#1070) keeps 146 and the wake-latency hold takes 147, which #1077 also claims, so whichever lands second renumbers. Then it fixes the counters instrument.

🤖 Generated with Claude Code

[Japabu]

Design round, as approved (orchestrator, 2026-10-27, under the owner's direction for this work that day). The rulings on it:

1. Processor _OSC: commit ToyOS's claim (Intel bits 4 and 9) with no query; \_SB._OSC keeps query-then-commit.
2. A separate WAKE_LATENCY right; a machine-wide WakeLatencyHold; soundserver's mix thread holds period/2 while streaming, a policy stage 4 replaces.
3. AMD's 0x415: withheld only if no vendor document establishes coherence. None for family 17h was found (the PR body says what was searched); it is withheld with an exit. _LPI versus _CST is settled by ACPI 6.5 §8.4.3.5: ToyOS claims no LPI support, so _CST and its figures are the interface.
4. Residency = 2 × latency and the 4-of-8 history are labelled policies at their sites. A platform-declared residency is used where one is authoritative, which on these benches none is (_CST declares none).
5. The shallowest-sleeper placement tiebreak is included.

One addition since the design: a CPU that only its history holds shallow revisits after the deeper state's residency. Without it, an idle CPU could stay in C1 indefinitely and hold the T14's bin down.

The design as written:

Deep idle: design (stage 5), for architecture check before implementation

Worktree /Users/jan/Dev/jan/toyos-deep-idle, branch deep-idle at main 8348c2a2c. Nothing is built yet.

0. Facts this rests on (M1, read tonight)

Each fact is taken from the vendor document or the spec, or measured with our own interpreter on the benches' dumps. That measurement used #875's out-of-tree harness, rebuilt against this worktree (m1/, run logs m1/run-*.log, summaries m1/out/*/summary.txt). Linux's sysfs is used as a black-box comparison only.

	T14 (Tiger Lake)	AMD bench (Zen 2)
Processor _OSC	Intel UUID. 14 of the 24 processors answer any query (even an empty one) with DWORD2 = 0x1	not Intel's; no processor _OSC is negotiated
_CST under ToyOS's rule as built (query, then commit the offer; bits 4 and 9)	C1 FFH; C2 and C3 SystemIO 0x1816/0x1819	n/a
_CST with Intel bits 4 and 9 committed, no query	all 24 the same as Linux's: C1 FFH hint 0x0 (type 1, 1 µs); C2 FFH MWAIT 0x31 (type 2, 253 µs); C3 FFH MWAIT 0x60 (type 3, 1048 µs); Arg1 = 1 (HW-coordinated, no bus-master avoidance)	—
_CST (no claim involved)	—	C1 FFH vendor 2/class 2/hint 0 (type 1, 1 µs); C2 SystemIO 0x414 (type 2, 400 µs); C3 SystemIO 0x415 (type 3, 350 µs). The latencies are not ascending. Linux reports 350/400 (unexplained; ours are the bytes' values).
_CSD	absent	one per SMT pair, CoordType 0xFE (HW_ALL), Index 0 (0-based, so C1 only)
_LPI	absent	present: 0x414 = 60 µs min residency / 18 µs wake; 0x415 = 1000 / 350
FADT	WBINVD=1, PM2_CNT_LEN 4	WBINVD=1, P_LVL2_UP=1, PM2_CNT_LEN 4
ARAT, invariant TSC	yes (cpuinfo fixture cited in #870)	ARAT Fixed 1, TscInvariant Fixed 1 (PPR 55922)
MWAIT	sub-states per CPUID.5	CPUID.5:EDX = 0x11, so C0 and C1 only (PPR 55922): MWAIT reaches only C1

What the documents say:

• Intel 302223-007.
  • Table 1's bits say what OSPM "is capable of". Bit 4 is independent C2/C3 per processor; bit 9 is native C-state instructions for C2/C3.
  • FFH _CST (Table 4) is vendor 1, class 2, Arg0 = MWAIT hint, Arg1 bit 0 = HW-coordinated, bit 1 = "OSPM should use bus master avoidance".
• ACPI 6.5.
  • §8.4.1.1: _CST is ascending, and its latency is the "worst-case latency to enter and exit". Notify(proc, 0x81) means re-evaluate _CST and _CSD (§8.4.1.2).
  • §8.1.4: for a type-3 state, OSPM is responsible for cache coherency. On multiprocessors that means flushing caches before entry.
  • §6.2.11.2 note: claiming LPI support in \_SB._OSC asserts support for processor containers.
• PPR 55922. Core::X86::Msr::CStateBaseAddr (C001_0073): the core traps I/O reads of CstateAddr..+7 as C-state entry requests, so the read reaches no device.
• SDM MWAIT.
  • An interrupt "that would be delivered to software (as if HLT had been executed)" exits MWAIT, so sti; mwait has sti; hlt's atomicity.
  • MWAIT does not enter a state without an armed MONITOR.
  • Leaf 5's sub-state counts bound the hints.

1. Ownership

Layer	Owns
acpiserver (userland, trusted platform code)	ACPI semantics. It evaluates each processor's _CST/_CSD, and again on Notify 0x81. It decides which states ToyOS can honour and drops the rest by name. It sets each state's residency (policy, §4). It hands the kernel only enterable states: {entry, exit latency, target residency}. C1 is never handed over, because C1 is the kernel's own sti; hlt (ACPI §8.1.2).
kernel boundary (toyos-userbound::idle, pure, host-tested)	Hardware safety, as a bug fence. It checks that the entry is executable and harmless on that CPU (MWAIT hint enumerated by leaf 5; port inside that CPU's CStateBaseAddr window on AMD). It checks the machine prerequisites (ARAT, invariant TSC). Only a checked IdleTable can be constructed.
kernel idle loop	Choosing a state each idle period (kernel/pure/sched/idle.rs, pure) and entering it (arch::hw::idle(state)).
kernel object WakeLatencyHold	The machine-wide wake-latency bound, a minimum over the holds that are live (§3).

Without a declaration, or with one refused, every CPU stays in C1. That is logged once; it is expected absence, not an error. No kernel thread is added, and all work stays in the idle loop and is bounded.

2. The kernel/acpiserver interface

• The operation. A new op on the x86 acpi claim, beside op::S5: op::IDLE with {processor_uid: u32, count: u8, states: [AbiIdleState; 8]}, where AbiIdleState = {entry: Mwait{hint} | Port{port}, latency_us: u32, residency_us: u32}.
  • One call per processor UID.
  • The kernel maps UID → APIC ID through the MADT (ACPI 6.5 §5.2.12.2: _UID or the Processor op's ProcessorID), then APIC ID → CpuId through the roster.
• Interpreter change. The interpreter must keep each Processor object's ProcessorID. Today Object::Processor drops it.
• The same rule as S5. The declaration is a fact of the tables, so it outlives the holder. A later holder, or a re-evaluation on Notify 0x81, replaces it. The per-CPU table is published under the existing seqlock, so the idle path never takes a lock.
• Why the claim and not a new right. The authority is "the process that interprets this machine's AML", which is exactly the acpi claim. AArch64 has no claim today, and its record says the channel is part of its exit (§7).
• What acpiserver withholds, each by name, in one log line per machine:
  • type 1 (C1 is hlt);
  • an Intel FFH state asking bus-master avoidance (no BM_STS tracking);
  • an FFH class other than Intel class 2;
  • a SystemIO state on Intel (a chipset P_LVLx read; no bench needs it);
  • a type-3 SystemIO state (ACPI §8.1.4's cache flush, which ToyOS does not do; WBINVD on Zen would write back the CCX's shared L3 under its siblings);
  • a state a _CSD entry coordinates SW_ALL/SW_ANY.
• Result per bench:
  • T14: C2 MWAIT 0x31 and C3 MWAIT 0x60.
  • AMD: C2 SystemIO 0x414 only. Its 0x415 is withheld, recorded with an exit: an AMD document that its I/O C-states keep caches coherent, or LPI support with processor containers.
• Processor _OSC (needs your ruling; it changes The AML interpreter finished to ACPI 6.5: Load and _OSC, the namespace initialised, every address space, every event, and the bounds stated #875's rule):
  • ToyOS claims Intel bits 4 and 9 and commits them with no query for the Intel processor UUID. \_SB._OSC keeps query-then-commit.
  • Why:
    • Table 1's bits declare what OSPM does; they are not grants of control. 302223 §2.1 says the firmware "interprets and records the set bits".
    • ACPI only "strongly recommends" the query.
    • The T14's processor query is meaningless: it offers 0x1 even when nothing is asked.
  • The alternative, keeping the query and committing the claim when the offer is incoherent, is a tolerance written for one firmware's defect.
  • Without one or the other, the T14 has no FFH deep state.
  • We claim no bit 8, so C1 stays hlt. We claim no bit 6, so firmware expects no software _CSD coordination.

3. The wake-latency limit

• The contract. SysCap::hold_wake_latency(bound: Duration) -> WakeLatencyHold, a new kernel object.
  • While a hold lives, no CPU enters a state whose declared exit latency exceeds the smallest bound held.
  • C1 is always permitted.
  • Dropping or closing the handle ends the hold, and so does the process's death, because handle teardown ends it.
• Authority. A new right, Rights::WAKE_LATENCY (manifest syscap = ["rt", "wake-latency"]). Only the supervisor endows it, from system.toml; packages never receive a syscap.
  • It is not folded into rt: they are different costs. rt takes CPU from others; this takes power from the whole machine.
  • A later audio worker, or a network service, may need one without the other.
• Scope: machine-wide. An interrupt and the thread it wakes are on different CPUs (HDA IRQ → hda process → soundserver), and nothing pins either, so a per-CPU bound protects nothing today. The object shape survives a later per-CPU scope, which would add a CPU-set argument when stage 2/4 placement can pin.
• Kernel side. A multiset of bounds under a spinlock on the syscall path only, with the minimum mirrored into one AtomicU64 that the idle loop reads. A hold is a handle, so its count is bounded by the handle table.
• The real caller now: soundserver's mix thread.
  • It holds period/2 while a stream plays (2.9 ms periods give 1.45 ms) and drops the hold when it suspends.
  • Half a period leaves the other half for the render and the rest of the wake chain. That is a policy, and stage 4's workers replace it with their own deadline arithmetic.
  • On both benches this bound admits every declared state, so it changes nothing measurable there. The contract is still the right shape: an explicit, revocable, authority-bearing bound that stage 4's workers inherit.
  • Device DMA latency in package states (HDA FIFO, LTR) is outside a CPU wake bound. That is a recorded risk, and the audio metal rows judge it.

4. The selection rule (pure, kernel/pure/sched/idle.rs)

Each idle period, after the final interrupts-off recheck that exists today, the CPU takes the deepest declared state s (declaration order is depth; ACPI §8.4.1.1) for which all three hold:

1. s.latency ≤ limit (§3; unlimited when nothing is held);
2. s.residency ≤ horizon, where horizon = SleepToken::armed() − now, and no timer means unbounded;
3. fewer than 4 of this CPU's last 8 unknown-wake idles (kick, device or IPI, not its own timer) were shorter than s.residency. That is one 8-bit shift register per state, updated once per wake.

If no state qualifies, the CPU halts in C1.

• Known deadlines are met, not made late. When s is chosen with a timer armed, the one-shot is pulled in to deadline − s.latency (arm_within, which exists). The CPU then wakes early, passes, finds nothing due and spends the rest of the period in a shallower state.
  • The time and wake.periodic groups therefore see C1-like lateness.
  • The cost is one extra pass (about 1–2 µs) per deep timer wake.
• Why rule 3 exists. Rule 3 is the predictor for wakes the timer cannot see. IPC ping-pong idles with no timer armed; without rule 3 it would pay 1048 µs per wake on the T14. With rule 3 a burst after a long quiet spell pays at most 4 deep exits before it adapts. The host test fixes that count.
• Residency, which _CST does not give. acpiserver sets residency = 2 × latency.
  • _CST's latency is entry plus exit, so requiring twice that means the CPU spends at least half the period in the state in the worst case.
  • It is labelled a policy at its site. The metal power and clock readings judge it.
  • A future _LPI reader fills in the platform's own Min Residency.
• AMD's non-ascending latencies. The rule does not need ascending latencies: wherever 0x414 (400 µs) qualified, 0x415 (350 µs) would too. That case does not arise anyway, because 0x415 is withheld (§2).
• Entry.
  • Mwait: monitor on a per-CPU line nobody writes (so wakes are interrupts alone, as today; this is not the killed doorbell design), then sti; mwait hint with ECX = 0.
  • Port: sti; in al, port. That the trapped read honours the STI shadow as hlt does is an inference from PPR's "trapped as a C-state entry request", checked on AMD metal (a lost wake is a stall the boot's deadline catches).
• The bound on idle-path cost. One seqlock read of at most 8 states, a scan of at most 8, one atomic load for the limit, one clock read after the wake, and at most 8 shift-register updates.
• Placement accounts for wake cost (requirements line 181).
  • Each CPU publishes the chosen state's exit latency in its CpuHandle (one relaxed store before it sleeps, 0 for C1).
  • Within each group and free-core tier, CpuHandles::idle_near prefers the shallowest sleeper, then the rotation.
  • Wakes then go to "hot" CPUs, and deep sleepers stay deep.
  • It is pure and simulated, and the sched group judges it.

5. Tests by tier

• Host, pure:
  • idle.rs selection:
    • each rule's arm;
    • the limit;
    • the prewake arithmetic;
    • ping-pong history converging to C1;
    • a long quiet spell reaching the deepest state;
    • burst adaptation within 4 wakes.
  • toyos-userbound::idle:
    • a hint above leaf 5's sub-state count refused;
    • a port outside the window refused;
    • no ARAT or no invariant TSC refuses the table;
    • more than 8 states, or latency 0, refused;
    • compile_fail for the private constructor.
  • Hold aggregation (minimum, release).
  • idle_near's shallow-sleeper tiebreak.
• Host, acpiserver (synthetic AML, as s5_package does; decoded extracts of both benches' _CST/_CSD as fixtures under "Extracts only"):
  • the _CST/_CSD decode;
  • each withholding rule;
  • residency;
  • the processor _OSC commit-the-claim rule.
• QEMU:
  • QEMU declares no _CST, so the existing boots exercise expected absence (C1, one log line).
  • No new guest test: the boundary and the selection are host-tested, and the behaviour that remains (entry atomicity, real exit latency) is hardware.
• Mutations (applied as checked patches, built, run, restored):
  • always the deepest state: host selection tests red; on metal, ipc/sched latency red;
  • rule 3 removed: the ping-pong host test red;
  • no prewake: the host prewake test red, and time.* late on metal;
  • the hold ignored: the host limit test red.

6. Metal plan (requests via --metal-readback; I touch no machine)

• Rows. benchcase (osbench, every group) on the T14 and the AMD. The arms:
  • A: main, the whole change reverted, as the control;
  • B: the branch;
  • M: the always-deepest mutation, T14 only.
• What is read.
  • The logs: acpiserver's declaration line, and the kernel's accepted or refused states per CPU.
  • Per-state entry counts and residency, exposed through counters (Rights::COUNTERS).
• Expected.
  • T14 B: cpu.clock at about 4189 MHz (ratio 42), and crunch.single about 1.10× A's.
  • sched/time/ipc/tail no worse than A beyond noise: time.* within C1's lateness thanks to the prewake; ipc ping-pong at A's.
  • M worse than B in ipc/tail. That shows rule 3 and the limit matter.
  • AMD B: lower energy package power than A and an idle-core boost read in cpu.clock. Its state is 0x414 only.
• Audio. The T14's audio metal rows with the other CPUs idle: no underrun.
• Oracles. The turbo bin (hardware's own count of active cores), RAPL energy, and APERF/MPERF. Intel's core/package residency MSRs are not read today. I propose not adding them in this PR; the bin is the independent oracle. Say if you want them.

7. Records

• issues/idle-cpus-enter-the-c-states-cst-names.md: closed by the T14 and AMD readings, with what remains moved to two records:
  • AArch64 idle enters only wfi. Its equivalent is _LPI with PSCI CPU_SUSPEND (Arm DEN0022, FFH per Arm DEN0048), with the same selector and hold. No fleet machine declares _LPI, and AArch64 has no AML server or claim to hand it over. Exit: a machine that declares one, with the _LPI decode and the retention-state entry built, and power-down states with a resume path.
  • The AMD's type-3 0x415 withheld (§2), with its exit.
• issues/the-scheduler-and-the-clocks-never-talk.md: the idle-state-awareness bullet updated to what placement now does.
• issues/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md: the processor _OSC rule, if ruled.

8. Decisions for you or the owner

1. Processor _OSC: commit ToyOS's claim with no query (§2). This changes The AML interpreter finished to ACPI 6.5: Load and _OSC, the namespace initialised, every address space, every event, and the bounds stated #875's built rule. The T14 has no FFH deep state otherwise.
2. A new wake-latency right separate from rt, machine-wide in scope, with soundserver holding period/2 while streaming (§3).
3. Type-3 SystemIO withheld, so the AMD gets C2 (0x414) only (§2). The alternatives are spec-violating (no flush) or harmful (WBINVD of the shared L3).
4. _CST residency = 2× latency as acpiserver's policy, and rule 3's 4-of-8 threshold (§4).
5. The shallow-sleeper placement tiebreak in this PR (§4). It is small, but it touches stage-2/3 placement code.

Risks:

• The AMD's I/O-entry atomicity is inferred (§4).
• The AMD might gain no boost from C2 alone.
• Package C-states may not follow the cores without LTR/ASPM, which affects power but not the T14 turbo bin, since the bin is per core.
• A burst after a quiet spell pays up to 4 deep exits.

[Japabu]

Review r1: deep idle (head 59dcd1f)

Architecture. The ownership is right. acpiserver owns ACPI semantics: it evaluates _CST/_CSD, withholds states by name and sets residency. The kernel takes typed states over the existing acpi claim, and the declaration's ABI type lives only at the kernel/program boundary. A pure bug fence (toyos_userbound::idle) is the only constructor of Table. Selection is pure and lock-free on the idle path: one seqlock snapshot, atomics, no kernel thread. The wake bound is a handle-scoped kernel object behind its own right, endowed only through system.toml. The placement tiebreak is a small, bounded change: with no declaration every exit is 0 and idle_near keeps its old early return. The revisit timer is a compensating mechanism, but a necessary one. A wake-driven predictor cannot learn anything without wakes, so I do not count it as a workaround. The rulings (processor _OSC commit with no query, WAKE_LATENCY, 0x415 withheld, the two policies, the tiebreak) are implemented as ruled and I do not reopen them. Growth is +2012/−140, roughly 40 % of it tests. The new production code is about 900 lines across server, boundary, selector and glue, which is proportionate.

What I verified by reading the code.

• The prewake cannot cause a lost deadline. Each try_sleep re-applies the plan through set_timer, and arm_within only shortens the timer on both architectures, so an early fire is a spurious pass and the next pass re-arms the deadline.
• The revisit timer cannot push a deadline later, because min with the current count keeps the earlier one.
• enter leaves interrupts open as halt does.
• MONITOR uses a per-CPU line aligned to 64 bytes.
• The seqlock has a single publisher, serialised by HOLDER.
• No userland input reaches a kernel panic. decode's panic and Bounds::release's panic are reachable only through kernel bugs.
• The ABI structs have no padding.
• The hold's lifetime is correct: a failed install drops the object, and the drop releases the bound.

Blockers

1. kernel/pure/cpu.rs:235: the CStateBaseAddr gate excludes every AMD part after family 17h. traps_cstate_ports admits family 17h only, "a family no document in hand defines it for is left out". Zen 3, Zen 4 and Zen 5 (families 19h and 1Ah) are the 2020+ AMD target, so every one of them gets a refused declaration and C1. Root CLAUDE.md says a bug fence in the trusted base "never withholds a feature for want of a vendor document". As far as I know the family 19h PPRs (for example 55898 and 56214) define the same MSRC001_0073 CStateBaseAddr; please verify that against the documents. Then either extend the gate to the families whose PPRs define the register, citing them, or file the gap with its exit in issues/. Today the gap is unrecorded (rg 19h issues/ finds nothing on it), which is silent debt.

2. system/acpiserver/src/cst.rs:13-25: nothing at the site justifies handing over a type-3 FFH state. The header withholds a type-3 SystemIO state because ACPI 6.5 §8.1.4 makes OSPM responsible for cache coherency in type 3. The T14's MWAIT 0x60 is also type 3 and is handed over, and the site gives no reason why §8.1.4's flush does not apply to it. The T14's deepest state depends on that reason. Cite the Intel statement (302223-007 or SDM Vol. 2B/3) that the processor keeps its caches coherent itself when C-states are entered through native MWAIT. Then state the distinction in one clause beside the SystemIO rule. Without that citation the PR applies the rule asymmetrically and nothing establishes the claim.

3. lib/userbound/src/idle.rs:137-143 and kernel/pure/sched/idle.rs:102-113: the prewake arithmetic relies on a relation the boundary never checks. choose admits a state when residency <= horizon and then asks the CPU to wake exit early. That is sound only if residency > exit, which acpiserver's 2× policy gives but the kernel never checks: residency_us 0 or below latency_us is accepted. With such a declaration, fires = armed − exit lies at or before now. The CPU then wakes every MIN_ONE_SHOT (10 µs), chooses the same state again and loops until the deadline. A buggy acpiserver makes idle CPUs spin. That is not a crash, but it is a defect reachable from the boundary. Refuse residency_us < latency_us, or zero, in table() with a named Refused variant and a test. Alternatively, have permitted require horizon > exit.

4. sdk/abi/src/syscall.rs:1764 (and kernel/src/object/wake_latency.rs:1-4, sdk/toyos/src/syscap.rs): the hold's contract is false for CPUs already asleep. The documentation says "for as long as the returned handle lives: no CPU waits in a state slower to leave than the least bound held". crate::idle::hold only stores LIMIT, so a CPU that entered a deeper state before the hold stays there until its next wake and pays the old exit. The hold returns before the bound holds, and the caller cannot tell when it does. There are two ways to fix this:

   • Make the contract true. On a tightening hold, kick each sleeping CPU whose published idle_exit exceeds the new least bound. The published exit already exists, so this is a few lines.
   • Or state the weaker contract at all three sites: the bound applies from each CPU's next idle period.

   A latency bound conventionally takes effect at once, so I would take the first.

5. kernel/src/arch/aarch64/idle.rs:1-4: the module header is misleading. "so every CPU reports a timer that does not run in a deep state" reads as a claim about AArch64 hardware; what actually happens is that CpuIdle::default() has always_running_timer: false. Nothing reaches that check anyway, because cpu_of returns None first. Cut it to the fact that matters: nothing declares a state on AArch64, so facts admits nothing and cpu_of names no CPU.

6. issues/idle-cpus-enter-the-c-states-cst-names.md:9: the record goes stale on landing. "Built on branch deep-idle" becomes false at merge. Name the PR, or say it is built, so the record stays true on main while the metal exit is outstanding.

Suggestions

• kernel/src/idle.rs:157-198: move the timer plan into the pure crate and test it there. The glue is untested and is not covered by any mutation. That glue covers which instant is armed (early versus revisit, min), whether a wake was foreseen (now + slept >= fires), and the round trip of encode/decode. All eight mutations hit kernel::sched::idle or the server. A max for min, or an inverted foreseen, would pass every host test and degrade silently on metal. One option is for choose to return the absolute arm instant, plus a pure foreseen(fires, now, slept), with a host test each.
• The foreseen classification has no margin. A scheduler timer that fires a few nanoseconds before fires (the one-shot's tick truncation) counts as an unforeseen wake shorter than the deeper state's residency. That feeds the history wakes the design excludes. In my estimate the arming latency outweighs the truncation on both benches, so this is not a blocker, but a small tolerance would make the classification robust.
• kernel/src/arch/x86_64/idle.rs:52 and kernel/src/idle.rs:155: "returns once an interrupt has been taken". MWAIT may also exit for implementation-specific events (SDM Vol. 2B, MWAIT). The code handles a spurious exit correctly as a short unforeseen wake, but the stated contract is stronger than the hardware's.
• system/acpiserver/src/cst.rs:259: _CSD's Index is treated as 0-based into the states after _CST's count. Cite the ACPI 6.5 §8.4.2.2 wording at the site. If the index is in fact 1-based, a software-coordinated last state would be handed over.
• kernel/src/idle.rs:66: a declaration for a CPU that has not reached its idle loop is answered NotFound. acpiserver counts that as "naming no CPU" and never retries it. Reading the facts at CPU bring-up instead would remove the ordering dependency, though in practice every AP is idle before userland starts.
• kernel/pure/cpu.rs:294-303: the new test sits above the module's use super::*;. Move it below.

Evidence

• Host checks. --ci host ran at b392ba8, and the two later hunks (mix.rs, cst.rs) had their suites rerun at head. That reuse is sound: git diff b392ba8a8 59dcd1f03 touches only those two files.
• QEMU. The QEMU boots show only that QEMU declares no _CST, and that soundserver's hold works when a stream plays. That is all QEMU can show here.
• Mutations. The mutation logs match the table in the PR.
• Metal. The metal expectations are labelled as expectations, which is correct for a draft. Blocker 3's check is a host test; none of the blockers needs metal to settle.

SEND BACK

[Japabu]

Review r2: deep idle (head f6510de)

Architecture. The fold is the right shape. The pass asks Machine::idle_plan against the timer it would sleep under. The plan's wake is folded into no_later_than, and the SleepToken carries the plan to the wait, which no longer touches the timer. That removes r1's second timer write and the compensating arm_within. Moving words/read into toyos_userbound::idle puts the encoding next to the table it encodes, and it now has a round-trip test. The hold kick is a real Dekker: wait publishes the exit, fences and re-reads LIMIT; hold stores LIMIT, fences and reads every exit. The sibling rule sits in the pure scheduler, beside the doorbell it pokes through, which is the right owner. Since r1 the branch has grown by +873/−248. About 250 of those lines are tests; osbench's around.rs and the placement plumbing make up most of the rest. That is proportionate to the four mechanisms added.

r1 blockers.

• B1: resolved per the ruling. traps_cstate_ports and the CStateBaseAddr read are gone. The fence asks firmware::port(pio::standing(..)) for Through under the holder's own row (acpi_mode.rs:556-558), and the trust argument is at lib/userbound/src/idle.rs:12-20.
• B2: resolved. cst.rs:24-30 now gives the distinction from 302223-007 Table 4 and the SDM in one clause. I cannot check the SDM section number against a document in hand, because dl/intel holds only 302223. The reasoning itself is sound.
• B3: resolved. Refused::Residency covers both zero and a residency below the exit, has a test, and has a mutation (residency-unchecked). Equality is admitted. It costs one immediate timer fire at most, never a loop.
• B4: resolved in code. hold kicks every CPU whose published exit exceeds the bound, and the contract at all three sites is now true. See blocker 4 for the stale comment this left behind.
• B5 and B6: resolved.

Verified by reading.

• No deadline is made late.
  • try_sleep's plan deadline is the same TimerPlan::compute(None, earliest).no_later_than(rearm) that apply_timer_no_later_than arms when nothing is running.
  • wake_at filters any instant at or after it.
  • deadline − exit ≥ now + residency − exit ≥ now follows from B3's check.
  • wait's bound re-check only makes the state shallower under an already-earlier timer.
  • A C1 plan's wake_at is None, so QEMU and every machine without a declaration arm exactly as on main. That is why the earlier QEMU evidence still stands.
• No lost wake in the sibling protocol. The hold check is held=1; SeqCst fence; read SLEEPING. The sleeper does fetch_or SLEEPING (AcqRel) … confirm; SeqCst fence; read held, so one side sees the other. A poke that lands before the held CPU's arm_sleep raises KICK_PENDING, confirm fails, and the repeated pass finds the sibling sleeping.
• Topology is shared with placement. The sibling rule reads the same CPUID 0BH/1FH-derived Topology the placement already uses on both vendors. A one-thread core has no siblings, so it never holds. Nothing on AArch64 plans a deep state, so the rule never engages there.
• The revisit climb is correct. next_deeper is permitted and deeper than the chosen state, so it is necessarily refused by history alone. quiet clears exactly what the period covered, and the climb test and the three revisit mutations hold it.
• Evidence reuse is sound. --ci host exited 0 at 6a0fa3f (104 steps), and git diff 6a0fa3f29 f6510defd touches only the track issue. The nine r2 mutation logs match the summary.

Blockers

1. kernel/pure/sched/cpu.rs:2173-2175: held_by_sibling stays set across C1 choices.

   • The early return for a plan with no deep state leaves the flag as the last held period set it. Only dispatch clears it (:1757).
   • Suppose a CPU was held once, and its next pass plans C1 because the horizon is short or the history refuses. From then on, every sleep of its sibling pokes it, until it next runs a task.
   • Each spurious wake ends as Ended::Unforeseen and is learned as a short period (blocker 2). That keeps the history refusing, so the plan stays at C1 and the flag stays stale. The loop sustains itself on an idle CPU beside a sibling that sleeps often.
   • Fix: store false on that path too, or compute the flag on every pass. Add a host test: held once, then a C1 plan, then the sibling sleeps, and no kick follows.

2. kernel/pure/sched/cpu.rs:2183 and kernel/src/idle.rs:192-195: the history learns the kernel's own poke.

   • A held plan is Plan::c1 with the real declaration kept, so wait updates history. ended classifies the sibling's poke as Unforeseen, and the history records a period as short as the sibling's run.
   • That is the class of wake the design excludes: the module says the history estimates wakes the CPU did not foresee, and it already excludes the scheduler's own timer and the revisit for that reason. The poke is equally self-made.
   • Consequence: a CPU beside a sibling that runs in bursts shorter than C2's residency accumulates early bits. When the sibling finally sleeps, it waits shallow and climbs by revisits instead of going deep at once. That works against the turbo bin this PR exists for.
   • Fix: mark a held plan, for example with Plan.held or an Ended arm, and do not learn from its end. Add a test in sched::idle or the cpu tests.

3. kernel/pure/sched/cpu.rs:2172-2185, :2616: two idle siblings can hold each other, with nothing that bounds it.

   • busy_siblings counts any sibling that is not SLEEPING, including one in an idle pass that is about to sleep.
   • If both threads of a core are in passes with deep plans at once, each sees the other awake and holds. Each then publishes SLEEPING and pokes the other, and both wake together and repeat. The rule converges only when one confirms before the other's held check.
   • The offset between the two wakes is mirrored, not shrunk, from round to round. Escape therefore depends on jitter, and no termination argument is given (compare probe_still_owed's).
   • The start condition is ordinary: wake_slower_than kicks both threads of every deep core at the same instant, and so does any shared wake.
   • Consequence: bursts of mutual IPIs, with both threads kept in C0/C1, on exactly the machines the rule is for. Neither the host test (sequential passes) nor B3 would show it.
   • Fix: make "busy" mean a sibling with a task to run, not one in an idle pass. Two idle passes then both go deep, and a deep thread beside a sibling that turns out to run is the case the rule already accepts at its next wake. Alternatively, give a bound on the alternation.

4. kernel/pure/sched/cpu.rs:2361-2365: the idle_exit contract is now false.

   • It says a reader "may see the previous idle period's: a tiebreak, never a correctness input".
   • The value is now reset to 0 on wake (kernel/src/idle.rs:176).
   • It is also the correctness input of the hold's Dekker (wake_slower_than, kernel/src/sched/driver.rs:774). The hold's contract is only true because wait publishes it before its fence.
   • A maintainer who trusts this comment could weaken the publish or move it after the bound check.
   • Fix: state the current contract briefly. It is published before the wait and fenced against the bound (crate::idle), 0 while awake, and a tiebreak for idle_near.

5. kernel/pure/cpu.rs:205-224: the core C6/C7 residency admission is not shown against the document, and a wrong admission is a #GP at every CPU's counters::bring_up.

   • The list reuses SMI_COUNT_MODELS' Table 2-20 run on the premise that every model a section names as supporting Table 2-20 has both 3FDH and 3FEH.
   • The run includes Xeon parts: 0x2d, 0x3e, 0x3f, 0x4f, 0x56, 0x55, 0x6a, 0x6c, 0x8f, 0xcf, 0xad, 0xae, 0xaf. Server cores commonly expose no core C7. Sierra Forest (0xaf) is an E-core-only part. Their SDM Vol. 4 sections may except 3FEH.
   • The implementer's downloaded documents contain no SDM Vol. 4, and the PR states only "the Table 2-20 models". I cannot confirm the premise either way.
   • Resolve by quoting, per section, that 3FDH and 3FEH apply unexcepted. Otherwise admit C6 and C7 separately and only where a section confirms each. A bug fence may lose a count; it must not panic a 2020+ server at boot.

6. kernel/pure/sched/idle.rs:78-82 and :339-346: History::forget is dead. Production now gets a fresh history from kernel/src/idle.rs::history, which returns History::default() on a generation change. Nothing outside the test calls forget, so delete it and its test.

Suggestions

• kernel/src/idle.rs:193: quiet with the revisit's own residency, not the measured period.
  • Ended::Revisit admits a wake up to FORESEEN_SLACK early. quiet(costs, woke − chosen_at) then fails residency <= slept for the very state the revisit was armed for, the CPU revisits again with the same choice, and the climb stalls.
  • Today the C1 exit and the interrupt path outweigh the APIC count's truncation by a sub-30 ns tick, so this is not live. But the arithmetic depends on that margin rather than on the classification it already made.
  • quiet(costs, plan.choice.revisit_after), or a max with it, removes the dependence.
• kernel/src/sched/driver.rs:774: give the hold kick a test. wake_slower_than is the whole of B4's fix and is covered by no test or mutation. As a CpuHandles method in the pure crate, like poke_held_siblings, it gets a host test cheaply: kick the CPUs above the bound, not those at or below it, and not self.
• kernel/pure/sched/idle.rs:182-216: deadline_fires, wake_at, foreseen and ended are public free functions, each wrapped once by a Plan method. One spelling would do. Default on Choice and Plan has no user.
• kernel/pure/sched/cpu.rs:4423: "pays nothing for the rule" is not quite true, because every sleep runs poke_held_siblings' SeqCst fence and loop. Either skip both for a one-thread core and with nothing held, or soften the claim.
• kernel/pure/sched/cpu.rs:2166: "Idle CPUs enter the deep states their firmware declares #1073's readings" is a measurement reference in code. The rule's reason, that a core enters a core C-state only when all its threads ask, stands without it.
• For the orchestrator (PR body): the Checks row gives the nine r2 mutations' revision as 7b78def. The sibling mutations patch code from e14605a, and the handoff says "7b78def41 and later heads".

Evidence

• Host checks. Sound at 6a0fa3f for the current head.
• QEMU. The QEMU result predates the fold, but every QEMU plan is C1, so the fold changes nothing there. It still establishes only that QEMU has no _CST and that the hold works.
• Sibling rule. Only sequential host passes support it today. The concurrent case in blocker 3 needs either a changed predicate or an argument. B3 cannot settle it.
• Metal claims. The AMD deflate claim and the T14's group starts and .ran remain B3's to measure, as the PR says.
• Blockers 1, 2, 4 and 6. Each needs a code change and, for 1 and 2, a host test. Blocker 5 needs a document quotation, not a boot.

SEND BACK

[Japabu]

Review r3: deep idle (head d286ddd)

Architecture. The round removes code. d286ddd is +124/−150, and the branch now stands at +2664/−193 against origin/main. The SMT-sibling mechanism is gone: the held flag, the poke, the hold-shallow wrapper, its TestHw deep knob and its two tests. With it, the pass no longer pays a SeqCst fence and a sibling loop on every sleep. The hold kick moved to its proper owner. CpuHandles::wake_slower_than sits in the pure scheduler beside idle_near, which reads the same word, and the driver delegates to it in one line. The history update moved too: Plan::learn sits beside ended, which classifies the period, and kernel::idle::wait is now straight-line. Both moves turn kernel-only logic into host-tested logic, which is the right direction under the testing policy. I did not reopen the ruling that removed the rule.

r2 blockers

• B1-B3 (sibling rule): resolved by removal, with no residue.

  • I searched the whole tree for held_by_sibling, poke_held, held_shallow, running_sibling and related phrasing. There are no hits outside unrelated code.
  • busy_siblings predates the branch: origin/main's cpu.rs:2539 already uses it for placement.
  • e14605a touched only kernel/pure/sched/cpu.rs and one around.rs comment. That comment is about deep_idles counting and is true independently of the rule.
  • AtomicBool is no longer imported. The Kicker import moved from the tests to the module, where wake_slower_than uses it.
  • No issue record mentions the rule. The PR body says once, correctly, that r2's two sibling mutations covered code that has since been removed.

• B4 (idle_exit contract): resolved. kernel/pure/sched/cpu.rs:2321-2326 now says:

  • it is published before the wait and fenced before the bound read;
  • it is the hold's input and the tiebreak.

  This matches kernel/src/idle.rs:172-181. There is one narrow imprecision, given as a suggestion below.

• B5 (core C6/C7 admission): resolved. I checked against speed-ruling/sdm-093.txt (sha256 c36093b8…, 325462-093US):

  • "3FEH" appears exactly twice, in Table 2-20 (332824) and Table 2-57 (351728).
  • "3FDH" appears five times: Tables 2-6 (Atom), 2-15 (Nehalem), 2-20 and 2-57, and Table 2-61, where it is Xeon Phi's MSR_MC6_RESIDENCY.
  • None of 2-6, 2-15 and 2-61 applies to an admitted model.
  • The openings of §2.11, §2.12, §2.13, §2.14, §2.15 and §2.17 name Table 2-20 for their models. So do the subsection openings §2.11.2 (06_2DH), §2.12.2 (06_3EH), §2.16.1 (06_56H) and §2.16.2 (06_4FH).
  • §2.17's model paragraph lists exactly the remaining 25 models, 0x4e through 0xbd.
  • Its footnote excepts 3F7H and others for the 12th-generation E-core, but not 3FDH or 3FEH. No Vol. 4 text excepts either address.
  • My r2 worry about Sierra Forest (06_AFH) is answered: Table 2-57 defines both addresses.
  • The comment at kernel/pure/cpu.rs:212-217 says "section's opening" where three of the namings sit in subsection openings. That is close enough; the evidence quotes them.

• B6: resolved. History::forget and its test are deleted. Production gets a fresh history from kernel/src/idle.rs::history on a generation change.

Suggestions taken

• Plan::learn (kernel/pure/sched/idle.rs:144-151).

  • On a revisit it quiets slept.max(revisit_after), which removes the dependence on C1 exit margin that I noted in r2.
  • The test a_revisit_a_tick_early_clears_the_state_it_was_armed_for wakes 1 µs before C2's 506 µs residency. It asserts that C2 is admitted again and that C3 is not, so it pins both the max and quiet's per-state scope.
  • The learn-slept-only mutation turns it red (exit 101, rev d286ddd).

• wake_slower_than in pure (kernel/pure/sched/cpu.rs:2568-2578). The test covers:

  • the holder excluded, even with the deepest exit;
  • a CPU exactly at the bound and one at 0, both excluded;
  • one CPU 1 ns above the bound, kicked;
  • a second hold that sends no duplicate kick.

  The mutations kick-at-bound, kick-self and kick-ignores-ring each target one of these claims, and each is red.

• Private helpers and the Default derives. deadline_fires, wake_at, foreseen and ended are now private. Choice and Plan lost Default. The r2 mutations on ended, foreseen and quiet patch pure code that is still present, with the same tests, so "7b78def41 onward" in the PR body holds.

Evidence

• cargo run -- --ci host. Exit 0 with 104 steps, all green (r3/logs/ci-host-8.log, .exit reads EXIT=0). Its UTC timestamps (03:10-03:20) end at the commit time (04:20 +0100), which is consistent with the run being at d286ddd.
• Mutations. Four, at d286ddd, each exit 101 with its named test red. The patches are in r3/mutations/.
• AMD deflate placement, checked in the r3 readbacks.
  • amd-1-As ran on apic 4 at 66.59 ms. The others ran on apics 10, 12 and 2, at 63.37 to 63.59 ms.
  • cpu equals apic on this bench, so "CPU 4" is unambiguous.
  • In amd-1-As every other CPU, sibling 5 included, reads busy_permille=0 across the row.
• T14 cpu.clock.ran, checked in t14-2 and t14-4.
  • acpiserver ran 5494 to 5626 µs of each 5.5 ms window for boot, identity, energy and system.
  • At net and procs nothing beside osbench exceeds 0.9 ms.
  • The track's statements match.
• B4. Staged and not yet booted, as the PR says. The orchestrator's boots supply it.

Findings

No blockers.

SUGGESTIONS

1. kernel/src/idle.rs:172-178, with the contract at kernel/pure/sched/cpu.rs:2321-2322: "0 for C1" is not quite true.
   • When the bound re-check or a changed declaration sends the wait to halt(), plan.exit stays published while the CPU sits in C1.
   • Over-publishing is the safe direction for the Dekker, so this is not a correctness defect.
   • The cost is a later hold sending that CPU one needless kick, and idle_near ranking it behind its true cost.
   • A publish_idle_exit(0) after the fence when state is None and plan.exit != 0 makes the contract exact. Otherwise say "the state it chose to wait in".
2. kernel/pure/sched/idle.rs:144-151: only the Revisit arm of Plan::learn has a test. A swapped Unforeseen or Deadline arm would pass the current suite. The match was untested kernel code before this round, so this is not a regression. One extra assertion in the new test would cover it: an unforeseen 10 µs wake learns, and a deadline wake does not.
3. Records: label the inference as one.
   • Measured: the slow deflate followed placement on CPU 4 and came back with every idle CPU in C1.
   • Inferred: "a property of the bench, not of the idle states", in the track (issues/idle-cpus-enter-the-c-states-cst-names.md:68) and the PR body. The finding itself leaves open "CPU 4 itself" against co-located work.
   • "follows its placement on CPU 4, not the idle states" states only what was measured.
   • The finding could also record what the .cpus notes already show: in amd-1-As the sibling and every other CPU were idle across the row. That narrows its exit.
4. The track's "about 12.2 s after power-on" (:73) has no source in r3's evidence. The readback notes carry no timestamps. Cite the log it came from or drop it. The material fact is that acpiserver ran each window.
5. kernel/pure/cpu.rs tests: a blank line was added after mod tests {. It is stray churn in the diff.

LAND

The landing waits on the B4 boots the PR has already staged, and that decision is the orchestrator's.

[Japabu]

Review r4: deep-idle at d60d727 (focused: d286ddd..d60d727, plus the uncommitted records edit in the worktree)

Architectural assessment

The defect fix has the right shape. The process takes its one SysCap endowment exactly once, in sys::syscap() (a OnceLock<Option<SysCap>>), and Diag, identity::pci and fair's Grouping::Job borrow &'static SysCap from it. It also deletes fair's private OnceLock, which duplicated the mechanism. grep finds no other SYSCAP_LABEL taker in tests/guest/osbench/src. The type system can't stop a future second take, so an image-level check of the two capability groups is justified: whether the image and supervisor endow the job is not host-buildable.

The C1-fallback publish in kernel/src/idle.rs:175-178 is correct. It only lowers idle_exit after the fence. A holder that reads the stale nonzero value sends one unneeded kick, which is safe. The new Plan::learn test covers the Unforeseen arm and the Deadline arm, and the arms-swapped mutation turns it red (r4 mutations).

Size: about +20 lines of production code, and the net change in osbench is close to zero. Tests: +13 unit lines and +34 lines for the QEMU check.

Metal at d60d727 (run5), which I checked against the readbacks:

• Harness judge: exit 0 on all four boots, with PASS osbench on both B5 boots. Every ===TEST_END osbench_* exit=0=== on the T14 and the AMD.
• T14 B5: deflate 54.80 ms, ipc.reqrep.8 6969 ns. cpu.clock max 4189 at 5 of 17 group starts (sched, files, ipc, work, crunch).
• AMD B5: deflate 63.70 ms (the fast mode), IPC 6642 ns.
• AMD A* testcases: soundserver worst_irq_late_us 0 to 2661. The 2661 µs window recurs with every idle CPU in C1, so it is not the deep states.

Findings

BLOCKER 1. Records at the head don't state the final measured state.

• Committed state. At d60d727, issues/idle-cpus-enter-the-c-states-cst-names.md:84-87 says The T14's one-thread clock is held to its three-active-core turbo bin by hlt idle; deep idle lifts it 10 % #1066's exit stands at "8 to 10 of 17". B5 read 5. The AMD finding counts nine boots. The worktree holds an uncommitted edit of all three records that adds B5 ("5 to 10", ten boots, the A* lateness control), which is the right content. It must land as a commit. Fix these first:
• Lateness range. idle-cpus-…md (worktree) gives the T14 C1 lateness as "457 to 572 µs". The six soundserver lines in run5/t14-Astar/testcases/kernel.log read 530, 540, 181, 572, 486, 457, so the range is 181 to 572. 457 to 572 only holds for the clients=0 lines. The deep-state side, "166 to 564", also disagrees with the PR body's "331–564". Give one selection rule for both sides, or the full ranges. The conclusion, that the deep states move neither, holds either way.
• Wrap. The rewrap left one overlong line: "nothing beside osbench for more than 0.9 ms of the 5.5 ms window. Mem stays unexplained. Net, procs, tail". Re-wrap it.

BLOCKER 2. tests/toyos.rs:502-508: the registration comment carries history and names no QEMU configuration.

• History. "osbench_identity and osbench_fair once exited 1 on every bench because an instrument took the one capability first" is history. Root CLAUDE.md puts that in commits, and 49a98e3's message already says it. Delete the sentence.
• Configuration. Root requires the supported QEMU configuration at the registration. The neighbouring entries state theirs, for example "on QEMU x86-64's q35 …". The PR body says x86-64, and that is the only evidence (logs/qemu-osbench-{fixed,defect}.log). The comment should either say x86-64 q35, or the aarch64 lane should be shown to run it.

BLOCKER 3 (PR body; the orchestrator owns the correction). The body describes a pre-B5 state.

• Risks. The section still says "Staged at d60d727, not yet booted", and "The T14's one-thread clock is held to its three-active-core turbo bin by hlt idle; deep idle lifts it 10 % #1066's exit is unmet: 8–10 of 17". The table has no B5 column.
• Lateness. The testcases bullet still says "There is no A* testcases boot to compare against".
• Required content. The body should give:
  • B5's readings (T14 5 of 17, deflate 54.80 ms, IPC 6969 ns; AMD deflate 63.70 ms, IPC 6642 ns);
  • the run5 judge (exit 0 on four boots, PASS osbench on both benches);
  • the 5–10 of 17 spread, with The T14's one-thread clock is held to its three-active-core turbo bin by hlt idle; deep idle lifts it 10 % #1066's exit unmet;
  • the A* lateness control (AMD 0–2661 µs with C1 only; T14 181–572 µs).
• Checks table. Add the run5 judge row next to the B4 judge's exit 1 row.

SUGGESTION. kernel/pure/sched/idle.rs:433: in the assertion message "the early wake before the deadline is the scheduler's own", the wake is the deadline firing, pulled in by the chosen state's exit. "the deadline, pulled in by the exit, is the scheduler's own" would say that.

No other issues in the changed code.

SEND BACK

[Japabu]

The merge resolution at 145e659 was checked by the orchestrator: syscall renumbering only (VM_REMAP 146, WAKE_LATENCY_HOLD 147, 149 profile bins), with host CI, the osbench QEMU check and the image build all exit 0. Metal at d60d727 (B5 plus A* testcases, both benches) passed the harness judge; the merge touches neither the idle path, the scheduler nor osbench's instruments. Review r3 and r4 LAND. Landing.

[Japabu]

Review r5: deep-idle at 55269e9 (focused: b1ae80c, and the merge of main 23f34b5)

Architectural assessment

The fix has the right shape. It takes each CPU's window from the stamps its own two readings carry, and every fraction is computed over that window. window_us and row_us are both printed. The refusal is a pure rule in osbench_rules::window, tested on the host, and the window-unchecked mutation is red (r6 mutations/summary.txt). deep_ended/deep_ended_us now say what the kernel counts. Size: about +75 production lines and +45 test lines. The merge of main touches nothing in this PR's subject.

I checked the cases you asked about against the code:

• Timestamp domain. Counter::Stamp is the CPU's TSC/CNTVCT_EL0, which is the counter the clock page describes (sdk/abi/src/arch/x86_64.rs:27). Both readings come from the same CPU, so no cross-CPU sync is assumed. ticks_to_nanos is exact for periods under 18 ms.
• Missing CPU. counters::read always writes one record for every index 0..cpu_count, so the continue for a missing CPU in around.rs:67 can't be reached.
• Stale or never-answered CPU. A CPU that never answered carries its last block and that block's stamp, which gives OneReading or NotTheRow. A block that was never published has no stamp, which gives NoStamp.
• Counter wrap. All counters are 64-bit and use wrapping_sub. Stamps can't run backwards within one CPU's block. SDM 17.2's APERF/MPERF reset on overflow would take more than a century at these clocks.
• Refused lines. A refused line carries refused= and no figure keys, so it can't be read as a measured line.

What remains wrong is not the window but the rate the instrument assumes for MPERF.

Findings

BLOCKER 1. tests/guest/osbench/rules/src/lib.rs:311-315 (run_mhz) and tests/guest/osbench/src/around.rs:88 (busy_permille): MPERF is assumed to count at the stamp counter's rate. The SDM does not grant that, and on the T14 it is measurably false. The corrected instrument still prints wrong figures there.

• What the SDM says. SDM 325462-093 Vol. 3B §17.2 (the copy at dl/sdm/sdm-093.txt:245188-245198): "IA32_MPERF ... increments in proportion to a fixed frequency, which is configured when the processor is booted ... Only the IA32_APERF/IA32_MPERF ratio is architecturally defined". §22.7.3 (:270839): "The TSC, IA32_MPERF, and IA32_FIXED_CTR2 operate at close to the maximum non-turbo frequency".

• Busy reads 989 on any T14 window. A fully busy T14 CPU reads busy_permille=989 whatever the window's length:

  • every deflate row, about 4 s, in run2 to run5 and D1 to D3;
  • D1's 5 ms identity and system windows (cpu 6, 989).

• The AMD reads 999. That rules out windowing as the cause.

• run_mhz sits 1.0 % above osbench's own cpu.clock for the same row, in every T14 boot:

  Boots	run_mhz	mhz= (cpu.clock)
  deep (run4, run5, D1-D3)	4229-4230	4188-4189
  A*	3827-3828	3788-3789

  That gap is 1/0.989.

• Why the fix didn't change it. Where the window equals the row, the corrected formula gives the same value as the old one, so 4230 is what B6 will print. 4230 is also above the 4.2 GHz bin that cpu.clock puts this CPU at.

• Probable cause (inference). The kernel log gives the TSC as 2419.2 MHz by CPUID. If MPERF counts at a 2.4 GHz-class reference, 4230 × 0.989 = 4184, which is cpu.clock's 4188.

• Cheapest check. The B6 T14 crunch.single.deflate.cpus lines. If window_us equals row_us within 0.3 % while busy still reads 989 and run_mhz still reads 4230, the assumption is refuted.

• Fix. Either take the clock from the architecturally defined ratio and a vendor-defined reference frequency, cited at the site, or print the ratio and leave MHz to cpu.clock. Busy then has to be either counted in the reference's ticks or labelled as MPERF over TSC. The run_mhz doc sentence "MPERF counting at that rate" is false on this machine and goes with the fix.

BLOCKER 2. issues/idle-cpus-enter-the-c-states-cst-names.md:59-64 (and PR body line 77, which is the orchestrator's): the re-derived T14 claims are not supported.

• The clock figures. "4.23 GHz against 3.83" are the figures with Blocker 1's error in them. The same rows' cpu.clock field reads 4188 against 3789, which is the independent reading.
• The proof of matching. "The thread busy 989 ‰ across deflate's" is offered as evidence that the windows matched their rows. 989 is what every busy T14 CPU reads over any window, so it shows nothing about the window. The old lines carry no window or row figures, and r6 has no analysis file that re-derives them.
• The churn range. malloc_churn.1048576.t1's "4.19 to 4.21 GHz against 3.79" does not match what the readbacks print. The busy threads read 4187-4231 on the deep boots (mostly 4225-4231) and 3793-3827 on A*. The record doesn't say how the range was obtained.
• Fix. State the clock step from cpu.clock's row field (4188-4189 against 3788-3789), or from B6 once Blocker 1 is fixed. Drop the "windows that matched" justification unless a printed window_us/row_us supports it.
• The C7 claim stands. "99.5 to 99.9 % in C7" holds: core C6/C7 residency counts in TSC ticks, and the AMD-style 999/998 readings agree.

BLOCKER 3. tests/guest/osbench/src/around.rs:5: the edited header line is 94 columns among lines of 71-77. Rewrap it. This is the same class of fix as r4's.

SUGGESTION 1. rules/src/lib.rs:257 (SLACK_PERMILLE): the 10 % is stated as a number without a basis.

• Size of a legitimate mismatch. A joined before read is the only one that occurs, and it is bounded by JOIN, 10 ms, which is 0.25 % of a 4 s crunch row. On such a row, 10 % admits 40 times more non-row time than any mechanism produces.
• Alignment. The rule checks length, not alignment. An after that predates the row's end gives a window of about the right length, shifted earlier. That happens when another reader (inspect) reads within JOIN, or when a CPU doesn't answer within ANSWER.
• The stale flag. CpuCounts drops the stale flag that the kernel sets for the second case.
• Options. Bound the mismatch by JOIN plus the read's cost and refuse records the kernel marks stale. Or say in one line what 10 % bounds.

SUGGESTION 2. issues/a-counters-read-within-10-ms-of-another-returns-the-older-reading-as-new.md: the content is accurate.

• Checked against the code. counters.rs:202 joins the read. record() marks only unserved CPUs stale, so a joined reply looks fresh. Two reads 5 ms apart get the same values. The exit is concrete and testable.
• Status and owner. It is status: assigned to "the idle track's follow-up", but the track is status: open and does not name this issue. Its The T14's one-thread clock is held to its three-active-core turbo bin by hlt idle; deep idle lifts it 10 % #1066 paragraph still calls the diagnostic boots "next", although D1-D3 have run and the new issue cites D1. Either link the issue from that sentence or make the issue open, so its holder is findable as issues/README.md requires.

SEND BACK

[Japabu]

Review r6: deep-idle at 5c812f1 (focused on 1d46a3c, 862f68c and 18fe4e1, plus the records-only efe09aa and 5c812f1)

Architectural assessment

r5's points are resolved, and the instrument now prints only figures that have a documented rate.

• APERF/MPERF. aperf_mperf_permille is the only figure taken from the pair. busy_permille and run_mhz are gone from the tree. git grep finds no derivation of a clock or busy fraction from MPERF in code. The clock is cpu.clock's.
• Residency. Core C6/C7 is still a fraction of the TSC window. I checked this against the SDM copy:
  • Table 2-20 says "Count at the same frequency as the TSC" (sdm-093.txt:332815-332830).
  • Table 2-57 counts in "units compatible to P1 clock frequency" (:351712-351730).
  • So leaving out 06_AFH is correct, and the test signature 0x000a_06f0 decodes to family 6, model AFH.
• The window bound is principled. I checked it against kernel/src/counters.rs:190-221:
  • A fresh round is issued at clock::now() inside the call. Its stamps and any later round's snapshot all land before the call returns, so the window is at most the row, and at least the row less the two calls' cost.
  • A joined read is served a round issued under JOIN before its call, so the window can grow by at most JOIN_NS.
  • Stale records are refused before anything else.
• Mutations. All four (join-unbounded, reads-unbounded, stale-ignored, afh-admitted) exit 101 with the expected test red, at 862f68c. Nothing after that commit touches the rule or the kernel list.
• Size. About +25 production lines and +20 test lines since r5. The ABI is unchanged.

B7 readbacks (metal/run7, which I parsed independently):

• Line counts. 96 T14 and 192 AMD instrument lines. Each bench has one refused= match, and it is mem.munmap_touched_remote.4k's windows_refused=0/10, a field of that row. No instrument line was refused. Every window lies in [row - reads_us, row + 10 ms].
• Ratios are in range.
  • T14: 188-1749. The deflate thread reads 1748; times the 2419 MHz TSC-class reference that is about 4.2 GHz.
  • AMD: 776-2358. The thread reads 2354; times the 4800U's 1.8 GHz P0 that is about 4.24 GHz, against cpu.clock's 4214-4239.
• C7 is in range. T14 C7 reads 0-999, with C6+C7 never above 1000. Only the deflate thread's core (CPUs 1 and 5) reads 0. On the AMD, C7 is absent.
• Windows longer than their rows. On 24 T14 and 48 AMD lines the window exceeds its row by 0.1-0.7 ms. These are sha256, matmul and parse: each row's begin read joined the previous row's closing round. This is the join case the bound admits, and it shows the bound is needed.

Records. I checked the records at 5c812f1 against the readbacks:

• The clock step. Deflate and churn t1 read mhz= 4187-4189 in the eight deep boots (B2×2, B3×2, B4-B7) and 3788-3789 in the four A* boots.
• Group starts. B6 and B7 each have 10 of 17 above 4100.
• The AMD deflate count. Twelve boots: 3 slow, B6, and 8 fast including B7 on CPU 12. That matches the readbacks, excluding run2's plain A, as the record has done since r3.
• The track links the stale-reading issue.

Findings

r5 B1-B3 and S1-S2 are resolved. No new blocker.

SUGGESTION 1. tests/guest/osbench/rules/src/lib.rs:261 (JOIN_NS): a private copy of a kernel-private constant that the ABI does not state.

• The join is not documented in toyos_abi::counters. The bound is only correct while kernel/src/counters.rs's JOIN stays at 10 ms.
• The stale-reading issue's exit will change that contract.
• Naming osbench_rules::window::JOIN_NS in that exit would stop the copy from being left behind. Either the ABI states what replaces the join, or the rule deletes the constant.

SUGGESTION 2. sdk/abi/src/counters.rs:57 (DeepIdleNanos): since 1d46a3c, nothing reads this counter by name.

• Only inspect's generic dump of every counter reads it.
• Its doc is true for a total since boot. But a difference between two reads, the only use the ABI header suggests, includes time from before the window.
• If no reader needs the total, deleting the counter and its kernel accounting removes an ABI field with no reader.

SUGGESTION 3. issues/the-amd-benchs-deflate-runs-6-percent-slow-in-some-boots-at-the-same-clock.md:21-23: "no other CPU's MPERF advanced by a thousandth of its TSC's count" still compares MPERF with the TSC.

• The margin is three orders of magnitude, so the conclusion stands.
• Comparing each idle CPU's MPERF with the busy thread's MPERF over the same row would need no assumption about MPERF's rate. APM §17.3 has MPERF counting only in C0, at one reference rate.

LAND

[Japabu]

Focused r6 LAND at 5c812f1, which adds only records after 18fe4e1, the code B7 booted. B7: the harness judge exits 0 on both benches, and every instrument figure is in range. r6's suggestions S1-S3 (naming JOIN_NS in the stale-reading issue's exit, the unread DeepIdleNanos counter, the AMD record's MPERF comparison) go to the counters follow-up on this track, which rewrites that contract. Landing.