Skip to content

crypto: ChaCha20-Poly1305 and OpenSSH in one stitched AVX2 pass, eight blocks a batch - #1056

Merged
Japabu merged 5 commits into
mainfrom
wt/chacha-avx2
Oct 24, 2026
Merged

Japabu merged 5 commits into
mainfrom
wt/chacha-avx2

Conversation

@Japabu

@Japabu Japabu commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

On AVX2-only CPUs, the common case (every AMD before Zen 4, and Intel desktops since Alder Lake), toyos-crypto's
ChaCha20-Poly1305 trailed the fastest library. On the AMD bench (Ryzen 7 4800U, Zen 2), crypto-oracles r2 measured
seal 64 KiB at 1.15× AWS-LC, open 64 KiB at 1.34× ring and seal 1 KiB at 1.48× ring. AVX2 ran 6 ChaCha20 blocks a
batch. Open made two passes, the vector Poly1305 then the keystream. Block 0, the Poly1305 key, was a separate
scalar block. The seal stitch left 2 of every 10 double rounds and the tail unstitched. OpenSSH's construction made
two passes in both directions.

Change

AVX2 machines without AVX-512VL only. The T14's AVX-512 dispatch is untouched.

  • Eight blocks per AVX2 batch (C2). They run in the word-per-register layout that AVX-512 already used: one
    register per state word of all eight blocks. Sixteen registers spill a few words a round, but the layout needs
    none of the row layout's diagonal shuffles, which share Zen 2's two shuffle pipes (FP1/FP2) with its one shift pipe
    (FP2) (uops.info, Zen 2). Round 1 measured the four-pair row layout too, and it lost (table). It is gone. The
    stream (xor, which SSH also uses) takes the same layout.
  • One stitched pass for seal and open (C1, C3, C4), in arch::x86_64::chacha::avx2::chacha20_poly1305:
    • The key batch. Block 0 comes out of the first SIMD batch: with seven data blocks when sealing, or with one
      when an open runs past the batch, since an open's MAC can't start before the key.
    • The MAC beside the rounds. The scalar Poly1305 runs 3 blocks in each of the 10 double rounds, and any past
      thirty after them. Sealing absorbs each batch's ciphertext beside the next batch; opening absorbs a batch beside
      its own rounds, before the keystream overwrites it.
    • Tails. The tail batch is stitched as well, and short first and tail batches run one or two pairs.
    • Short records. Records of 1–192 B take the existing short path first, since it is cheaper there (round 4).
    • MULX. The stitched code compiles with BMI2, and the dispatch requires it with AVX2.
  • A refused ChaCha20-Poly1305 open zeroes the buffer on every path (aead::chacha_open, through
    secret::wipe), because the one-pass open has written plaintext before the tag is known. Paths without the one
    pass (AVX-512, portable, AArch64, and 1–192 B on AVX2) still decrypt only an authentic record and zero a refused one. AES-GCM's refusal
    zeroes since toyos-crypto: AES-GCM in one pass, 48 blocks to a reduction on VAES-512 #1054, so aead's module contract is now one rule for both. Every test of it (differential,
    Wycheproof, the RFC 8439 vector, the rustls adapter, the guest parity instrument) asks for zeros.
  • OpenSSH through the same pass. Its MAC runs over the encrypted length and the payload unpadded:
    • its blocks start 12 bytes into the payload;
    • an open reads up to 12 bytes past its batch, which is still ciphertext;
    • a partial last block ends in a one byte, as Poly1305's own message end does.
      A refused OpenSSH open zeroes the payload (ssh module contract).
  • The 16-bit rotation's mask passes through an empty asm block. LLVM otherwise lowers the constant VPSHUFB as
    VPSHUFLW+VPSHUFHW, two FP1/2 uops for one. Round 1's control without it was slower on the AMD at every size.
  • Deleted: chacha::seal, arch::chacha_seal, the six-block kernels and pairs_of. Poly1305::parts gets an
    expect(dead_code) on targets other than hosted x86-64, its only user. The attribute names an architecture in a
    generic module but holds no architecture code, as field25519.rs's selector does.

Measurements

Linux probes on each bench's own Linux (crypto-oracles' probe: ring 0.17.14, OpenSSL 3.5.5, AWS-LC 5.11.0;
toyos-crypto built incrementally in [profile.toyos] settings). Ratios are ours over ring; above 1 is slower.
The clock line reads 0.234 ns a cycle on the AMD (4.27 GHz).

Round 1, AMD:

variant seal 1 KiB open 1 KiB seal 64 KiB open 64 KiB
main b48c7dd 1.480 1.426 1.138 1.347
A: rows, four pairs 1.236 1.478 1.262 1.259
A without the VPSHUFB fix 1.248 1.509 1.293 1.299
A + BMI2 1.247 1.461 1.268 1.264
B: word layout 1.191 1.426 1.103 1.123
B + BMI2 (kept) 1.165 1.409 1.036 1.048
  • Each half alone (AMD, 64 KiB).
    • The ChaCha20 keystream: main 20.6 µs, B 19.4 µs, OpenSSL 18.6 µs, AWS-LC 18.5 µs.
    • B+BMI2's seal is 23.2 µs, so the stitched Poly1305 costs about 3.8 µs on top of the keystream (AWS-LC's whole
      AEAD: 22.3 µs).
  • T14. Every variant left the rows where they were (seal/open 64 KiB 0.862/0.872 of ring), except reject at
    64 KiB: 8.21 → 8.80 µs, the wipe of the refused 64 KiB buffer, still 0.33 of ring. That figure is 2c00ebb's,
    whose wipe was a fill; since the rebase it is toyos-crypto: AES-GCM in one pass, 48 blocks to a reduction on VAES-512 #1054's 16-byte volatile wipe, which costs more (on the AMD bench
    reject 64 KiB reads 0.75 µs above open at 035c36d, against 0.58 µs below it at 2c00ebb).
  • Reject on the AMD rises from 0.41 to ring's level (1.05 at 64 KiB): a forgery now costs the full one-pass
    decrypt and a wipe, as every other library's does. This is the approved C1 tradeoff.
  • After round 1 the head changed in three ways: the one-block first batch for an open that runs on (round 1's
    open 1 KiB barely moved, 1.426 → 1.409, because it hashed 448 bytes unstitched while waiting for the key); one pair
    for short first and tail batches (round 1's 64 B rows rose); and OpenSSH.

Round 2: main b48c7dd against the head 2c00ebb, both benches, two alternated runs each (ours over ring):

row AMD main AMD head T14 main T14 head
seal 64 B 1.244 1.306 1.105 1.118
open 64 B 1.148 1.186 1.014 1.016
seal 1 KiB 1.481 1.252 1.039 1.039
open 1 KiB 1.427 1.185 1.047 1.054
reject 1 KiB 0.700 1.170 0.605 0.635
seal 64 KiB 1.139 1.042 0.862 0.860
open 64 KiB 1.347 1.044 0.872 0.871
reject 64 KiB 0.411 1.050 0.302 0.326
OpenSSH seal 1 KiB 0.690 0.600 0.570 0.565
  • Short open: the fix took open at 1 KiB from 1.41 (round 1) to 1.19.
  • Seal at 1 KiB: 1.25, against round 1's B+BMI2 at 1.17. The 64 B rows stay a little above main's.
    • Two causes are candidates since round 1: one pair rather than two for a short stitched tail, and the word layout
      rather than four pairs for the first batch.
    • Both are staged as variants for one AMD run. This is unresolved until then.

The rebase. The head was then rebased onto main (035c36d) past #1054, whose AES-GCM refusal wipe makes the
contract the same for both AEADs. The ChaCha code is 2c00ebb's, but its refusal now goes through #1054's
secret::wipe (16-byte volatile stores) rather than a fill. The probe numbers above are 2c00ebb's.

Round 3: the head 035c36d (run twice) and two variants of it, AMD bench, ours over ring:

build seal 1 KiB open 1 KiB seal 64 KiB open 64 KiB
head 035c36d 1.259 / 1.261 1.209 / 1.207 1.100 / 1.098 1.101 / 1.100
+ a short stitched tail in two pairs 1.239 1.190 1.072 1.074
+ an eight-block first or tail batch in four pairs of rows 1.241 1.237 1.097 1.103
  • No variant chosen, and no code changed. Neither recovers round 1's seal at 1 KiB (1.17).
  • The spread is larger than the variants. The tail variant moves the 64 KiB rows by 0.03, but its only change is
    the 64-byte tail, which costs far less than the 600 ns that implies.
  • The same loop reads differently in two builds. The head reads 1.10 at 64 KiB where 2c00ebb read 1.04 in
    round 2, with ring unchanged (22.0–22.4 µs) and the same clock (0.234 ns a cycle). The bulk loop is the same
    4,503 bytes in both builds and differs only in alignment.
  • Code placement is the suspect, not verified. That and the bench's run-to-run state are both unmeasured. The
    bulk rows read 1.04 to 1.10 of ring at these two heads.

Round 4 (review r1's suggestion): 1–192 B through the short path first, AMD bench, head ca676da against the
variant, alternated twice (ours over ring):

row head short first
seal 64 B 1.290 / 1.292 1.231 / 1.228
open 64 B 1.193 / 1.184 1.113 / 1.116
reject 64 B 1.104 / 1.091 1.035 / 1.035
seal 0 B (control) 1.355 / 1.362 1.383 / 1.384
seal 1 KiB (control) 1.265 / 1.262 1.258 / 1.259
  • Short first kept (79764c8). Every 64 B row gains about 10 ns, in both runs, with the controls unchanged.
  • What it makes stale, on the AMD bench only:
    • the 1–192 B rows of round 3's cryptocase;
    • round 4's 64 B ChaCha20-Poly1305 timing rows, which then ran the one-pass kernel and now run the short path.
  • What stays: the 1 KiB timing rows, which run the same code. On the T14 the AVX-512 dispatch never takes the
    one pass, so its paths are unchanged.
  • Boots: one AMD cryptocase and one AMD cryptoctcase at 79764c8 are staged; the orchestrator decides
    whether they are owed.

Checks and evidence

  • Unit tests that call the AVX2 kernels directly, so they run on any CPU with the instructions whatever the dispatch
    picks (the one-pass test asks for AVX2 and BMI2, and refuses by name without them):
    • arch::x86_64::chacha::tests::the_avx2_stream_matches_one_block_at_a_time: every length 0..1600 and batch edges,
      across counter carries;
    • ...::the_avx2_one_pass_matches_both_macs_composed: RFC 8439's MAC and OpenSSH's, sealing and opening, every
      length 1..1600 and up to 16 KiB, against portable blocks and poly1305::tag.
  • Through the dispatch:
    • aead::tests::a_forged_chacha_record_opens_to_zeros;
    • ssh::tests::packets_match_the_construction_composed_from_blocks.
  • The differential sweeps gain the new batch edges. The refusal expectations in differential, Wycheproof, the RFC
    8439 vector test and the rustls adapter test follow the new contract.
  • Mutants: 11 patches, each a plausible defect in a new path (absorb schedule, MAC skew, OpenSSH's last block, the
    tail emit, the rotation mask, the open's read-ahead, the seal's lag, the pair choice, the wipe, the OpenSSH
    header order).
    • On QEMU TCG with AVX2 and no AVX-512, every one fails the unit tests, each by the tests named in the evidence.
    • Bench runs: all eleven exit 101 on both benches (round 2). m11 fails on the T14 too: it moves the header encryption into the one-pass arm, so the T14's fallback arm sends the header unencrypted.
  • QEMU TCG (AVX2, BMI2, no AVX-512; a Linux 6.6 guest, correctness only) at the head 2c00ebb, all exit 0:
    • toyos-crypto's lib tests (80);
    • vectors (10);
    • Wycheproof (4, the ChaCha20-Poly1305 file included);
    • differential (13);
    • the probe's agreement with ring, OpenSSL and AWS-LC at every AEAD and OpenSSH length to 1153 B and on every row's
      input.
  • Benches: round 1's lib and differential tests exited 0 on both benches; round 2 is lib and differential tests exit 0 on both benches at 2c00ebb.
  • Machine code (Linux probe build of ca676da, published as r4/inspect/): 20 MULX and 4 IMUL (h2·r0) in each of chacha20_poly1305 and keystream, and no MUL. The full-batch path
    calls nothing. Inside the bulk loop's range, the only calls are keystream and block::xor on the tail branch, once
    per message; keystream also runs once for the first batch.
  • cargo run -- --ci host: 103 steps, all green, exit 0, at ca676da and at 79764c8. clippy's three toyos-crypto shapes on x86_64 and aarch64: clean.
  • cargo run -- --build-only: exit 0 at 035c36d, ca676da and 79764c8.
  • 9c68ba5 changes comments and the track issue only; no check above is affected.
  • Metal cryptocase (crypto_instrument) and cryptoctcase (crypto_secret_timing) on the T14 and the AMD bench:
    round 2 at 2c00ebb: every boot rc=0. crypto_secret_timing passed on both benches, a pass that rests only on the orchestrator's judge summary ("1 passed, 1 failed, 2 boot(s)"): round 2's readbacks were deleted before they were published and are lost. crypto_instrument failed on both: the guest parity test still asked a refused ChaCha20-Poly1305 record to be left as it was, and it panicked before timing any row. That test now asks for zeros, for OpenSSH too, and passes under QEMU (78 rows in each build). cryptocase at 035c36d passed crypto_instrument on both benches (round 3, "1 passed, 0 failed, 1 boot(s)" each; readbacks published).
  • Constant time.
    • Until ca676da, crypto_constant_time timed ChaCha20-Poly1305 and OpenSSH at 64 B only. That never reaches the
      stitched batch, and OpenSSH at 64 B never reaches the kernel at all, so no timing boot so far covers the new path.
    • It now times both at 1 KiB too (seal and refusal), as it already timed AES-GCM's stitched paths.
    • Round 4, cryptoctcase at ca676da: crypto_secret_timing passed on the T14 and the AMD bench, 1/1 each.
      Every ChaCha20-Poly1305 and OpenSSH row passed at 64 B and at 1 KiB, max |t| 0.59–2.93; the leaky control leaks
      and the null control passes.
    • By inspection, every branch and index depends on len, seal, pad or prefix.len(), the tag goes through
      ct::equal, and the wipe's length is the record's.
    • MULX is on Intel's data-operand-independent timing list. AMD publishes no such list. MULX uses the multiplier MUL
      uses, and the crate's BMI2 Montgomery kernels already use MULX on the AMD bench under
      crypto_secret_timing_curves. That last point is an inference, not a measurement.
  • Evidence: https://github.com/ToyOSOrg/evidence/tree/89c6f54163a79f3b4327ab21405b8191eb2a1bf6 (branch pr-1056).

Also in this branch

  • issues/toyos-crypto-is-in-the-tree-on-par-with-ring-and-secure.md:
    • the AVX2 exception describes the one-pass open, an owner decision of 2026-10-23, from the guest table at
      035c36d, with round 3's per-build spread;
    • the superseded two-pass breakdown is gone;
    • the per-message entry adds the 0 B and 64 B rows at 035c36d.
  • issues/a-metal-judge-says-on-the-t14-for-every-bench.md (new, tooling): Readback::job_passed words every
    bench's failure as "on the T14". Round 2's AMD judge line said so.

Risks

  • Per-build spread. The AVX2 bulk rows vary by about 0.06 of ring from build to build with the same vector loop
    (round 3). An unrelated change elsewhere may move them either way.

  • Reject cost on AVX2. One-pass open makes reject cost an open, as above.

  • Inlining. batch and eight are cost-based #[inline] #[target_feature] functions, as everywhere in the
    crate. Each closure type gives each generic instance one call site, which is what inlines them today. A future
    second caller should be checked in the machine code.

  • Spills. The word layout's spills are LLVM's choice. A different LLVM could spill differently on Zen 2.

  • Intel AVX2-only CPUs (Alder Lake and later desktops) are unmeasured: no such bench. Their shift ports differ
    from Zen 2's, so the layout choice is Zen 2's measurement.

🤖 Generated with Claude Code

On AVX2 without AVX-512VL (the AMD bench, Zen 2), seal 64 KiB ran 1.14x
ring and open 1.35x: six blocks a batch, open in two passes, block 0
apart from the batches.

- Eight blocks a batch, in the word-per-register layout AVX-512 already
  used, for the stream and the AEAD. Sixteen registers spill a few words
  a round, but the layout needs none of the row layout's diagonal
  shuffles, which share Zen 2's two shuffle pipes with its one shift
  pipe. Measured against four pairs of rows on the AMD bench, it won at
  every size.
- One pass for seal and open. Block 0, the Poly1305 key, comes out of
  the first batch: with seven data blocks when sealing, or with one when
  an open runs past it, since an open's MAC waits for the key. The scalar
  Poly1305 runs three blocks in each of the ten double rounds and the rest
  after them. Sealing absorbs a batch's ciphertext beside the next batch,
  opening beside its own before the keystream overwrites it. Tails are
  stitched too, and short batches run one or two pairs.
- The stitched code compiles with BMI2, so its multiplies are MULX.
- OpenSSH's MAC goes through the same pass: its blocks begin 12 bytes into
  the payload, an open reads up to 12 bytes past its batch, and a partial
  last block ends in a one byte.
- A refused ChaCha20-Poly1305 record and a refused OpenSSH payload are
  wiped, as AES-GCM's record is since #1054. The one-pass open has written
  plaintext before the tag is known. Every test of the refusal contract,
  the guest's parity instrument's included, now asks for zeros.
- The 16-bit rotation's mask passes through an empty asm block. Otherwise
  LLVM lowers the VPSHUFB as VPSHUFLW and VPSHUFHW, two shuffle uops for
  one.
- Unit tests call the AVX2 kernels directly, so they run on any AVX2 CPU
  whatever the dispatch picks. An OpenSSH unit test composes the packet
  from portable blocks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Review r1: #1056 at 4c853eb (code 035c36d, on main 3a19907)

Architecture

The design matches the analyst's spec (C1–C4) and stays inside its boundary:

  • All ISA code lives in arch/x86_64/chacha.rs.
  • The selector in arch/mod.rs returns Option<Poly1305>.
  • chacha::aead and chacha::openssh are thin, arch-neutral entries.
  • aead and ssh keep their fallback arms, now with one refusal rule.

One kernel serves both RFC 8439 and OpenSSH. The only difference is the pad/skew handling of the MAC's block grid, which is a single mechanism. Superseded code is deleted: the six-block path, pairs_of, chacha::seal, arch::chacha_seal, and the 8-of-10-rounds stitch. The AVX-512 dispatch still declines (avx512vl() → None). Its xor reaches the same eight and emit_eight code through the macro, and the T14's bulk rows did not move (0.862/0.872 → 0.860/0.871).

Size: the crate's sources grow by about +360 lines, of which about 165 are the new unit tests. That leaves about +200 net production lines. For that, the AMD bench moves open 64 KiB from 1.35 to 1.04–1.10 of ring and seal 1 KiB from 1.47 to 1.25, and OpenSSH open 16 KiB from 0.51 to 0.40. I accept the size for that gain under the hot-path direction.

Correctness: traced, no defect found

I traced avx2::chacha20_poly1305 by hand at the boundaries, for both directions and both pad modes:

  • The first batch is first = 448, or 64 for an open past 448 bytes. The counter then advances by 1 + first/64.
  • reach() keeps the MAC on the grid that starts skew bytes in. That makes prefix.len() + hashed ≡ 0 (mod 16) wherever parts() is called, so the two unreachable! calls are true invariants. Lengths alone cannot reach them.
  • Opening: every batch absorbs data[hashed..reach(end)] before emit_eight/xor_bytes overwrites data[done..end]. Since reach(end) ≥ end and hashed ≥ done, nothing is hashed after it is decrypted.
  • OpenSSH's read-ahead of up to 12 bytes stays ahead of the decrypted region (I checked len = 580 by hand). The final partial block is terminated(…, 1) with no 2^128 bit, so finish sees filled == 0.
  • Sealing lags by one batch. The final ≤ 512 bytes are absorbed unstitched by update/update_padded.
  • The 64-bit counter: lanes split counter + j into two words. IETF input is bounded by check_chacha_length, which allows 2^32 − 1 blocks after block 0, so no counter used for data carries into the nonce word. A tail batch's surplus lanes may carry, but their output is never used. OpenSSH's payload is at most u32::MAX bytes.
  • The unit test covers every length 1..1600 in both modes and both directions, plus the 2 KiB/4 KiB/16 KiB edges. AAD lengths there are 0, 5 and 17; the differential test covers 0..33. Length 0 never enters the kernel (data.is_empty() → None).

Refusal wipe. I found no path that can leak plaintext:

  • aead::chacha_open and ssh::OpeningKey::open both end in if !opened { secret::wipe(..) }.
  • wipe uses write_volatile plus a SeqCst compiler fence, so the compiler cannot remove it.
  • On a refusal the fallback arms leave the buffer as ciphertext, and it is then zeroed.
  • The only other error returns are the length checks, which run before any write, and an infallible xor after authentication.
  • OpenSSH's header is decrypted only on success.
  • The keystream buffer is a Secret, and Poly1305 erases itself on drop.

Constant time by inspection. Every branch and index depends only on len, seal, pad or prefix.len(). The tag goes through ct::equal, and the wipe's length is the record's.

  • MULX is on Intel's DOIT list (doit-list.txt:493).
  • AMD 71030 (Feb 2026) offers no DOIT-style list or control.
  • [inference] MULX uses the same multiplier as MUL, and the crate's BMI2/ADX Montgomery kernels already use MULX on the AMD bench under crypto_secret_timing_curves.

Dispatch. The AVX2 entry requires avx2 && bmi2 && !avx512vl, and the kernel compiles with avx2,bmi2. UEFI and AArch64 return None.

Evidence

  • I re-derived the medians from the AMD and T14 oracle outputs for round 2 and from the AMD outputs for round 3. The PR tables are accurate.
  • All eleven mutants exit 101 on both benches at 2c00ebb, each failing in the named tests. m10 removed data.fill(0), which has since become secret::wipe. By inspection the same four zero-asserting tests still guard it, so I don't need it re-run.
  • The round-3 cryptocase readbacks are published and passed on both benches at 035c36d. That is direct metal evidence for the new path on the AMD bench under ToyOS.
  • No identifiers: paths and the AMD bench's host are masked. Model and BIOS strings and image partition GUIDs are not among sourcegate's shapes.

BLOCKERS

  1. lib/crypto/src/arch/x86_64/chacha.rs:788 and :708 — the test's safety claim "every AVX2 CPU has BMI2" is false, and the helper checks AVX2 only.

    • ToyOS's own QEMU CPU disproves it: build/arch.rs:167 is qemu64,…,+avx,+avx2 with no BMI2. The handoff says the guest instrument took the fallback for exactly that reason.
    • The dispatch checks bmi2() separately, so the code itself contradicts the comment.
    • On such a CPU, the_avx2_one_pass_matches_both_macs_composed calls a #[target_feature(enable = "avx2,bmi2")] unsafe fn and dies on #UD in MULX, instead of failing clearly.
    • Fix: require features().bmi2() for the one-pass test (the stream test needs only AVX2), and make the SAFETY comment say what was checked.
  2. PR "Checks and evidence", Constant time — the cited crypto_secret_timing pass does not cover the new code.

    • crypto_constant_time.rs times ChaCha20-Poly1305 only at 64 B (lines 66–79) and OpenSSH only at 64 B (95–105).

    • At 64 B the one-pass kernel never enters batch or the stitched MULX loop (first = len, so the loop is skipped). OpenSSH at 64 B takes Keys::short and never reaches the kernel at all.

    • The instrument's own rule, RECORD at line 29 ("long enough for AES-GCM's stitched paths, whose refusal zeroes what it decrypted"), now applies to ChaCha20-Poly1305 too, yet line 81 still limits 1 KiB to aeads[..2].

    • Your reasoning about what changed since 2c00ebb is right: the wipe's length is public, and the rebase didn't touch this code. The flaw is the premise that the 2c00ebb boot covered the stitched path; lost or not, it never timed it.

    • Fix, either of:

      • (a) extend the 1 KiB rows to all three AEADs and add an OpenSSH 1 KiB seal/refusal row. Then one crypto_secret_timing boot on the AMD bench, the only bench that takes this path.
      • (b) say in the PR that constant time of the stitched path rests on inspection plus the inference above, not on a timing pass.

      I recommend (a): it's a two-line test change, and it is the repository's established guard for exactly this property.

  3. issues/toyos-crypto-is-in-the-tree-on-par-with-ring-and-secure.md, lines 214 and 219–232 — the record is now partly false, and it carries review history.

SUGGESTIONS

  • The 64 B and 0 B rows. aead now tries the kernel before short(), while ssh keeps short first; nothing in the PR explains the difference.
    • On the AMD bench, 64 B reads +0.04–0.07 of ring. The T14's seal/0 also moved 1.17 → 1.24, on a path whose code is effectively unchanged. So these deltas are within the per-build spread and not an established regression.
    • Still: when the AMD bench is next booted, one probe with short first for 1..=192 would settle it. The 512-byte Secret stream, zeroed and erased per call, is the obvious cost.
  • aead.rs (chacha::aead(..., true)), chacha.rs (openssh(..., seal)), x86_64.rs. The two bare bools (seal, pad) are opaque at the call sites. A Direction enum, or naming the MAC grid, would read better.
  • poly1305.rs:105. The cfg_attr(not(all(target_arch = "x86_64", …)), expect(dead_code)) names an architecture in a generic module. It holds no arch code and field25519.rs has a precedent, so this is not a blocker, but say so if you keep it.
  • PR body. The T14 reject figure (8.21 → 8.80 µs) was measured with 2c00ebb's fill. The head's 16-byte volatile wipe costs more: on the AMD bench, reject 64 KiB is 0.75 µs above open at 035c36d, against 0.58 µs below it at 2c00ebb. Say that the T14 figure predates the wipe.
  • PR body. The machine-code claim ("no call and no MUL, MULX throughout") is backed by inspect/norm-035c36d30.s in scratch, which matches it, but that file is unpublished. Publish it masked, or mark the claim as unpublished.
  • chacha.rs:1–13 module header. The new first sentence threads two layouts, a register count and Zen 2 pipe sharing into one clause, and it is hard to parse. Two short sentences, one for each layout, would do.

Open items as recorded

  • Seal 1 KiB at 1.24–1.26 (round 1: 1.17) remains unexplained.
  • The bulk rows spread from 1.04 to 1.10 between builds with the same loop.
  • Neither blocks landing: the gain over main (1.47 → 1.25 and 1.35 → 1.04–1.10) is far larger than the spread, and both are recorded honestly.
  • On the T14 there is no regression beyond the approved reject cost.

SEND BACK

instrument reaches the stitched paths

- The one-pass unit test checked AVX2 alone before calling a kernel
  compiled with BMI2, and claimed every AVX2 CPU has BMI2. ToyOS's own
  QEMU CPU doesn't. It now requires both and says so; on AVX2 without
  BMI2 it fails by name, where it died on an invalid opcode before.
- crypto_constant_time timed ChaCha20-Poly1305 and OpenSSH at 64 B only,
  which never reaches the stitched batch: it now times both at 1 KiB too,
  as AES-GCM's stitched paths already were.
- The track issue's AVX2 exception describes the one-pass open from the
  guest table at 035c36d, and drops the superseded two-pass breakdown.
- The module header says each layout in its own sentence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Response to review r1 (head ca676da, code commit 035c36d plus ca676da)

Blockers

  1. The one-pass test's BMI2 check: fixed.

    • The new avx2_bmi2() helper requires both features and refuses by name otherwise; the stream test keeps
      avx2(). The SAFETY comment now says that CPUID and XCR0 admitted AVX2 and BMI2.
    • On QEMU TCG with -cpu max,-bmi2, the test fails with "this host's CPU lacks AVX2 or BMI2". The control, the old
      test on the same CPU, dies on an invalid opcode. Both logs are in evidence r4/vm/.
  2. Constant-time coverage: option (a).

    • crypto_constant_time now times ChaCha20-Poly1305 seal and refusal at 1 KiB, as AES-GCM's already were, and
      adds an OpenSSH seal and refusal at 1 KiB. RECORD's doc names all three stitched paths.
    • At 1 KiB, a seal runs the first batch, one full stitched batch and a stitched tail, and an open runs the
      one-block first batch and the same. OpenSSH at 1 KiB is past Keys::short.
    • The job took about 465 s and 434 s with its 19 rows after toyos-crypto: AES-GCM in one pass, 48 blocks to a reduction on VAES-512 #1054. Four more rows stay well inside its 1,455 s
      addition, so the bound is unchanged.
    • cryptoctcase at ca676da is staged for both benches, since the instrument changed on the T14 too. The PR no
      longer cites the 2c00ebb pass for this path.
  3. The track issue: fixed.

    • The two-pass breakdown (2.5 µs of Poly1305 plus 5.5 µs of ChaCha20) is gone, and so is the review-history clause.
    • The entry gives the one-pass rows from the guest table at 035c36d: AMD 1 KiB seal 1.25 and open 1.19; 16 KiB
      seal 1.11 and open 1.09; reject 1.14 and 1.17.
    • The approval is cited as an owner decision of 2026-10-23. The claim is limited to the one-pass open.
    • The per-message entry adds the current 0 B and 64 B rows.

Suggestions

  • short() first: taken, as a measurement. It can't ride the cryptoctcase image, which times no parity rows,
    so it's an optional two-minute AMD Linux probe (head against aead trying short() first; evidence
    r4/README-probe.txt).
    • If it wins, the change only returns 1–192 B to main's short path. The new 1 KiB timing rows cover the stitched
      path either way.
  • bool parameters: declined for now. A Direction or grid enum would touch the kernel, the dispatch and both
    callers. That would invalidate the passed cryptocase boot for readability alone. I'll do it if you want it in this
    PR.
  • poly1305.rs cfg_attr: kept, and the PR now says why. It holds no architecture code, as field25519.rs's
    selector does.
  • T14 reject figure: the PR now says it was 2c00ebb's fill, and that the rebased wipe costs more (the AMD
    reject-against-open gap you measured).
  • Machine code: published (evidence r4/inspect/). The claim is restated exactly. The full-batch path calls
    nothing. Inside the loop's range, the only calls are on the tail branch, keystream and block::xor, once per
    message.
  • Module header: rewritten, one sentence per layout.

Evidence: https://github.com/ToyOSOrg/evidence/tree/a276e825e3e297b9ebe2e133f1af10676c4fa6d0

Checks at ca676da:

  • clippy's three shapes on x86_64 and aarch64: clean;
  • --build-only: exit 0;
  • --ci host: 103 steps, all green, exit 0, at ca676da;
  • AVX2 VM: the ChaCha, Poly1305, AEAD and SSH tests pass, and the probe agrees at every length.

Requesting focused review of chacha.rs's test helpers and header, crypto_constant_time.rs, and the issue edits.

…e one pass

Review r1 of #1056 asked whether short records cost more through the
one-pass kernel. On the AMD bench (Linux probe, head and variant
alternated twice) the short path first takes 64 B seal from 193 to 183
ns, open from 188 to 176 and reject from 183 to 171 (1.29, 1.19 and 1.10
of ring to 1.23, 1.11 and 1.04); 0 B and the 1 KiB control are unchanged.
The T14, whose AVX-512 dispatch declines the one pass, is unaffected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Review r2: #1056 at 79764c8 (since r1: ca676da, 79764c8)

Architecture

Neither commit changes the design. 79764c8 moves 1–192 B ahead of the kernel in aead::chacha_seal and chacha_open. That matches ssh, which already tried Keys::short first, so the two callers now share one order. The match on Option becomes if short / else if let Some / else, which reads more directly, and the commit's net size is −1 line. ca676da adds a test helper (+11), four timing rows (+14) and doc edits.

r1's blockers

  • B1, resolved. avx2_bmi2() (arch/x86_64/chacha.rs:717) requires both features and fails by name. Only the one-pass test uses it, and the stream test keeps avx2(). The SAFETY comment at :798 now says what was checked. Evidence r4/vm/:
    • vm-w8-nobmi2.log panics at chacha.rs:719 with the named message;
    • the control, vm-035c36d30-nobmi2.log, traps on an invalid opcode;
    • vm-w8-max.log passes.
  • B2, resolved. The new rows reach the stitched path:
    • crypto_constant_time.rs:82 now iterates all three AEADs at 1 KiB. ChaCha20-Poly1305 at 1024 B is past short(), so on the AMD bench (AVX2, BMI2, no AVX-512VL) it enters avx2::chacha20_poly1305.
    • The new OpenSSH rows (:108–118) use header [0,0,4,0] and 1024 B, which is past Keys::short's 128 B, so they go to one_pass.
    • Both refusals are a full one-pass decrypt followed by secret::wipe. AMD refusal 1 KiB averages 1269.5 ticks against 1311.8 for seal, consistent with a full decrypt.
    • Evidence 89c6f541 r4/metal/{amd,t14}/: both kernel.logs carry ca676da…, and both verdicts read passed. The judge logs carry every ChaCha20-Poly1305 and OpenSSH row at 64 B and 1 KiB as pass, max |t| 0.59–2.93. The leaky control leaks and the null control passes ("1 passed, 0 failed, 1 boot(s)" on each bench).
    • The PR's figures match the logs.
  • B3, resolved at ca676da, then partly reintroduced by 79764c8. See blocker 1 below.

The short-path-first switch

I traced each boundary in both directions:

length path
0 not short → chacha::aead → None (data.is_empty()) → fallback: MAC first, an empty xor
1, 191, 192 short: a 256-byte stream, blocks 0–3. The open is MAC first and XORs only when authentic
193 and up the kernel on AVX2+BMI2 without AVX-512VL, otherwise the fallback, as before
  • chacha_short takes 1 + ceil(len/64) ≤ 4 blocks, which fits the 256-byte buffer.
  • Wipe on refusal. Every branch reaches the single if !opened { secret::wipe(data) }. On the short path the buffer is still ciphertext and is then zeroed. The ? on chacha::xor runs only after authentication, and the only earlier return is check_chacha_length.
  • a_forged_chacha_record_opens_to_zeros covers 1, 63, 64, 65, 192, 193 and on.
  • The kernel's 1–192 B handling with pad = true no longer runs in production. The direct unit test still checks it over 1..1600, and OpenSSH still sends 129 B and up to the kernel, so nothing became dead.
  • The T14 claim holds. Before, aead::chacha::aead returned None (avx512vl()) and then fell to None if short(..), which is the same code that now runs first. AArch64 and the portable build are likewise unchanged.
  • What the implementer listed as stale is right for the code: the AMD cryptocase rows at 1–192 B and the AMD 64 B ChaCha20-Poly1305 timing rows. Those 64 B rows now run main's short code, which main's AMD timing boots already covered. The orchestrator judges the staged boots, as settled.
  • What it also made stale, not listed: the two record statements in blocker 1, and the test doc in suggestion 1.

Accepted suggestions

  • Module header (chacha.rs:1–11). Accurate, and one sentence per layout.
  • T14 reject note and cfg_attr explanation in the PR. Accurate.
  • Machine code (r4/inspect/chacha20_poly1305-ca676da20.s).
    • It has 40 mulxq, as the PR says, and no mul.
    • It also has four imulq %r12, … (for example at 0x30408f), the h2·r term on accumulator and key data.
    • "No MUL" is literally true, but r1's "MULX throughout" was not. See suggestion 2.

BLOCKERS

  1. issues/toyos-crypto-is-in-the-tree-on-par-with-ring-and-secure.md — 79764c8 leaves the record false and puts review history back in it.
  2. tests/guest/rust-tests/src/bin/crypto_constant_time.rs:3 — the module header is stale.
    • "(AES-GCM's at 1 KiB too)" says that only AES-GCM is timed at 1 KiB. ca676da added 1 KiB rows for ChaCha20-Poly1305 and OpenSSH, and edited RECORD's doc five lines away without updating this one.
    • Fix: make the parenthesis cover all of them ("at 64 B and 1 KiB"), or delete it, since RECORD's doc already says why 1 KiB exists.

SUGGESTIONS

  1. lib/crypto/src/aead.rs:259–261. The test doc "an AVX2 host decrypts as it authenticates" now holds only past 192 B. ssh's test doc gives its bound ("past 128 bytes"); this one could give its own.
  2. PR body, machine code. Say "40 MULX and 4 IMUL (h2·r), no MUL", so the constant-time inference names every multiply. IMUL r64 is on Intel's DOIT list, and the 1 KiB timing pass now covers it empirically, so this is a wording fix.
  3. PR body line 32 (orchestrator). "Paths without the one pass (AVX-512, portable, AArch64) still decrypt only an authentic record": 1–192 B on AVX2 now belongs in that list too.
  4. crypto_constant_time.rs:29–31. "the stitched paths, AES-GCM's and ChaCha20-Poly1305's and OpenSSH's on AVX2" is hard to parse. "long enough for every AEAD's stitched path" would say the same.

Declined suggestions

The bool→enum change was declined because it would invalidate a passed boot. That reason is true, and the decline stands.

SEND BACK

- The track issue's AVX2 exception says the one pass serves records past
  192 B; its per-message rows give the landing code's readings alone.
- crypto_constant_time's header says the AEADs and OpenSSH are timed at
  64 B and 1 KiB; RECORD's doc says why in one line.
- The forged-record test's doc gives its bound: past 192 bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Response to review r2 (head 9c68ba5, text only over 79764c8)

Blockers:

  1. Track issue.
    • The AVX2 exception now says the one pass serves records past 192 B, and shorter records take the short path.
    • The per-message rows give only the landing code's readings: 0 B and 64 B on the AMD bench 1.04–1.38 (the
      short-first Linux probe), the T14 0.83–1.25, and the T14's 1 KiB rows 1.04–1.05. The T14 figures come from the
      guest table; the T14's paths are unchanged by the switch.
    • The 035c36d intermediate and the review clause are gone.
  2. crypto_constant_time.rs header: it now reads "sealing and refusal at 64 B and 1 KiB".

Suggestions, all taken:

  • aead.rs's forged-record test doc gives its bound: past 192 bytes.
  • RECORD's doc reads "long enough for every AEAD's stitched path".
  • Machine code: the claim now names every multiply, and corrects the count. The published listing holds 20 MULX
    and 4 IMUL (h2·r0) in each of chacha20_poly1305 and keystream, and no MUL. The earlier "40 MULX" summed both
    functions.
  • PR body: "1–192 B on AVX2" joins the list of paths that decrypt only an authentic record.

9c68ba5 changes comments and the track issue only. No build, test or boot is affected; the AMD cryptocase and
cryptoctcase at 79764c8 remain the code's boots.

@Japabu
Japabu marked this pull request as ready for review October 24, 2026 11:01
@Japabu
Japabu added this pull request to the merge queue Oct 24, 2026
Merged via the queue into main with commit e8f2432 Oct 24, 2026
8 checks passed
@Japabu
Japabu deleted the wt/chacha-avx2 branch October 24, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant