diff --git a/docs/ghostty-engine.md b/docs/ghostty-engine.md index 587d463..988e111 100644 --- a/docs/ghostty-engine.md +++ b/docs/ghostty-engine.md @@ -53,7 +53,13 @@ onto a dedicated thread with an explicit `maxStackSize`. Ghostty owns VT parsing, modes, viewport state, resize/reflow, scrollback, terminal replies, titles, and bells. Its render-state API is projected into the -shared immutable cell model consumed by the existing Skia renderer. This keeps +shared immutable cell model consumed by the existing Skia renderer. + +Projection is lazy. A feed only marks the projection stale. The next read of +`Buffer`, the cursor or a snapshot (normally once per frame) re-projects the +rows that Ghostty's render state reports dirty, then cleans that state. Title +and working-directory changes are still raised during the feed. See +[performance benchmarks](performance-benchmarks.md) for the measured effect. This keeps Avalonia tabs, panes, accessibility, clipboard, selection, search, and window behavior common across engines. diff --git a/docs/performance-benchmarks.md b/docs/performance-benchmarks.md new file mode 100644 index 0000000..fe6bb1a --- /dev/null +++ b/docs/performance-benchmarks.md @@ -0,0 +1,391 @@ +# Performance benchmarks vs Ghostty + +This page explains how Ghostty backs up its "fastest terminal" positioning, +what its own benchmarks do and do not measure, and how DT benchmarks its +**builtin** engine and renderer against it. The goal is a claim that holds up +under review: "with the default engine, DT matches or beats Ghostty at +workload X, measured this way, on this machine." + +For renderer-only details (caches, pixel verification, historical numbers) +see [renderer benchmarks](renderer-benchmarks.md). + +## How Ghostty supports its performance claims + +Ghostty's public claims are mostly qualitative. Few of them come with a method +someone else could rerun. + +| Claim | Where | Evidence | +| --- | --- | --- | +| "Same performance category" as the fastest terminals; "~100x faster than Terminal.app and iTerm" | README | None. Ghostty's own devlog measured about 6x against iTerm2 and under 2x against Terminal.app | +| `cat` of plain text 4x faster than iTerm/Kitty, ~60 fps under load | v1.0 README | No method given. The README itself says "we need better benchmarks" | +| SIMD parser: 7.3x faster ASCII parse, 16.6x faster UTF-8 decode; `time cat` of a 5.4 MB Japanese text: Ghostty 73 ms, Alacritty 66 ms, Kitty 392 ms, iTerm2 470 ms | [Devlog 006](https://mitchellh.com/writing/ghostty-devlog-006) | **The only write-up with a real method**: hyperfine microbenchmarks with noop/memcpy floors, plus 10-run `time cat` averages. Alacritty beat Ghostty in that table | +| 480–500 fps in DOOM-fire | [Zig patterns talk](https://mitchellh.com/writing/ghostty-and-useful-zig-patterns) | Anecdote. The same talk says input latency was never measured | +| 1.3: IO 20–30% faster on a 4 GB asciinema replay; frame builds 2–5x faster | [1.3.0 notes](https://ghostty.org/docs/install/release-notes/1-3-0) | Real-world replay, but the corpus is not public | +| `cat` of 150 MB ASCII in 575 ms (nightly) | Social post, second-hand | One run, no hardware listed. Unverified | + +The [about page](https://ghostty.org/docs/about) itself is careful: Ghostty is +"in some benchmarks faster, in others slower." + +### What Ghostty's in-tree benchmarks measure + +Ghostty's benchmarks live in `src/benchmark/` at the pinned commit (see +`native/ghostty/ghostty-upstream.json`). They are built with +`zig build -Demit-bench`, which produces two binaries: + +- `ghostty-bench +` runs one component microbenchmark. It prints + nothing; the documented method is to time the **whole process** with + `hyperfine --warmup N`. That time includes process startup and setup. This is + why many actions have a `noop` mode to subtract. +- `ghostty-gen ` writes synthetic input: `ascii`, `utf8`, `osc`, + `kitty` and `styled`. Seeds are time-based by default, so you save the output + once and reuse the file. + +The actions that matter for a comparison with DT: + +| Action | What it times | Closest DT measurement | +| --- | --- | --- | +| `+terminal-stream --data=F` | Full VT parse and terminal state update, read in 64 KiB chunks | `vt` (`ghostty-vt` target) | +| `+terminal-parser` | State machine only, actions discarded | none (DT's parser is not separable) | +| `+screen-clone --mode=render\|render-clean\|render-partial` | Updates to the CPU `RenderState` while holding the lock | `render` snapshot/plan | +| `+codepoint-width`, `+grapheme-break` | Unicode tables | none | +| `+terminal-resize` | Reflow | none yet | + +**CI only builds these benchmarks.** Nothing runs them, Ghostty publishes no +result history, and nothing gates regressions. Ghostty does **not** measure: + +- input latency +- frame presentation or dropped frames +- PTY-to-screen throughput outside of hand runs of `vtebench` or `time cat` +- memory per line of scrollback +- startup time + +On Windows its IO path is a serial `ReadFile` loop, and no Ghostty GUI exists +there. + +### Why Ghostty is fast (what the builtin engine has to match) + +- **SIMD ground state**: Ghostty scans for ESC and decodes UTF-8 to UTF-32 in + bulk (simdutf/Highway). Each printable run goes to the screen as one slice + (`stream.zig`), not one byte at a time. +- **Bulk cell writes**: `printSlice` writes same-width runs against a template + cell. Widths come from a 3-level lookup table, with a shortcut for + codepoints up to 0xFF. +- **8-byte cells in pooled, offset-addressed pages**: when scrollback is full, + the oldest page is reused. Steady-state scrolling allocates nothing. +- **Separate IO and render threads**: the renderer rebuilds only dirty rows, + once per frame, never once per write. + +## DT's benchmark layers + +All modes live in `tools/Devolutions.Terminal.Bench`. That tool runs on the +JIT with tiered compilation disabled. It is **not** the NativeAOT shipping +build, so treat its numbers as relative. + +| Layer | Mode | Ghostty counterpart | Comparable on Windows? | +| --- | --- | --- | --- | +| 1. VT ingest | `vt` | `ghostty-bench +terminal-stream` | **Yes, in-process on identical bytes** | +| 2. Frame preparation | `render` (snapshot, plan) | `+screen-clone --mode=render*` | Diagnostic only; the two do different work | +| 3. CPU paint | `render` (paint-*) | none (GPU renderer) | No | +| 4. PTY to UI coalescing | `control` | termio/renderer wakeups | No | +| 5. End-to-end | manual: `vtebench`, `time cat` | same | Linux/macOS only, same machine | + +### Layer 1: `vt`, same-process ingest A/B + +```powershell +dotnet run --project tools\Devolutions.Terminal.Bench -c Release -- vt --mb 8 --runs 7 --json artifacts\bench\vt.json +``` + +Three targets consume the same bytes in the same chunks (16 KiB by default, +the ConPTY read size; `--chunk-kb 64` matches `+terminal-stream`): + +- `builtin`: DT's `TerminalEngine`, the engine under test. +- `ghostty-vt`: raw `ghostty_terminal_vt_write` from the pinned + `libghostty-vt`, with only the scrollback limit set to the builtin's 9,001 + lines. This is the bar to beat. It is the same work as `+terminal-stream`, + without process startup. +- `ghostty`: DT's `GhosttyTerminalEngine`, which adds the projection into DT's + cell model. This is what the opt-in Ghostty mode actually costs. Projection + is lazy, so measure it with `--frame-kb`. + +`--frame-kb N` reads one frame every N KiB fed, inside the timed region: + +- builtin and `ghostty` take a `CreateSnapshot`; +- `ghostty-vt` does what Ghostty's renderer does per frame, a render-state + update followed by a clean. + +The default of `0` measures pure ingest. `256` approximates a busy screen at a +frame rate the renderer can sustain. + +Methodology: + +- **Identical, deterministic bytes.** Each workload is seeded, and its + SHA-256 is printed. `--export ` writes the `.vt` files so + `ghostty-bench` can consume exactly the same input. `--data ` replays + a capture, for example an asciinema or vim session. +- **Equivalence gate.** Before timing, both engines consume the workload, and + the visible text, cursor and history are compared. A throughput number only + counts when the screens are identical. The `utf8` workload enables DEC mode + 2027 because, without it, Ghostty measures ZWJ emoji per codepoint while + builtin clusters them. Ghostty trims history by page, so its history count + can be a few hundred lines below 9,001; that difference is expected. +- **Fair timing.** + - Engines are built outside the timed region. + - Runs interleave the targets in rotating order and include a discarded + warmup. + - A sample replays the corpus into the same engine until at least `--min-ms` + (500 ms by default) has elapsed, so fast targets are not dominated by + timer noise. +- **Conservative verdict.** + - `faster` only when builtin's *slowest* run beats Ghostty's *fastest*. + - `slower` when the reverse holds. + - `overlap` otherwise. + - Without `--no-gate`, the exit code is non-zero unless every workload is + `faster`. +- **Cost metrics**: managed bytes allocated per MiB fed, GCs per MiB, and + retained private memory after the run. Retained memory is a process-wide + delta, so read it as an order of magnitude. Native allocations by Ghostty + appear only in that column. + +Workloads (`--workloads a,b,...`): + +| Workload | Shape | Source of the idea | +| --- | --- | --- | +| `ascii` | unbroken 0x21–0x7E stream relying on autowrap | `ghostty-gen ascii` | +| `ascii-lines` | printable lines with CRLF | `cat` of a log | +| `utf8` | Latin, Greek, Cyrillic, CJK, emoji, ZWJ, combining marks | `ghostty-gen utf8`, restricted to assigned characters | +| `cjk` | lines of CJK ideographs only (wide, 3-byte UTF-8) | `time cat` of CJK text | +| `emoji` | lines of emoji only (wide, 4-byte UTF-8) | emoji-heavy logs and prompts | +| `styled` | short SGR runs, OSC 8, wide text | `ghostty-gen styled` | +| `dense-cells` | every cell gets its own truecolor/256 SGR | vtebench `dense_cells` | +| `light-cells` | full-screen plain text from home | vtebench `light_cells` | +| `cursor-motion` | random CUP, short writes, EL | vtebench `cursor_motion`, TUIs | +| `scrolling` | `y\n` | vtebench `scrolling` | +| `scroll-region` | DECSTBM region scrolling | vtebench `scrolling_*_region` | +| `osc` | titles, OSC 133 marks, OSC 8 links | `ghostty-gen osc` (valid sequences only) | +| `mixed` | the legacy `engine`/`render` corpus | DT history | + +### Running the whole suite + +```powershell +pwsh scripts/Invoke-TerminalBenchmarks.ps1 # vt + render, results in artifacts/bench/ +pwsh scripts/Invoke-TerminalBenchmarks.ps1 -Quick # fast local signal +pwsh scripts/Invoke-TerminalBenchmarks.ps1 -Baseline artifacts/bench//vt.json # fails if builtin/ghostty-vt drops >15% on identical inputs +pwsh scripts/Invoke-TerminalBenchmarks.ps1 -RequireBeatGhostty # fails unless builtin wins every vt workload +pwsh scripts/Invoke-TerminalBenchmarks.ps1 -GhosttyBench \ghostty-bench.exe # also replays the corpora through upstream +terminal-stream +``` + +`-GhosttyBench` times whole processes, as Ghostty does, so its figures include +process startup. It cross-checks the in-process `ghostty-vt` target; it does +not replace it. Build `ghostty-bench` as described in +[renderer benchmarks](renderer-benchmarks.md#ghostty-reference-measurements). + +### End-to-end (layer 5) on Linux/macOS + +Ghostty's only cross-terminal numbers come from `time cat` and vtebench. To +reproduce them, run on one native Linux or macOS machine, with both apps in +Release and matching font, size, grid and scrollback: + +- `time cat` of a large ASCII file and of a large CJK file, 10 runs each. +- vtebench's 12 workloads, run inside each terminal. vtebench measures how fast + the terminal drains the PTY (backpressure), not paint time. + +Never run both terminals at the same time; alternate their order and report +the spread. The checklist in +[renderer benchmarks](renderer-benchmarks.md#full-renderer-comparison-linuxmacos) +lists the full set of gates. + +## Current results + +Setup for these numbers: + +- One Windows 11 x64 workstation (24 logical CPUs, AVX2), .NET 10.0.13 (JIT, + tiered compilation off), and the pinned `libghostty-vt` (ReleaseFast). +- 120x30 grid, 16 KiB chunks, 8 MiB per workload, 7 timed runs plus 1 warmup. +- Each workload ran in its own process, via: + + ```powershell + pwsh scripts/Invoke-TerminalBenchmarks.ps1 -Suite vt,render + ``` + +This machine's absolute speed drifted by up to 2.5x between invocations +(Ghostty itself moved from ~540 to ~1,600 MB/s on `light-cells`). So read the +**builtin / ghostty-vt ratio**, which is measured within each run, and treat +anything within about ±10% of 1.0 as a tie. Throughput is the median MB/s. + +### Ingest: builtin engine vs raw `libghostty-vt` + +| Workload | builtin | ghostty-vt | ghostty (DT) | ratio | verdict | builtin alloc / MiB | screens | +| --- | ---: | ---: | ---: | ---: | --- | ---: | --- | +| ascii | 1,144 | 922 | 411 | **1.24x** | overlap | 31 KB | identical | +| ascii-lines | 879 | 741 | 361 | **1.19x** | **faster** | 40 KB | identical | +| light-cells | 2,024 | 1,647 | 60 | **1.23x** | **faster** | 0 | identical | +| scrolling | 42.5 | 31.9 | 29.5 | **1.33x** | overlap | 0.7 MB | identical | +| dense-cells | 207 | 144 | 53 | **1.44x** | **faster** | 0 | identical | +| styled | 78.4 | 26.9 | 24.8 | **2.91x** | **faster** | 5.4 MB | 1 row differs | +| osc | 124 | 27.0 | 25.1 | **4.59x** | **faster** | 6.9 MB | identical | +| mixed | 667 | 362 | 227 | **1.84x** | **faster** | 54 KB | identical | +| scroll-region | 641 | 669 | 56 | 0.96x | overlap | 0 | identical | +| cursor-motion | 152 | 168 | 94 | 0.90x | slower | 33 KB | identical | +| emoji | 588 | 736 | 267 | 0.80x | overlap | 61 KB | identical | +| utf8 | 61.3 | 79.6 | 70.1 | 0.77x | slower | 8.7 MB | identical | +| cjk | 458 | 596 | 299 | 0.77x | slower | 78 KB | identical | + +`ghostty (DT)` skips projection without `--frame-kb`. The one-row difference in +`styled` is a behavior difference, not a measurement artifact: builtin drops a +combining mark that follows a space, and Ghostty keeps it. + +How builtin got here, as the median builtin/ghostty-vt ratio per round (each +round measured against its own in-run Ghostty control): + +| Workload | `dc1cd26` | round 1 | round 2 (packed cells) | round 3 | round 4 | +| --- | ---: | ---: | ---: | ---: | ---: | +| ascii | 0.009 | 0.20 | 0.55 | 1.21 | 1.24 | +| ascii-lines | 0.009 | 0.20 | 0.50 | 1.13 | 1.19 | +| light-cells | 0.009 | 0.32 | 0.48 | 1.20 | 1.23 | +| scrolling | 0.015 | 0.14 | 0.40 | 1.11 | 1.33 | +| dense-cells | 0.42 | 0.63 | 0.62 | 1.57 | 1.44 | +| styled | 0.30 | 1.33 | 1.14 | 2.82 | 2.91 | +| osc | 0.29 | 2.60 | 4.52 | 4.57 | 4.59 | +| mixed | 0.03 | 0.39 | 0.45 | 1.56 | 1.84 | +| scroll-region | 0.018 | 0.22 | 0.34 | 0.90 | 0.96 | +| cursor-motion | 0.11 | 0.23 | 0.17 | 0.94 | 0.90 | +| utf8 | 0.10 | 0.36 | 0.27 | 0.57 | 0.77 | +| cjk | — | — | — | 0.53 | 0.77 | +| emoji | — | — | — | 0.50 | 0.80 | + +The `-Baseline` gate flagged real regressions twice: + +- **Round 2:** utf8 and cursor-motion. +- **Round 4:** dense-cells and cursor-motion, from caching packed templates on + the short-ASCII path. + +Each was confirmed with an interleaved A/B build and fixed before moving on. +Moves of about ±10% between rounds (dense-cells and cursor-motion from round 3 +to round 4, for example) are within this machine's noise. + +### DT's Ghostty mode vs raw `libghostty-vt`, with frames + +These numbers use `--frame-kb 256`, so a projection is taken every 256 KiB. + +| Workload | ghostty (DT) before | after | ghostty-vt | ratio before → after | +| --- | ---: | ---: | ---: | ---: | +| ascii | 49.8 | 214.4 | 960 | 0.05 → 0.22 | +| ascii-lines | 46.5 | 220.0 | 660 | 0.07 → 0.33 | +| cursor-motion | 36.9 | 79.9 | 146 | 0.25 → 0.55 | +| mixed | 41.1 | 146.5 | 295 | 0.12 → 0.50 | +| light-cells | 45.6 | 57.5 | 1,580 | 0.03 → 0.04 | + +`light-cells` rewrites every row between frames, so every row is dirty and the +per-cell P/Invoke projection still dominates. + +### Frame preparation and CPU paint + +`render`, 120x30, median p50 / p95: + +| Stage | before packed cells | now | +| --- | --- | --- | +| snapshot | 0.009 / 0.011 ms | 0.040 / 0.044 ms | +| snapshot + plan | 0.074 / 0.082 ms | 0.113 / 0.129 ms | +| static paint | 0.69 / 1.08 ms | 0.75 / 1.00 ms | +| one-row update | 0.69 / 1.70 ms | 0.76 / 1.06 ms | +| scrolling paint | 3.29 / 9.72 ms | 2.90 / 5.54 ms | + +Snapshots now unpack each packed cell into a public `Cell`, which costs about +0.03 ms per 120x30 frame. Every stage still fits well inside a 16.7 ms frame. +No Ghostty counterpart exists on Windows. + +### What changed in the builtin engine + +Every fast path is checked against the per-character path by differential +tests (`AsciiFastPathTests`, `CsiFastPathTests`, `CellRowModelTests`, +`PackedCellStorageTests`). Each new branch was also checked by deliberately +breaking it and confirming a test fails. + +- **Packed, reference-free storage** (`CellRow`, `PackedCell`). + - A cell is 16 bytes with no GC references, down from about 40 with two + `string?` fields. + - Combining text and hyperlinks live in per-line side arrays that exist only + while needed and are freed when the line is recycled. + - The public `Cell` and `TermColor` are unchanged, and every value round-trips + losslessly. +- **Logically blank cells are never written.** + - A line stores only its written range; everything outside it reads as the + blank it was cleared with. + - Recycling a scrollback line, the coldest memory in the buffer, is O(1). + - Erasing to the end of a line is O(1) for left-aligned text. + - Writes fill only the gap between the stored range and the new cells. +- **Bulk printing.** + - **ASCII:** the parser hands whole printable-ASCII runs to + `IVtDispatch.PrintAscii`. They are stored a row segment at a time, two + 64-bit (or, with AVX2, two 256-bit per four cells) stores per cell batch. + - **UTF-8 runs:** well-formed UTF-8 runs (2–4 byte sequences decoded inline) + go to `IVtDispatch.PrintRunes`. + - **Grapheme joins:** precomputed tables for planes 0 and 1 rule out joins, + so runes that cannot join are stored a segment at a time. That includes + emoji after a cell without a trailing ZWJ. + - **Rows without wide glyphs:** a conservative per-row "may hold wide cells" + flag means classifying and storing happen in one pass, with no clearing + around wide glyphs. Every freshly recycled scrollback line qualifies. +- **CSI fast path.** A complete plain CSI sequence is parsed in one pass, then + goes through the same `DispatchCsi` as the byte path. Parameters are copied + into reused arrays. +- **C0 fast path.** CR, LF and other controls in ground state go straight to + `ExecuteC0` instead of through the string-state checks. +- **Scroll regions** rotate line references with `Array.Copy`, not one + indexer call per line. + +### What still separates builtin from Ghostty + +- **Non-ASCII text** (`cjk`, `emoji`, `utf8`: about 0.8x). On CJK, per scalar: + + | Stage | Cost | + | --- | --- | + | Builtin: decode | ~2.0 ns | + | Builtin: storage (two 16-byte cells per wide character) | ~3.6 ns | + | Builtin: dispatch | ~0.6 ns | + | Ghostty: everything | ~5 ns | + + What was tried and measured: + - **Runtime SIMD transcoder.** Swapping the scalar decoder for the runtime's + SIMD `Utf8.ToUtf16` (plus a UTF-16-to-rune pass) was *slower* at these run + lengths, and was reverted. + - **Unchecked scalar decoding.** A `ref`-based decoder with unchecked `Rune` + construction measured the same as the indexed one. + + Closing the rest needs either a purpose-built SIMD UTF-8 decoder or 8-byte + cells. +- **cursor-motion and scroll-region** are ties within noise (0.90–0.96x). Per + escape sequence, builtin still pays for an interface call into the engine + and that call's own checks before each short write. +- **Cell size.** Packed cells are 16 bytes against Ghostty's 8, so a cleared + or copied row moves twice the memory. Getting to 8 bytes means interning + styles, which is a larger change. + +**Variance.** Within isolated processes the ratio repeats to about ±10% on +this machine. In one long process running every workload, ratios drifted by +up to 50%, because heap state carried over between workloads. For that reason +the script runs one process per workload, and `-Baseline` only compares +workloads with the same SHA-256 and gates on the ratio, not on absolute MB/s. + +## Definition of "matches or beats Ghostty" + +A claim needs every row below. A missing or failed row means the claim has +not been established. + +1. `vt`: `faster` (or, for "matches", `overlap` with a median ratio ≥ 0.95) + on every workload, with identical screens, on at least one x64 and one + arm64 host. +2. The same corpora through upstream `ghostty-bench +terminal-stream` do not + contradict layer 1. This rules out a P/Invoke artifact. +3. Scrolling allocation is bounded. Steady-state `scrolling` should allocate + close to 0 B/MiB once history is full, as Ghostty's page reuse does. +4. On Linux/macOS, `time cat` and vtebench place DT at or above Ghostty, run + under the conditions described in the end-to-end section. +5. Frame latency and dropped frames: measure these even though Ghostty does + not publish them. Input latency (key to PTY echo to presented frame) is + unmeasured on both sides, and a DT measurement would be new evidence rather + than a comparison. + +Run `scripts/Invoke-TerminalBenchmarks.ps1 -Baseline` on every engine change +so wins are kept, and record the commit, machine and `vt.json` with each +published number. diff --git a/docs/renderer-benchmarks.md b/docs/renderer-benchmarks.md index 8899138..77dda02 100644 --- a/docs/renderer-benchmarks.md +++ b/docs/renderer-benchmarks.md @@ -4,6 +4,9 @@ The `render` mode of `tools/Devolutions.Terminal.Bench` measures the CPU work of preparing and rasterizing terminal frames. It is not a GPU or presentation benchmark. The existing `engine` and `control` modes measure different parts of the pipeline; `control` uses Avalonia headless and does **not** paint. +For the engine-level comparison against Ghostty (`vt` mode), the full suite +script and the "beats Ghostty" criteria, see +[performance benchmarks](performance-benchmarks.md). ## Reproducible input diff --git a/scripts/Invoke-TerminalBenchmarks.ps1 b/scripts/Invoke-TerminalBenchmarks.ps1 new file mode 100644 index 0000000..c771b3a --- /dev/null +++ b/scripts/Invoke-TerminalBenchmarks.ps1 @@ -0,0 +1,219 @@ +[CmdletBinding()] +param( + # vt: builtin vs Ghostty ingest; render: frame prep + CPU raster; control: PTY->UI coalescing. + [string[]] $Suite = @("vt", "render"), + + # Smaller corpora and fewer runs for a fast local signal (not for published numbers). + [switch] $Quick, + + [string] $OutputDirectory, + + # Comma-separated vt workloads; "all" runs every workload. + [string] $Workloads = "all", + + # Read a frame (snapshot / render-state update) every N KiB fed; 0 measures pure ingest. + [ValidateRange(0, 1048576)] + [int] $FrameKb = 0, + + # A previous run's vt.json; medians that drop by more than -RegressionPercent fail the run. + [string] $Baseline, + + [ValidateRange(1, 100)] + [int] $RegressionPercent = 15, + + # Upstream ghostty-bench built from the pinned commit (see docs/performance-benchmarks.md). + # When given, every vt workload is also replayed through `+terminal-stream`. + [string] $GhosttyBench, + + # Fail when builtin is not faster than raw libghostty-vt on every workload. + [switch] $RequireBeatGhostty +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = "Stop" + +# `pwsh -File` passes "vt,render" as one string; accept both forms. +$Suite = @($Suite | ForEach-Object { $_ -split "," } | ForEach-Object { $_.Trim() } | Where-Object { $_ }) +foreach ($name in $Suite) { + if ($name -notin @("vt", "render", "control")) { + throw "Unknown suite '$name' (expected vt, render or control)." + } +} + +$repo = Split-Path -Parent $PSScriptRoot +$project = Join-Path $repo "tools\Devolutions.Terminal.Bench" +if (-not $OutputDirectory) { + $OutputDirectory = Join-Path $repo ("artifacts\bench\" + (Get-Date -Format "yyyyMMdd-HHmmss")) +} + +New-Item -ItemType Directory -Force -Path $OutputDirectory | Out-Null +$OutputDirectory = (Resolve-Path -LiteralPath $OutputDirectory).Path + +$megabytes = if ($Quick) { 2 } else { 8 } +$runs = if ($Quick) { 3 } else { 7 } + +Write-Host "Building $project (Release)..." +dotnet build $project -c Release --nologo -v quiet +if ($LASTEXITCODE -ne 0) { + throw "Benchmark build failed." +} + +$bench = Get-ChildItem -Path (Join-Path $project "bin\Release") -Recurse -File | + Where-Object { $_.BaseName -eq "Devolutions.Terminal.Bench" -and $_.Extension -in @(".exe", "") } | + Select-Object -First 1 +if (-not $bench) { + $bench = Get-ChildItem -Path (Join-Path $project "bin\Release") -Recurse -Filter "Devolutions.Terminal.Bench.dll" | + Select-Object -First 1 +} + +function Invoke-Bench { + param([string] $Name, [string[]] $Arguments) + + $log = Join-Path $OutputDirectory "$Name.txt" + Write-Host "" + Write-Host "== $Name ==" + # Out-Host keeps the transcript out of the function's return value. + if ($bench.Extension -eq ".dll") { + & dotnet $bench.FullName @Arguments 2>&1 | Tee-Object -FilePath $log | Out-Host + } + else { + & $bench.FullName @Arguments 2>&1 | Tee-Object -FilePath $log | Out-Host + } + + return $LASTEXITCODE +} + +$failures = [System.Collections.Generic.List[string]]::new() + +if ($Suite -contains "vt") { + $vtJson = Join-Path $OutputDirectory "vt.json" + $names = if ($Workloads -eq "all") { + if ($bench.Extension -eq ".dll") { & dotnet $bench.FullName vt --list } else { & $bench.FullName vt --list } + } + else { + $Workloads -split "," | ForEach-Object { $_.Trim() } | Where-Object { $_ } + } + + # One process per workload: GC heap and JIT state left behind by an earlier + # workload shifted later ratios by up to ~50% in a shared process, which made + # runs with different -Workloads subsets incomparable. + $merged = $null + foreach ($name in $names) { + $partial = Join-Path $OutputDirectory "vt-$name.json" + $vtArgs = @("vt", "--mb", $megabytes, "--runs", $runs, "--frame-kb", $FrameKb, "--workloads", $name, "--json", $partial) + if (-not $RequireBeatGhostty) { + $vtArgs += "--no-gate" + } + + if ((Invoke-Bench -Name "vt-$name" -Arguments $vtArgs) -ne 0) { + $failures.Add("builtin did not beat ghostty-vt on vt workload '$name'") + } + + $report = Get-Content -LiteralPath $partial -Raw | ConvertFrom-Json + if ($merged) { + $merged.workloads = @($merged.workloads) + @($report.workloads) + } + else { + $merged = $report + } + } + + $merged | ConvertTo-Json -Depth 8 | Set-Content -LiteralPath $vtJson + + if ($GhosttyBench) { + $corpora = Join-Path $OutputDirectory "corpora" + Invoke-Bench -Name "vt-export" -Arguments @("vt", "--mb", $megabytes, "--workloads", $Workloads, "--export", $corpora) | Out-Null + $upstream = foreach ($file in Get-ChildItem -LiteralPath $corpora -Filter "*.vt") { + $times = foreach ($run in 1..$runs) { + $watch = [System.Diagnostics.Stopwatch]::StartNew() + & $GhosttyBench "+terminal-stream" "--data=$($file.FullName)" "--terminal-rows=30" "--terminal-cols=120" | Out-Null + $watch.Stop() + if ($LASTEXITCODE -ne 0) { + throw "ghostty-bench failed on $($file.Name)." + } + + $watch.Elapsed.TotalSeconds + } + + $sorted = @($times | Sort-Object) + $median = $sorted[[int][Math]::Floor($sorted.Count / 2)] + [pscustomobject]@{ + Workload = $file.BaseName + MedianMBps = [Math]::Round($file.Length / 1MB / $median, 1) + Note = "process wall time, includes startup and file load" + } + } + + $upstream | Format-Table -AutoSize | Out-String | Tee-Object -FilePath (Join-Path $OutputDirectory "ghostty-bench.txt") + $upstream | ConvertTo-Json | Set-Content -LiteralPath (Join-Path $OutputDirectory "ghostty-bench.json") + } + + if ($Baseline) { + $previous = Get-Content -LiteralPath $Baseline -Raw | ConvertFrom-Json + $current = Get-Content -LiteralPath $vtJson -Raw | ConvertFrom-Json + Write-Host "" + Write-Host "== regression check vs $Baseline (threshold $RegressionPercent%) ==" + foreach ($workload in $current.workloads) { + $before = $previous.workloads | Where-Object { $_.name -eq $workload.name } | Select-Object -First 1 + if (-not $before) { + continue + } + + if ($before.sha256 -ne $workload.sha256) { + Write-Host ("{0,-14} skipped: input differs from the baseline (use the same -Quick/-Workloads settings)" -f $workload.name) + continue + } + + foreach ($target in $workload.targets) { + $old = $before.targets | Where-Object { $_.target -eq $target.target } | Select-Object -First 1 + if ($old) { + $change = ($target.medianMBps - $old.medianMBps) / $old.medianMBps * 100 + Write-Host ("{0,-14} {1,-10} {2,9:F1} -> {3,9:F1} MB/s ({4:+0.0;-0.0}%)" -f $workload.name, $target.target, $old.medianMBps, $target.medianMBps, $change) + } + } + + # Absolute MB/s drifts with clocks and background load between sessions, so + # gate on builtin relative to ghostty-vt measured in the same run: Ghostty is + # the control, and only the engine under our control can regress. + $oldRatio = $before.PSObject.Properties["builtinVsGhosttyVt"] + $newRatio = $workload.PSObject.Properties["builtinVsGhosttyVt"] + if ($oldRatio -and $newRatio -and $oldRatio.Value -gt 0) { + $change = ($newRatio.Value - $oldRatio.Value) / $oldRatio.Value * 100 + $line = "{0,-14} builtin/ghostty-vt {1:F3} -> {2:F3} ({3:+0.0;-0.0}%)" -f $workload.name, $oldRatio.Value, $newRatio.Value, $change + if ($change -lt -$RegressionPercent) { + $failures.Add("regression: $line") + Write-Host "$line REGRESSION" + } + else { + Write-Host $line + } + } + } + } +} + +if ($Suite -contains "render") { + $renderArgs = if ($Quick) { + @("render", "--mb", $megabytes, "--runs", $runs, "--samples", "60", "--warmup", "2", "--frames", "64") + } + else { + @("render", "--mb", $megabytes, "--runs", "5", "--samples", "128", "--warmup", "2", "--frames", "256") + } + + if ((Invoke-Bench -Name "render" -Arguments $renderArgs) -ne 0) { + $failures.Add("render benchmark failed") + } +} + +if ($Suite -contains "control") { + if ((Invoke-Bench -Name "control" -Arguments @("control", "--mb", $megabytes, "--runs", $runs)) -ne 0) { + $failures.Add("control benchmark failed") + } +} + +Write-Host "" +Write-Host "Results: $OutputDirectory" +if ($failures.Count -gt 0) { + $failures | ForEach-Object { Write-Warning $_ } + exit 1 +} diff --git a/src/Devolutions.Terminal.Core/CellRow.cs b/src/Devolutions.Terminal.Core/CellRow.cs new file mode 100644 index 0000000..399ffa7 --- /dev/null +++ b/src/Devolutions.Terminal.Core/CellRow.cs @@ -0,0 +1,631 @@ +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; +using System.Text; + +namespace Devolutions.Terminal.Core; + +/// +/// The 16-byte, reference-free storage form of a . Text, +/// width, flags and both colors are packed losslessly; combining text and +/// hyperlink URIs live in the owning 's side arrays, so +/// scrollback holds no GC references per cell and filling or copying cells is a +/// plain memory operation without write barriers. +/// +internal readonly struct PackedCell +{ + // _meta: rune (21 bits) | stored width (3) | wide continuation (1) | + // protected (1) | shell integration (2) | foreground kind (2) | background kind (2). + private const int WidthShift = 21; + private const int WideShift = 24; + private const int ProtectedShift = 25; + private const int ShellShift = 26; + private const int ForegroundKindShift = 28; + private const int BackgroundKindShift = 30; + private const uint RuneMask = 0x1F_FFFF; + + private readonly uint _meta; + private readonly uint _foreground; + private readonly uint _background; + private readonly ushort _flags; + + private PackedCell(uint meta, uint foreground, uint background, ushort flags) + { + _meta = meta; + _foreground = foreground; + _background = background; + _flags = flags; + } + + public bool IsWideContinuation => ((_meta >> WideShift) & 1) != 0; + + public Rune Rune => Unsafe.BitCast(_meta & RuneMask); + + public byte StoredWidth => (byte)((_meta >> WidthShift) & 7); + + /// Matches . + public int DisplayWidth => + IsWideContinuation + ? 0 + : StoredWidth == 0 + ? Math.Max(1, WcWidth.Width(Rune)) + : StoredWidth; + + /// + /// Matches except for combining text, which lives + /// in the row's side array. + /// + public bool IsBlankIgnoringCombining => + // Rune is a space; not a continuation, not protected; both colors default. + (_meta & (RuneMask | (1u << WideShift) | (1u << ProtectedShift) | (3u << ForegroundKindShift) | (3u << BackgroundKindShift))) == ' ' && + _flags == 0; + + public static PackedCell Blank(CellAttributes attributes) => + From(new Rune(' '), attributes, isProtected: false, isWideContinuation: false, storedWidth: 1, ShellIntegrationKind.None); + + public static PackedCell From(in Cell cell) => + From(cell.Rune, cell.Attributes, cell.IsProtected, cell.IsWideContinuation, cell.StoredWidth, cell.ShellIntegration); + + public static PackedCell From( + Rune rune, + CellAttributes attributes, + bool isProtected, + bool isWideContinuation, + byte storedWidth, + ShellIntegrationKind shellIntegration) + { + var meta = (uint)rune.Value | + (Math.Min((uint)storedWidth, 7u) << WidthShift) | + (isWideContinuation ? 1u << WideShift : 0) | + (isProtected ? 1u << ProtectedShift : 0) | + (((uint)shellIntegration & 3) << ShellShift) | + (((uint)attributes.Foreground.Kind & 3) << ForegroundKindShift) | + (((uint)attributes.Background.Kind & 3) << BackgroundKindShift); + return new PackedCell( + meta, + PackColor(attributes.Foreground), + PackColor(attributes.Background), + (ushort)attributes.Flags); + } + + public PackedCell WithRune(Rune rune) => + new((_meta & ~RuneMask) | (uint)rune.Value, _foreground, _background, _flags); + + public PackedCell WithWidth(byte storedWidth, bool isWideContinuation) => + new( + (_meta & ~((7u << WidthShift) | (1u << WideShift))) | + ((uint)storedWidth << WidthShift) | + (isWideContinuation ? 1u << WideShift : 0), + _foreground, + _background, + _flags); + + public bool SameBits(PackedCell other) => + _meta == other._meta && + _foreground == other._foreground && + _background == other._background && + _flags == other._flags; + + /// + /// Stores once per byte with that byte as its rune, + /// as two 64-bit writes per cell: on little-endian hardware the first word holds + /// the rune bits and the foreground, the second the background and flags. + /// + public static void FillAscii(Span destination, ReadOnlySpan text, PackedCell template) + { + if (!BitConverter.IsLittleEndian) + { + for (var cell = 0; cell < destination.Length; cell++) + { + destination[cell] = template.WithRune(new Rune(text[cell])); + } + + return; + } + + ref var words = ref Unsafe.As(ref System.Runtime.InteropServices.MemoryMarshal.GetReference(destination)); + ref var templateWords = ref Unsafe.As(ref template); + var low = templateWords & ~(ulong)RuneMask; + var high = Unsafe.Add(ref templateWords, 1); + var index = 0; + if (Avx2.IsSupported) + { + // Four cells per iteration: widen four bytes to four 64-bit lanes, add + // the template's rune-free low word, then interleave with the constant + // high word into two 32-byte stores. + ref var bytes = ref System.Runtime.InteropServices.MemoryMarshal.GetReference(text); + var lows = Vector256.Create(low); + var highs = Vector256.Create(high); + for (; index + 4 <= destination.Length; index += 4) + { + var four = Vector128.CreateScalarUnsafe(Unsafe.ReadUnaligned(ref Unsafe.Add(ref bytes, index))).AsByte(); + var cellLows = Avx2.Or(Avx2.ConvertToVector256Int64(four).AsUInt64(), lows); + var even = Avx2.UnpackLow(cellLows, highs); + var odd = Avx2.UnpackHigh(cellLows, highs); + Avx2.Permute2x128(even, odd, 0x20).StoreUnsafe(ref words, (nuint)(2 * index)); + Avx2.Permute2x128(even, odd, 0x31).StoreUnsafe(ref words, (nuint)((2 * index) + 4)); + } + } + + for (; index < destination.Length; index++) + { + Unsafe.Add(ref words, 2 * index) = low | text[index]; + Unsafe.Add(ref words, (2 * index) + 1) = high; + } + } + + public Cell ToCell(string? combining, string? hyperlink) => new() + { + Rune = Unsafe.BitCast(_meta & RuneMask), + Attributes = new CellAttributes + { + Foreground = UnpackColor((ColorKind)((_meta >> ForegroundKindShift) & 3), _foreground), + Background = UnpackColor((ColorKind)((_meta >> BackgroundKindShift) & 3), _background), + Flags = (CellFlags)_flags, + }, + IsProtected = ((_meta >> ProtectedShift) & 1) != 0, + IsWideContinuation = IsWideContinuation, + StoredWidth = (byte)((_meta >> WidthShift) & 7), + CombiningCharacters = combining, + HyperlinkUri = hyperlink, + ShellIntegration = (ShellIntegrationKind)((_meta >> ShellShift) & 3), + }; + + private static uint PackColor(TermColor color) => + color.Index | ((uint)color.R << 8) | ((uint)color.G << 16) | ((uint)color.B << 24); + + private static TermColor UnpackColor(ColorKind kind, uint value) => + new(kind, (byte)value, (byte)(value >> 8), (byte)(value >> 16), (byte)(value >> 24)); +} + +/// +/// One line of packed cells. The indexer converts to and from the public +/// so buffer code keeps its value semantics; hot paths use the +/// bulk members. Side arrays exist only while some cell of the line carries +/// combining text or a hyperlink, and are bounded by the line width. +/// +/// +/// Only cells in [_lo, _hi) are stored; every other cell is logically +/// _blank with no side entries, whatever the array still holds. Blanking a +/// recycled scrollback line, the coldest memory in the buffer, is therefore O(1), +/// and writes touch only the cells they store plus any gap between them and the +/// stored range, which is filled to keep that range contiguous. +/// +internal sealed class CellRow +{ + private readonly PackedCell[] _cells; + private string?[]? _combining; + private string?[]? _hyperlinks; + private PackedCell _blank; + private int _lo; + private int _hi; + // Conservative: false guarantees no cell is a wide glyph or its continuation, + // so writes cannot split one and need no clearing around them. + private bool _mayHoldWide; + + public CellRow(int columns, CellAttributes attributes) + { + _cells = GC.AllocateUninitializedArray(columns); + _blank = PackedCell.Blank(attributes); + } + + private CellRow(PackedCell[] cells) + { + _cells = cells; + } + + public int Length => _cells.Length; + + public Cell this[int x] + { + get => CellAt(x).ToCell(_combining?[x], _hyperlinks?[x]); + set + { + PrepareWrite(x, 1); + var packed = PackedCell.From(value); + _cells[x] = packed; + NoteStored(packed); + SetSide(ref _combining, x, value.CombiningCharacters); + SetSide(ref _hyperlinks, x, value.HyperlinkUri); + } + } + + /// Equivalent to the indexer storing a cell with these fields, without building one. + public void Store( + int x, + Rune rune, + CellAttributes attributes, + bool isProtected, + bool isWideContinuation, + byte storedWidth, + string? combining, + string? hyperlink, + ShellIntegrationKind shellIntegration) + { + PrepareWrite(x, 1); + var packed = PackedCell.From(rune, attributes, isProtected, isWideContinuation, storedWidth, shellIntegration); + _cells[x] = packed; + NoteStored(packed); + SetSide(ref _combining, x, combining); + SetSide(ref _hyperlinks, x, hyperlink); + } + + /// + /// Stores runes of the given widths (1 or 2) from to + /// : each narrow rune as , each + /// wide one as followed by , + /// all without combining text and with . + /// + public void WriteRunes( + int start, + int end, + ReadOnlySpan runes, + ReadOnlySpan widths, + PackedCell narrow, + PackedCell wide, + PackedCell continuation, + string? hyperlink) + { + PrepareWrite(start, end - start); + var cells = _cells.AsSpan(start, end - start); + var x = 0; + for (var index = 0; index < runes.Length; index++) + { + var rune = runes[index]; + if (widths[index] == 2) + { + cells[x] = wide.WithRune(rune); + cells[x + 1] = continuation; + x += 2; + _mayHoldWide = true; + } + else + { + cells[x++] = narrow.WithRune(rune); + } + } + + ClearSide(_combining, start, end - start); + if (hyperlink is null) + { + ClearSide(_hyperlinks, start, end - start); + } + else + { + _hyperlinks ??= new string?[_cells.Length]; + _hyperlinks.AsSpan(start, end - start).Fill(hyperlink); + } + } + + /// Stores a packed cell with no combining text. + public void StorePacked(int x, PackedCell cell, string? hyperlink) + { + PrepareWrite(x, 1); + _cells[x] = cell; + NoteStored(cell); + SetSide(ref _combining, x, null); + SetSide(ref _hyperlinks, x, hyperlink); + } + + // Field reads that avoid materializing a whole Cell on hot paths. + public bool IsWideContinuation(int x) => CellAt(x).IsWideContinuation; + + public int DisplayWidth(int x) => CellAt(x).DisplayWidth; + + public Rune RuneAt(int x) => CellAt(x).Rune; + + public string? CombiningAt(int x) => _combining?[x]; + + public bool IsBlank(int x) => CellAt(x).IsBlankIgnoringCombining && string.IsNullOrEmpty(_combining?[x]); + + public static CellRow FromCells(ReadOnlySpan cells, int columns) + { + var row = new CellRow(columns, CellAttributes.Default); + var count = Math.Min(cells.Length, columns); + for (var x = 0; x < count; x++) + { + row[x] = cells[x]; + } + + return row; + } + + public Cell[] ToArray() + { + var result = new Cell[_cells.Length]; + for (var x = 0; x < result.Length; x++) + { + result[x] = this[x]; + } + + return result; + } + + /// Blanks the whole line and releases its side arrays, in O(1). + public void Reset(CellAttributes attributes) + { + _blank = PackedCell.Blank(attributes); + _lo = 0; + _hi = 0; + _mayHoldWide = false; + _combining = null; + _hyperlinks = null; + } + + /// Equivalent to storing a blank with in each cell. + public void FillBlank(int start, int count, CellAttributes attributes) + { + if (count <= 0) + { + return; + } + + if (start == 0 && count == _cells.Length) + { + Reset(attributes); + return; + } + + ClearSide(_combining, start, count); + ClearSide(_hyperlinks, start, count); + if (start + count == _cells.Length) + { + // Erasing through the end of the line becomes the new blank tail. + // [0, start) must keep reading as it does now, including cells that + // are blank only logically, so it is stored first; for text written + // from column 0 that costs nothing. + Materialize(0, start); + _hi = start; + _blank = PackedCell.Blank(attributes); + return; + } + + PrepareWrite(start, count); + _cells.AsSpan(start, count).Fill(PackedCell.Blank(attributes)); + } + + /// Equivalent to storing in each cell of the range. + public void Fill(int start, int count, in Cell cell) + { + if (count <= 0) + { + return; + } + + PrepareWrite(start, count); + var packed = PackedCell.From(cell); + _cells.AsSpan(start, count).Fill(packed); + NoteStored(packed); + FillSide(ref _combining, start, count, cell.CombiningCharacters); + FillSide(ref _hyperlinks, start, count, cell.HyperlinkUri); + } + + /// Equivalent to storing with each byte as its rune. + public void WriteAscii(int start, ReadOnlySpan text, in Cell template) => + WriteAscii(start, text, PackedCell.From(template), template.HyperlinkUri); + + /// + /// Stores once per byte with that byte as its rune, + /// no combining text and . + /// + public void WriteAscii(int start, ReadOnlySpan text, PackedCell template, string? hyperlink) + { + if (text.IsEmpty) + { + return; + } + + PrepareWrite(start, text.Length); + PackedCell.FillAscii(_cells.AsSpan(start, text.Length), text, template); + ClearSide(_combining, start, text.Length); + if (hyperlink is null) + { + ClearSide(_hyperlinks, start, text.Length); + } + else + { + _hyperlinks ??= new string?[_cells.Length]; + _hyperlinks.AsSpan(start, text.Length).Fill(hyperlink); + } + } + + public void CopyTo(int sourceIndex, CellRow destination, int destinationIndex, int count) + { + if (count <= 0) + { + return; + } + + Materialize(sourceIndex, sourceIndex + count); + destination.PrepareWrite(destinationIndex, count); + destination._mayHoldWide |= _mayHoldWide; + _cells.AsSpan(sourceIndex, count).CopyTo(destination._cells.AsSpan(destinationIndex, count)); + CopySide(_combining, ref destination._combining, sourceIndex, destinationIndex, count, destination.Length); + CopySide(_hyperlinks, ref destination._hyperlinks, sourceIndex, destinationIndex, count, destination.Length); + } + + public CellRow Clone() => new((PackedCell[])_cells.Clone()) + { + _combining = (string?[]?)_combining?.Clone(), + _hyperlinks = (string?[]?)_hyperlinks?.Clone(), + _blank = _blank, + _lo = _lo, + _hi = _hi, + _mayHoldWide = _mayHoldWide, + }; + + public bool MayHoldWide => _mayHoldWide; + + /// + /// Whether may start at : the + /// stored range is empty or starts at or before it, so no gap can open on the + /// right of the cells appended. + /// + public bool CanAppendAt(int start) => _lo == _hi || start >= _lo; + + /// + /// Returns the cells from to the end of the line for + /// the caller to overwrite from the start, after storing the blank in any gap + /// before them. must follow with the end written. + /// + public Span BeginAppend(int start) + { + if (_lo != _hi && start > _hi) + { + _cells.AsSpan(_hi, start - _hi).Fill(_blank); + _hi = start; + } + + return _cells.AsSpan(start); + } + + /// Completes : the cells written carry no combining text. + public void EndAppend(int start, int end, string? hyperlink, bool wroteWide) + { + if (_lo == _hi) + { + _lo = start; + _hi = end; + } + else if (end > _hi) + { + _hi = end; + } + + ClearSide(_combining, start, end - start); + if (hyperlink is null) + { + ClearSide(_hyperlinks, start, end - start); + } + else + { + _hyperlinks ??= new string?[_cells.Length]; + _hyperlinks.AsSpan(start, end - start).Fill(hyperlink); + } + + _mayHoldWide |= wroteWide; + } + + private void NoteStored(PackedCell cell) + { + if (cell.IsWideContinuation || cell.DisplayWidth == 2) + { + _mayHoldWide = true; + } + } + + private ref readonly PackedCell CellAt(int x) + { + ref readonly var stored = ref _cells[x]; + return ref (uint)(x - _lo) < (uint)(_hi - _lo) ? ref stored : ref _blank; + } + + // [start, start + count) is about to be overwritten: grow the stored range + // to include it, filling any gap between them with the blank they read as. + private void PrepareWrite(int start, int count) + { + var end = start + count; + if (_lo == _hi) + { + _lo = start; + _hi = end; + return; + } + + if (start > _hi) + { + _cells.AsSpan(_hi, start - _hi).Fill(_blank); + } + + if (end < _lo) + { + _cells.AsSpan(end, _lo - end).Fill(_blank); + } + + _lo = Math.Min(_lo, start); + _hi = Math.Max(_hi, end); + } + + // [start, end) is about to be read from the array: store the blank in the + // cells that are blank only logically, then grow the stored range. + private void Materialize(int start, int end) + { + if (start >= end) + { + return; + } + + if (_lo == _hi) + { + _cells.AsSpan(start, end - start).Fill(_blank); + } + else + { + if (start < _lo) + { + _cells.AsSpan(start, Math.Min(end, _lo) - start).Fill(_blank); + } + + if (end > _hi) + { + var from = Math.Max(start, _hi); + _cells.AsSpan(from, end - from).Fill(_blank); + } + } + + PrepareWrite(start, end - start); + } + + private void SetSide(ref string?[]? side, int x, string? value) + { + if (value is not null) + { + side ??= new string?[_cells.Length]; + side[x] = value; + } + else if (side is not null) + { + side[x] = null; + } + } + + private void FillSide(ref string?[]? side, int start, int count, string? value) + { + if (value is not null) + { + side ??= new string?[_cells.Length]; + side.AsSpan(start, count).Fill(value); + } + else + { + ClearSide(side, start, count); + } + } + + private static void ClearSide(string?[]? side, int start, int count) + { + if (side is not null) + { + Array.Clear(side, start, count); + } + } + + private static void CopySide( + string?[]? source, + ref string?[]? destination, + int sourceIndex, + int destinationIndex, + int count, + int destinationLength) + { + if (source is not null) + { + destination ??= new string?[destinationLength]; + Array.Copy(source, sourceIndex, destination, destinationIndex, count); + } + else if (destination is not null) + { + Array.Clear(destination, destinationIndex, count); + } + } +} diff --git a/src/Devolutions.Terminal.Core/CircularBuffer.cs b/src/Devolutions.Terminal.Core/CircularBuffer.cs index 462a739..ffadab7 100644 --- a/src/Devolutions.Terminal.Core/CircularBuffer.cs +++ b/src/Devolutions.Terminal.Core/CircularBuffer.cs @@ -36,7 +36,7 @@ public bool AddLast(T item) } _items[_start] = item; - _start = (_start + 1) % Capacity; + _start = Wrap(_start + 1); return true; } @@ -49,7 +49,7 @@ public T RemoveFirst() var item = _items[_start]; _items[_start] = default!; - _start = (_start + 1) % Capacity; + _start = Wrap(_start + 1); Count--; return item; } @@ -78,7 +78,16 @@ public void Clear() public void ResetCapacity(int capacity, IReadOnlyList items) { capacity = Math.Max(1, capacity); - _items = new T[capacity]; + if (_items.Length == capacity) + { + // Projected engines replace the viewport every frame; keep the array. + Array.Clear(_items); + } + else + { + _items = new T[capacity]; + } + _start = 0; Count = 0; var first = Math.Max(0, items.Count - capacity); @@ -99,7 +108,70 @@ public List ToList() return result; } - private int PhysicalIndex(int logicalIndex) => (_start + logicalIndex) % Capacity; + /// + /// Moves the items at logical indexes +1 through + /// down by one and puts the former first item last; + /// it returns that item. This is a scroll region scrolling up. + /// + public T RotateDown(int first, int last) + { + ValidateIndex(first); + ValidateIndex(last); + var item = this[first]; + Move(first + 1, first, last - first); + _items[PhysicalIndex(last)] = item; + return item; + } + + /// The inverse of : a scroll region scrolling down. + public T RotateUp(int first, int last) + { + ValidateIndex(first); + ValidateIndex(last); + var item = this[last]; + Move(first, first + 1, last - first); + _items[PhysicalIndex(first)] = item; + return item; + } + + // Copies count items between logical ranges, as Array.Copy (memmove) when + // neither range wraps around the end of the backing array. + private void Move(int source, int destination, int count) + { + if (count <= 0) + { + return; + } + + var physicalSource = PhysicalIndex(source); + var physicalDestination = PhysicalIndex(destination); + if (physicalSource + count <= _items.Length && physicalDestination + count <= _items.Length) + { + Array.Copy(_items, physicalSource, _items, physicalDestination, count); + return; + } + + if (destination < source) + { + for (var index = 0; index < count; index++) + { + _items[PhysicalIndex(destination + index)] = _items[PhysicalIndex(source + index)]; + } + } + else + { + for (var index = count - 1; index >= 0; index--) + { + _items[PhysicalIndex(destination + index)] = _items[PhysicalIndex(source + index)]; + } + } + } + + // _start and a valid logical index are each below the capacity, so one + // conditional subtraction replaces an integer division. + private int PhysicalIndex(int logicalIndex) => Wrap(_start + logicalIndex); + + private int Wrap(int index) => index >= _items.Length ? index - _items.Length : index; private void ValidateIndex(int index) { diff --git a/src/Devolutions.Terminal.Core/Devolutions.Terminal.Core.csproj b/src/Devolutions.Terminal.Core/Devolutions.Terminal.Core.csproj index 84d0557..62fa1cf 100644 --- a/src/Devolutions.Terminal.Core/Devolutions.Terminal.Core.csproj +++ b/src/Devolutions.Terminal.Core/Devolutions.Terminal.Core.csproj @@ -5,4 +5,8 @@ false + + + + diff --git a/src/Devolutions.Terminal.Core/IVtDispatch.cs b/src/Devolutions.Terminal.Core/IVtDispatch.cs index 15f36d8..b6ba438 100644 --- a/src/Devolutions.Terminal.Core/IVtDispatch.cs +++ b/src/Devolutions.Terminal.Core/IVtDispatch.cs @@ -5,6 +5,22 @@ namespace Devolutions.Terminal.Core; public interface IVtDispatch { void Print(Rune rune); + // A ground-state run of printable ASCII (0x20-0x7E); dispatchers may store it in bulk. + void PrintAscii(ReadOnlySpan text) + { + foreach (var value in text) + { + Print(new Rune(value)); + } + } + // A ground-state run of decoded printable scalars; dispatchers may store it in bulk. + void PrintRunes(ReadOnlySpan runes) + { + foreach (var rune in runes) + { + Print(rune); + } + } void ExecuteC0(byte control); void EscDispatch(char final, byte intermediate); void EscDispatch(char final, ReadOnlySpan intermediates) => diff --git a/src/Devolutions.Terminal.Core/TerminalEngine.cs b/src/Devolutions.Terminal.Core/TerminalEngine.cs index a4bfa70..03397eb 100644 --- a/src/Devolutions.Terminal.Core/TerminalEngine.cs +++ b/src/Devolutions.Terminal.Core/TerminalEngine.cs @@ -378,13 +378,23 @@ _drcsDesignator is not null && Buffer.WrapPending = false; } - var advance = Buffer.GetPrintAdvance(rune); - if (InsertMode && advance > 0) + int advance; + if (InsertMode) { - Buffer.InsertCharacters(advance); + // IRM shifts the line by the advance before the glyph is stored. + advance = Buffer.GetPrintAdvance(rune); + if (advance > 0) + { + Buffer.InsertCharacters(advance); + } + + Buffer.Print(rune, AutoWrap); + } + else + { + advance = Buffer.PrintAndGetAdvance(rune, AutoWrap); } - Buffer.Print(rune, AutoWrap); if (advance > 0) { _lastPrintedRune = rune; @@ -392,6 +402,69 @@ _drcsDesignator is not null && } } + // Differential tests turn this off to get the per-character reference path. + internal bool BulkPrintEnabled { get; set; } = true; + + void IVtDispatch.PrintAscii(ReadOnlySpan text) + { + // Bulk storage applies only when Print would store each byte unchanged: no + // VT52 or DEC Special Graphics/DRCS remapping, single shift, insert mode or + // DECAWM-off overwrite. A lone byte may still join a Prepend; the buffer + // reports that so REP repeats the same rune as the per-byte path. + var gset = _gsets[_gl]; + if (!BulkPrintEnabled || + !AnsiMode || + !AutoWrap || + InsertMode || + _singleShift >= 0 || + string.Equals(gset, "0", StringComparison.Ordinal) || + (_drcsDesignator is not null && string.Equals(gset, _drcsDesignator, StringComparison.Ordinal))) + { + foreach (var value in text) + { + ((IVtDispatch)this).Print(new Rune(value)); + } + + return; + } + + if (Buffer.PrintAscii(text)) + { + _lastPrintedRune = new Rune(text[^1]); + _hasLastPrintedRune = true; + } + } + + void IVtDispatch.PrintRunes(ReadOnlySpan runes) + { + // The same conditions as PrintAscii, plus DRCS on the GR set, which maps + // the Latin-1 range: with none of them every rune is stored unchanged. + var gl = _gsets[_gl]; + if (!BulkPrintEnabled || + !AnsiMode || + !AutoWrap || + InsertMode || + _singleShift >= 0 || + string.Equals(gl, "0", StringComparison.Ordinal) || + (_drcsDesignator is not null && + (string.Equals(gl, _drcsDesignator, StringComparison.Ordinal) || + string.Equals(_gsets[_gr], _drcsDesignator, StringComparison.Ordinal)))) + { + foreach (var rune in runes) + { + ((IVtDispatch)this).Print(rune); + } + + return; + } + + if (Buffer.PrintRunes(runes, out var lastAdvanced)) + { + _lastPrintedRune = lastAdvanced; + _hasLastPrintedRune = true; + } + } + void IVtDispatch.ExecuteC0(byte control) { switch (control) diff --git a/src/Devolutions.Terminal.Core/TextBuffer.cs b/src/Devolutions.Terminal.Core/TextBuffer.cs index d3e6c40..d2997a5 100644 --- a/src/Devolutions.Terminal.Core/TextBuffer.cs +++ b/src/Devolutions.Terminal.Core/TextBuffer.cs @@ -1,5 +1,6 @@ using System.Text; using System.Globalization; +using System.Runtime.CompilerServices; namespace Devolutions.Terminal.Core; @@ -43,16 +44,22 @@ public sealed class TextBuffer { private sealed class BufferLine { - public BufferLine(Cell[] cells, long logicalLineId) + public BufferLine(CellRow cells, long logicalLineId) { Cells = cells; LogicalLineId = logicalLineId; } - public Cell[] Cells { get; } + public CellRow Cells { get; } public bool Wrapped { get; set; } public LineRendition Rendition { get; set; } - public List Marks { get; } = []; + private List? _marks; + + // Most lines never carry a mark; allocate the list on first write only. + public List Marks => _marks ??= []; + public IReadOnlyList MarksOrEmpty => (IReadOnlyList?)_marks ?? []; + + public void ClearMarks() => _marks?.Clear(); public long LogicalLineId { get; set; } public int LogicalOffset { get; set; } } @@ -65,6 +72,9 @@ public BufferLine(Cell[] cells, long logicalLineId) private ShellMark? _activeMark; private bool _activeMarkHasOutput; private string? _pendingPrepend; + // Printing always completes a Prepend with the scalar after it, so a cell + // holding a lone Prepend can only come from a projected viewport. + private bool _mayHoldLonePrepend; private long _nextLogicalLineId; public TextBuffer(int columns, int rows, int historySize, bool hasHistory) @@ -225,7 +235,8 @@ public void Resize( ScrollOffset = Math.Clamp(previousOffset, 0, HistoryCount); } - public Cell[] GetRow(int viewportY) => GetVisibleLine(viewportY).Cells; + /// A copy of the visible row; writes to it do not reach the buffer. + public Cell[] GetRow(int viewportY) => GetVisibleLine(viewportY).Cells.ToArray(); public Cell GetCell(int x, int y) { @@ -234,7 +245,7 @@ public Cell GetCell(int x, int y) return Cell.Blank; } - return GetRow(y)[x]; + return GetVisibleLine(y).Cells[x]; } public void ApplyColors( @@ -260,15 +271,18 @@ public void ApplyColors( column <= Math.Clamp(lastColumn, 0, Columns - 1); column++) { + var cell = line.Cells[column]; if (foreground is { } foregroundColor) { - line.Cells[column].Attributes.Foreground = foregroundColor; + cell.Attributes.Foreground = foregroundColor; } if (background is { } backgroundColor) { - line.Cells[column].Attributes.Background = backgroundColor; + cell.Attributes.Background = backgroundColor; } + + line.Cells[column] = cell; } } } @@ -281,11 +295,11 @@ public TextBufferSnapshot CreateSnapshot(bool includeHistory = false) for (var i = 0; i < count; i++) { var sourceIndex = Math.Min(first + i, _lines.Count - 1); - var cells = Array.AsReadOnly((Cell[])_lines[sourceIndex].Cells.Clone()); + var cells = Array.AsReadOnly(_lines[sourceIndex].Cells.ToArray()); copies[i] = new TextBufferLineSnapshot( cells, _lines[sourceIndex].Wrapped, - _lines[sourceIndex].Marks + _lines[sourceIndex].MarksOrEmpty .Select(static mark => new ShellMark(mark.StartColumn, mark.ExitCode)) .ToArray()) { @@ -310,16 +324,19 @@ public void ReplaceViewport( var projection = y < rows.Count ? rows[y] : new TextBufferProjectionRow([], false, []); - var source = projection.Cells; - var cells = Enumerable.Repeat(Cell.Blank, Columns).ToArray(); - Array.Copy(source, cells, Math.Min(source.Length, cells.Length)); + var cells = CellRow.FromCells(projection.Cells, Columns); + _mayHoldLonePrepend = true; var line = new BufferLine(cells, ++_nextLogicalLineId) { Wrapped = projection.Wrapped, Rendition = projection.Rendition, }; - line.Marks.AddRange(projection.Marks.Select(static mark => - new ShellMark(mark.StartColumn, mark.ExitCode))); + if (projection.Marks.Count > 0) + { + line.Marks.AddRange(projection.Marks.Select(static mark => + new ShellMark(mark.StartColumn, mark.ExitCode))); + } + replacement.Add(line); } @@ -410,9 +427,443 @@ public int GetPrintAdvance(Rune rune) : 0; } - public void Print(Rune rune, bool autoWrap = true) + /// + /// Prints printable ASCII (0x20-0x7E) with auto-wrap, producing exactly what + /// per byte would. Only the first byte can extend the + /// previous grapheme (a pending or completed Prepend); every later byte + /// follows an ASCII cell written here, and ASCII never extends a grapheme, so + /// the rest skip the join checks and are stored a row segment at a time. + /// Returns whether the last byte advanced the cursor (only a lone byte that + /// joined a Prepend does not), which is what REP repeats. + /// + public bool PrintAscii(ReadOnlySpan text) { - if (IsPrepend(rune)) + if (text.IsEmpty) + { + return false; + } + + if (LeftRightMargins) + { + // DECLRMM can leave the cursor outside the margins after a wrap; keep + // that rare geometry on the per-character path. + var advanced = false; + foreach (var value in text) + { + advanced = PrintAndGetAdvance(new Rune(value)) > 0; + } + + return advanced; + } + + var index = 0; + if (_pendingPrepend is not null || PreviousIsLonePrepend()) + { + var firstAdvanced = PrintAndGetAdvance(new Rune(text[0])) > 0; + if (text.Length == 1) + { + return firstAdvanced; + } + + index = 1; + } + + var template = PackedCell.From(new Rune(' '), CurrentAttributes, CurrentProtection, false, 1, _shellIntegration); + var hyperlink = CurrentHyperlinkUri; + while (index < text.Length) + { + if (WrapPending) + { + var source = GetLiveLine(CursorY); + source.Wrapped = true; + CarriageReturn(); + LineFeed(); + ContinueLogicalLine(source); + } + + var line = GetLiveLine(CursorY); + var effectiveColumns = RightBound(line) + 1; + if (CursorX + 1 > effectiveColumns) + { + line.Wrapped = true; + CarriageReturn(); + LineFeed(); + ContinueLogicalLine(line); + line = GetLiveLine(CursorY); + effectiveColumns = RightBound(line) + 1; + } + + var row = line.Cells; + var count = Math.Min(text.Length - index, effectiveColumns - CursorX); + // Wide glyphs inside a segment are overwritten cell by cell, so only + // glyphs straddling its ends need their other half blanked. A later + // segment starts after a cell written here or at column 0 of a new + // line, so only the first can straddle its start. + if (index == 0) + { + ClearGlyphAt(row, CursorX); + } + + ClearGlyphAt(row, CursorX + count - 1); + row.WriteAscii(CursorX, text.Slice(index, count), template, hyperlink); + index += count; + CursorX += count; + if (CursorX >= effectiveColumns) + { + CursorX = effectiveColumns - 1; + WrapPending = true; + } + } + + return true; + } + + /// + /// Prints runes with auto-wrap, producing exactly what + /// per rune would. A rune that cannot take part in a grapheme join (no + /// , not a Prepend, nothing pending and no lone + /// Prepend before it) and has a width of 1 or 2 is stored directly from packed + /// templates; every other rune takes the per-rune path. Returns whether any rune + /// advanced the cursor and the last one that did, which is what REP repeats. + /// + public bool PrintRunes(ReadOnlySpan runes, out Rune lastAdvanced) + { + lastAdvanced = default; + var advanced = false; + if (LeftRightMargins) + { + // DECLRMM can leave the cursor outside the margins after a wrap; keep + // that rare geometry on the per-character path. + foreach (var rune in runes) + { + if (PrintAndGetAdvance(rune) > 0) + { + lastAdvanced = rune; + advanced = true; + } + } + + return advanced; + } + + UpdateRuneTemplates(); + var hyperlink = CurrentHyperlinkUri; + var widths = _segmentWidths; + var line = GetLiveLine(CursorY); + var effectiveColumns = RightBound(line) + 1; + // Whether the cell the next rune would join has no combining text ending in + // ZWJ, so an Extended_Pictographic rune cannot join it (GB11): true after + // anything printed here advanced the cursor, since only a zero-width append + // (no advance) can leave a ZWJ at the end of a cell. + var previousEndsWithoutZwj = JoinTargetEndsWithoutZwj(); + var index = 0; + while (index < runes.Length) + { + var rune = runes[index]; + // Only the per-rune path below can set a pending Prepend. + var width = _pendingPrepend is null && !_mayHoldLonePrepend + ? SimpleWidth(rune, previousEndsWithoutZwj) + : 0; + if (width == 0) + { + previousEndsWithoutZwj = PrintAndGetAdvance(rune) > 0; + if (previousEndsWithoutZwj) + { + lastAdvanced = rune; + advanced = true; + } + + line = GetLiveLine(CursorY); + effectiveColumns = RightBound(line) + 1; + index++; + continue; + } + + // PrintCore for runes that cannot join, a row segment at a time. + if (WrapPending) + { + line.Wrapped = true; + CarriageReturn(); + LineFeed(); + ContinueLogicalLine(line); + line = GetLiveLine(CursorY); + effectiveColumns = RightBound(line) + 1; + } + + if (CursorX + width > effectiveColumns) + { + line.Wrapped = true; + CarriageReturn(); + LineFeed(); + ContinueLogicalLine(line); + line = GetLiveLine(CursorY); + effectiveColumns = RightBound(line) + 1; + } + + // Take every following rune that is also simple and still fits the row. + // Only the first can wrap; the rest follow cells stored here. + var row = line.Cells; + var end = CursorX; + var next = index; + if (!row.MayHoldWide && row.CanAppendAt(CursorX)) + { + // No wide glyph in the row can straddle the segment, so classify and + // store in one pass. + var cells = row.BeginAppend(CursorX); + var narrow = _narrowTemplate; + var wide = _wideTemplate; + var continuation = _continuationTemplate; + var limit = effectiveColumns - CursorX; + var written = 0; + var wroteWide = false; + while (true) + { + var stored = runes[next]; + if (width == 2) + { + cells[written] = wide.WithRune(stored); + cells[written + 1] = continuation; + wroteWide = true; + } + else + { + cells[written] = narrow.WithRune(stored); + } + + written += width; + next++; + if (next == runes.Length) + { + break; + } + + width = SimpleWidth(runes[next], previousEndsWithoutZwj: true); + if (width == 0 || written + width > limit) + { + break; + } + } + + end = CursorX + written; + row.EndAppend(CursorX, end, hyperlink, wroteWide); + } + else + { + // Record widths for the write. + do + { + widths[next - index] = (byte)width; + end += width; + next++; + width = next < runes.Length && next - index < widths.Length + ? SimpleWidth(runes[next], previousEndsWithoutZwj: true) + : 0; + } + while (width != 0 && end + width <= effectiveColumns); + + // As in PrintAscii, cells inside the segment are overwritten one by + // one, so only glyphs straddling its first and last cell need clearing. + ClearGlyphAt(row, CursorX); + ClearGlyphAt(row, end - 1); + row.WriteRunes( + CursorX, + end, + runes[index..next], + widths, + _narrowTemplate, + _wideTemplate, + _continuationTemplate, + hyperlink); + } + + lastAdvanced = runes[next - 1]; + advanced = true; + previousEndsWithoutZwj = true; + CursorX = end; + if (CursorX >= effectiveColumns) + { + CursorX = effectiveColumns - 1; + WrapPending = true; + } + + index = next; + } + + return advanced; + } + + // Packed cells for runes stored by PrintRunes. The narrow one is packed per + // call (a few integer operations) and doubles as the cache key: the wide and + // continuation cells are derived again only when it changes. + private PackedCell _narrowTemplate; + private PackedCell _wideTemplate; + private PackedCell _continuationTemplate; + private bool _templatesBuilt; + private readonly byte[] _segmentWidths = new byte[256]; + + private void UpdateRuneTemplates() + { + var narrow = PackedCell.From(new Rune(' '), CurrentAttributes, CurrentProtection, false, 1, _shellIntegration); + if (_templatesBuilt && narrow.SameBits(_narrowTemplate)) + { + return; + } + + _narrowTemplate = narrow; + _wideTemplate = narrow.WithWidth(2, isWideContinuation: false); + _continuationTemplate = narrow.WithWidth(0, isWideContinuation: true); + _templatesBuilt = true; + } + + /// + /// The width of a rune may store without the per-rune + /// path, or 0. Such a rune is not a Prepend and cannot extend the previous + /// grapheme: either it has no join property at all, or it is an + /// Extended_Pictographic scalar whose only possible join (GB11) needs combining + /// text ending in ZWJ, which the previous cell is known not to have. The caller + /// has ruled out a pending or lone Prepend. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int SimpleWidth(Rune rune, bool previousEndsWithoutZwj) + { + var simpleClass = rune.Value < SimpleClasses.Length ? SimpleClasses[rune.Value] : ClassifySimple(rune); + if ((simpleClass & NeverSimple) != 0 || + ((simpleClass & SimpleAfterZwjFreeCell) != 0 && !previousEndsWithoutZwj)) + { + return 0; + } + + return WcWidth.ResolveWidth((byte)(simpleClass & 3)); + } + + // SimpleWidth's inputs in one load per rune for planes 0 and 1: the width class + // in the low two bits, plus whether the rune needs the per-rune path or may + // skip it only after a cell without a trailing ZWJ. + private const byte SimpleAfterZwjFreeCell = 4; + private const byte NeverSimple = 8; + private static readonly byte[] SimpleClasses = BuildSimpleClasses(); + + private static byte[] BuildSimpleClasses() + { + var classes = new byte[0x20000]; + for (var value = 0; value < classes.Length; value++) + { + if (value is < 0xD800 or > 0xDFFF) + { + classes[value] = ClassifySimple(new Rune(value)); + } + } + + return classes; + } + + // Built from ClassifyJoin, not the JoinClasses table, which static + // initialization order may not have filled yet. + private static byte ClassifySimple(Rune rune) + { + var joinClass = ClassifyJoin(rune); + var widthClass = WcWidth.WidthClass(rune); + if ((joinClass & PrependScalar) != 0 || + ((joinClass & MayExtendPrevious) != 0 && (joinClass & ExtendsOnlyAfterZwj) == 0)) + { + return (byte)(NeverSimple | widthClass); + } + + return (joinClass & ExtendsOnlyAfterZwj) != 0 + ? (byte)(SimpleAfterZwjFreeCell | widthClass) + : widthClass; + } + + /// + /// Whether the cell a printed rune would join (as + /// finds it) has no combining text ending in ZWJ, or there is none. + /// + private bool JoinTargetEndsWithoutZwj() + { + var y = CursorY; + var x = WrapPending ? CursorX : CursorX - 1; + if (!WrapPending && x < 0 && y > 0 && GetLiveLine(y - 1).Wrapped) + { + y--; + x = Columns - 1; + } + + if (x < 0) + { + return true; + } + + var row = GetLiveLine(y).Cells; + if (row.IsWideContinuation(x) && x > 0) + { + x--; + } + + return row.CombiningAt(x)?.EndsWith('\u200D') != true; + } + + /// + /// Whether the cell a printed rune would join holds a lone Prepend scalar + /// (GB9b). That is the only join open to runes without + /// , printable ASCII included. Conservative: + /// true sends the rune through the full join check. + /// + private bool PreviousIsLonePrepend() + { + if (!_mayHoldLonePrepend) + { + return false; + } + + var y = CursorY; + var x = WrapPending ? CursorX : CursorX - 1; + if (!WrapPending && x < 0 && y > 0 && GetLiveLine(y - 1).Wrapped) + { + y--; + x = Columns - 1; + } + + if (x < 0) + { + return false; + } + + var row = GetLiveLine(y).Cells; + if (row.IsWideContinuation(x) && x > 0) + { + x--; + } + + var previous = row.RuneAt(x); + return IsPrependScalar(previous) && row.CombiningAt(x) is null; + } + + public void Print(Rune rune, bool autoWrap = true) => PrintCore(rune, autoWrap, out _); + + /// + /// Prints like and returns what + /// would have returned just before, resolving the grapheme join target once + /// instead of once per call. + /// + public int PrintAndGetAdvance(Rune rune, bool autoWrap = true) + { + if (_pendingPrepend is not null || IsPrependScalar(rune)) + { + // GetPrintAdvance ignores a pending Prepend while Print consumes it; + // keep both calls so the reported advance stays the same. + var advance = GetPrintAdvance(rune); + Print(rune, autoWrap); + return advance; + } + + PrintCore(rune, autoWrap, out var printed); + return printed; + } + + private void PrintCore(Rune rune, bool autoWrap, out int advance) + { + advance = 0; + if (IsPrependScalar(rune)) { _pendingPrepend = (_pendingPrepend ?? string.Empty) + rune; return; @@ -420,12 +871,17 @@ public void Print(Rune rune, bool autoWrap = true) var prepend = _pendingPrepend; _pendingPrepend = null; - if (prepend is null && TryAppendJoinedRune(rune)) + // Most runes cannot extend the previous grapheme unless it is a lone + // Prepend; skip the full join evaluation for them. + if (prepend is null && + ((JoinClass(rune) & MayExtendPrevious) != 0 || PreviousIsLonePrepend()) && + TryAppendJoinedRune(rune, out advance)) { return; } var width = WcWidth.Width(rune); + advance = width; if (width == 0) { AppendCombining(rune); @@ -441,7 +897,8 @@ public void Print(Rune rune, bool autoWrap = true) ContinueLogicalLine(source); } - var effectiveColumns = RightBound(GetLiveLine(CursorY)) + 1; + var line = GetLiveLine(CursorY); + var effectiveColumns = RightBound(line) + 1; if (CursorX + width > effectiveColumns && !autoWrap) { // DECAWM reset: never wrap or scroll; overwrite at the right edge. @@ -449,50 +906,52 @@ public void Print(Rune rune, bool autoWrap = true) } else if (CursorX + width > effectiveColumns) { - var source = GetLiveLine(CursorY); - source.Wrapped = true; + line.Wrapped = true; CarriageReturn(); LineFeed(); - ContinueLogicalLine(source); - effectiveColumns = RightBound(GetLiveLine(CursorY)) + 1; + ContinueLogicalLine(line); + line = GetLiveLine(CursorY); + effectiveColumns = RightBound(line) + 1; } - var row = GetLiveLine(CursorY).Cells; + var row = line.Cells; ClearGlyphAt(row, CursorX); if (width == 2 && CursorX + 1 < Columns) { ClearGlyphAt(row, CursorX + 1); } - row[CursorX] = new Cell - { - Rune = rune, - Attributes = CurrentAttributes, - IsProtected = CurrentProtection, - StoredWidth = (byte)width, - HyperlinkUri = CurrentHyperlinkUri, - ShellIntegration = _shellIntegration, - }; + var storedRune = rune; + string? combining = null; if (prepend is not null) { var runes = prepend.EnumerateRunes().ToArray(); - row[CursorX].Rune = runes[0]; - row[CursorX].CombiningCharacters = - string.Concat(runes.Skip(1).Select(static value => value.ToString())) + rune; - } - + storedRune = runes[0]; + combining = string.Concat(runes.Skip(1).Select(static value => value.ToString())) + rune; + } + + row.Store( + CursorX, + storedRune, + CurrentAttributes, + CurrentProtection, + isWideContinuation: false, + (byte)width, + combining, + CurrentHyperlinkUri, + _shellIntegration); if (width == 2 && CursorX + 1 < Columns) { - row[CursorX + 1] = new Cell - { - Rune = new Rune(' '), - Attributes = CurrentAttributes, - IsProtected = CurrentProtection, - IsWideContinuation = true, - StoredWidth = 0, - HyperlinkUri = CurrentHyperlinkUri, - ShellIntegration = _shellIntegration, - }; + row.Store( + CursorX + 1, + new Rune(' '), + CurrentAttributes, + CurrentProtection, + isWideContinuation: true, + storedWidth: 0, + combining: null, + CurrentHyperlinkUri, + _shellIntegration); } CursorX += width; @@ -713,7 +1172,7 @@ public void FillAlignmentPattern() var line = GetLiveLine(y); line.Wrapped = false; line.Rendition = LineRendition.SingleWidth; - Array.Fill(line.Cells, glyph); + line.Cells.Fill(0, line.Cells.Length, glyph); } } @@ -953,7 +1412,7 @@ public ushort ChecksumRectangle(int top, int left, int bottom, int right) var row = GetLiveLine(y).Cells; for (var x = left; x <= right; x++) { - ref readonly var cell = ref row[x]; + var cell = row[x]; if (!cell.IsWideContinuation) { checksum = SubtractChecksum(checksum, cell.Rune.Value == 0x2426 ? 0x1B : cell.Rune.Value); @@ -1044,7 +1503,7 @@ public void ShiftColumns(int count) var row = GetLiveLine(y).Cells; if (count < 0) { - Array.Copy(row, left + shift, row, left, width - shift); + row.CopyTo(left + shift, row, left, width - shift); for (var x = right - shift + 1; x <= right; x++) { row[x] = BlankWith(CurrentAttributes); @@ -1052,7 +1511,7 @@ public void ShiftColumns(int count) } else { - Array.Copy(row, left, row, left + shift, width - shift); + row.CopyTo(left, row, left + shift, width - shift); for (var x = left; x < left + shift; x++) { row[x] = BlankWith(CurrentAttributes); @@ -1067,7 +1526,7 @@ public void InsertCharacters(int count) { var row = GetLiveLine(CursorY).Cells; count = Math.Min(Math.Max(1, count), Columns - CursorX); - Array.Copy(row, CursorX, row, CursorX + count, Columns - CursorX - count); + row.CopyTo(CursorX, row, CursorX + count, Columns - CursorX - count); for (var x = CursorX; x < CursorX + count; x++) { row[x] = BlankWith(CurrentAttributes); @@ -1080,7 +1539,7 @@ public void DeleteCharacters(int count) { var row = GetLiveLine(CursorY).Cells; count = Math.Min(Math.Max(1, count), Columns - CursorX); - Array.Copy(row, CursorX + count, row, CursorX, Columns - CursorX - count); + row.CopyTo(CursorX + count, row, CursorX, Columns - CursorX - count); for (var x = Columns - count; x < Columns; x++) { row[x] = BlankWith(CurrentAttributes); @@ -1107,19 +1566,22 @@ public void ScrollUp(int count) CoordinateVersion++; } - _lines.AddLast(NewBlankLine(CurrentAttributes)); + // Once history is full the oldest line is evicted anyway; reuse it so + // steady-state scrolling allocates nothing. + _lines.AddLast(_lines.Count == _lines.Capacity + ? RecycleLine(_lines.RemoveFirst(), CurrentAttributes) + : NewBlankLine(CurrentAttributes)); ScrollOffset = 0; } else { - var top = ViewportStart + ScrollTop; var bottom = ViewportStart + ScrollBottom; - for (var y = top; y < bottom; y++) + var scrolledOut = _lines.RotateDown(ViewportStart + ScrollTop, bottom); + var recycled = RecycleLine(scrolledOut, CurrentAttributes); + if (!ReferenceEquals(recycled, scrolledOut)) { - _lines[y] = _lines[y + 1]; + _lines[bottom] = recycled; } - - _lines[bottom] = NewBlankLine(CurrentAttributes); } } } @@ -1130,13 +1592,12 @@ public void ScrollDown(int count) for (var i = 0; i < count; i++) { var top = ViewportStart + ScrollTop; - var bottom = ViewportStart + ScrollBottom; - for (var y = bottom; y > top; y--) + var scrolledOut = _lines.RotateUp(top, ViewportStart + ScrollBottom); + var recycled = RecycleLine(scrolledOut, CurrentAttributes); + if (!ReferenceEquals(recycled, scrolledOut)) { - _lines[y] = _lines[y - 1]; + _lines[top] = recycled; } - - _lines[top] = NewBlankLine(CurrentAttributes); } } @@ -1321,26 +1782,34 @@ private void TrimHistory() ScrollOffset = 0; } - private BufferLine NewBlankLine(CellAttributes attributes) + private BufferLine NewBlankLine(CellAttributes attributes) => + new(new CellRow(Columns, attributes), ++_nextLogicalLineId); + + /// + /// Blanks a line that just left the buffer (or a scroll region) and gives it a + /// new logical identity, so image anchors and marks never resolve to it again. + /// Snapshots clone cells, so no reader still observes the old content. + /// + private BufferLine RecycleLine(BufferLine line, CellAttributes attributes) { - var row = new Cell[Columns]; - for (var x = 0; x < row.Length; x++) + if (line.Cells.Length != Columns) { - row[x] = BlankWith(attributes); + return NewBlankLine(attributes); } - return new BufferLine(row, ++_nextLogicalLineId); + line.Cells.Reset(attributes); + line.Wrapped = false; + line.Rendition = LineRendition.SingleWidth; + line.ClearMarks(); + line.LogicalLineId = ++_nextLogicalLineId; + line.LogicalOffset = 0; + return line; } private static BufferLine ResizeLineCells(BufferLine source, int columns) { - var cells = new Cell[columns]; - var copy = Math.Min(source.Cells.Length, columns); - Array.Copy(source.Cells, cells, copy); - for (var x = copy; x < columns; x++) - { - cells[x] = Cell.Blank; - } + var cells = new CellRow(columns, CellAttributes.Default); + source.Cells.CopyTo(0, cells, 0, Math.Min(source.Cells.Length, columns)); RepairWideCells(cells); var line = new BufferLine(cells, source.LogicalLineId) @@ -1349,7 +1818,7 @@ private static BufferLine ResizeLineCells(BufferLine source, int columns) Rendition = source.Rendition, LogicalOffset = source.LogicalOffset, }; - foreach (var mark in source.Marks) + foreach (var mark in source.MarksOrEmpty) { if ((uint)mark.StartColumn < (uint)columns) { @@ -1404,7 +1873,7 @@ private void EraseLineRange(int fromY, int toY, bool selective = false) } } - private void EraseCells(Cell[] row, int from, int to, bool selective = false) + private void EraseCells(CellRow row, int from, int to, bool selective = false) { from = Math.Clamp(from, 0, row.Length); to = Math.Clamp(to, 0, row.Length); @@ -1424,13 +1893,13 @@ private void EraseCells(Cell[] row, int from, int to, bool selective = false) ClearGlyphAt(row, to - 1); } - for (var x = from; x < to; x++) + if (from < to) { - row[x] = BlankWith(CurrentAttributes); + row.FillBlank(from, to - from, CurrentAttributes); } } - private static void SelectivelyEraseGlyphAt(Cell[] row, int x) + private static void SelectivelyEraseGlyphAt(CellRow row, int x) { var first = row[x].IsWideContinuation ? x - 1 : x; var last = row[x].DisplayWidth == 2 ? x + 1 : x; @@ -1485,7 +1954,9 @@ private void ChangeAttributesArea( var row = GetLiveLine(y).Cells; for (var x = left; x <= right; x++) { - ApplyRectangularAttributes(ref row[x].Attributes, attributes, reverse); + var cell = row[x]; + ApplyRectangularAttributes(ref cell.Attributes, attributes, reverse); + row[x] = cell; } } } @@ -1683,16 +2154,19 @@ private void AppendCombining(Rune rune) x--; } - if (row[x].IsBlank) + var cell = row[x]; + if (cell.IsBlank) { return; } - row[x].CombiningCharacters = (row[x].CombiningCharacters ?? string.Empty) + rune; + cell.CombiningCharacters = (cell.CombiningCharacters ?? string.Empty) + rune; + row[x] = cell; } - private bool TryAppendJoinedRune(Rune rune) + private bool TryAppendJoinedRune(Rune rune, out int advance) { + advance = 0; var target = GetJoinTarget(rune); if (target is null) { @@ -1700,13 +2174,16 @@ private bool TryAppendJoinedRune(Rune rune) } var (row, x, desiredWidth, lineColumns) = target.Value; - ref var previous = ref row[x]; + var previous = row[x]; var previousWidth = previous.DisplayWidth; + advance = Math.Max(0, desiredWidth - previousWidth); previous.CombiningCharacters = (previous.CombiningCharacters ?? string.Empty) + rune; + row[x] = previous; if (desiredWidth > previousWidth && x + desiredWidth <= lineColumns) { ClearGlyphAt(row, x + 1); previous.StoredWidth = (byte)desiredWidth; + row[x] = previous; row[x + 1] = new Cell { Rune = new Rune(' '), @@ -1731,7 +2208,7 @@ private bool TryAppendJoinedRune(Rune rune) return true; } - private (Cell[] Row, int Column, int DesiredWidth, int LineColumns)? GetJoinTarget(Rune rune) + private (CellRow Row, int Column, int DesiredWidth, int LineColumns)? GetJoinTarget(Rune rune) { var y = CursorY; var x = WrapPending ? CursorX : CursorX - 1; @@ -1747,37 +2224,48 @@ private bool TryAppendJoinedRune(Rune rune) } var row = GetLiveLine(y).Cells; - if (row[x].IsWideContinuation && x > 0) + if (row.IsWideContinuation(x) && x > 0) { x--; } - ref var previous = ref row[x]; - if (previous.IsBlank) + // Most runes can only join a lone Prepend (GB9b); rule them out with + // table loads before evaluating the rules one by one. + if ((JoinClass(rune) & MayExtendPrevious) == 0 && + (!IsPrependScalar(row.RuneAt(x)) || row.CombiningAt(x) is not null)) { return null; } + // Read only the fields the rules need; whole cells are built for the + // rare ZWJ and Indic checks that walk the combining text. + if (row.IsBlank(x)) + { + return null; + } + + var previousRune = row.RuneAt(x); + var previousCombining = row.CombiningAt(x); var joinedByZwj = - previous.CombiningCharacters?.EndsWith('\u200D') == true && + previousCombining?.EndsWith('\u200D') == true && IsExtendedPictographic(rune) && - HasExtendedPictographicBeforeZwj(previous); + HasExtendedPictographicBeforeZwj(previousRune, previousCombining); var emojiPresentationSelector = rune.Value == 0xFE0F && - IsExtendedPictographic(previous.Rune); + IsExtendedPictographic(previousRune); var regionalPair = - IsRegionalIndicator(previous.Rune) && + IsRegionalIndicator(previousRune) && IsRegionalIndicator(rune) && // GB12/13 count immediately adjacent RIs only. An intervening // Extend or ZWJ terminates the run, even if it joined the RI cell. - previous.CombiningCharacters is null; - var hangulSyllable = IsHangulContinuation(previous, rune); + previousCombining is null; + var hangulSyllable = IsHangulContinuation(previousRune, previousCombining, rune); var spacingMark = Rune.GetUnicodeCategory(rune) == UnicodeCategory.SpacingCombiningMark; // A Prepend scalar at the front of a completed cell is no longer // pending: it applies only to the immediately following grapheme. - var prepend = IsPrepend(previous.Rune) && - previous.CombiningCharacters is null; - var indicConjunct = IsIndicConsonant(rune) && EndsWithIndicLinker(previous); + var prepend = IsPrepend(previousRune) && + previousCombining is null; + var indicConjunct = IsIndicConsonant(rune) && EndsWithIndicLinker(row[x]); if (!joinedByZwj && !emojiPresentationSelector && !regionalPair && @@ -1791,7 +2279,7 @@ private bool TryAppendJoinedRune(Rune rune) var desiredWidth = joinedByZwj || emojiPresentationSelector || regionalPair ? 2 - : Math.Max(previous.DisplayWidth, Math.Max(1, WcWidth.Width(rune))); + : Math.Max(row.DisplayWidth(x), Math.Max(1, WcWidth.Width(rune))); var lineColumns = EffectiveColumns(GetLiveLine(y)); return x + desiredWidth <= lineColumns ? (row, x, desiredWidth, lineColumns) : null; } @@ -1799,16 +2287,25 @@ private bool TryAppendJoinedRune(Rune rune) private static bool IsRegionalIndicator(Rune rune) => rune.Value is >= 0x1F1E6 and <= 0x1F1FF; - private static bool HasExtendedPictographicBeforeZwj(Cell cell) + // Walks the cell's text (base rune, then combining text ending in ZWJ) + // backwards past the ZWJ and any Extend scalars, without building arrays. + private static bool HasExtendedPictographicBeforeZwj(Rune baseRune, string combining) { - var runes = cell.Text.EnumerateRunes().ToArray(); - var index = runes.Length - 2; // The final rune is ZWJ. - while (index >= 0 && IsGraphemeExtend(runes[index])) + var text = combining.AsSpan(); + Rune.DecodeLastFromUtf16(text, out _, out var consumed); // The final rune is ZWJ. + text = text[..^consumed]; + while (!text.IsEmpty) { - index--; + Rune.DecodeLastFromUtf16(text, out var rune, out consumed); + if (!IsGraphemeExtend(rune)) + { + return IsExtendedPictographic(rune); + } + + text = text[..^consumed]; } - return index >= 0 && IsExtendedPictographic(runes[index]); + return !IsGraphemeExtend(baseRune) && IsExtendedPictographic(baseRune); } private static bool IsGraphemeExtend(Rune rune) => @@ -1875,12 +2372,19 @@ private enum HangulClass : byte Lvt, } - private static bool IsHangulContinuation(Cell previous, Rune current) + private static bool IsHangulContinuation(Rune previousRune, string? previousCombining, Rune current) { - var prior = Hangul(previous.CombiningCharacters is { Length: > 0 } - ? previous.CombiningCharacters.EnumerateRunes().Last() - : previous.Rune); + // Every continuation needs a Hangul jamo or syllable next; check that + // before walking the previous cell's combining text. var next = Hangul(current); + if (next == HangulClass.Other) + { + return false; + } + + var prior = Hangul(previousCombining is { Length: > 0 } + ? previousCombining.EnumerateRunes().Last() + : previousRune); return prior == HangulClass.L && next is HangulClass.L or HangulClass.V or HangulClass.Lv or HangulClass.Lvt || prior is HangulClass.Lv or HangulClass.V && next is HangulClass.V or HangulClass.T || prior is HangulClass.Lvt or HangulClass.T && next == HangulClass.T; @@ -1912,6 +2416,72 @@ private static HangulClass Hangul(Rune rune) return HangulClass.Other; } + // Grapheme-join properties of every scalar in planes 0 and 1 (which hold the + // emoji), precomputed from the rule predicates below so most printed runes + // skip them with one table load. + private const byte MayExtendPrevious = 1; + private const byte PrependScalar = 2; + private const byte ExtendsOnlyAfterZwj = 4; + private static readonly byte[] JoinClasses = BuildJoinClasses(); + + private static byte JoinClass(Rune rune) => + rune.Value < JoinClasses.Length ? JoinClasses[rune.Value] : ClassifyJoin(rune); + + private static bool IsPrependScalar(Rune rune) => (JoinClass(rune) & PrependScalar) != 0; + + private static byte[] BuildJoinClasses() + { + var classes = new byte[0x20000]; + for (var value = 0; value < classes.Length; value++) + { + if (value is < 0xD800 or > 0xDFFF) + { + classes[value] = ClassifyJoin(new Rune(value)); + } + } + + return classes; + } + + /// + /// marks the runes for which some rule in + /// other than GB9b (a lone Prepend before them) + /// can hold: a ZWJ target, VS16, a regional indicator, Hangul, a spacing mark + /// or an Indic consonant. + /// + private static byte ClassifyJoin(Rune rune) + { + byte joinClass = 0; + if (IsExtendedPictographic(rune) || + rune.Value == 0xFE0F || + IsRegionalIndicator(rune) || + Hangul(rune) != HangulClass.Other || + Rune.GetUnicodeCategory(rune) == UnicodeCategory.SpacingCombiningMark || + IsIndicConsonant(rune)) + { + joinClass |= MayExtendPrevious; + } + + if (IsPrepend(rune)) + { + joinClass |= PrependScalar; + } + + // An Extended_Pictographic scalar with no other join property extends the + // previous cell only through GB11 (combining text ending in ZWJ) or GB9b. + if (IsExtendedPictographic(rune) && + rune.Value != 0xFE0F && + !IsRegionalIndicator(rune) && + Hangul(rune) == HangulClass.Other && + Rune.GetUnicodeCategory(rune) != UnicodeCategory.SpacingCombiningMark && + !IsIndicConsonant(rune)) + { + joinClass |= ExtendsOnlyAfterZwj; + } + + return joinClass; + } + private static bool IsPrepend(Rune rune) => rune.Value is >= 0x0600 and <= 0x0605 or 0x06DD or @@ -2002,32 +2572,34 @@ 0x116B6 or 0x11930 or 0x1193D or 0x11F41 or >= 0x1D165 and <= 0x1D166 or >= 0x1D16D and <= 0x1D172; - private static void ClearGlyphAt(Cell[] row, int x) + private static void ClearGlyphAt(CellRow row, int x) { - if ((uint)x >= (uint)row.Length) + // Without wide glyphs in the row there is nothing to split. + if (!row.MayHoldWide || (uint)x >= (uint)row.Length) { return; } - if (row[x].IsWideContinuation) + // FillBlank with default attributes stores exactly Cell.Blank. + if (row.IsWideContinuation(x)) { - row[x] = Cell.Blank; + row.FillBlank(x, 1, CellAttributes.Default); if (x > 0) { - row[x - 1] = Cell.Blank; + row.FillBlank(x - 1, 1, CellAttributes.Default); } } - else if (row[x].DisplayWidth == 2) + else if (row.DisplayWidth(x) == 2) { - row[x] = Cell.Blank; - if (x + 1 < row.Length && row[x + 1].IsWideContinuation) + row.FillBlank(x, 1, CellAttributes.Default); + if (x + 1 < row.Length && row.IsWideContinuation(x + 1)) { - row[x + 1] = Cell.Blank; + row.FillBlank(x + 1, 1, CellAttributes.Default); } } } - private static void RepairWideCells(Cell[] row) + private static void RepairWideCells(CellRow row) { for (var x = 0; x < row.Length; x++) { @@ -2113,7 +2685,7 @@ private List Reflow( } } } - foreach (var mark in line.Marks) + foreach (var mark in line.MarksOrEmpty) { paragraphMarks.Add((DisplayWidth(paragraphCells) + mark.StartColumn, mark)); } @@ -2237,8 +2809,9 @@ private void EmitParagraph( cursorColumn = x; } - line.Cells[x] = cell; - line.Cells[x].IsWideContinuation = false; + var stored = cell; + stored.IsWideContinuation = false; + line.Cells[x] = stored; if (width == 2) { line.Cells[x + 1] = new Cell @@ -2274,8 +2847,9 @@ private static int LastTextLine( for (var index = lines.Count - 1; index >= 0; index--) { var line = lines[index]; - foreach (var cell in line.Cells) + for (var x = 0; x < line.Cells.Length; x++) { + var cell = line.Cells[x]; // Match conhost: only text counts, so rows that were merely erased // with a colored background are trimmed like any other blank row. if (!cell.IsWideContinuation && @@ -2300,7 +2874,7 @@ private static int LastTextLine( return -1; } - private static int LastContentColumn(Cell[] cells) + private static int LastContentColumn(CellRow cells) { for (var x = cells.Length - 1; x >= 0; x--) { diff --git a/src/Devolutions.Terminal.Core/VtParser.cs b/src/Devolutions.Terminal.Core/VtParser.cs index 2c922f5..0f9284e 100644 --- a/src/Devolutions.Terminal.Core/VtParser.cs +++ b/src/Devolutions.Terminal.Core/VtParser.cs @@ -1,3 +1,4 @@ +using System.Buffers; using System.Text; namespace Devolutions.Terminal.Core; @@ -41,6 +42,11 @@ private enum State private readonly IVtDispatch _dispatch; private readonly int[] _parameters = new int[MaxParameters]; private readonly bool[] _subparameters = new bool[MaxParameters]; + private readonly int[] _csiParameters = new int[MaxParameters]; + private readonly bool[] _csiSubparameters = new bool[MaxParameters]; + private int _csiDispatchDepth; + private readonly Rune[] _runeScratch = new Rune[256]; + private int _printRunesDepth; private bool _nextIsSubparameter; private readonly List _osc = []; private readonly List _apc = []; @@ -69,12 +75,249 @@ public VtParser(IVtDispatch dispatch) public void Process(ReadOnlySpan data) { - foreach (var value in data) + var index = 0; + while (index < data.Length) { - ProcessByte(value); + if (_state == State.Ground && _utf8Needed == 0 && IsPrintableAscii(data[index])) + { + // Hand whole printable-ASCII runs to the dispatcher. Single characters + // between escape sequences are common, so only start the vectorized + // scan once a second printable byte follows. + var run = 1; + if (index + 1 < data.Length && IsPrintableAscii(data[index + 1])) + { + run = data[index..].IndexOfAnyExceptInRange((byte)0x20, (byte)0x7E); + run = run < 0 ? data.Length - index : run; + } + + _dispatch.PrintAscii(data.Slice(index, run)); + index += run; + continue; + } + + if (data[index] >= 0xC2 && _state == State.Ground && _utf8Needed == 0) + { + var consumed = DecodePrintableRun(data[index..]); + if (consumed > 0) + { + index += consumed; + continue; + } + } + + if (data[index] == 0x1B && _state == State.Ground && _utf8Needed == 0) + { + var consumed = TryProcessCsi(data[index..]); + if (consumed > 0) + { + index += consumed; + continue; + } + } + + if (data[index] < 0x20 && data[index] is not (0x18 or 0x1A or 0x1B) && + _state == State.Ground && _utf8Needed == 0) + { + // What ProcessByte does with a C0 control in ground state with no + // UTF-8 pending, without its string-state checks: CR and LF end + // every line. + _dispatch.ExecuteC0(data[index++]); + continue; + } + + ProcessByte(data[index++]); } } + /// + /// Decodes a run that starts with a well-formed UTF-8 sequence and continues + /// with well-formed sequences and short printable-ASCII stretches, and hands it + /// to . Each well-formed sequence is exactly + /// what the byte path would decode into one Print, and each printable ASCII byte + /// one Print too. The run stops before controls, malformed or split input, and + /// long ASCII stretches (left to the ASCII path). Returns the bytes consumed. + /// + private int DecodePrintableRun(ReadOnlySpan data) + { + // A handler that feeds this parser again must not overwrite the run the + // outer call is still printing. + var runes = _printRunesDepth == 0 ? _runeScratch : new Rune[_runeScratch.Length]; + var count = 0; + var index = 0; + while (index < data.Length && count < runes.Length) + { + var value = data[index]; + if (IsPrintableAscii(value)) + { + // One look per ASCII stretch: 16 or more printable bytes go to the + // ASCII path, a shorter stretch joins the run whole. Stretches inside + // non-ASCII text are short, so a scalar count beats a vector search. + var limit = Math.Min(16, data.Length - index); + var stretch = 1; + while (stretch < limit && IsPrintableAscii(data[index + stretch])) + { + stretch++; + } + + if (stretch == 16) + { + break; + } + + stretch = Math.Min(stretch, runes.Length - count); + for (var offset = 0; offset < stretch; offset++) + { + runes[count++] = new Rune(data[index + offset]); + } + + index += stretch; + continue; + } + + // Two- to four-byte sequences, decoded inline with the validity rules + // of the byte path (no overlong forms, no surrogates, nothing past + // U+10FFFF); everything else goes through the general decoder. + if (value <= 0xDF && value >= 0xC2 && index + 1 < data.Length && (data[index + 1] & 0xC0) == 0x80) + { + runes[count++] = new Rune(((value & 0x1F) << 6) | (data[index + 1] & 0x3F)); + index += 2; + continue; + } + + if (value is >= 0xE0 and <= 0xEF && index + 2 < data.Length && + (data[index + 1] & 0xC0) == 0x80 && (data[index + 2] & 0xC0) == 0x80) + { + var scalar = ((value & 0x0F) << 12) | ((data[index + 1] & 0x3F) << 6) | (data[index + 2] & 0x3F); + if (scalar >= 0x800 && scalar is < 0xD800 or > 0xDFFF) + { + runes[count++] = new Rune(scalar); + index += 3; + continue; + } + } + + if (value is >= 0xF0 and <= 0xF4 && index + 3 < data.Length && + (data[index + 1] & 0xC0) == 0x80 && (data[index + 2] & 0xC0) == 0x80 && (data[index + 3] & 0xC0) == 0x80) + { + var scalar = ((value & 0x07) << 18) | ((data[index + 1] & 0x3F) << 12) | + ((data[index + 2] & 0x3F) << 6) | (data[index + 3] & 0x3F); + if (scalar is >= 0x10000 and <= 0x10FFFF) + { + runes[count++] = new Rune(scalar); + index += 4; + continue; + } + } + + if (value < 0xC2 || + Rune.DecodeFromUtf8(data[index..], out var rune, out var length) != OperationStatus.Done) + { + break; + } + + runes[count++] = rune; + index += length; + } + + if (count > 0) + { + _printRunesDepth++; + try + { + _dispatch.PrintRunes(runes.AsSpan(0, count)); + } + finally + { + _printRunesDepth--; + } + } + + return index; + } + + /// + /// Parses a complete, plain CSI sequence (ESC [, optional private + /// marker, digits, ';' and ':' separators, final byte) in one pass instead of + /// feeding each byte through the state machine. Parameters go through the same + /// helpers and as the byte path. Anything else + /// (intermediates, embedded controls, ignored or overflowing sequences, or a + /// sequence that continues in the next chunk) returns 0 without consuming + /// input, and the state machine handles it. + /// + private int TryProcessCsi(ReadOnlySpan data) + { + if (!_ansiMode || data.Length < 3 || data[1] != (byte)'[') + { + return 0; + } + + var start = 2; + var privateMarker = data[2] is >= 0x3C and <= 0x3F ? data[2] : (byte)0; + if (privateMarker != 0) + { + start = 3; + } + + var final = -1; + var separators = 0; + for (var index = start; index < data.Length; index++) + { + var value = data[index]; + if (value is >= (byte)'0' and <= (byte)'9') + { + continue; + } + + if (value is (byte)';' or (byte)':') + { + separators++; + continue; + } + + if (IsFinal(value)) + { + final = index; + } + + break; + } + + // A separator pushing past the limit puts the byte path in CsiIgnore and + // drops the sequence. An overflowing final push behaves the same in both + // paths (DispatchCsi still dispatches), so only that case is excluded. + if (final < 0 || separators > MaxParameters) + { + return 0; + } + + // The state after ESC and '[' (and the private marker, if any). + ClearSequence(); + _state = State.CsiParam; + _privateMarker = privateMarker; + for (var index = start; index < final; index++) + { + var value = data[index]; + if (value == (byte)';') + { + PushParameter(); + _nextIsSubparameter = false; + } + else if (value == (byte)':') + { + PushParameter(); + _nextIsSubparameter = true; + } + else + { + _currentParameter = _currentParameter < 0 ? 0 : _currentParameter; + _currentParameter = Math.Min((_currentParameter * 10) + (value - '0'), 65535); + } + } + + DispatchCsi(data[final]); + return final + 1; + } + public void Reset() { _state = State.Ground; @@ -509,15 +752,35 @@ private void DispatchCsi(byte final) var parameterCount = _parameterCount; var intermediate = _intermediate; var privateMarker = _privateMarker; - var parameters = _parameters.AsSpan(0, parameterCount).ToArray(); - var subparameters = _subparameters.AsSpan(0, parameterCount).ToArray(); + // Copies keep the dispatched values stable if the handler resets the + // parser. Reused scratch arrays avoid per-sequence heap arrays and the + // zeroing a stackalloc pays; a handler that feeds this parser again + // gets fresh copies so it cannot overwrite the outer sequence. + var nested = _csiDispatchDepth > 0; + var parameters = nested ? new int[parameterCount] : _csiParameters; + var subparameters = nested ? new bool[parameterCount] : _csiSubparameters; + // Usually a handful of values: a loop beats a Memmove call. + for (var index = 0; index < parameterCount; index++) + { + parameters[index] = _parameters[index]; + subparameters[index] = _subparameters[index]; + } ClearSequence(); - _dispatch.CsiDispatch( - (char)final, - parameters, - subparameters, - intermediate, - privateMarker); + _csiDispatchDepth++; + try + { + _dispatch.CsiDispatch( + (char)final, + parameters.AsSpan(0, parameterCount), + subparameters.AsSpan(0, parameterCount), + intermediate, + privateMarker); + } + finally + { + _csiDispatchDepth--; + } + if (enterVt52) { _ansiMode = false; @@ -903,13 +1166,18 @@ private void PushParameterIfNeeded() private void ClearSequence() { + // PushParameter sets flags only below the count, so only those can be set. + for (var index = 0; index < _parameterCount; index++) + { + _subparameters[index] = false; + } + _parameterCount = 0; _currentParameter = -1; _intermediate = 0; _privateMarker = 0; _escIntermediateCount = 0; _nextIsSubparameter = false; - Array.Clear(_subparameters); } private void AppendEscIntermediate(byte value) @@ -946,4 +1214,6 @@ private void ResetUtf8() } private static bool IsFinal(byte value) => value is >= 0x40 and <= 0x7E; + + private static bool IsPrintableAscii(byte value) => value is >= 0x20 and <= 0x7E; } diff --git a/src/Devolutions.Terminal.Core/WcWidth.cs b/src/Devolutions.Terminal.Core/WcWidth.cs index 29a4ad7..93ce412 100644 --- a/src/Devolutions.Terminal.Core/WcWidth.cs +++ b/src/Devolutions.Terminal.Core/WcWidth.cs @@ -7,25 +7,62 @@ public static class WcWidth { public static bool AmbiguousAsWide { get; set; } + private const byte Zero = 0; + private const byte Narrow = 1; + private const byte Wide = 2; + private const byte Ambiguous = 3; + + // The width class of every scalar in planes 0 and 1 (which hold the emoji), + // from the same rules as the rest, so printing does not repeat category and + // range lookups per character. Ambiguous stays a class of its own because + // AmbiguousAsWide can change. + private static readonly byte[] Classes = BuildClasses(); + public static int Width(Rune rune) { var value = rune.Value; + var widthClass = value < Classes.Length ? Classes[value] : Classify(value); + return widthClass == Ambiguous ? (AmbiguousAsWide ? 2 : 1) : widthClass; + } + + /// 0 zero width, 1 narrow, 2 wide, 3 ambiguous (2 when ). + internal static byte WidthClass(Rune rune) => + rune.Value < Classes.Length ? Classes[rune.Value] : Classify(rune.Value); + + internal static int ResolveWidth(byte widthClass) => + widthClass == Ambiguous ? (AmbiguousAsWide ? 2 : 1) : widthClass; + + private static byte[] BuildClasses() + { + var classes = new byte[0x20000]; + for (var value = 0; value < classes.Length; value++) + { + classes[value] = Classify(value); + } + + return classes; + } + + private static byte Classify(int value) + { if (value == 0) { - return 0; + return Zero; } if (value < 0x20 || (value >= 0x7F && value < 0xA0)) { - return 0; + return Zero; } if (value is >= 0x1F3FB and <= 0x1F3FF) { - return 0; + return Zero; } - var category = Rune.GetUnicodeCategory(rune); + var category = value is >= 0xD800 and <= 0xDFFF + ? UnicodeCategory.Surrogate + : Rune.GetUnicodeCategory(new Rune(value)); if (category is UnicodeCategory.NonSpacingMark or UnicodeCategory.EnclosingMark or UnicodeCategory.SpacingCombiningMark @@ -33,15 +70,15 @@ or UnicodeCategory.Control or UnicodeCategory.Format or UnicodeCategory.Surrogate) { - return 0; + return Zero; } - if (IsWide(value) || (AmbiguousAsWide && IsAmbiguous(value))) + if (IsWide(value)) { - return 2; + return Wide; } - return 1; + return IsAmbiguous(value) ? Ambiguous : Narrow; } private static bool IsWide(int v) => diff --git a/src/Devolutions.Terminal.Ghostty/GhosttyNative.cs b/src/Devolutions.Terminal.Ghostty/GhosttyNative.cs index 2210720..a55f906 100644 --- a/src/Devolutions.Terminal.Ghostty/GhosttyNative.cs +++ b/src/Devolutions.Terminal.Ghostty/GhosttyNative.cs @@ -108,6 +108,10 @@ internal static partial GhosttyResult RenderStateUpdate( nint state, nint terminal); + [LibraryImport(LibraryName, EntryPoint = "ghostty_render_state_clean")] + [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] + internal static partial GhosttyResult RenderStateClean(nint state); + [LibraryImport(LibraryName, EntryPoint = "ghostty_render_state_get")] [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] internal static partial GhosttyResult RenderStateGet( diff --git a/src/Devolutions.Terminal.Ghostty/GhosttyTerminalEngine.cs b/src/Devolutions.Terminal.Ghostty/GhosttyTerminalEngine.cs index 0c08735..3e3c926 100644 --- a/src/Devolutions.Terminal.Ghostty/GhosttyTerminalEngine.cs +++ b/src/Devolutions.Terminal.Ghostty/GhosttyTerminalEngine.cs @@ -23,8 +23,11 @@ public sealed unsafe class GhosttyTerminalEngine : ITerminalEngine private const int RenderDataCols = 1; private const int RenderDataRows = 2; + private const int RenderDataDirty = 3; private const int RenderDataRowIterator = 4; private const int RenderDataCursor = 18; + private const int RenderStateDirtyFull = 2; + private const int RenderRowDataDirty = 1; private const int RenderRowDataRaw = 2; private const int RenderRowDataCells = 3; private const int RenderCellDataStyle = 2; @@ -50,6 +53,15 @@ public sealed unsafe class GhosttyTerminalEngine : ITerminalEngine private string _title = "Devolutions Terminal"; private string? _workingDirectory; private GhosttyRenderCursor _cursor; + private readonly TextBuffer _buffer; + // Feeds only mark the projection stale; readers of the projected buffer or + // cursor rebuild it on demand, once per frame rather than once per PTY read. + private bool _projectionStale; + // The last projection of each viewport row, reused while Ghostty's render + // state reports the row clean. + private (Cell[] Cells, int? PromptStart)?[] _projectedRows = []; + private uint[] _codepointBuffer = new uint[16]; + private byte[] _hyperlinkBuffer = new byte[256]; private Exception? _callbackException; private SafeGhosttyTrackedGridRefHandle? _historyAnchor; private readonly int _historyLimit; @@ -95,7 +107,7 @@ public GhosttyTerminalEngine(int columns = 120, int rows = 30, int historySize = { GhosttyAbi.Validate(); _historyLimit = Math.Max(0, historySize); - Buffer = new TextBuffer(columns, rows, historySize, hasHistory: true); + _buffer = new TextBuffer(columns, rows, historySize, hasHistory: true); nint terminal = 0; nint renderState = 0; nint rowIterator = 0; @@ -168,7 +180,30 @@ public GhosttyTerminalEngine(int columns = 120, int rows = 30, int historySize = } } - public TextBuffer Buffer { get; } + public TextBuffer Buffer + { + get + { + lock (_gate) + { + EnsureProjection(); + } + + return _buffer; + } + } + + private GhosttyRenderCursor ProjectedCursor + { + get + { + lock (_gate) + { + EnsureProjection(); + return _cursor; + } + } + } public ColorScheme Scheme { @@ -192,10 +227,10 @@ public ColorScheme Scheme public TerminalEngineCapabilities Capabilities => TerminalEngineCapabilities.Vt52Keyboard; public string? WorkingDirectory => _workingDirectory; public bool AlternateBufferActive => QueryInt(TerminalDataActiveScreen) == 1; - public bool CursorVisible => _cursor.ViewportHasValue != 0 && _cursor.Visible != 0; - public bool CursorBlinking => _cursor.Blinking != 0; + public bool CursorVisible => ProjectedCursor is { ViewportHasValue: not 0, Visible: not 0 }; + public bool CursorBlinking => ProjectedCursor.Blinking != 0; public int CursorStyle => - _cursor.VisualStyle is >= 1 and <= 6 ? _cursor.VisualStyle : 0; + ProjectedCursor.VisualStyle is var style and >= 1 and <= 6 ? style : 0; public bool ApplicationCursorKeys => QueryMode(1); public bool BracketedPaste => QueryMode(2004); public bool MouseTracking => QueryMode(9) || QueryMode(1000) || QueryMode(1002) || QueryMode(1003); @@ -258,7 +293,11 @@ public void Feed(ReadOnlySpan data) callbackException); } - SynchronizeProjection(); + // Projecting the grid costs several native calls per cell, so it waits + // for a reader (normally one snapshot per frame). Title and directory + // changes still raise their events from the feed. + _projectionStale = true; + SynchronizeMetadata(); var historyAfter = ReadScrollbarCore().HistoryCount; var initialEvictionPossible = historyBefore == 0 && EstimateRows(data, Columns) >= historyAfter + Rows; @@ -269,7 +308,7 @@ public void Feed(ReadOnlySpan data) initialEvictionPossible || !trackedHistory) { - Buffer.AdvanceCoordinateVersion(); + _buffer.AdvanceCoordinateVersion(); } if (!trackedHistory || trackedHistoryEvicted) { @@ -303,7 +342,7 @@ public void Resize(int columns, int rows, double cellWidth = 1, double cellHeigh _cellWidthPixels, _cellHeightPixels), "ghostty_terminal_resize"); - Buffer.Resize(columns, rows); + _buffer.Resize(columns, rows); SynchronizeProjection(); } @@ -364,7 +403,8 @@ public TerminalSnapshot CreateSnapshot(bool includeHistory = false) lock (_gate) { ObjectDisposedException.ThrowIf(_disposed, this); - var buffer = includeHistory ? CreateHistorySnapshot() : Buffer.CreateSnapshot(); + EnsureProjection(); + var buffer = includeHistory ? CreateHistorySnapshot() : _buffer.CreateSnapshot(); return new TerminalSnapshot( buffer, Title, @@ -838,7 +878,23 @@ private void ApplyScheme() "set palette"); } - private void SynchronizeProjection() + private void EnsureProjection() + { + if (_projectionStale && !_disposed) + { + SynchronizeProjection(reprojectAll: false); + } + } + + /// + /// Copies Ghostty's render state into . Rows the render + /// state reports clean keep their previous projection: it only rebuilds a row + /// (marking it dirty) when its contents change, and a viewport move, resize, + /// screen switch or palette change rebuilds them all, so a clean row reads + /// back exactly what was projected last time. Paths other than feeding pass + /// to keep their previous, unconditional cost. + /// + private void SynchronizeProjection(bool reprojectAll = true) { ThrowIfFailed( GhosttyNative.RenderStateUpdate( @@ -848,9 +904,25 @@ private void SynchronizeProjection() var columns = GetRenderUInt16(RenderDataCols); var rows = GetRenderUInt16(RenderDataRows); - if (Buffer.Columns != columns || Buffer.Rows != rows) + if (_buffer.Columns != columns || _buffer.Rows != rows) + { + _buffer.Resize(columns, rows); + } + + var frameDirty = 0; + ThrowIfFailed( + GhosttyNative.RenderStateGet(_renderState.DangerousGetHandle(), RenderDataDirty, &frameDirty), + "get render dirty state"); + if (frameDirty == RenderStateDirtyFull || + _projectedRows.Length != rows || + (_projectedRows.Length > 0 && _projectedRows[0] is { } first && first.Cells.Length != columns)) + { + reprojectAll = true; + } + + if (_projectedRows.Length != rows) { - Buffer.Resize(columns, rows); + _projectedRows = new (Cell[] Cells, int? PromptStart)?[rows]; } var cursor = new GhosttyRenderCursor { Size = (nuint)sizeof(GhosttyRenderCursor) }; @@ -878,20 +950,41 @@ private void SynchronizeProjection() RenderRowDataRaw, &rawRow), "get raw row"); - var cellsValue = _rowCells.DangerousGetHandle(); - ThrowIfFailed( - GhosttyNative.RowGet( - _rowIterator.DangerousGetHandle(), - RenderRowDataCells, - &cellsValue), - "get row cells"); + byte rowDirty = 1; + if (!reprojectAll) + { + ThrowIfFailed( + GhosttyNative.RowGet( + _rowIterator.DangerousGetHandle(), + RenderRowDataDirty, + &rowDirty), + "get row dirty"); + } + byte wrapped = 0; ThrowIfFailed(GhosttyNative.RawRowGet(rawRow, 1, &wrapped), "get row wrap"); var semanticPrompt = 0; ThrowIfFailed( GhosttyNative.RawRowGet(rawRow, 6, &semanticPrompt), "get row semantic prompt"); - var projected = ProjectRow(columns, rowIndex); + var cached = rowIndex < _projectedRows.Length ? _projectedRows[rowIndex] : null; + if (rowDirty != 0 || cached is null) + { + var cellsValue = _rowCells.DangerousGetHandle(); + ThrowIfFailed( + GhosttyNative.RowGet( + _rowIterator.DangerousGetHandle(), + RenderRowDataCells, + &cellsValue), + "get row cells"); + cached = ProjectRow(columns, rowIndex); + if (rowIndex < _projectedRows.Length) + { + _projectedRows[rowIndex] = cached; + } + } + + var projected = cached.Value; projectedRows.Add(new TextBufferProjectionRow( projected.Cells, wrapped != 0, @@ -915,10 +1008,15 @@ private void SynchronizeProjection() var cursorY = _cursor.ViewportHasValue != 0 ? _cursor.ViewportY : QueryUInt16Core(4); - Buffer.ReplaceViewport( + _buffer.ReplaceViewport( projectedRows, cursorX, cursorY); + // Every row was consumed, so the next update reports only new changes. + ThrowIfFailed( + GhosttyNative.RenderStateClean(_renderState.DangerousGetHandle()), + "ghostty_render_state_clean"); + _projectionStale = false; SynchronizeMetadata(); } @@ -928,8 +1026,6 @@ private void SynchronizeProjection() var column = 0; var previousWidth = 0; int? promptStart = null; - var codepointBuffer = new uint[16]; - var hyperlinkBuffer = new byte[256]; while (column < columns && GhosttyNative.RowCellsNext(_rowCells.DangerousGetHandle())) { ulong rawCell = 0; @@ -959,7 +1055,7 @@ private void SynchronizeProjection() }, IsProtected = GetRawCellBool(rawCell, 8), HyperlinkUri = hasHyperlink - ? GetHyperlinkUri(column, rowIndex, ref hyperlinkBuffer) + ? GetHyperlinkUri(column, rowIndex, ref _hyperlinkBuffer) : null, ShellIntegration = ShellIntegrationKind.None, }; @@ -984,12 +1080,12 @@ private void SynchronizeProjection() continue; } - if (codepointBuffer.Length < graphemeLength) + if (_codepointBuffer.Length < graphemeLength) { - codepointBuffer = new uint[graphemeLength]; + _codepointBuffer = new uint[graphemeLength]; } - fixed (uint* codepoints = codepointBuffer) + fixed (uint* codepoints = _codepointBuffer) { ThrowIfFailed( GhosttyNative.RowCellsGet( @@ -999,19 +1095,25 @@ private void SynchronizeProjection() "get grapheme"); } - var rune = Rune.TryCreate(codepointBuffer[0], out var value) ? value : Rune.ReplacementChar; - var combining = new StringBuilder(); - for (var index = 1; index < graphemeLength; index++) + var rune = Rune.TryCreate(_codepointBuffer[0], out var value) ? value : Rune.ReplacementChar; + string? combining = null; + if (graphemeLength > 1) { - if (Rune.TryCreate(codepointBuffer[index], out var extra)) + var builder = new StringBuilder(); + for (var index = 1; index < graphemeLength; index++) { - combining.Append(extra); + if (Rune.TryCreate(_codepointBuffer[index], out var extra)) + { + builder.Append(extra); + } } + + combining = builder.Length == 0 ? null : builder.ToString(); } var width = wide == 1 ? 2 : 1; projected.Rune = rune; - projected.CombiningCharacters = combining.Length == 0 ? null : combining.ToString(); + projected.CombiningCharacters = combining; projected.StoredWidth = (byte)width; projected.ShellIntegration = semantic switch { @@ -1210,7 +1312,7 @@ private TextBufferSnapshot CreateHistorySnapshot() var scrollbar = ReadScrollbarCore(); if (scrollbar.HistoryCount == 0) { - return Buffer.CreateSnapshot(); + return _buffer.CreateSnapshot(); } var originalOffset = scrollbar.ScrollOffset; @@ -1219,7 +1321,7 @@ private TextBufferSnapshot CreateHistorySnapshot() for (var top = 0; top <= scrollbar.HistoryCount;) { ScrollToAbsoluteRow(top); - var viewport = Buffer.CreateSnapshot(); + var viewport = _buffer.CreateSnapshot(); var firstNewRow = Math.Max(0, nextAbsoluteRow - top); for (var row = firstNewRow; row < viewport.Lines.Count; row++) { diff --git a/tests/Devolutions.Terminal.Core.Tests/AsciiFastPathTests.cs b/tests/Devolutions.Terminal.Core.Tests/AsciiFastPathTests.cs new file mode 100644 index 0000000..5a1e1db --- /dev/null +++ b/tests/Devolutions.Terminal.Core.Tests/AsciiFastPathTests.cs @@ -0,0 +1,184 @@ +using System.Text; +using Devolutions.Terminal.Core; +using Xunit; + +namespace Devolutions.Terminal.Core.Tests; + +/// +/// The parser hands printable-ASCII runs to +/// and the engine stores them in bulk; with bulk storage disabled every byte +/// takes the per-character path, and both must leave the terminal in exactly the +/// same state. DEC Special Graphics maps to +/// ambiguous-width box drawing, so stay clear of tests toggling +/// . +/// +[Collection("Unicode cell width observations")] +public sealed class AsciiFastPathTests +{ + private const string Esc = "\u001b"; + + public static TheoryData Streams => new() + { + { "wrap", new string('a', 50) + "\r\n" + new string('b', 21) + "xyz" }, + { "prepend joins first byte", "ab\u0600cd" + "\u0600" + "efgh" }, + { "wrap over wide cells", "\u4E00\u4E00\u4E00\u4E00\r" + "abcdefghij" }, + { "run ends on wide head", "\u4E00\u4E00\u4E00\r" + "abc" }, + { "run starts on wide tail", $"\u4E00\u4E00\u4E00{Esc}[4Gxyz" }, + { "runes start on wide tail", $"\u4E00\u4E00\u4E00{Esc}[2G\u0436\u0438" }, + { "runes end on wide head", "\u4E00\u4E00\u4E00\r\u0436\u0438\u0437" }, + { "wide runes start on wide tail", $"\u4E00\u4E00\u4E00{Esc}[2G\u4E2D\u6587" }, + { "wide runes end on wide head", "\u4E00\u4E00\u4E00\r\u0436\u4E2D" }, + { "wide runes wrap", new string('a', 19) + "\u4E2D\u6587\u0436\U0001F600\U0001F680" }, + { "zwj family after runes", "\u0436\U0001F468\u200D\U0001F469\u200D\U0001F467\u0438" }, + { "dec graphics", $"{Esc}(0lqqk{Esc}(B plain" }, + { "single shift", $"{Esc}*0{Esc}Nqq plain" }, + { "insert mode", $"0123456789\r{Esc}[4hab{Esc}[4lcd" }, + { "autowrap off", $"{Esc}[?7l{new string('x', 30)}yz{Esc}[?7h" }, + { "left right margins", $"{Esc}[?69h{Esc}[3;8s{Esc}[1;3H{new string('m', 25)}{Esc}[?69l" }, + { "double width line", $"{Esc}#6{new string('d', 15)}" }, + { "repeat last", $"abc{Esc}[3b" }, + { "styled hyperlink", $"{Esc}[1;31mred{Esc}]8;;https://example.test\u0007link{Esc}]8;;\u0007{Esc}[0m" }, + { "region scroll", $"{Esc}[2;4r{Esc}[4;1H" + string.Concat(Enumerable.Range(0, 12).Select(i => $"line {i}\r\n")) }, + { "history overflow", string.Concat(Enumerable.Range(0, 40).Select(i => $"{Esc}[{31 + (i % 7)}mrow {i:D3} {new string('#', i % 23)}\r\n")) }, + }; + + [Theory] + [MemberData(nameof(Streams))] + public void BulkAsciiMatchesPerCharacterPrinting(string name, string stream) + { + AssertBulkMatchesReference(name, Encoding.UTF8.GetBytes(stream), 20, 5, 8); + } + + [Fact] + public void BulkAsciiMatchesPerCharacterPrintingOnRandomStreams() + { + string[] pieces = + [ + "\r\n", "\n", "\r", "\t", "\b", $"{Esc}[K", $"{Esc}[2J", $"{Esc}[H", $"{Esc}[3;5H", + $"{Esc}[1m", $"{Esc}[0m", $"{Esc}[38;5;200m", $"{Esc}[4h", $"{Esc}[4l", $"{Esc}[?7l", $"{Esc}[?7h", + $"{Esc}(0", $"{Esc}(B", $"{Esc}#6", $"{Esc}#5", $"{Esc}[2;4r", $"{Esc}[r", $"{Esc}[2b", $"{Esc}M", + "\u0600", "\u0301", "\u4E00", "\U0001F600", "\u200D", "e\u0301", + "\u0436\u0438\u0437\u043D\u044C", "\u4E2D\u6587\u5B57", "\U0001F600\U0001F680", "\U0001F468\u200D\U0001F469", + "\u2764\uFE0F", "\U0001F1EB\U0001F1F7", "\u1100\u1161\u11A8", "\u0915\u094D\u0937", "\u00E9\u00FC\u00DF", + "\u0928\u093F", "\u0600\u0628", + ]; + var random = new Random(20261026); + for (var iteration = 0; iteration < 200; iteration++) + { + var builder = new StringBuilder(); + for (var piece = 0; piece < 60; piece++) + { + if (random.Next(3) == 0) + { + builder.Append(pieces[random.Next(pieces.Length)]); + } + else + { + var length = random.Next(1, 30); + for (var index = 0; index < length; index++) + { + builder.Append((char)random.Next(0x20, 0x7F)); + } + } + } + + AssertBulkMatchesReference($"random #{iteration}", Encoding.UTF8.GetBytes(builder.ToString()), 17, 4, 6); + } + } + + [Fact] + public void AsciiAfterCompletedPrependCellJoinsLikePrint() + { + // Live feeds always complete a Prepend with the next scalar, but a + // projected screen can hold one alone; ASCII must then join it (GB9b). + static TextBuffer Create() + { + var buffer = new TextBuffer(8, 2, 0, hasHistory: false); + buffer.ReplaceViewport( + [new TextBufferProjectionRow([new Cell { Rune = new Rune(0x0600), StoredWidth = 1 }], false, [])], + 1, + 0); + return buffer; + } + + var lone = Create(); + Assert.False(lone.PrintAscii("x"u8)); + + var bulk = Create(); + var reference = Create(); + Assert.True(bulk.PrintAscii("xy"u8)); + reference.Print(new Rune('x')); + reference.Print(new Rune('y')); + + Assert.Equal("\u0600x", bulk.GetCell(0, 0).Text); + Assert.Equal(reference.GetRow(0), bulk.GetRow(0)); + Assert.Equal(reference.GetRow(0), lone.GetRow(0).Select((cell, x) => x == 1 ? reference.GetCell(1, 0) : cell).ToArray()); + Assert.Equal((reference.CursorX, reference.CursorY), (bulk.CursorX, bulk.CursorY)); + } + + // The reference prints every byte through the per-character path. The bulk + // engine gets the stream whole (long runs) and one byte at a time (runs of + // one, split anywhere); both must end in the reference's state. + private static void AssertBulkMatchesReference(string name, byte[] bytes, int columns, int rows, int history) + { + using var reference = new TerminalEngine(columns, rows, history) { BulkPrintEnabled = false }; + using var whole = new TerminalEngine(columns, rows, history); + using var single = new TerminalEngine(columns, rows, history); + reference.Feed(bytes); + whole.Feed(bytes); + foreach (var value in bytes) + { + single.Feed([value]); + } + + AssertSameState($"{name} (whole)", reference, whole); + AssertSameState($"{name} (byte by byte)", reference, single); + } + + [Fact] + public void RecycledHistoryLineDoesNotChangeEarlierSnapshots() + { + using var engine = new TerminalEngine(10, 2, historySize: 2); + engine.Feed("one\r\ntwo\r\nthree\r\nfour"); + var before = engine.CreateSnapshot(includeHistory: true); + var beforeText = Text(before.Buffer.Lines[0]); + + engine.Feed($"\r\n{Esc}[44mfive\r\nsix"); + + Assert.Equal("one", beforeText); + Assert.Equal("one", Text(before.Buffer.Lines[0])); + var after = engine.CreateSnapshot(includeHistory: true); + Assert.Equal(["three", "four", "five", "six"], after.Buffer.Lines.Select(Text)); + // The recycled line was blanked with the current (BCE) background. + Assert.Equal(ColorKind.Indexed, after.Buffer.Lines[3].Cells[9].Attributes.Background.Kind); + Assert.Equal(4, after.Buffer.Lines[3].Cells[9].Attributes.Background.Index); + } + + private static void AssertSameState(string name, TerminalEngine expected, TerminalEngine actual) + { + Assert.True( + (expected.CursorX, expected.CursorY, expected.HistoryCount) == + (actual.CursorX, actual.CursorY, actual.HistoryCount), + $"{name}: cursor/history ({expected.CursorX},{expected.CursorY},{expected.HistoryCount}) " + + $"vs ({actual.CursorX},{actual.CursorY},{actual.HistoryCount})"); + Assert.True(expected.Buffer.WrapPending == actual.Buffer.WrapPending, $"{name}: wrap pending"); + var left = expected.CreateSnapshot(includeHistory: true).Buffer.Lines; + var right = actual.CreateSnapshot(includeHistory: true).Buffer.Lines; + Assert.True(left.Count == right.Count, $"{name}: line count"); + for (var line = 0; line < left.Count; line++) + { + Assert.True(left[line].Wrapped == right[line].Wrapped, $"{name}: line {line} wrapped"); + Assert.True(left[line].Rendition == right[line].Rendition, $"{name}: line {line} rendition"); + Assert.True(left[line].LogicalOffset == right[line].LogicalOffset, $"{name}: line {line} logical offset"); + for (var column = 0; column < left[line].Cells.Count; column++) + { + Assert.True( + left[line].Cells[column].Equals(right[line].Cells[column]), + $"{name}: line {line} column {column}: '{left[line].Cells[column].Text}' vs '{right[line].Cells[column].Text}'"); + } + } + } + + private static string Text(TextBufferLineSnapshot line) => + string.Concat(line.Cells.Select(static cell => cell.Text)).TrimEnd(); +} diff --git a/tests/Devolutions.Terminal.Core.Tests/CellRowModelTests.cs b/tests/Devolutions.Terminal.Core.Tests/CellRowModelTests.cs new file mode 100644 index 0000000..cb8c474 --- /dev/null +++ b/tests/Devolutions.Terminal.Core.Tests/CellRowModelTests.cs @@ -0,0 +1,181 @@ +using System.Text; +using Devolutions.Terminal.Core; +using Xunit; + +namespace Devolutions.Terminal.Core.Tests; + +/// +/// packs cells, keeps rare strings in side arrays and +/// leaves a logically blank tail unwritten. Random operation sequences must +/// leave it indistinguishable from a plain array. +/// +public sealed class CellRowModelTests +{ + private const int Columns = 12; + + [Fact] + public void RandomOperationsMatchPlainCellArrays() + { + var random = new Random(4242); + for (var iteration = 0; iteration < 300; iteration++) + { + var attributes = RandomAttributes(random); + var rows = new[] { new CellRow(Columns, attributes), new CellRow(Columns, CellAttributes.Default) }; + var models = new[] { Blank(attributes), Blank(CellAttributes.Default) }; + for (var step = 0; step < 40; step++) + { + var target = random.Next(2); + var row = rows[target]; + var model = models[target]; + var start = random.Next(Columns); + var count = random.Next(0, Columns - start + 1); + string operation; + switch (random.Next(9)) + { + case 0: + var cell = RandomCell(random); + row[start] = cell; + model[start] = cell; + operation = $"set {start}"; + break; + case 1: + var fill = RandomCell(random); + row.Fill(start, count, fill); + model.AsSpan(start, count).Fill(fill); + operation = $"fill {start}+{count}"; + break; + case 2: + case 3: + // Half of the blank fills run to the end of the line. + var blankAttributes = RandomAttributes(random); + var blankCount = random.Next(2) == 0 ? Columns - start : count; + row.FillBlank(start, blankCount, blankAttributes); + model.AsSpan(start, blankCount).Fill(BlankCell(blankAttributes)); + operation = $"blank {start}+{blankCount}"; + break; + case 4: + var text = Encoding.ASCII.GetBytes(new string('a', count).Select(_ => (char)random.Next(0x20, 0x7F)).ToArray()); + var template = RandomCell(random); + template.CombiningCharacters = null; + template.IsWideContinuation = false; + template.StoredWidth = 1; + row.WriteAscii(start, text, template); + for (var index = 0; index < text.Length; index++) + { + model[start + index] = template with { Rune = new Rune(text[index]) }; + } + + operation = $"ascii {start}+{count}"; + break; + case 5: + var source = random.Next(2); + var sourceStart = random.Next(Columns - count + 1); + rows[source].CopyTo(sourceStart, row, start, count); + Array.Copy(models[source], sourceStart, model, start, count); + operation = $"copy {source}:{sourceStart} -> {start}+{count}"; + break; + case 6: + var reset = RandomAttributes(random); + row.Reset(reset); + Array.Fill(model, BlankCell(reset)); + operation = "reset"; + break; + case 7 when row.CanAppendAt(start): + // The single-pass path: narrow and wide cells, no combining text. + var appendTemplate = RandomCell(random); + appendTemplate.CombiningCharacters = null; + appendTemplate.IsWideContinuation = false; + var narrowCell = appendTemplate with { StoredWidth = 1 }; + var wideCell = appendTemplate with { StoredWidth = 2 }; + var tailCell = appendTemplate with { Rune = new Rune(' '), StoredWidth = 0, IsWideContinuation = true }; + var cells = row.BeginAppend(start); + var written = 0; + var wroteWide = false; + while (written < count) + { + var rune = new Rune(random.Next(0x4E00, 0x4F00)); + if (random.Next(2) == 0 && written + 2 <= count) + { + cells[written] = PackedCell.From(wideCell with { Rune = rune }); + cells[written + 1] = PackedCell.From(tailCell); + model[start + written] = wideCell with { Rune = rune }; + model[start + written + 1] = tailCell; + written += 2; + wroteWide = true; + } + else + { + cells[written] = PackedCell.From(narrowCell with { Rune = rune }); + model[start + written] = narrowCell with { Rune = rune }; + written++; + } + } + + row.EndAppend(start, start + written, appendTemplate.HyperlinkUri, wroteWide); + for (var x = start; x < start + written; x++) + { + model[x].HyperlinkUri = appendTemplate.HyperlinkUri; + } + + operation = $"append {start}+{written}"; + break; + default: + rows[target] = row.Clone(); + operation = "clone"; + break; + } + + AssertSame($"iteration {iteration} step {step} ({operation}) row {target}", models[target], rows[target]); + AssertSame($"iteration {iteration} step {step} ({operation}) row {1 - target}", models[1 - target], rows[1 - target]); + } + } + } + + private static void AssertSame(string context, Cell[] expected, CellRow actual) + { + Assert.True(expected.SequenceEqual(actual.ToArray()), context); + Assert.True( + actual.MayHoldWide || !expected.Any(static cell => cell.IsWideContinuation || cell.DisplayWidth == 2), + $"{context}: wide cell stored while MayHoldWide is false"); + for (var x = 0; x < Columns; x++) + { + var cell = expected[x]; + Assert.True(cell.IsWideContinuation == actual.IsWideContinuation(x), $"{context}: wide at {x}"); + Assert.True(cell.DisplayWidth == actual.DisplayWidth(x), $"{context}: width at {x}"); + Assert.True(cell.Rune == actual.RuneAt(x), $"{context}: rune at {x}"); + Assert.True(cell.CombiningCharacters == actual.CombiningAt(x), $"{context}: combining at {x}"); + Assert.True(cell.IsBlank == actual.IsBlank(x), $"{context}: blank at {x}"); + } + } + + private static Cell[] Blank(CellAttributes attributes) => + Enumerable.Repeat(BlankCell(attributes), Columns).ToArray(); + + private static Cell BlankCell(CellAttributes attributes) => new() + { + Rune = new Rune(' '), + Attributes = attributes, + StoredWidth = 1, + }; + + private static CellAttributes RandomAttributes(Random random) => random.Next(3) == 0 + ? CellAttributes.Default + : new CellAttributes + { + Foreground = TermColor.FromIndex(random.Next(256)), + Background = random.Next(2) == 0 ? TermColor.Default : TermColor.FromRgb(1, 2, (byte)random.Next(256)), + Flags = (CellFlags)random.Next(0, 0x800), + }; + + private static Cell RandomCell(Random random) => new() + { + Rune = random.Next(3) == 0 ? new Rune(' ') : new Rune(random.Next(0x21, 0x3000)), + Attributes = RandomAttributes(random), + IsProtected = random.Next(4) == 0, + IsWideContinuation = random.Next(5) == 0, + StoredWidth = (byte)random.Next(0, 3), + CombiningCharacters = random.Next(4) == 0 ? "\u0301" : null, + HyperlinkUri = random.Next(4) == 0 ? "https://example.test" : null, + ShellIntegration = (ShellIntegrationKind)random.Next(4), + }; +} diff --git a/tests/Devolutions.Terminal.Core.Tests/CsiFastPathTests.cs b/tests/Devolutions.Terminal.Core.Tests/CsiFastPathTests.cs new file mode 100644 index 0000000..fb16e87 --- /dev/null +++ b/tests/Devolutions.Terminal.Core.Tests/CsiFastPathTests.cs @@ -0,0 +1,168 @@ +using System.Text; +using Devolutions.Terminal.Core; +using Xunit; + +namespace Devolutions.Terminal.Core.Tests; + +/// +/// parses complete plain CSI sequences in one pass. Fed one +/// byte at a time it can never see a whole sequence, so it uses only the state +/// machine; every dispatch must be identical either way. +/// +public sealed class CsiFastPathTests +{ + private const string Esc = "\u001b"; + + public static TheoryData Streams => new() + { + { "plain", $"{Esc}[m{Esc}[1;31m{Esc}[38;2;10;20;30;48;5;200mX{Esc}[12;40H{Esc}[K" }, + { "private marker", $"{Esc}[?25l{Esc}[?1049h{Esc}[>4;2m{Esc}[=1c{Esc}[<0;10;5M" }, + { "colon subparameters", $"{Esc}[4:3m{Esc}[38:2::1:2:3m{Esc}[58:5:9m" }, + { "empty parameters", $"{Esc}[;5H{Esc}[3;;7r{Esc}[;m" }, + { "large values", $"{Esc}[99999;123456789H{Esc}[65535;65536m" }, + { "31 separators", $"{Esc}[{string.Join(';', Enumerable.Range(1, 32))}m" }, + { "32 separators", $"{Esc}[{string.Join(';', Enumerable.Range(1, 33))}m" }, + { "33 separators", $"{Esc}[{string.Join(';', Enumerable.Range(1, 34))}mafter" }, + { "intermediate", $"{Esc}[2 q{Esc}[!p{Esc}[1\"q" }, + { "control inside", $"{Esc}[1\r;2H{Esc}[3\n1m{Esc}[\u0007m" }, + { "delete inside", $"{Esc}[1\u007f;2H" }, + { "escape inside", $"{Esc}[1;{Esc}[2Jx" }, + { "cancel inside", $"{Esc}[1;3\u0018m{Esc}[1\u001am" }, + { "marker after parameter", $"{Esc}[1?2h{Esc}[1;<3m" }, + { "high byte inside", "\u001b[1\u00E9;2H" }, + { "unterminated", $"{Esc}[1;2" }, + { "vt52 mode", $"{Esc}[?2lAB{Esc}[H{Esc}<{Esc}[2J" }, + { "after utf8", $"\u00E9{Esc}[1m\u4E00{Esc}[0m" }, + }; + + [Theory] + [MemberData(nameof(Streams))] + public void WholeAndBytewiseFeedsDispatchIdentically(string name, string stream) => + AssertSameDispatch(name, Encoding.UTF8.GetBytes(stream), new Random(name.Length)); + + [Fact] + public void RandomCsiStreamsDispatchIdentically() + { + string[] pieces = + [ + "\u001b[", "\u001b[?", "\u001b[>", "0", "1", "5", "38", "48", "2", "255", "65535", "99999", + ";", ";", ":", "m", "H", "J", "K", "h", "l", "r", " ", "!", "?", "\r", "\n", "\u0007", + "\u007f", "\u001b", "\u0018", "x", "y", "\u00E9", "\u4E00", + ]; + var random = new Random(77); + for (var iteration = 0; iteration < 500; iteration++) + { + var builder = new StringBuilder(); + for (var piece = 0; piece < 40; piece++) + { + builder.Append(pieces[random.Next(pieces.Length)]); + } + + AssertSameDispatch($"random #{iteration}", Encoding.UTF8.GetBytes(builder.ToString()), random); + } + } + + [Fact] + public void RandomUtf8StreamsDispatchIdentically() + { + // Well-formed sequences take the one-pass decode; everything malformed or + // interrupted must still reach the state machine unchanged. + byte[][] pieces = + [ + [0xC3, 0xA9], [0xE4, 0xB8, 0x80], [0xF0, 0x9F, 0x98, 0x80], [0xEF, 0xBF, 0xBD], + [0xC0, 0xAF], [0xC1, 0xBF], [0xE0, 0x80, 0xAF], [0xED, 0xA0, 0x80], [0xF4, 0x90, 0x80, 0x80], + [0xF5, 0x80], [0xFF], [0x80], [0xBF], [0xE4, 0xB8], [0xF0, 0x9F], [0xC3], + [0x9B, (byte)'1', (byte)'m'], [0x9D], [0x1B, (byte)'[', (byte)'m'], [0x1B], [0x0A], [0x41], [0x7F], + ]; + var random = new Random(99); + for (var iteration = 0; iteration < 500; iteration++) + { + var bytes = new List(); + for (var piece = 0; piece < 30; piece++) + { + bytes.AddRange(pieces[random.Next(pieces.Length)]); + } + + AssertSameDispatch($"utf8 #{iteration}", [.. bytes], random); + } + } + + private static void AssertSameDispatch(string name, byte[] bytes, Random random) + { + var bytewise = new RecordingDispatch(); + var bytewiseParser = new VtParser(bytewise); + foreach (var value in bytes) + { + bytewiseParser.Process([value]); + } + + var whole = new RecordingDispatch(); + new VtParser(whole).Process(bytes); + + // Random chunk boundaries also split sequences mid-way. + var chunked = new RecordingDispatch(); + var chunkedParser = new VtParser(chunked); + for (var offset = 0; offset < bytes.Length;) + { + var length = Math.Min(bytes.Length - offset, random.Next(1, 12)); + chunkedParser.Process(bytes.AsSpan(offset, length)); + offset += length; + } + + Assert.True(bytewise.Calls.SequenceEqual(whole.Calls), $"{name} (whole):\n{Diff(bytewise.Calls, whole.Calls)}"); + Assert.True(bytewise.Calls.SequenceEqual(chunked.Calls), $"{name} (chunked):\n{Diff(bytewise.Calls, chunked.Calls)}"); + } + + private static string Diff(List expected, List actual) + { + var index = 0; + while (index < expected.Count && index < actual.Count && expected[index] == actual[index]) + { + index++; + } + + return $"first difference at call {index}: expected '{(index < expected.Count ? expected[index] : "")}' " + + $"actual '{(index < actual.Count ? actual[index] : "")}'"; + } + + private sealed class RecordingDispatch : IVtDispatch + { + public List Calls { get; } = []; + + public void Print(Rune rune) => Calls.Add($"print {rune.Value:X}"); + + public void ExecuteC0(byte control) => Calls.Add($"c0 {control:X2}"); + + public void EscDispatch(char final, byte intermediate) => Calls.Add($"esc {final} {intermediate}"); + + public void EscDispatch(char final, ReadOnlySpan intermediates) => + Calls.Add($"esc {final} [{string.Join(',', intermediates.ToArray())}]"); + + public void Vt52Dispatch(char final, byte row = 0, byte column = 0) => Calls.Add($"vt52 {final} {row} {column}"); + + public void CsiDispatch(char final, ReadOnlySpan parameters, byte intermediate, bool privateMarker) => + throw new InvalidOperationException("the parser dispatches the detailed overload"); + + public void CsiDispatch( + char final, + ReadOnlySpan parameters, + ReadOnlySpan subparameters, + byte intermediate, + byte privateMarker) => + Calls.Add( + $"csi {final} [{string.Join(',', parameters.ToArray())}] " + + $"[{string.Join(',', subparameters.ToArray())}] {intermediate} {privateMarker}"); + + public void DcsDispatch( + char final, + ReadOnlySpan parameters, + ReadOnlySpan intermediates, + byte privateMarker, + ReadOnlySpan data) => + Calls.Add($"dcs {final} [{string.Join(',', parameters.ToArray())}] {Convert.ToHexString(data)}"); + + public void OscDispatch(int command, ReadOnlySpan data) => Calls.Add($"osc {command} {data.ToString()}"); + + public void ApcDispatch(ReadOnlySpan data) => Calls.Add($"apc {data.ToString()}"); + } +} diff --git a/tests/Devolutions.Terminal.Core.Tests/PackedCellStorageTests.cs b/tests/Devolutions.Terminal.Core.Tests/PackedCellStorageTests.cs new file mode 100644 index 0000000..edcd65f --- /dev/null +++ b/tests/Devolutions.Terminal.Core.Tests/PackedCellStorageTests.cs @@ -0,0 +1,113 @@ +using System.Text; +using Devolutions.Terminal.Core; +using Xunit; + +namespace Devolutions.Terminal.Core.Tests; + +/// +/// The buffer stores lines as packed, reference-free cells with combining text +/// and hyperlinks in per-line side arrays. Whatever goes in through the public +/// must come back out unchanged, including colors that the +/// factories would never produce. +/// +public sealed class PackedCellStorageTests +{ + [Fact] + public void ArbitraryCellsRoundTripThroughBufferStorage() + { + const int columns = 40; + const int rows = 6; + var random = new Random(5150); + var buffer = new TextBuffer(columns, rows, 4, hasHistory: true); + var expected = new Cell[rows][]; + var projection = new List(); + for (var row = 0; row < rows; row++) + { + expected[row] = new Cell[columns]; + for (var column = 0; column < columns; column++) + { + expected[row][column] = RandomCell(random); + } + + projection.Add(new TextBufferProjectionRow((Cell[])expected[row].Clone(), random.Next(2) == 0, [])); + } + + buffer.ReplaceViewport(projection, 3, 2); + var snapshot = buffer.CreateSnapshot(); + + for (var row = 0; row < rows; row++) + { + for (var column = 0; column < columns; column++) + { + Assert.Equal(expected[row][column], snapshot.Lines[row].Cells[column]); + Assert.Equal(expected[row][column], buffer.GetCell(column, row)); + } + } + } + + [Fact] + public void GetRowReturnsACopy() + { + var buffer = new TextBuffer(4, 2, 0, hasHistory: false); + buffer.Print(new Rune('a')); + + var row = buffer.GetRow(0); + row[0].Rune = new Rune('z'); + + Assert.Equal(new Rune('a'), buffer.GetCell(0, 0).Rune); + } + + [Fact] + public void RecycledLinesDropCombiningTextAndHyperlinks() + { + using var engine = new TerminalEngine(8, 2, historySize: 1); + engine.Feed("\u001b]8;;https://example.test\u0007e\u0301x\u001b]8;;\u0007\r\n"); + engine.Feed("second\r\nthird\r\nfourth\r\nfifth"); + + // Every line has been recycled since the hyperlinked, combining first line. + var snapshot = engine.CreateSnapshot(includeHistory: true); + foreach (var line in snapshot.Buffer.Lines) + { + Assert.All(line.Cells, static cell => + { + Assert.Null(cell.HyperlinkUri); + Assert.Null(cell.CombiningCharacters); + }); + } + + Assert.Equal("fifth", string.Concat(snapshot.Buffer.Lines[^1].Cells.Select(static cell => cell.Text)).TrimEnd()); + } + + private static Cell RandomCell(Random random) + { + static TermColor Color(Random random) => new( + (ColorKind)random.Next(3), + (byte)random.Next(256), + (byte)random.Next(256), + (byte)random.Next(256), + (byte)random.Next(256)); + + return new Cell + { + Rune = random.Next(4) switch + { + 0 => new Rune(random.Next(0x20, 0x7F)), + 1 => new Rune(random.Next(0x4E00, 0x9FFF)), + 2 => new Rune(random.Next(0x1F300, 0x1FAFF)), + _ => new Rune(random.Next(0x100000, 0x10FFFE)), + }, + Attributes = new CellAttributes + { + Foreground = Color(random), + Background = Color(random), + Flags = (CellFlags)random.Next(0, 0x10000), + }, + IsProtected = random.Next(2) == 0, + IsWideContinuation = random.Next(4) == 0, + StoredWidth = (byte)random.Next(0, 3), + CombiningCharacters = random.Next(5) == 0 ? "\u0301\u0308" : null, + HyperlinkUri = random.Next(5) == 0 ? $"https://example.test/{random.Next(100)}" : null, + ShellIntegration = (ShellIntegrationKind)random.Next(4), + }; + } +} diff --git a/tests/Devolutions.Terminal.Ghostty.Tests/GhosttyProjectionTests.cs b/tests/Devolutions.Terminal.Ghostty.Tests/GhosttyProjectionTests.cs new file mode 100644 index 0000000..ef936a3 --- /dev/null +++ b/tests/Devolutions.Terminal.Ghostty.Tests/GhosttyProjectionTests.cs @@ -0,0 +1,91 @@ +using System.Text; +using Devolutions.Terminal.Core; +using Devolutions.Terminal.Ghostty; +using Xunit; + +namespace Devolutions.Terminal.Ghostty.Tests; + +/// +/// Feeding only marks the projection stale, and a later read re-projects just +/// the rows Ghostty's render state reports dirty. Reading after every chunk +/// exercises that incremental path; scrolling the viewport to the bottom then +/// forces a full re-projection, and both must describe the same screen. +/// +public sealed class GhosttyProjectionTests +{ + private const string Esc = "\u001b"; + + public static TheoryData Streams => new() + { + { "partial rows", $"one\r\ntwo\r\nthree{Esc}[2;1Hchanged{Esc}[5;3Hx" }, + { "scrolling", string.Concat(Enumerable.Range(0, 30).Select(i => $"line {i:D2} {new string('#', i % 15)}\r\n")) }, + { "region scroll", $"{Esc}[2;5r{Esc}[5;1H" + string.Concat(Enumerable.Range(0, 12).Select(i => $"r{i}\r\n")) + $"{Esc}[r" }, + { "styled wide", $"{Esc}[1;31mred{Esc}[0m \u4E00\u4E00 {Esc}[44mblue{Esc}[0m e\u0301" }, + { "hyperlink and prompt", $"{Esc}]133;A\u0007PS> {Esc}]133;B\u0007{Esc}]8;;https://example.test\u0007link{Esc}]8;;\u0007\r\n" }, + { "alternate screen", $"primary\r\n{Esc}[?1049h{Esc}[Halt{Esc}[3;3Hz{Esc}[?1049lback" }, + { "erase", $"abcdef\r\nghijkl{Esc}[1;3H{Esc}[K{Esc}[2;2H{Esc}[1P" }, + }; + + [Theory] + [MemberData(nameof(Streams))] + public void IncrementalProjectionMatchesFullProjection(string name, string stream) + { + var bytes = Encoding.UTF8.GetBytes(stream); + using var engine = new GhosttyTerminalEngine(16, 6, historySize: 20); + var random = new Random(name.Length); + for (var offset = 0; offset < bytes.Length;) + { + var length = Math.Min(bytes.Length - offset, random.Next(1, 9)); + engine.Feed(bytes.AsSpan(offset, length)); + offset += length; + _ = engine.CreateSnapshot(); + } + + var incremental = engine.CreateSnapshot(); + engine.SetScrollOffset(0); + var full = engine.CreateSnapshot(); + + AssertSameScreen(name, full, incremental); + } + + [Fact] + public void FeedDefersProjectionUntilRead() + { + using var reader = new GhosttyTerminalEngine(20, 4); + using var writer = new GhosttyTerminalEngine(20, 4); + foreach (var line in new[] { "first\r\n", "second\r\n", "third" }) + { + reader.Feed(line); + _ = reader.CreateSnapshot(); + writer.Feed(line); + } + + // The cursor and grid accessors project on demand. + Assert.Equal((reader.CursorX, reader.CursorY), (writer.CursorX, writer.CursorY)); + AssertSameScreen("deferred", reader.CreateSnapshot(), writer.CreateSnapshot()); + Assert.Equal("third", writer.Buffer.GetText(0, 2, 19, 2).TrimEnd()); + } + + private static void AssertSameScreen(string name, TerminalSnapshot expected, TerminalSnapshot actual) + { + Assert.True( + (expected.Buffer.CursorX, expected.Buffer.CursorY) == (actual.Buffer.CursorX, actual.Buffer.CursorY), + $"{name}: cursor"); + Assert.Equal(expected.Buffer.Lines.Count, actual.Buffer.Lines.Count); + for (var row = 0; row < expected.Buffer.Lines.Count; row++) + { + var left = expected.Buffer.Lines[row]; + var right = actual.Buffer.Lines[row]; + Assert.True(left.Wrapped == right.Wrapped, $"{name}: row {row} wrapped"); + Assert.True( + left.Marks.Select(static mark => mark.StartColumn).SequenceEqual(right.Marks.Select(static mark => mark.StartColumn)), + $"{name}: row {row} marks"); + for (var column = 0; column < left.Cells.Count; column++) + { + Assert.True( + left.Cells[column].Equals(right.Cells[column]), + $"{name}: row {row} column {column}: '{left.Cells[column].Text}' vs '{right.Cells[column].Text}'"); + } + } + } +} diff --git a/tools/Devolutions.Terminal.Bench/Devolutions.Terminal.Bench.csproj b/tools/Devolutions.Terminal.Bench/Devolutions.Terminal.Bench.csproj index c43046a..bb18397 100644 --- a/tools/Devolutions.Terminal.Bench/Devolutions.Terminal.Bench.csproj +++ b/tools/Devolutions.Terminal.Bench/Devolutions.Terminal.Bench.csproj @@ -7,6 +7,7 @@ false enable false + true @@ -15,7 +16,9 @@ + + diff --git a/tools/Devolutions.Terminal.Bench/Program.cs b/tools/Devolutions.Terminal.Bench/Program.cs index d92e40a..bb4d84b 100644 --- a/tools/Devolutions.Terminal.Bench/Program.cs +++ b/tools/Devolutions.Terminal.Bench/Program.cs @@ -23,6 +23,8 @@ namespace Devolutions.Terminal.Bench; /// render — snapshot/planning and CPU Skia rasterization, measured separately /// for a static screen, one-row updates, and scrolling screens. /// corpus — write the deterministic VT input for reuse with Ghostty benchmarks. +/// vt — same-process ingest comparison of the builtin engine against raw +/// libghostty-vt and DT's Ghostty engine over per-workload corpora. /// /// Caveat: control mode does not paint. Render mode uses a raster surface, not /// Avalonia's GPU/compositor or Ghostty's OpenGL/Metal renderer. @@ -34,6 +36,11 @@ internal static class Program private static int Main(string[] args) { var mode = args.Length > 0 ? args[0] : "control"; + if (mode == "vt") + { + return VtBench.Run(args); + } + var megabytes = GetOption(args, "--mb", 8); var runs = GetOption(args, "--runs", 5); var chunkKb = GetOption(args, "--chunk-kb", 16); @@ -103,7 +110,7 @@ private static int Main(string[] args) { "engine" => RunEngine(corpus, chunkKb * 1024), "control" => RunControl(corpus, chunkKb * 1024), - _ => throw new ArgumentException($"unknown mode '{mode}' (expected engine|control|render|corpus)"), + _ => throw new ArgumentException($"unknown mode '{mode}' (expected engine|control|render|corpus|vt)"), }; samples.Add(mbPerSec); Console.WriteLine($" run {run + 1}: {mbPerSec:F1} MB/s"); @@ -114,7 +121,7 @@ private static int Main(string[] args) return 0; } - private static int GetOption(string[] args, string name, int fallback, bool allowZero = false) + internal static int GetOption(string[] args, string name, int fallback, bool allowZero = false) { var index = Array.IndexOf(args, name); if (index < 0) @@ -132,7 +139,7 @@ private static int GetOption(string[] args, string name, int fallback, bool allo return value; } - private static string? GetStringOption(string[] args, string name) + internal static string? GetStringOption(string[] args, string name) { var index = Array.IndexOf(args, name); if (index < 0) @@ -148,6 +155,8 @@ private static int GetOption(string[] args, string name, int fallback, bool allo return args[index + 1]; } + internal static byte[] BuildMixedCorpus(int targetBytes) => Corpus.Build(targetBytes); + private static double RunEngine(byte[] corpus, int chunkSize) { using var engine = new TerminalEngine(); diff --git a/tools/Devolutions.Terminal.Bench/VtBench.cs b/tools/Devolutions.Terminal.Bench/VtBench.cs new file mode 100644 index 0000000..7da7156 --- /dev/null +++ b/tools/Devolutions.Terminal.Bench/VtBench.cs @@ -0,0 +1,478 @@ +using System.Diagnostics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Security.Cryptography; +using System.Text.Json; +using System.Text.Json.Serialization; +using Devolutions.Terminal.Core; +using Devolutions.Terminal.Ghostty; + +namespace Devolutions.Terminal.Bench; + +/// +/// Same-process, same-bytes VT ingest comparison between the builtin engine and +/// Ghostty. Three targets consume identical chunks: +/// builtin — DT's own (the default engine). +/// ghostty-vt — raw ghostty_terminal_vt_write: Ghostty's parser and +/// screen with nothing else, the bar the builtin engine must clear. +/// ghostty — DT's , i.e. ghostty-vt plus +/// the per-feed render-state projection DT needs to paint it. +/// Runs interleave the targets in rotating order so thermal drift and +/// background load hit every target alike; fresh engines are built outside the +/// timed region. A verdict is only "faster"/"slower" when the run ranges do +/// not overlap. +/// +internal static partial class VtBench +{ + public const string Builtin = "builtin"; + public const string GhosttyVt = "ghostty-vt"; + public const string GhosttyDt = "ghostty"; + + public static int Run(string[] args) + { + var megabytes = Program.GetOption(args, "--mb", 8); + var runs = Program.GetOption(args, "--runs", 7); + var warmup = Program.GetOption(args, "--warmup", 1, allowZero: true); + var chunkSize = checked(Program.GetOption(args, "--chunk-kb", 16) * 1024); + var minSeconds = Program.GetOption(args, "--min-ms", 500, allowZero: true) / 1000.0; + // 0 measures pure ingest; N reads a frame every N KiB, as a renderer would. + var frameBytes = checked(Program.GetOption(args, "--frame-kb", 0, allowZero: true) * 1024); + var columns = Program.GetOption(args, "--columns", 120); + var rows = Program.GetOption(args, "--rows", 30); + var workloads = Split(Program.GetStringOption(args, "--workloads"), VtWorkloads.Names); + var targets = Split(Program.GetStringOption(args, "--targets"), [Builtin, GhosttyVt, GhosttyDt]); + var exportDirectory = Program.GetStringOption(args, "--export"); + var jsonPath = Program.GetStringOption(args, "--json"); + var dataPath = Program.GetStringOption(args, "--data"); + if (Array.IndexOf(args, "--list") >= 0) + { + Console.WriteLine(string.Join(Environment.NewLine, VtWorkloads.Names)); + return 0; + } + + foreach (var target in targets) + { + if (target is not (Builtin or GhosttyVt or GhosttyDt)) + { + throw new ArgumentException($"unknown target '{target}' (expected {Builtin}|{GhosttyVt}|{GhosttyDt})"); + } + } + + Console.WriteLine( + $"mode=vt grid={columns}x{rows} chunk={chunkSize / 1024} KiB " + + $"frame={(frameBytes == 0 ? "off" : $"{frameBytes / 1024} KiB")} runs={runs} warmup={warmup} " + + $"targets={string.Join(',', targets)} runtime={RuntimeInformation.FrameworkDescription} " + + $"os={RuntimeInformation.OSDescription} arch={RuntimeInformation.ProcessArchitecture} " + + $"cpus={Environment.ProcessorCount}"); + + var inputs = dataPath is null + ? workloads.Select(name => (Name: name, Bytes: VtWorkloads.Build(name, checked(megabytes * 1024 * 1024), columns, rows))).ToArray() + : [(Name: Path.GetFileNameWithoutExtension(dataPath), Bytes: File.ReadAllBytes(dataPath))]; + + if (exportDirectory is not null) + { + Directory.CreateDirectory(exportDirectory); + foreach (var (name, bytes) in inputs) + { + var path = Path.Combine(exportDirectory, $"{name}.vt"); + File.WriteAllBytes(path, bytes); + Console.WriteLine($"wrote {path} ({bytes.Length:N0} B sha256={Convert.ToHexString(SHA256.HashData(bytes))})"); + } + + return 0; + } + + var report = new VtReport( + DateTimeOffset.UtcNow, + Environment.MachineName, + RuntimeInformation.OSDescription, + RuntimeInformation.ProcessArchitecture.ToString(), + Environment.ProcessorCount, + RuntimeInformation.FrameworkDescription, + columns, + rows, + chunkSize, + frameBytes, + runs, + []); + var exitCode = 0; + foreach (var (name, bytes) in inputs) + { + var sha = Convert.ToHexString(SHA256.HashData(bytes)); + Console.WriteLine(); + Console.WriteLine($"[{name}] {bytes.Length / (1024.0 * 1024):F1} MiB sha256={sha[..16]}… {VtWorkloads.Describe(name)}"); + var equivalence = targets.Contains(Builtin) && (targets.Contains(GhosttyVt) || targets.Contains(GhosttyDt)) + ? CheckEquivalence(bytes, columns, rows, chunkSize) + : null; + if (equivalence is not null) + { + Console.WriteLine($" screen equivalence vs ghostty: {equivalence}"); + } + + var samples = targets.ToDictionary(target => target, _ => new List()); + for (var run = -warmup; run < runs; run++) + { + for (var slot = 0; slot < targets.Length; slot++) + { + // Rotate the starting target every run to cancel ordering bias. + var target = targets[(slot + Math.Max(0, run)) % targets.Length]; + var sample = Measure(target, bytes, columns, rows, chunkSize, minSeconds, frameBytes); + if (run >= 0) + { + samples[target].Add(sample); + } + } + } + + var results = targets.Select(target => Summarize(target, samples[target])).ToArray(); + foreach (var result in results) + { + Console.WriteLine( + $" {result.Target,-10} median={result.MedianMBps,8:F1} MB/s min={result.MinMBps,8:F1} " + + $"max={result.MaxMBps,8:F1} spread={result.SpreadPercent,5:F1}% " + + $"alloc={result.AllocatedBytesPerMiB / 1024.0,9:F1} KiB/MiB gc0/1/2 per MiB={result.Gen0PerMiB:F2}/{result.Gen1PerMiB:F2}/{result.Gen2PerMiB:F2} " + + $"retained={result.RetainedMiB:F1} MiB"); + } + + string? verdict = null; + double? ratio = null; + var builtin = results.FirstOrDefault(result => result.Target == Builtin); + var reference = results.FirstOrDefault(result => result.Target == GhosttyVt); + if (builtin is not null && reference is not null) + { + ratio = builtin.MedianMBps / reference.MedianMBps; + verdict = builtin.MinMBps > reference.MaxMBps ? "faster" + : builtin.MaxMBps < reference.MinMBps ? "slower" + : "overlap"; + Console.WriteLine($" builtin vs ghostty-vt: {ratio:F2}x median throughput, verdict={verdict}"); + if (verdict != "faster") + { + exitCode = 1; + } + } + + report.Workloads.Add(new VtWorkloadReport(name, bytes.Length, sha, equivalence, results, ratio, verdict)); + } + + if (jsonPath is not null) + { + Directory.CreateDirectory(Path.GetDirectoryName(Path.GetFullPath(jsonPath))!); + File.WriteAllText(jsonPath, JsonSerializer.Serialize(report, JsonOptions)); + Console.WriteLine($"wrote {jsonPath}"); + } + + // A non-zero exit means "builtin did not beat ghostty-vt on every workload"; + // scripts use it as a gate, so --no-gate keeps a measurement-only run green. + return Array.IndexOf(args, "--no-gate") >= 0 ? 0 : exitCode; + } + + private static readonly JsonSerializerOptions JsonOptions = new() + { + WriteIndented = true, + PropertyNamingPolicy = JsonNamingPolicy.CamelCase, + DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull, + }; + + private static string[] Split(string? value, string[] fallback) => + value is null || value == "all" + ? fallback + : value.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries); + + [MethodImpl(MethodImplOptions.NoInlining)] + private static VtSample Measure( + string target, + byte[] bytes, + int columns, + int rows, + int chunkSize, + double minSeconds, + int frameBytes) + { + GC.Collect(); + GC.WaitForPendingFinalizers(); + GC.Collect(); + var memoryBefore = PrivateBytes(); + using var engine = Create(target, columns, rows); + var gen0 = GC.CollectionCount(0); + var gen1 = GC.CollectionCount(1); + var gen2 = GC.CollectionCount(2); + var allocated = GC.GetAllocatedBytesForCurrentThread(); + var passes = 0; + var sinceFrame = 0L; + var start = Stopwatch.GetTimestamp(); + // Fast targets replay the corpus into the same engine until the sample is + // long enough for timer and scheduler noise to stop dominating it. + do + { + for (var offset = 0; offset < bytes.Length; offset += chunkSize) + { + var length = Math.Min(chunkSize, bytes.Length - offset); + engine.Feed(bytes.AsSpan(offset, length)); + sinceFrame += length; + if (frameBytes > 0 && sinceFrame >= frameBytes) + { + engine.Frame(); + sinceFrame = 0; + } + } + + passes++; + } + while (Stopwatch.GetElapsedTime(start).TotalSeconds < minSeconds); + + var elapsed = Stopwatch.GetElapsedTime(start); + var sample = new VtSample( + (long)bytes.Length * passes, + elapsed.TotalSeconds, + GC.GetAllocatedBytesForCurrentThread() - allocated, + GC.CollectionCount(0) - gen0, + GC.CollectionCount(1) - gen1, + GC.CollectionCount(2) - gen2, + 0); + GC.Collect(); + GC.WaitForPendingFinalizers(); + GC.Collect(); + return sample with { RetainedBytes = Math.Max(0, PrivateBytes() - memoryBefore) }; + } + + private static long PrivateBytes() + { + using var process = Process.GetCurrentProcess(); + return process.PrivateMemorySize64; + } + + private static IVtTarget Create(string target, int columns, int rows) => target switch + { + Builtin => new EngineTarget(new TerminalEngine(columns, rows)), + GhosttyDt => new EngineTarget(new GhosttyTerminalEngine(columns, rows)), + GhosttyVt => new RawGhosttyTarget(columns, rows), + _ => throw new ArgumentOutOfRangeException(nameof(target), target, null), + }; + + /// + /// Throughput is only comparable if both engines did the same work, so the + /// final visible screen and cursor must match. A mismatch is reported (with the + /// first differing row) rather than failing, because some workloads hit + /// documented engine differences such as grapheme-width policy. + /// + private static string CheckEquivalence(byte[] bytes, int columns, int rows, int chunkSize) + { + using var builtin = new TerminalEngine(columns, rows); + using var ghostty = new GhosttyTerminalEngine(columns, rows); + for (var offset = 0; offset < bytes.Length; offset += chunkSize) + { + var chunk = bytes.AsSpan(offset, Math.Min(chunkSize, bytes.Length - offset)); + builtin.Feed(chunk); + ghostty.Feed(chunk); + } + + var differing = 0; + var first = -1; + for (var row = 0; row < rows; row++) + { + var expected = builtin.Buffer.GetText(0, row, columns - 1, row).TrimEnd(); + var actual = ghostty.Buffer.GetText(0, row, columns - 1, row).TrimEnd(); + if (!string.Equals(expected, actual, StringComparison.Ordinal)) + { + differing++; + first = first < 0 ? row : first; + } + } + + var cursor = builtin.CursorX == ghostty.CursorX && builtin.CursorY == ghostty.CursorY + ? "cursor equal" + : $"cursor builtin=({builtin.CursorX},{builtin.CursorY}) ghostty=({ghostty.CursorX},{ghostty.CursorY})"; + var history = builtin.HistoryCount == ghostty.HistoryCount + ? $"history equal ({builtin.HistoryCount})" + : $"history builtin={builtin.HistoryCount} ghostty={ghostty.HistoryCount}"; + return differing == 0 + ? $"identical text, {cursor}, {history}" + : $"MISMATCH {differing}/{rows} rows (first row {first}), {cursor}, {history}"; + } + + private static VtTargetReport Summarize(string target, List samples) + { + static double Mebibytes(VtSample sample) => sample.Bytes / (1024.0 * 1024.0); + var throughput = samples.Select(sample => Mebibytes(sample) / sample.Seconds).Order().ToArray(); + var median = Median(throughput); + var fed = samples.Sum(Mebibytes); + return new VtTargetReport( + target, + median, + throughput[0], + throughput[^1], + (throughput[^1] - throughput[0]) / median * 100, + Median(samples.Select(sample => sample.AllocatedBytes / Mebibytes(sample)).Order().ToArray()), + samples.Sum(sample => sample.Gen0) / fed, + samples.Sum(sample => sample.Gen1) / fed, + samples.Sum(sample => sample.Gen2) / fed, + Median(samples.Select(sample => sample.RetainedBytes / (1024.0 * 1024.0)).Order().ToArray()), + throughput); + } + + private static double Median(double[] sorted) => + sorted.Length % 2 == 1 + ? sorted[sorted.Length / 2] + : (sorted[(sorted.Length / 2) - 1] + sorted[sorted.Length / 2]) / 2; + + private interface IVtTarget : IDisposable + { + void Feed(ReadOnlySpan data); + + // What a renderer does once per frame to read the screen. + void Frame(); + } + + private sealed class EngineTarget(ITerminalEngine engine) : IVtTarget + { + public void Feed(ReadOnlySpan data) => engine.Feed(data); + + public void Frame() => GC.KeepAlive(engine.CreateSnapshot()); + + public void Dispose() => engine.Dispose(); + } + + /// + /// The pinned libghostty-vt with only the scrollback limit configured to match + /// the builtin engine's 9,001 lines; no effects or projection. A frame is what + /// Ghostty's renderer does per frame: update its render state, then clean it. + /// + private sealed unsafe partial class RawGhosttyTarget : IVtTarget + { + private const int ScrollbackBytesOption = 27; + private const int ScrollbackLinesOption = 28; + private nint _terminal; + private nint _renderState; + + public RawGhosttyTarget(int columns, int rows) + { + Check(TerminalNew(0, out _terminal, checked((ushort)columns), checked((ushort)rows)), "ghostty_terminal_new"); + try + { + Check(TerminalSet(_terminal, ScrollbackBytesOption, null), "remove scrollback byte limit"); + var lines = (nuint)9001; + Check(TerminalSet(_terminal, ScrollbackLinesOption, &lines), "set scrollback line limit"); + Check(RenderStateNew(0, out _renderState), "ghostty_render_state_new"); + } + catch + { + Dispose(); + throw; + } + } + + public void Feed(ReadOnlySpan data) + { + fixed (byte* bytes = data) + { + TerminalWrite(_terminal, bytes, (nuint)data.Length); + } + } + + public void Frame() + { + Check(RenderStateUpdate(_renderState, _terminal), "ghostty_render_state_update"); + Check(RenderStateClean(_renderState), "ghostty_render_state_clean"); + } + + public void Dispose() + { + if (_renderState != 0) + { + RenderStateFree(_renderState); + _renderState = 0; + } + + if (_terminal != 0) + { + TerminalFree(_terminal); + _terminal = 0; + } + } + + private static void Check(int result, string operation) + { + if (result != 0) + { + throw new InvalidOperationException($"{operation} failed with {result}"); + } + } + + [LibraryImport("ghostty-vt", EntryPoint = "ghostty_terminal_new")] + [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] + private static partial int TerminalNew(nint allocator, out nint terminal, ushort columns, ushort rows); + + [LibraryImport("ghostty-vt", EntryPoint = "ghostty_terminal_set")] + [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] + private static partial int TerminalSet(nint terminal, int option, void* value); + + [LibraryImport("ghostty-vt", EntryPoint = "ghostty_terminal_vt_write")] + [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] + private static partial void TerminalWrite(nint terminal, byte* bytes, nuint length); + + [LibraryImport("ghostty-vt", EntryPoint = "ghostty_terminal_free")] + [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] + private static partial void TerminalFree(nint terminal); + + [LibraryImport("ghostty-vt", EntryPoint = "ghostty_render_state_new")] + [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] + private static partial int RenderStateNew(nint allocator, out nint state); + + [LibraryImport("ghostty-vt", EntryPoint = "ghostty_render_state_update")] + [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] + private static partial int RenderStateUpdate(nint state, nint terminal); + + [LibraryImport("ghostty-vt", EntryPoint = "ghostty_render_state_clean")] + [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] + private static partial int RenderStateClean(nint state); + + [LibraryImport("ghostty-vt", EntryPoint = "ghostty_render_state_free")] + [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] + private static partial void RenderStateFree(nint state); + } + + private readonly record struct VtSample( + long Bytes, + double Seconds, + long AllocatedBytes, + int Gen0, + int Gen1, + int Gen2, + long RetainedBytes); + + private sealed record VtReport( + DateTimeOffset Timestamp, + string Machine, + string Os, + string Architecture, + int ProcessorCount, + string Runtime, + int Columns, + int Rows, + int ChunkBytes, + int FrameBytes, + int Runs, + List Workloads); + + private sealed record VtWorkloadReport( + string Name, + long Bytes, + string Sha256, + string? Equivalence, + VtTargetReport[] Targets, + double? BuiltinVsGhosttyVt, + string? Verdict); + + private sealed record VtTargetReport( + string Target, + double MedianMBps, + double MinMBps, + double MaxMBps, + double SpreadPercent, + double AllocatedBytesPerMiB, + double Gen0PerMiB, + double Gen1PerMiB, + double Gen2PerMiB, + double RetainedMiB, + double[] RunsMBps); +} diff --git a/tools/Devolutions.Terminal.Bench/VtWorkloads.cs b/tools/Devolutions.Terminal.Bench/VtWorkloads.cs new file mode 100644 index 0000000..82ce6dd --- /dev/null +++ b/tools/Devolutions.Terminal.Bench/VtWorkloads.cs @@ -0,0 +1,397 @@ +using System.Text; + +namespace Devolutions.Terminal.Bench; + +/// +/// Deterministic VT byte streams for engine-ingest comparisons. Each workload +/// isolates one kind of terminal work so a regression or a gap against Ghostty +/// points at a code path instead of an average. The shapes follow +/// alacritty/vtebench (dense/light cells, cursor motion, scrolling regions, +/// unicode) and Ghostty's own `ghostty-gen` generators (+ascii, +utf8, +osc). +/// Generation is seeded, so the printed SHA-256 identifies the exact bytes. +/// +internal static class VtWorkloads +{ + private const string Esc = "\u001b"; + private const int Seed = 0x5EED; + + public static readonly string[] Names = + [ + "ascii", "ascii-lines", "utf8", "cjk", "emoji", "styled", "dense-cells", "light-cells", "cursor-motion", + "scrolling", "scroll-region", "osc", "mixed", + ]; + + public static byte[] Build(string name, int targetBytes, int columns, int rows) => name switch + { + "ascii" => AsciiStream(targetBytes), + "ascii-lines" => Ascii(targetBytes, columns), + "utf8" => Utf8(targetBytes, columns), + "cjk" => SingleScript(targetBytes, columns, 0x4E00, 0x9FFF), + "emoji" => SingleScript(targetBytes, columns, 0x1F600, 0x1F64F), + "styled" => Styled(targetBytes), + "dense-cells" => DenseCells(targetBytes, columns, rows), + "light-cells" => LightCells(targetBytes, columns, rows), + "cursor-motion" => CursorMotion(targetBytes, columns, rows), + "scrolling" => Scrolling(targetBytes), + "scroll-region" => ScrollRegion(targetBytes, columns, rows), + "osc" => Osc(targetBytes, columns), + "mixed" => Program.BuildMixedCorpus(targetBytes), + _ => throw new ArgumentException( + $"unknown workload '{name}' (expected {string.Join('|', Names)})"), + }; + + public static string Describe(string name) => name switch + { + "ascii" => "unbroken 0x21-0x7E stream relying on autowrap (`ghostty-gen ascii`)", + "ascii-lines" => "printable ASCII lines with CRLF (`cat` of a log or source file)", + "utf8" => "mixed-width UTF-8 lines: Latin, Greek, CJK, emoji, ZWJ, combining marks", + "cjk" => "lines of CJK ideographs only (wide, 3-byte UTF-8)", + "emoji" => "lines of emoji only (wide, 4-byte UTF-8, Extended_Pictographic)", + "styled" => "CRLF lines of short styled runs, SGR + OSC 8 + wide text (`ghostty-gen styled`)", + "dense-cells" => "full-screen repaints, every cell with its own truecolor/256 SGR", + "light-cells" => "full-screen repaints of plain text from the home position", + "cursor-motion" => "TUI-style CUP jumps with short writes and EL, no scrolling", + "scrolling" => "short lines that scroll the whole screen into history (`yes`)", + "scroll-region" => "DECSTBM partial-screen region scrolling (`less`, vim splits)", + "osc" => "OSC titles, OSC 8 hyperlinks and OSC 133 prompt marks around text", + "mixed" => "the legacy `engine`/`render` corpus (70% ASCII, SGR, CJK, CUP)", + _ => name, + }; + + private static byte[] AsciiStream(int targetBytes) + { + var random = new Random(Seed); + return Emit(targetBytes, writer => + { + for (var index = 0; index < 1024; index++) + { + writer.Append((char)random.Next(0x21, 0x7F)); + } + }); + } + + /// + /// Mirrors the defaults of Ghostty's `styled` generator: 40-80 character lines + /// split into 4-16 character runs, an SGR at a run boundary half of the time + /// drawn uniformly from eight kinds (quantized RGB keeps the style set + /// bounded), occasional OSC 8 links, and ASCII/Latin/CJK/emoji text with rare + /// combining marks. Style and link are closed at the end of each line. + /// + private static byte[] Styled(int targetBytes) + { + var random = new Random(Seed); + int[] levels = [0, 51, 102, 153, 204, 255]; + int[] attributes = [1, 3, 4, 7, 9]; + var link = 0; + return Emit(targetBytes, writer => + { + var length = random.Next(40, 81); + var linked = false; + var written = 0; + while (written < length) + { + if (random.Next(2) == 0) + { + writer.Append(random.Next(8) switch + { + 0 => $"{Esc}[0m", + 1 => $"{Esc}[{random.Next(30, 38)}m", + 2 => $"{Esc}[{random.Next(40, 48)}m", + 3 => $"{Esc}[38;5;{random.Next(256)}m", + 4 => $"{Esc}[48;5;{random.Next(256)}m", + 5 => $"{Esc}[38;2;{levels[random.Next(6)]};{levels[random.Next(6)]};{levels[random.Next(6)]}m", + 6 => $"{Esc}[48;2;{levels[random.Next(6)]};{levels[random.Next(6)]};{levels[random.Next(6)]}m", + _ => $"{Esc}[{attributes[random.Next(attributes.Length)]}m", + }); + } + + if (random.Next(50) == 0) + { + writer.Append(linked + ? $"{Esc}]8;;\u0007" + : $"{Esc}]8;;https://example.test/{link++}\u0007"); + linked = !linked; + } + + var run = Math.Min(random.Next(4, 17), length - written); + for (var index = 0; index < run; index++) + { + var roll = random.Next(100); + if (roll < 80) + { + writer.Append(random.Next(100) < 15 ? ' ' : (char)random.Next(0x21, 0x7F)); + } + else if (roll < 90) + { + writer.Append((char)random.Next(0x00C0, 0x0250)); + } + else if (roll < 97) + { + writer.Append((char)random.Next(0x4E00, 0x5000)); + } + else + { + writer.Append(char.ConvertFromUtf32(random.Next(0x1F600, 0x1F650))); + } + + if (random.Next(100) == 0) + { + writer.Append((char)random.Next(0x0300, 0x0370)); + } + } + + written += run; + } + + if (linked) + { + writer.Append($"{Esc}]8;;\u0007"); + } + + writer.Append($"{Esc}[0m\r\n"); + }); + } + + private static byte[] Ascii(int targetBytes, int columns) + { + var random = new Random(Seed); + return Emit(targetBytes, writer => + { + var length = random.Next(0, columns * 2); + for (var index = 0; index < length; index++) + { + writer.Append((char)random.Next(0x20, 0x7F)); + } + + writer.Append("\r\n"); + }); + } + + private static readonly (int Start, int End)[] Utf8Ranges = + [ + (0x00A1, 0x00FF), // Latin-1 supplement + (0x0391, 0x03C9), // Greek + (0x0410, 0x044F), // Cyrillic + (0x4E00, 0x4FFF), // CJK ideographs (wide) + (0x3041, 0x3096), // Hiragana (wide) + (0x1F600, 0x1F64F), // emoji (wide) + (0x2500, 0x257F), // box drawing + ]; + + private static byte[] Utf8(int targetBytes, int columns) + { + var random = new Random(Seed); + var started = false; + return Emit(targetBytes, writer => + { + if (!started) + { + // DECSET 2027 (grapheme clustering): without it Ghostty measures a ZWJ + // sequence per codepoint while the builtin engine clusters it, and the + // two screens would wrap differently. + writer.Append($"{Esc}[?2027h"); + started = true; + } + + var length = random.Next(0, columns); + for (var index = 0; index < length; index++) + { + var roll = random.Next(100); + if (roll < 30) + { + writer.Append((char)random.Next(0x21, 0x7F)); + } + else if (roll < 33 && writer.Length > 0) + { + // Combining acute accent on the previous base character. + writer.Append('\u0301'); + } + else if (roll < 35) + { + // ZWJ family sequence: one grapheme cluster, several codepoints. + writer.Append("\U0001F468\u200D\U0001F469\u200D\U0001F467"); + } + else + { + var (start, end) = Utf8Ranges[random.Next(Utf8Ranges.Length)]; + writer.Append(char.ConvertFromUtf32(random.Next(start, end + 1))); + } + } + + writer.Append("\r\n"); + }); + } + + private static byte[] SingleScript(int targetBytes, int columns, int first, int last) + { + var random = new Random(Seed); + return Emit(targetBytes, writer => + { + // Wide characters: at most a row's worth per line. + var length = random.Next(0, columns / 2); + for (var index = 0; index < length; index++) + { + writer.Append(char.ConvertFromUtf32(random.Next(first, last + 1))); + } + + writer.Append("\r\n"); + }); + } + + private static byte[] DenseCells(int targetBytes, int columns, int rows) + { + var random = new Random(Seed); + var frame = 0; + return Emit(targetBytes, writer => + { + writer.Append($"{Esc}[H"); + for (var row = 0; row < rows; row++) + { + for (var column = 0; column < columns; column++) + { + var attribute = random.Next(4) switch + { + 0 => "1;", + 1 => "3;", + 2 => "4;", + _ => "", + }; + if (random.Next(2) == 0) + { + writer.Append( + $"{Esc}[{attribute}38;2;{random.Next(256)};{random.Next(256)};{random.Next(256)};" + + $"48;2;{random.Next(256)};{random.Next(256)};{random.Next(256)}m"); + } + else + { + writer.Append($"{Esc}[{attribute}38;5;{random.Next(256)};48;5;{random.Next(256)}m"); + } + + writer.Append((char)('A' + ((frame + row + column) % 26))); + } + } + + writer.Append($"{Esc}[0m"); + frame++; + }); + } + + private static byte[] LightCells(int targetBytes, int columns, int rows) + { + var frame = 0; + return Emit(targetBytes, writer => + { + writer.Append($"{Esc}[H"); + for (var row = 0; row < rows; row++) + { + for (var column = 0; column < columns; column++) + { + writer.Append((char)('a' + ((frame + row + column) % 26))); + } + } + + frame++; + }); + } + + private static byte[] CursorMotion(int targetBytes, int columns, int rows) + { + var random = new Random(Seed); + return Emit(targetBytes, writer => + { + var row = random.Next(1, rows + 1); + var column = random.Next(1, columns + 1); + writer.Append($"{Esc}[{row};{column}H"); + switch (random.Next(4)) + { + case 0: + writer.Append($"{Esc}[K"); + break; + case 1: + writer.Append($"{Esc}[{random.Next(31, 38)}m{random.Next(1000):D3}%{Esc}[0m"); + break; + default: + writer.Append((char)random.Next(0x21, 0x7F)); + break; + } + }); + } + + private static byte[] Scrolling(int targetBytes) => + Emit(targetBytes, writer => writer.Append("y\n")); + + private static byte[] ScrollRegion(int targetBytes, int columns, int rows) + { + var random = new Random(Seed); + var top = Math.Max(1, rows / 4); + var bottom = Math.Max(top + 1, rows - 2); + var header = $"{Esc}[{top};{bottom}r{Esc}[{bottom};1H"; + var started = false; + return Emit(targetBytes, writer => + { + if (!started) + { + writer.Append(header); + started = true; + } + + var length = random.Next(1, columns); + for (var index = 0; index < length; index++) + { + writer.Append((char)random.Next(0x20, 0x7F)); + } + + writer.Append("\r\n"); + }); + } + + private static byte[] Osc(int targetBytes, int columns) + { + var random = new Random(Seed); + var line = 0; + return Emit(targetBytes, writer => + { + switch (random.Next(4)) + { + case 0: + writer.Append($"{Esc}]0;job {line} - {random.Next(100)}% complete\u0007"); + break; + case 1: + writer.Append($"{Esc}]133;A\u0007PS C:\\src\\{line}> {Esc}]133;B\u0007dir{Esc}]133;C\u0007"); + break; + case 2: + writer.Append($"{Esc}]133;D;0\u0007"); + break; + default: + writer.Append( + $"{Esc}]8;id={line};https://example.test/item/{random.Next(1_000_000)}{Esc}\\" + + $"item-{line}{Esc}]8;;{Esc}\\"); + break; + } + + var length = random.Next(0, columns / 2); + for (var index = 0; index < length; index++) + { + writer.Append((char)random.Next(0x20, 0x7F)); + } + + writer.Append("\r\n"); + line++; + }); + } + + private static byte[] Emit(int targetBytes, Action next) + { + var builder = new StringBuilder(); + using var stream = new MemoryStream(targetBytes + 64 * 1024); + while (stream.Length < targetBytes) + { + builder.Clear(); + next(builder); + var bytes = Encoding.UTF8.GetBytes(builder.ToString()); + stream.Write(bytes); + } + + return stream.ToArray(); + } +}