Optimize parser hot paths: token window, lexer scanning, and allocations

[kyleconroy]

A profiling-driven performance pass over the parse benchmarks. No observable behavior changes.

Benchmarks

Medians over 6 runs on go1.26:

Benchmark	Time	Allocations
SysbenchSelect	3.33µs → 2.80µs (−16%)	20 → 15 (−25%)
ParseComplex	147µs → 112µs (−24%)	592 → 468 (−21%)
ParseSimple	10.0µs → 8.4µs (−16%)	56 → 49 (−12%)

Changes

• parser/rd_parser.go — current-token cache. advance and rewind are the only movers of the cursor, so the parser keeps a pointer to the current token's window slot and tok()/cur() — the hottest calls in the parser (~36% of CPU cumulative before this change) — become call-free field loads. at() is split into an inline-friendly fast path plus a fill() slow path, and lexOne writes each lexed token into its window slot in place instead of building the ~112-byte rdToken on the stack and copying it into the window.
• parser/lexer.go, parser/misc.go — table-driven scanning. Identifier, digit, and whitespace runs were scanned through incAsLongAs, an indirect closure call per byte. They now use dedicated loops over 256-entry lookup tables; identifier and digit characters can't contain \n, so position tracking simplifies to offset/column bumps.
• parser/parse_func.go — one block per column reference. ColumnNameExpr and its ColumnName — the most-allocated pair in typical queries, 28% of all objects — are allocated together as a single block; the two objects always live and die together.
• ast/base.go — SetText skips the sync.Once when conversion is a no-op. The per-node &sync.Once{} was 10% of all objects. When the lazy text conversion is provably the identity (no quote characters, and the encoding is utf8-with-valid-text, binary, or latin1 — all of which return their input unchanged from Transform), Text() serves the original string directly. No exported field or behavior of the ast package changes.

What remains on the profile is allocator/GC work for the AST nodes themselves plus NewCIStr lowercasing, both inherent to the AST's public shape, so the pass stops here rather than reach for arena-style node allocation, which would change memory-retention behavior for consumers.

Validation

• go test ./... -count=1 -timeout 120s — all packages pass, including the error-fidelity goldens (TestRDErrorFidelity) and the file-driven parser tests.
• go vet ./... clean.
• go test -race ./parser ./ast passes.

🤖 Generated with Claude Code

https://claude.ai/code/session_014LQ3NoipqqU17kRVkBcDRY

---

Generated by Claude Code