Repository navigation
perf(executor): compile a crosswalk once and fuse its single-path rules - #66
Merged
Merged
Conversation
The bench builds documents of several sizes and runs encode and decode for crosswalks of identity moves, with and without format, scale and enum rules, beside a hand-written function giving the same output. It prints the ratio of the two medians from one run on one machine and the input reads each side makes per node, counted through a Proxy, so the figures are work done rather than time.
The reference is the executor as it stands, with its gap search, kept in test support and not shipped. Property tests built on fast-check generate crosswalks of every op, path shape and gate, and documents with absent paths, empty and sparse arrays, fixed indices and values that are not JSON, then require the same output (key order included) or the same error class, message, rule index and path from both, in both directions. Each property also requires that every kind of outcome was met.
…bject compileCrosswalk validates a crosswalk, parses its paths and resolves each rule's sides and operation for both directions, returning a codec whose encode and decode take only the document. encode, decode and assertRoundTrip keep the compiled form in a WeakMap keyed by the crosswalk object, so existing callers validate once however many documents they run. An invalid crosswalk is not kept, so every attempt throws again.
…ad it When every two-way rule in a direction is a single-path op over keys and at most one wildcard, gated only on the whole rule, with write paths that never run through one another and each output array fed by one source array, the rules sharing an array are applied element by element in one walk, writing straight into the output element. A move carries its value with no call, a record moved whole is stored in one step, and no trail is built per element. Anything the fused run does not decide for itself (an absent path, a hole, a value an op rejects, a when that throws) makes it give way to the rule-at-a-time run, which then gives the exact error or result. The gap search runs only where a stored value or a one-way write can leave a gap. A property test requires that the fused run never gives way where the reference completes.
The output used to go through the JSON value schema's parse, which traverses and copies the whole document. isExactJson accepts exactly the values that parse would give back unchanged (finite numbers, plain records and ordinary arrays with nothing the schema drops or rejects), so an output it accepts is returned as it is and any other is parsed, which throws or copies just as before. The fused run checks each value as it stores it and gives way on one that is not exact, so a completed fused run needs neither this check nor the gap search. A record or array a move carries whole is now shared with the input rather than copied.
…on failure The gap search copied its trail into a new array at every element and record it entered, and listed every record's entries; it now keeps one trail for the whole walk, enters only records and arrays, and returns the trail it found. A write formatted its full path, and every intermediate container's, for an error that cannot be thrown; the path is now formatted only when that error is raised. Both serve the rule-at-a-time executor and any run with a one-way write.
…s from A move, scale, enum, format or split rule may omit to, meaning the same path as from, so an identity rule names its path once. Parsing fills the path in, so every parsed crosswalk carries to and nothing past the schema changes. The JSON Schema is now generated from the input side of the Zod schemas, which also stops it requiring the rules and oneWay lists that parseCrosswalk already defaults.
keep is an optional top-level list of paths, each meaning the identity move from that path to itself. parseCrosswalk appends those moves after rules in the order listed and drops the list, so validation, execution, round trips, rule numbering and the path-uniqueness check see only rules. A field that no rule or kept path names still does not survive encode. A property test generates each crosswalk twice, once with keep and once with explicit moves, and requires the same encode and decode outcome from both and from the reference executor.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins. |
|
🎉 This PR is included in version 0.9.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
encodeon a crosswalk of identity moves ran about 360 times a hand-written copy over 300k nodes, because every call re-validated the crosswalk, ran each rule over the whole document with a trail per element, and parsed the output through the JSON schema (a full copy). It is now about 3 to 9 times the copy for identity moves and about 2 times with format, scale and enum rules (pnpm bench, ratio to a same-machine baseline plus input reads per node).The crosswalk is compiled once per crosswalk object (
compileCrosswalk, memoised in a WeakMap, so existing callers ofencode,decodeandassertRoundTripvalidate once). When every two-way rule is a single-path op over keys and at most one wildcard, the rules reading the same array are applied in one walk and write straight into the output element. Anything the fused run cannot decide itself (absent path, hole, a value an op rejects, a throwingwhen) gives way to the rule-at-a-time run, so errors and results are unchanged. The output is checked withisExactJson, which accepts exactly what the schema parse returns unchanged, instead of being copied.A frozen copy of the old executor lives in test support, and fast-check properties require the same output (key order included) or the same error class, message, rule index and path from both, in both directions. A moved record or array is now shared with the input rather than copied.
Two additions that fell out of writing identity crosswalks:
tomay be omitted on a single-path rule when it equalsfrom, and a top-levelkeeplist of paths means the identity move for each.keepis expanded at parse time, so validation and rule numbering only see rules. The JSON Schema is now generated from the input side of the Zod schemas.