Repository navigation
HermitCrab optimization round record: what was tried, and a faster package that is not mergeable - #490
HermitCrab optimization round record: what was tried, and a faster package that is not mergeable#490johnml1135 wants to merge 34 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #490 +/- ##
=======================================
Coverage 74.57% 74.57%
=======================================
Files 453 453
Lines 38314 38314
Branches 5280 5280
=======================================
Hits 28573 28573
Misses 8576 8576
Partials 1165 1165 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
004e199 to
3252bbc
Compare
johnml1135
changed the title
|
Two further optimization ideas were investigated against this ledger, on a branch based on Neither appears in the existing 22 rows. Row 5 covers deterministic synthesis fold sharing and the rejected two-pass experiment covers nondeterministic traversal; neither is the idea below. Row 6 is the synthesis-side sibling of the second idea. Idea A — a
|
| grammar | words | rule attempts off → on | Σ det arc checks off → on | suppressed | Σ nondet arc checks | det share of FST work |
|---|---|---|---|---|---|---|
| Indonesian | 121 | 15,040 → 15,040 | 34,386 → 34,386 | 0 | 990,111 | 3.4 % |
| Mbugwe | 37 | 4,351,113 → 4,351,113 | 760,833 → 760,829 | 4 | 21,334,695 | 3.4 % |
| Sena | 35 | 22,416,707 → 22,416,707 | 194,014 → 192,515 | 192 | 124,891,129 | 0.16 % |
| Amharic | 37 | 71,315 → 71,315 | 9,979 → 9,979 | 0 | 44,559,287 | 0.02 % |
| Aweti | 22 | 24,274,820 → 24,274,820 | 192,827 → 192,779 | 8 | 97,502,953 | 0.20 % |
Parse-signature multisets identical on every completed word. The reason it is a no-op is structural: the deterministic traversal only runs on the synthesis side (analysis affix matchers set AllSubmatches, analysis rewrite/metathesis set Nondeterministic), and synthesis shapes carry almost no optional annotations, so the skip-vs-consume fork it would collapse rarely occurs. Deterministic traversal is 0.02–3.4 % of all CheckInputMatch calls, and dedup removes ≤ 0.8 % of that.
Where the duplicate explosion the idea was aiming at actually lives is the nondeterministic set, which already exists: on Amharic it suppressed 30.1 M candidate instances against 15.2 M pushed (66 % duplicates), and on one heavy word 82.7 M against 38.5 M.
Two latent issues noticed while auditing the existing keys, neither reachable from HermitCrab today:
NondeterministicFstTraversalMethod's key carries the output action list but not which annotations the actions applied to. Two paths through optional annotations can share state, index, registers and action sequence while having modified different output annotations. Unreachable because no HermitCrab FST carriesIFstOperations.- Both nondeterministic keys omit
VariableBindings, whichCheckInputMatchreads. Whether two equal(State, AnnotationIndex, Registers)keys can carry different bindings on a real pattern is untested. - Both store the mutable
Register[,]directly in the key whileExecuteCommandslater mutates it. This can only cause a missed dedup, never a wrong one.
Idea B — canonicalize the order of independent rules in an Unordered analysis stratum
The analysis-side counterpart of row 6. In an Unordered stratum the cascade explores both [A,B] and [B,A]; if A and B commute, one order is redundant work. The prune skips a dispatch when the candidate is orthogonal to the trail's last rule and precedes it in source order — adjacent-transposition normalization, with non-orthogonal rules and templates acting as barriers.
Orthogonality was implemented as three cumulative levels — L0 shape (pure prefix vs pure suffix, single application, edge-blind stem pattern, no ModifyFromInput/variables), L1 + syntactic-feature commutation decided by enumerating the finite feature lattice the two rules mention in both the analysis (IsUnifiable gate then Add/Clear) and synthesis (Unify then PriorityUnion) directions, L2 + MPR/stem-name/partial/blocking. A second, independently written census over the XML agreed with the object-model implementation on every unordered stratum: 176 L0 pairs, 60 L1, 36 L2 across the five grammars.
Measured: 0.00–0.05 % of rule-unapplication attempts, parity clean on every word.
| grammar | words | attempts off → on | reduction | dispatches | skipped |
|---|---|---|---|---|---|
| Indonesian | 121 | 15,040 → 15,040 | 0.00 % | 21,030 | 0 |
| Mbugwe | 37 | 4,351,113 → 4,351,113 | 0.00 % | 1,631,322 | 858 |
| Sena | 35 | 22,416,707 → 22,404,555 | 0.05 % | 6,603,760 | 12,104 |
| Amharic | 29 | 36,774 → 36,758 | 0.04 % | 27,264 | 16 |
| Aweti | 12 | 3,014,439 → 3,014,439 | 0.00 % | 1,213,430 | 0 |
| Sena at L0 (unsound control, same 35 words) | 35 | 25,823,709 → 24,179,995 | 6.37 % | 7,442,610 | 67,090 |
The L0 control is what makes this a finding rather than a null result: widening Sena from 2 qualifying pairs to 52 moves the reduction from 0.05 % to 6.37 %, so the limit is the predicate, not the hook. But each narrowing step is load-bearing — of Sena's 52 L0 pairs, 30 fail L1 because their syntactic features genuinely do not commute, and 20 fail L2 on the partial-rule clause, which is necessary there because that stratum holds 24 affix templates so IsLastAppliedRuleFinal takes a value and both final-template gates in SynthesisAffixProcessRule can fire. 6.37 % is what an incorrect prune wins.
A trail-depth census explains why widening cannot reach further. Instrumenting every dispatch with the length of the trail it extends:
- Mbugwe, 155,672 dispatches: 68 % sit on trails 9+ rules deep, but at only 0.8 % does the trail's last rule have any orthogonal partner even at L0.
- A heavy Sena word, 76,777 dispatches: 67 % at depth 9+, 16.6 % partner-available, 2.3 % actually skipped → the 6.37 %.
So the permutation blow-up is real and deep, but the rules filling those trails are ones the criterion excludes on shape (Mbugwe rejects 57 pairs MultiInsert, 17 AllomorphOutput, 21 RuleKind). The largest excluded class is same-side pairs (Sena: 84 rejections, the C(13,2) suffix pairs), and those can never commute — stem+A+B ≠ stem+B+A is arithmetic, not conservatism.
One consequence worth separating from performance. The prune is not signature-preserving and cannot be: a parse signature carries MorphemesInApplicationOrder, so unapplying a prefix before a suffix and the reverse are two distinct parses of one word even when the rules commute. What it preserves is the morph sequence and the root; what it drops is derivational history. Those two rows are indistinguishable to a reader, so this is an ambiguity reduction as much as an optimization — and it would need the correctness treatment rather than being presented as invisible.
This is now the fifth instance of the pattern in this ledger's "lesson that generalises": apparent redundancy that evaporates once the key is complete.
On rows 10 and 11 — reachability
The first reading of our data looked like it reopened the lexical reachability gate: the heavy Sena word made 272,152 unapplication attempts and the Aweti word 505,600, each finding zero roots. It does not reopen it. Those lexical-lookup counts come from runs truncated by a step budget, so they show the search never reached lookup within budget, not that lookup would have failed. Row 11's finding — that the expensive words fail on checks running after lexical lookup succeeds — is untouched by anything measured here.
On the complexity-cap step budget
b3fd2b55 was cherry-picked to bound the pathological words. It works where the cost is many rule applications: an Aweti word that previously crashed a test host at 12.1 GB after 1,717 s completes in 50 s at 200,000 steps, and whole-corpus runs go from 22/40 words to 40/40 at 757 MB peak instead of 12 GB.
It does not bound a word whose cost is one expensive traversal. An Amharic word ran 2,280 s under a 200,000-step budget with the nondeterministic counters frozen at 217,493 traversals and 53.9 M arc checks across successive samples — stuck inside a single matcher call. The budget is checked between leaf-rule applications and cannot interrupt one. Anyone relying on it as a safety bound should know that.
Also worth flagging for consumers: the budget ships on by default (2,000,000 steps, 10 s). A 10-second default silently truncates parses.
Reproduction
Branch based on master a4b2974. Measurements used a census harness that parses each word twice in one process with MaxDegreeOfParallelism: 1 and tracing on (so the analysis memo and MergeEquivalentAnalyses are off and counts are the unmemoized search), compares complete parse-signature multisets, and reports MorphologicalRuleAnalysis trace nodes as attempts and CheckInputMatch calls as arc checks. Grammars are FieldWorks GenerateHCConfig exports of five projects; word lists are the first 40 entries except Indonesian (all 121) and Sena (entries 1001–1040, since the head of that list is punctuation). Machine-safety aborts and test-host crashes are recorded as such and excluded from ratios, never counted as agreement.
|
Two observations from the same investigation that belong in the record but are not optimizations, so they did not fit the rows above. A repeatable memory wall at 12.2–12.6 GB, independent of any capAcross the five-grammar runs, 13 words (Sena and Aweti) ended as test-host crashes rather than completions, and they did so at a strikingly consistent managed heap: 12,225 / 12,259 / 12,270 / 12,278 / 12,282 / 12,312 / 12,323 / 12,324 / 12,336 / 12,340 / 12,359 / 12,365 / 12,563 MB. The consistency holds regardless of the per-process limit in force — including runs whose limit was 20 GB, so these are not the harness stopping them. The clustering at ~12.3 GB rather than at whatever ceiling was configured is the interesting part; it is the signature one would expect from a single collection outgrowing the 2 GB per-object limit rather than from address-space or RAM exhaustion. A crash dump on one of these words would name the collection. Reported as an observation only — no diagnosis attempted. For scale, one Aweti word (line 183 of its list) reached 13.2 GB and 30.1 M nondeterministic traversals / 139.3 M arc checks in 53 minutes without leaving analysis, and never reported a lexical lookup. Under a 200,000-step budget the same word completes in 231 s at 676 MB with 505,600 rule-unapplication attempts and 0 deterministic traversals. Where analysis time goes, by traversal familyThe counters added for this work split every
This is offered as calibration for anyone sizing a synthesis-side optimization: on the two grammars with the heaviest analysis, 99.8 % or more of FST work is on the nondeterministic side. It is consistent with the ledger's Amharic breakdown (analysis cascade ~95 % of analysis, all synthesis 0.3 %), measured independently and at the arc-check level rather than by wall-time bucket. |
|
Added one record from a September round that post-dates this set: Three things it contributes to this ledger:
Also closes the two extensions before anyone builds them — environment matchers are 0.20%/1.18% of traversal instances on Amharic/Mbugwe (this is #515), and the analysis rules' 93.95% is 99.7% alternate captures because
🤖 Generated with Claude Code |
2026-09-23 round: the missing Aweti speedup was reduplication, not traversalOne new row closes the gap #511 opened. Aweti index 182 had never completed here, on any stack we tried (#491, #494, #511, FieldWorks #1148, forced final templates, FLEx's parallel/no-memo mode). A copy-agreement prune in reduplication unapplication makes it finish in 9 s, and the whole Aweti list runs 4.95× faster with 0 divergences across five grammars. Proposed as #519 (on by default) and shipped in PanGloss v0.4.0. New row
Why the ledger missed it: every prior row measured Sena, Mbugwe and Amharic, where full-copy rules are absent or rare. The 19 docs mention reduplication once (the probe-design classifier, marked "Unknown"). What else this round settled
PanGloss v0.4.0 (same prune, Rust)Release binaries, same machine, v0.3.3 vs v0.4.0: identical analyses on all 7,455 words both complete across the five grammars. Aweti: 172 -> 191 words complete within 20 s, 5.3x on the 172 shared words, median word 188 -> 26 ms, word-list index 182 60 s timeout -> 2.7 s. Sena, Amharic and Mbugwe are within noise to about 10% slower; a per-call allocation on rules that copy nothing is being removed. Open since this round
Method notes worth keeping
🤖 Generated with Claude Code |
47318eb to
5bbfa01
Compare
johnml1135
changed the title
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Archived from HermitCrab search cost, 2026-10-21 roundDisposition: three large reductions in search work stand up. Two of them were bugs in the Rust This record follows the #490 (comment): one entry per Data and unitsGrammars. The public reference grammars are Sena, Amharic, Mbugwe, Aweti and Indonesian. The Units. The owner rule for this round is that gains are stated as parser work: "N% less work
Parity comparators. HC-Rust used Runs. Unless noted, HC-Rust runs were What stands up
The census instrumentation that settled the rejected lead is The lesson: diff the port on the oracle's own export before optimizing the algorithmThe bugs were found by a comparison, not by a profile. HC-Rust normally loads a FieldWorks project
These counts were supplied by the round's brief; this write-up did not reproduce them. A ~90x gap
Neither fix is a C# optimization. Before the fixes HC-Rust did more work than C#; afterwards it
One residual is not explained. On the slowest word, HC-Rust on C#'s export made 18 lookups where Method rule. Before measuring an algorithmic lead on a port, run the port on the oracle's own C1: prune analysis branches that cannot reach a root (research lead #1) — rejected on its ceilingExpected. A sound summary of the remaining transformations would prune analysis nodes from Measured. A hindsight census charged each unapplication's steps and work to its input node.
Only the plain bucket is what a sound, compound-aware summary could actually remove at a profit. A
The census builds include fix A but predate change C. Template-heavy shares (Sena, and possibly Verdict: not built.
What is different from prior attempts.
C# transfer. C#'s Per-parse distinct-shape lookup — rejectedThe brief's "97,575 lookups over 56 shapes" came from trace mode, where analysis merging is off. The
C#'s C2: share trailing slot batches inside one battery call (research lead #2) — kept, ported to C#Expected. The battery walks each template's slots last-first. Templates whose trailing slots Battery share of all parser work (HC-Rust, before C):
Static census. This counts rule applications on the battery's own input, before and after
Dynamic ceiling, Sena. A trie-exact key (same trailing slots and same full word state) predicts HC-Rust (PanGloss entry 095)Run with
Tests.
C# (Machine PR #546)
The figures below are C# slot batches on deterministic 200-row samples, single-threaded and warmed,
These two engines measure different things, so do not compare the columns. C#'s "31.4% fewer slot Conformance pins. The C# 60/60 self-check below is What is different from prior attempts.
Rejected alongside. A rule-level memo per battery call was no bigger than slot-level sharing on Conformance state at the round's closeThe run used the PR #480 head, Machine
How today's measurements bear on the six research reportsThe integrated ranking is in the #490 (comment). This
Prediction scored. The research doc said the trace-mode lookup ratio could not establish an The finding the reports could not have made. Where it was measured, much of the private Open leads, not pursuedEach lead is tagged [C# analogue] where the same question applies to Work reductions, with a ceiling to measure first:
C# port follow-ons to #546 [C#]:
Time-only costs (no work units charged, so outside the owner metric): Identity over-approximations left in HC-Rust [HC-Rust]. Synthesis re-validates all of them, so
Unexplained observations: Data handlingPrivate-project word lists, per-word logs and exports stay on the measuring machine. This record |
|
Archived from HermitCrab analysis-side optimization, 2026-10-27 roundDisposition: seven candidates are PRs (R1 #575, R2 #572, T1 #571, R5 #574, R3 #576, R1 on #546 #577, R1 purity extension #579), one is a This record follows the #490 (comment). Ledger rows 38-46 point here. Unit and method
Decisions
Earlier-round PRs: #560 (E1, allomorph-loop cutoff) and #570 (MPR and stem-name trail replay). E3 (template What the census said to aim atFrom the #490 (comment) the plan took these facts:
Eight issues were filed in sillsdev/machine (#561-#568), each checked against open and closed issues and PRs for R1. Required-slot look-ahead: KEEP (issue #561, PR #575)Claim. A template partial that reaches a required slot where no rule can unapply dies there, after paying for Soundness. Shadow mode, 0 violations and 0 completions under a fired check:
Parses were equal per word, traced and merged. Mutation checks: removing the purity gate fails the impurity Event-based result (analysis events, fewer is better; synthesis steps unchanged):
Unification-work result. The unit is top-level
Mbugwe's large event saving is mostly walk and trace work that costs few unifications, so little of it shows in Waived kill rule. The plan said to close R1 if check unifications exceeded 3x the events saved on the grammar Tried and rejected. Skipping slots whose every allomorph inserts a single segment: it did not cut Mbugwe's Gates. HermitCrab tests 168 passed, 0 failed on the branch (master 151 passed). Conformance 146 passed and 4 Risks. Amharic is net more work in unification calls, on 24 words. The stress fixture is synthetic. PR #534 R2. Root-substring prune: KEEP (issue #562, PR #572)Claim. When a node's shape contains no root allomorph as a contiguous substring, and nothing that can still be Why it is not row 27. Row 27 closed root reachability summaries (the prunable bucket was 0-5% on 6 of 8 lists, Where the flag is on (first failing condition per stratum): Sena all three strata; the stress fixture's main Net of probes. After review, each probe is charged as one event:
The conformance result is the honest cost: it is a suite of tiny words and small lexicons, so the probes exceed the Probe skip. Added after review: the probe is skipped when the output keeps at least as many segments as its Rejected: a root-count gate. Turning the prune off for strata with fewer than 16 roots would have hidden the Gates. HermitCrab tests 161 passed, 0 failed (master 151). 10 new tests in Risks. Pruned analyses no longer count toward the analysis-side alternative limit, so a parse that threw T1. Lean trace: KEEP, lands after #534 (issue #563, PR #571)Claim. The default Retained trace memory, same word, same run, product arm against lean arm:
yɔɔna: 1,035.0 to 22.6 MiB (97.8% less); peak falls from 1,038 to 89 MiB. The kill criterion (under 50% on Steps are identical. Parse counts and node counts equal for every compared word; the census rows for Indonesian Landing order. PR #534 rewrites nearly every Gates. HermitCrab tests 156 passed, 0 failed (master 151). 5 new Risks. Lean text of an unfrozen word is taken at callback time; full mode formats at read time. An unfrozen R5. Edge-segment prefilter: KEEP (issue #564, PR #574)Claim. Port ledger row 22's analysis affix prefilter to master. A pattern anchored at both ends must match its The plan scored it 0, and that was wrong. Census steps (analysis events and synthesis steps) are unchanged Census mode (unmerged), pairs completed in both arms:
Parity. All 1,796 completed pairs per mode have identical parse counts and signature sets. The Memory. Mbugwe has 874 compiled analysis affix-rule instances, 1,620 allomorph specs and 3,648 list-capacity Gates. HermitCrab tests 169 passed (master 151); tests cover the exact failure trace on matcher and prefilter R4. Subsequence fallback and insertion classes: DEAD END (issue #568)Claim tested. R2 is off on four of five real grammars because their rules restore, reorder or duplicate stem Built. Cost accounting. One probe is 1 step and each trie arc examined is 1 step ( Sound subsequence tier with gaps, net:
With the order-free tier added (rules that also reorder parts):
Kill criterion. Under 5% net on every one of Mbugwe, Indonesian, Aweti, Amharic: met (0.0, -182.9, 0.0, 0.0). Why.
Revival would need a test that costs well under one step per saved event on Indonesian, position-specific gaps Known nondeterminism. Conformance parse signatures differ for the same words between off-arm census and R6, R3 and R1 on #546
R3 and R1 on #546: KEEP, and the two stack (issues #567, #566; PRs #576, #577)
Combined stack and R2's costMeasurement. One census binary, one harness and one word set, every arm against master (result 9; census mode, 150 s per word; words aborted or erroring in any arm are dropped from all arms). The stack is #546 with R1 and R3 (PRs #577 and #576), R2 (PR #572) and R5 (PR #574). Work is analysis events plus synthesis steps. Unification is top-level FeatureStruct calls, look-ahead checks and the edge prefilter included. Matcher work is FST traversals, arc inspections and prefilter unifications. The stack needs one test-only commit: 11 older tests on toy grammars with no root set
Parity. Per-word parse signatures equal master on every completed word in every arm, the full stack included: 0 differences. Shadow on the full stack: 0 violations. Additivity (work, in points of master; the sum is of the single arms #546, R1 with R3, R2 and R5).
R2's cost: root cause. The extra unification and matcher work is R2's own probe, not a change in merging or order. With R2 on in the stack, every non-probe counter is equal or lower (events, lookups, nodes, look-ahead checks, edge-prefilter unifications). The added unifications track the added trie arcs almost one for one (conformance +37,582 unifications against +37,518 arcs). The prune rate is too low to pay for the probe: conformance prunes 13 of 7,214 probes, and Sena prunes 72 of 1,225 probes, where each prune saves about 575 unifications and a probe inspects about 70 arcs. Most probes pass, because they find a root, and a pass still walks the arcs up to the first accepting path. Bantu-tam prunes late, after R3 has cut nearly all the work, so each prune saves about 50 unifications against about 47 spent per probe. Witness-reuse fix (folded into #572 @ Residual, 1 to 4 points. With the fix, R2 in the stack still does more unification work than no R2: 1.03 points more on conformance, 1.52 on Sena and 3.71 on bantu-tam. Matcher work is 1.25, 2.05 and 4.28 points more. R2 still saves events: 4.74 points of Sena work on top of #546 with R3. The residual is one input probe per stratum entry, the decisive probes, and probes where the stem root is cut and another root must be searched. No sound grammar-independent rule removes it. A gate on the measured prune rate would be a runtime heuristic with run-to-run nondeterminism under the parallel cascade; it was not prototyped. Standalone. Alone on master, #572 with the fix is still a unification loss: 9.05% more on Sena, 1.00% more on conformance and 37.48% more on bantu-tam. Its event savings are 17.29% less on Sena, 0.22% less on conformance and 6.37% less on bantu-tam. It is a gain only on the early-lexicon fixture, which is synthetic: 58.20% less unification and 67.72% less work. Landing decision. #572 stays a draft and lands only on top of #577 and #574. Standalone it is a net unification and matcher loss on Sena, conformance and bantu-tam. Stacked, the fixed R2 is below master on every counter, but still above no R2 by the residual above. If the owner requires R2 never to cost more than no R2, the alternative is dropping #572, which gives up Sena's 4.74 points of work. Stress grammars. The stress fixtures are now PR #578: opt-in ( Impure slots: unbounded runs restore nothing (PR #579, stacked on #576)
Cross-cutting lessons
Open questions
|
|
Archived from HermitCrab synthesis-to-analysis pruning, 2026-10-27 roundDisposition: round finished; no product code is proposed for this PR. This round answers John This record follows the #490 (comment). Ledger rows 31-37 Answer to John Maxwell's questionsFailure-reason statistics. They exist now (census below, harness Excluded co-occurrence (E4): sound, cheap, effective only where exclusions are dense. The check DisjunctiveAllomorph: already mostly early; no experiment. The environment-free Environments (E5, the input-word prefilter): dead end as specified. The soundness scan has to ExcludedMprFeatures (covered by E2, the MPR and stem-name trail replay): sound; worth keeping, What the data says matters instead. Synthesis steps are a small fraction of total work, which is The questionHermitCrab is generate-and-test. Analysis unapplies rules to generate candidate analyses, and
His principle is that an excluded property can be checked early if nothing later can remove it, and Scope: the target is the space of grammars, not the five samplesThe owner set the scope during the round. The five sampled reference grammars (Sena, Amharic,
Every experiment's measurement must include its stress fixture as well as the conformance suite and Failure-reason censusReproduce. Branch Definitions.
Corpora. The real grammars are samples, not whole corpora: Indonesian all 70 words, Aweti every Headline numbersReal grammars, pooled (27,912 failed and 422 successful terminal outcomes; 279,872 synthesis steps):
Rule-attempt events, real grammars: Per grammar, the step shares differ widely. Unpooled:
The conformance suite is dominated by one pathological fixture. Caveats on the methodThese are read before any number above is used as a ceiling.
Direct answers to Maxwell's four candidatesAll four are sound to move in at least a restricted form, and the census predicts little benefit
Hook points (origin/master One principle covers all four. A successful candidate applies every entry of its trail Excluded co-occurrence
Benefit: zero on the samples. In a grammar that uses co-prohibitions it is the only candidate here DisjunctiveAllomorphMost of this check already runs early. An environment-free earlier allomorph with no required EnvironmentsEnvironments are checked on the final synthesized word, after every stratum's phonology
Excluded environments cannot be prefiltered this way, as Maxwell said. Benefit: on the samples, 23 ExcludedMprFeaturesOn a successful path, MPR features evolve deterministically along the trail. They start from the Literature that bears on thisThe full survey is in this round's working notes, and only the parts that change a decision are kept
No published source was found on tracking morph spans through rule inversion. That is what a Ranked experimentsRanked by soundness plus benefit across the grammar space, with cost as the tiebreak. "Ceiling" is E1. Stop the synthesis allomorph loop after the recorded allomorph
E2. Replay MPR features and stem names along the trail at lexical lookup
E3. Replay the template and final-state outcome at lexical lookup (after a dead-candidate census)
E4. Excluded co-occurrence during analysis (Maxwell)
E5. Required-environment prefilter on the input word (Maxwell)
E6. Intermediate-shape check for SurfaceFormMismatch (research only, no brief)
Not an experiment
Results of the roundFive experiments were built, each on its own unpushed branch from Decisions
None of the branches is pushed. A branch name and SHA is the citation. Synthesis share of total workSynthesis steps and analysis events for the identical words, switch off:
The E1 run used a Mbugwe word set with 3,451,016 analysis events (a different drop set), which is E1. Allomorph-loop cutoff: KEEP
Cost not in the counters: one E2. MPR and stem-name trail replay: KEEP
Replay checks are set operations, a different unit from synthesis steps (a step is a pattern match), E3. Template replay: KEEP, without memoisation
The three stress rows do not give analysis event counts in the result files, so their total-work The brief's blow-up kill rule says to kill if walk states exceed steps saved, and it fired on every E4. Excluded co-occurrence during analysis: PARK
The brief's kill criterion (under 25% on its own fixture) fires on 3.2%. The two dense fixtures were E5. Required-environment input prefilter: DEAD END as specifiedThe code is recoverable at
Forced on, it is net negative on two of the three fixtures. Soundness findings: the brief's What would have to change to revive it. (1) A scan that stops rejecting digraph grammars, which E6. Intermediate-shape check: not runNo soundness argument yet; ConclusionEach of E1-E3 is sound on every corpus and cuts synthesis steps by a large amount on grammars that Open problems found at baseline
Open questions
|
|
Archived from HermitCrab optimization round, September 2026: final dispositionThis is the final record for the count-driven round that followed Measurement scopeThe exploration used sequential parsing, one warm-up followed by repeated samples, and canonical analysis The probe counters are intentionally absent from the final diff. They put volatile gates in hot code across Retained optimizationsEach retained mechanism has one record with its invariant, implementation, evidence, parity coverage,
Only the deferred clone was timed alone. Profile movement and deterministic work removal justify the other Rejected or removed experiments
All associated production switches, alternate traversal code, and research counters were deleted. The two Five-language branch-wide sampleEach row used three fixed word-list indices and two reverse-ordered process rounds (baseline/candidate, then
The ratios are deliberately unpooled and are not extrapolated to whole corpora. An initially selected heavy Correctness gateThe minimized code tip was merged into the conformance integration branch and run through the complete
What remainsThe edge prefilter leaves roughly 44% of remaining traversals successful, so another edge check has little |
|
Archived from HermitCrab probe design — enough to rebuild itCompanion to Why this exists: the seven probe branches that produced the ledger's numbers were deleted after 1. Instrumentation architectureTwo probe classes, in two assemblies, with two independent gates.
Everything is insertion-only: a The wall-time splitSeven mutually exclusive top-level slices that sum to wall (six measured plus the remainder),
Nested inside Always print the remainder as its own column. It caught the misattributed split, the dead probe The per-node decompositionInside
Measured outcome, so a rebuild can sanity-check itself: on Amharic CensusesRule attempts, split by which guard rejected them ( Die points, as an enum with a counter array and a parallel tick array — timing each individual Die points, classified for the abstract-replay question — shape-free: synthesis-side 2. The keys
|
|
Archived from HermitCrab analysis search — asymptotic researchDisposition: count ceilings before building. The first two candidates below fit the two open work This integrates the six supplied reports: R1 partial-order reduction and rewriting; R2 packed forests The problem in one pageThe motivating case is a grammar with 16 extra affix processes in one Unordered stratum. Analysis There are three different sources of growth:
All numbers here are supplied by the common brief, not reproduced in this job:
These are ceilings to investigate, not interchangeable denominators. Tracing disables the ordinary Verified code anchorsLine numbers below were read directly, not copied from the reports. C# means the requested main
In particular, at There is a second relevant implementation difference: Rust's common dispatch applies the count gate and Ranked candidatesRank is the order in which to spend research effort under the memory constraint, not a promised speedup.
1. Residual viability: prune only a proved-empty continuationLet The target set must include everything ordinary lookup could supply. Compound non-head lookup is already Start with a grammar-compiled coarse domain: mandatory/optional segment-length intervals, safe segment R4's precedence idea is a special case with a stricter obligation. To install This must remain grammar-sized: use finite transfer tables and stack-local query work, not a table keyed 2. Make the template battery cheaper by factoring its controlThe code provides a useful, smaller target than a whole-search chart. C# builds a non-disjunctive battery Compile a trie of those reversed control programs. Share a prefix only when its slot operation has the For an admitted template, the original slot path has an identical path through the trie. Conversely, Two implementation boundaries matter. First, retain the existing slot/battery deduplication and representative The new trie has at most one node per original slot-prefix position: This can reduce the factor due to duplicated template control. It does not turn the 43,641 battery calls 3. Partial-order reduction: preserve the trace class, then recover its ordersR1 supplies the essential correction to canonicalization. A class contains paths related by swapping For every certified pair and reachable eligible state, require (1) either transition preserves the other's Under these conditions, adjacent swaps neither lose a path nor invent an impossible one. Their transitive Store the grammar's conservative pair matrix ( For fully independent, single-use rules with one outcome per occurrence there is one class per subset, 4. Finite-state compilation: exact only for a certified fragmentAn exact compiler would encode a relation from surface input to complete accepted analysis labels. It If each primitive's encoding is exact, relational union preserves all branches and composition preserves The compiler must refuse constructs with no established regular encoding. In particular arbitrary copying For an accepted finite relation with R3 is right that a compact proposer plus exact reparse does not solve the confirmation cascade. Its Where the six reports disagreed
Considered and not recommended
Implications for the two work itemsLexicon-driven pruning during unapplication. Start with candidate 1 in shadow mode at C1/C2. The Making the affix-template battery cheaper without memoization. Start with candidate 2's static For either item, compare complete analysis sets with ordered morpheme/allomorph identities, root position, Evidence limits and referencesNot verified: the brief's timings/counters, the earlier POR eligibility percentage, exact applicability All entries below are [memory; bibliographic details and applicability not independently verified].
The reports also invoke Held–Karp sequencing DP, Tomita-style shared forests, Minato's ZDDs, graph |
|
Archived from HermitCrab optimization ledger — tried, closed, do not retryPurpose: stop this work being redone. One row per optimization attempted, what was expected, Deliberately in-repo and durable, not evicted into a PR accordion. A closed avenue is only closed This is the only report. The working docs it was distilled from
No instrumentation is merged, by design. The probes gate on Unmeasured search-asymptotics research#490 (comment) integrates six independent reports, Final disposition of the September follow-upThe follow-up branch does deliver product improvements, retained on Four attractive experiments were deliberately excluded from the final product diff and are recorded under The minimized branch passed all 603 runnable tests in the conformance-inclusive suite (one expected Windows Earlier ledger round: net speed delivered noneNo optimization in the "closed" section shipped, and none of them made anything faster. The Earlier shipped results
Retracted historical prototypeRow 3 in the original ledger claimed that shape sharing through Closed — measured, does not pay
Closed before building — motivation refuted by measurement
Reopened this round, and their outcomes
2026-10-21 search-cost roundFull record: #490 (comment).
Lesson of the round: two of its three large reductions (rows 24 and 25) were port bugs, found 2026-10-27 synthesis-to-analysis round (finished)Full record: #490 (comment).
2026-10-27 analysis-side roundFull record: #490 (comment).
Lessons of the round (detail in the round doc):
The lesson that generalisesFour independent measurements, four boundaries, same collapse:
The redundancy in HermitCrab's synthesis is apparent, not real. The trail is what makes each That closes a family: packed parse forests of the merge-by-key kind, fold-step sharing and Where the time actually isMeasured with seven exclusive buckets summing to wall. Amharic is a single run (
Every optimization in this ledger that worked reduced how many nodes are visited. None touched Inside one node (Amharic, 28 words, both probe layers live)
Censuses: 27,385 rule attempts — guards reject 19.5% (application-count 6.3%, unifiability 13.2%), The headline: only ~16% of the Amharic analysis cascade is identified as linguistic Gate outcomes, written before the run. "Any bucket >=50% becomes the sole build target" — clone So there is no declared build target yet. The next probe splits the 37.4%: A caution on the 46.7% before anyone acts on it. One durable code constraintIt is not expressible in code, and it bites silently. HermitCrab's morphological rules are not order-invariant. Same pending-rule multiset, Calibration record — predictions vs outcomesWritten before the measurements, scored after. Kept because the misses are informative.
Method rules earned here
Addendum, September 2026: the round that followed, measured by countsBranch
|
|
Archived from HermitCrab optimization round, September 2026: final disposition of the branchDate: 2026-09-03. Branch: Why the package is not merged
Disposition per mechanism
The isolated prefilterStatus update: Branch Standalone measurement (same binary, toggle off = master behavior, toggle on; sequential, memo on)Signature: gloss + allomorph index + syntactic feature structure, compared as multisets. Counters: throwaway
Reading. The count reduction reproduces (Sena −85% traversals, matching the branch's −87%), and parity holds Raw logs: session scratchpad Branch and worktree inventory for the round (2026-08-20 to 2026-09-03)Every branch below is local unless marked. Worktrees for archived branches were removed on 2026-09-03; the
Records in this directory: Four further records of the round were deliberately left out of this set, because they concern another author's
Privacy: word forms and glosses from the private grammars appear only as word-list indices and rule ids in these What to do next, in order
|
Start here:
docs/hermitcrab-perf-2026-09-disposition.md: why the faster September package was archived rather than merged, and what any revival must fix first.Status: open as a record; not intended to merge code. The only non-skipped check is green (the
claudejob is skipped).This is a documentation-only record of the HermitCrab optimization rounds (22 files, all under
docs/): the attempt ledger, nine retained mechanism records, five rejected-optimization records, the measurement method, and the round write-ups for September 2026 and October 2026. No parser behaviour changes, and the archived mechanisms (including the edge prefilter) stay off master.Why keep a record of work that was not merged? The September package measured faster in wall-clock terms (1.76x Amharic, 1.83x Mbugwe; secondary figures, later rounds count steps), but the figure cannot be attributed to any one mechanism: only the deferred template clone was ever timed alone, at 1.054x. Its parity gate also compared morph counts, not analyses. The record lets the next attempt reuse the evidence instead of repeating it.
Where to look:
docs/hermitcrab-perf-2026-09-counts.md("legacyWordAnalysisSignaturecheck") shows the signature compared morph counts only on the FLEx exports, becauseMorpheme.Idis empty.perf/hc-optimization-archive, which still holds the edge prefilter (git grep EdgePrefilterEnabled).docs/hermitcrab-optimization-ledger.md, one row per attempt.docs/rejected-optimizations/; the traversal-dedupe record covers Add allMatches to Traverse function to improve performance #511 (still open).Deliberately not here: source changes and re-measurement. Reviving any archived mechanism needs a complete parse-identity signature first (gloss, allomorph index, syntactic feature structure, ideally the full multiset with multiplicities), not this PR.
Related PRs: #491 (merged) is the larger win the September disposition redirected to; #546 (open) is the template-slot sharing the October search-cost round ported to C#.
Verification:
git diff --check origin/master...HEADclean; comment-hygiene found no in-scope files (docs-only diff).dotnet build,dotnet test,dotnet csharpier check .and./local_check.shwere not run: no.csor.csprojtouched.Next: confirm the archive branch
perf/hc-optimization-archiveshould stay as the home for the unmerged package, then close or merge this record.Reading this a year from now -- start here
This record covers the September 2026 optimization round, the October 2026 search-cost round, and two October 27 rounds (synthesis-to-analysis pruning, then analysis-side optimization), plus a search-asymptotics research note. "Retained" does not mean merged or production-ready: the September package was rejected for merge.
The package's branch-wide sample was faster on all five languages measured, "stable" on two of them per the probe rules, but not attributable to any mechanism. That non-attribution, not an absence of speedup, is why it was archived.
Correctness limit: the branch-wide parity gate used
MorpherTests.WordAnalysisSignature, which joinsMorpheme.Id. That is empty for every FLEx export used, so on Sena, Mbugwe and Amharic it degenerates to a morph count per analysis: it verified counts, not analyses.Decisions and why
SIL.Machine.FeatureModel.FeatureStruct, the rest in FST traversal internals,Word,Shapeand the analysis rules: not reviewable as one change.AlwaysEnforceFinalTemplates = true), gave identical analysis sets with far larger gains on Mbugwe and Sena than the package; Amharic barely moves because its per-word cost is not in template interleaving. The package is worth keeping only where Filter final templates in analysis #491 cannot fire, and only in its most reviewable piece.docs/hermitcrab-probe-design.md.Surprising findings
Paths not taken
Five rejection records under
docs/rejected-optimizations/:alternatives-memo-and-hoist.md,compounding-split-prefilter.md,final-template-prune.md,two-pass-nondeterministic-traversal.md, andstate-position-traversal-dedup.md(the traversal dedupe of #511, added after the first round closed). Lexically-dead branch pruning, ranked first indocs/hermitcrab-search-asymptotics-research.md, was measured and rejected on its ceiling. Template replay (E3) was dropped on cost.Deferred, and what would unblock it
October analysis-side candidates live as separate PRs (#571, #574, #575, #576, #577, #579 open; #572 closed) with their own records in
docs/hermitcrab-2026-10-27-analysis-round.md. Merging-while-tracing is an owner product decision, not built. Exact finite-state compilation and partial-order reduction are larger projects described only as research in the asymptotics note. Any renewed branch-wide parity claim needs the full parse-identity signature first.Evidence, scope, and provenance
Durable records:
docs/hermitcrab-optimization-ledger.md,docs/hermitcrab-perf-2026-09-disposition.md,docs/hermitcrab-perf-2026-09-counts.md,docs/hermitcrab-probe-design.md, nine mechanism records underdocs/optimizations/, five rejection records underdocs/rejected-optimizations/, and the round write-upsdocs/hermitcrab-2026-10-21-search-cost-round.md,docs/hermitcrab-2026-10-27-synthesis-to-analysis-round.md,docs/hermitcrab-2026-10-27-analysis-round.md,docs/hermitcrab-search-asymptotics-research.md.The unmerged package is archived on
perf/hc-optimization-archive. The separately isolatedperf/hc-edge-prefilterbranch and its archive tag were deleted, but the edge-prefilter mechanism (source and tests) still lives in the whole-round archive, found withgit grep EdgePrefilterEnabled.The earlier body quoted a historical NUnit result from the prior measurement; it is not carried forward because it was not rerun.
🤖 Generated with Claude Code
This change is