[johnml1135]

Start here: src/SIL.Machine.Morphology.HermitCrab/Morpher.cs, ParseWords: the whole public surface, plus the guard that requires a degree-1 Morpher.

Status: ready for review; all checks green. #546 (shared template optimization, still open) is not required; this PR is independent of it.

FieldWorks consumer: FLEx's parser worker adopts this through ParseWords on a degree-1 Morpher, tracked in LT-22885.

Morpher.ParseWords(words, maxWordConcurrency = 4, guessRoot = false) parses a batch of words concurrently and returns one WordParseResult per input position, in input order, duplicates kept. Each result carries materialized analyses, a trace, and any per-word exception, so an invalid shape or resource-limit failure on one word does not discard the others. The single-word API and the constructor default (maxDegreeOfParallelism = 0) are unchanged.

What could break existing callers? Nothing opts in implicitly. ParseWords throws unless the Morpher was built with maxDegreeOfParallelism: 1, which keeps template slots, analysis cascades and synthesis sequential inside each word. The one shared-state change: alternative limits now belong to the invocation, removing writes to shared compiled rule objects during parsing (AnalysisStratumRule.cs, AnalysisLanguageRule.cs).

var morpher = new Morpher(traceManager, language, maxDegreeOfParallelism: 1);
IReadOnlyList<WordParseResult> results = morpher.ParseWords(pendingWords);

Where to look:

• Per-invocation limits replacing the old _maxAlternatives field: AnalysisStratumRule.cs. Pinned by ParseWords_CapturesMaxAlternativesPerWord.
• Results match single-word parsing, in order: ParseWords_PreservesInputOrderAndSingleWordAnalysisSets, ParseWords_MatchesSingleWordAnalysisSetsWithCompounding.
• Degree-0 Morphers are refused: ParseWords_RequiresSequentialMorpher.
• Tracing runs the whole batch serially on the calling thread: ParseWords_TracingRunsOnCallingThreadAndRetainsPerWordTraces.

Deliberately not here: FieldWorks adoption (it needs a caller change; the default stays 0); a change to the single-word constructor default; hard or capped words are unmeasured for parity and throughput.

Speed figures are measured wall timings used only as motivation, not a guarantee for this implementation or every Language (details below).

Verification: offline Release build passes with zero warnings or errors; the full HermitCrab NUnit project passes, including the new ParseWords_* cases; CSharpier checks the five changed files; comment hygiene is clean. Repository-wide ./local_check.sh and coverage were not run locally.

Next: review the ParseWords signature and the degree-1 requirement; say if you want the requirement relaxed.

---

Reading this a year from now -- start here

ParseWords exists so a caller with a queue of pending words can spend parallelism across words instead of inside one word. The measurements, FieldWorks adoption notes and the PanGloss comparison below are the reasoning behind the API shape; the tree only shows the API.

Evidence: measured speedups (wall time, secondary)

A local benchmark harness, not part of this PR, measured four concurrent degree-1 words at 3.05x on Sena and 2.20x on Mbugwe versus the degree-0 serial-word default. Those figures include the separate #546 shared template optimization. The same experiment with sharing disabled measured 2.42x and 1.47x.

Caveats: three-pass median wall timings on a shared machine, excluding construction, over small completed cohorts (62 Sena positions; 15 of 42 sampled Mbugwe positions). The harness used one Language/Morpher per worker, whereas this API shares one fixed Morpher. Hard or capped words remain unmeasured; the larger Sena run was unstable under its watchdog. Twenty concurrent words performed worse than four on both primary real-input cohorts, hence the default of 4.

Decisions and why

• Degree-1 Morpher required. Bounded parallelism across words with sequential work inside each word. The evidence does not justify silently changing the existing single-word constructor default.
• Differs from PanGloss on purpose. Its memory architecture differs, so the C# API puts parallel work across words and sequential paths within each word.
• Tracing is serial. Traced calls sharing a trace manager must not overlap, so a traced batch runs on the calling thread.

Deferred: what FieldWorks adoption needs

FieldWorks's parser worker currently submits one word at a time and constructs Morpher without a degree argument (Src/LexText/ParserCore/HCParser.cs:195 in the FieldWorks repo; not checked here). It would need to: construct a degree-1 Morpher; gather a finite batch from its pending queue; call ParseWords; map each indexed result, error and trace back to the original request. Language reloads and option changes must wait for the batch to finish; selectors and custom rules must support concurrent calls. FieldWorks is not changed here.

🤖 Generated with Claude Code

---

This change is

[codecov-commenter]

Codecov Report

❌ Patch coverage is 96.59091% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.06%. Comparing base (9f284a3) to head (9485394).

Files with missing lines	Patch %	Lines
...chine.Morphology.HermitCrab/AnalysisStratumRule.cs	91.89%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #582      +/-   ##
==========================================
+ Coverage   74.93%   75.06%   +0.12%     
==========================================
  Files         454      455       +1     
  Lines       38490    38545      +55     
  Branches     5322     5328       +6     
==========================================
+ Hits        28843    28934      +91     
+ Misses       8495     8463      -32     
+ Partials     1152     1148       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[claude]

FYI: F2. A maxWordConcurrency below 1 throws ArgumentOutOfRangeException. Everywhere else in this class a parallelism below 1 means unbounded (MaxDegreeOfParallelism doc, CreateParallelOptions), and ParseWords_ValidatesBatchArguments asserts the throw for 0 and -1. A caller following that convention gets an exception.

[johnml1135]

Keeping this one too. Below-1-means-unbounded suits work inside one parse, but a batch needs a real cap: in our measurements 20 concurrent words were slower than 4. ParseWords_ValidatesBatchArguments pins the throw on purpose.

[claude]

FYI: F3. With MaxAlternatives > 0, every Apply call copies both rule lists into a new cascade and a new RuleBatch. AnalysisLanguageRule makes that call once per input word per stratum, so the allocations grow with ambiguity. Building the capped pair once per stratum per parse, before the inData loop, gives the same limits. Unverified cost: not measured.

[johnml1135]

Not changing this in this PR. The cost is unmeasured and only applies when MaxAlternatives > 0, so I'd want a measurement before reshaping the cascade setup.

[claude]

1. Verdict: approve. No new findings this round. The open F2 and F3 are both Low and need no change before merge.
2. Most important: F1 (src/SIL.Machine.Morphology.HermitCrab/Morpher.cs:247) is addressed. The MaxDegreeOfParallelism doc now says ParseWords requires degree 1, so the throw is a documented contract rather than a contradiction.
3. Counts: Critical 0, Important 0 open (F1 addressed), Low 2 open (F2, F3).
4. Ran: the 25 new HermitCrab tests, filtered with dotnet test, passed 25/25 on net10.0 locally. gh pr checks 582: both Builds and Comment hygiene (advisory) passed; NuGet package and claude-review are pending.
5. Not verified: history was rewritten since d47ef0a (compare reports diverged, and that commit is not fetchable here), so I re-reviewed the whole PR against master. I still could not run ParseWords_UsesBoundedWordConcurrency with a reduced processor count to check its stability on a starved runner.

Public API: adds Morpher.ParseWords and WordParseResult; removes the internal AnalysisStratumRule.MaxAlternatives. Target framework, package, and sillsdev/machine.py parity: none verified.

Findings: F1 addressed, F2 open, F3 open, unverified (cost not measured).

Reviewed at 9485394

[jtmaxwell3]

FieldWorks currently displays what word is being parsed and the number of words remaining in the lower left corner of the display.  This is very useful feedback for a long running process. Seeing what word is being parsed lets you see which words are taking a long time. Maybe ParseWords could take an optional callback function that is called when a word starts parsing and when it stops? The set of words currently being parsed could be displayed in the lower left corner.

[jtmaxwell3]

An alternative to passing limits like MaxAlternatives in as an argument to internal functions is to make them be readonly once Morpher has been initialized. In FieldWorks, the Hermit Crab grammar gets recompiled whenever any of the parameters are changed. Are there situations where we want to change MaxAlternatives on the fly? If so, maybe we should make it be an argument to ParseWord(s) rather than a variable in Morpher.