From 8f5438a0c6aa745d243963614cb73b0bd8d88547 Mon Sep 17 00:00:00 2001 From: John Lambert Date: Fri, 17 Jul 2026 07:07:04 -0400 Subject: [PATCH] Memoize HermitCrab's sequential analysis cascade Adds AnalysisStateKey/AnalysisScope/Word.ReplayOnto and wires a memo table into both the mrule cascade and the affix-template battery, so repeated analysis-cascade states reached via different rule-unapplication orders are computed once and replayed rather than re-expanded. Off by default (the existing parallel cascade is unchanged); opt in via Morpher(maxDegreeOfParallelism: 1), which selects the new sequential+memo path. Ported from an archived prototype (parse-optimization-archive) with stronger verification: the acceptance gate is analysis-set equality (canonical morpheme-signature sets), not byte-identical objects, since a memo-replayed Word is not guaranteed field-for-field identical to a freshly-computed one. Verified via unit tests (key order-invariance, replay graft correctness, in-flight re-entry guard) plus corpus runs against three real grammars (Sena, Indonesian, Amharic) with zero analysis-set divergences. On a known-pathological word, sequential+memo measured 6.2x faster than the parallel default (isolated to ~6.3x attributable to the memo itself, not threading); aggregate corpus evidence and the typical-word tradeoff are in memoization.md, along with honestly-reported open gaps. --- memoization.md | 420 ++++++++++++++++++ .../AnalysisScope.cs | 72 +++ .../AnalysisStateKey.cs | 110 +++++ .../AnalysisStratumRule.cs | 68 ++- .../MemoizedCombinationRuleCascade.cs | 122 +++++ .../Morpher.cs | 19 +- src/SIL.Machine.Morphology.HermitCrab/Word.cs | 71 +++ .../AnalysisStateKeyTests.cs | 148 ++++++ .../MemoCorpusVerification.cs | 234 ++++++++++ .../MemoizedCombinationRuleCascadeTests.cs | 189 ++++++++ .../MorpherTests.cs | 298 +++++++++++++ 11 files changed, 1736 insertions(+), 15 deletions(-) create mode 100644 memoization.md create mode 100644 src/SIL.Machine.Morphology.HermitCrab/AnalysisScope.cs create mode 100644 src/SIL.Machine.Morphology.HermitCrab/AnalysisStateKey.cs create mode 100644 src/SIL.Machine.Morphology.HermitCrab/MemoizedCombinationRuleCascade.cs create mode 100644 tests/SIL.Machine.Morphology.HermitCrab.Tests/AnalysisStateKeyTests.cs create mode 100644 tests/SIL.Machine.Morphology.HermitCrab.Tests/MemoCorpusVerification.cs create mode 100644 tests/SIL.Machine.Morphology.HermitCrab.Tests/MemoizedCombinationRuleCascadeTests.cs diff --git a/memoization.md b/memoization.md new file mode 100644 index 000000000..2ce3359b8 --- /dev/null +++ b/memoization.md @@ -0,0 +1,420 @@ +# Memoization for HermitCrab analysis — minimal, verified port to master + +**Branch:** `feature/memoization` (at master `d9deb167`). +**Goal:** capture the prototype's demonstrated speedup (a pathological Bantu-template +word ~5×; a 312-word Sena heavy set 1,051s → 388s) with a small, human-reviewable diff, +no RUSTIFY rearchitecture, and stronger verification than the prototype had. + +Real grammar/corpus content (Sena, Amharic, Indonesian) is never committed to this repo, +including individual word forms in prose — this doc refers to specific test words only by +generic labels (heavy word H1, H2, ...) rather than by their actual spelling. + +## 1. Provenance and evidence base + +The design is not new. It was built, measured, and corpus-verified on the +`parse-optimization` branch (2026-07-02/03), which was stacked on the unmerged RUSTIFY +commit and later deleted; the full 23-commit chain has been recovered and preserved as the +local branch **`parse-optimization-archive`**. The Rust port (PanGloss `hc-memo`, +`hc-rules/stratum.rs`) faithfully re-implements the same design and adds test patterns and +two hygiene lessons, but no new architecture. Primary references: + +- `parse-optimization-archive` — `AnalysisScope.cs`, `AnalysisStateKey.cs`, + `MemoizedCombinationRuleCascade.cs`, `Word.ReplayOnto` (commits `3522fc41` Phases 0-3, + `6ea0536a` Phase 3b), plus `parse-optimization.md` and + `docs/hermitcrab-parse-algorithm-analysis.md` in that tree. +- PanGloss — `hc-memo/src/lib.rs`, `hc-rules/src/stratum.rs:751-955`, + `hc-rules/tests/memo_gate.rs`, `hc-parse/tests/simultaneous_conformance.rs`. +- fst-advisor branch docs — verification-gate methodology (golden signature dumps, + per-grammar gating order, soundness batteries). + +**Where the speedup actually is (fair sequential baselines, measured on the archive):** + +| Mechanism | Effect | +|---|---| +| mrule-cascade positive memo + nogood cache (Phases 2/3) | ~10% (30.5s → 27.4s on heavy word H1). 2,555 real expansions vs 118,162 hits — hits are cheap because guard clauses already reject fast. | +| **template-battery memo (Phase 3b)** | **The real win: 93% of instrumented wall time.** 38,840 battery invocations vs ~2,581 unique keys. H1 27.4s → 6.1s; H2 102.7s → 26.9s; Sena 312-set 1,051s → 388s. | + +Both mechanisms share one key type and one replay primitive, so the minimal port includes +both; the template memo is the piece that must not be dropped if scope is ever cut. + +## 2. What the design is (one page) + +All memoization is scoped to a single `Morpher.ParseWord` call and applies only to the +**sequential** Unordered-order (template/Bantu) analysis cascade. + +- **`AnalysisScope`** — carrier object threaded through `Word` clones like `CurrentTrace` + (reference-shared; excluded from `FreezeImpl`/`ValueEquals`). Holds two tables + + in-flight guards + a 100,000-entry growth cap (OOM guard; correctness-neutral): + - `Memo` — mrule-cascade results (empty list = nogood, non-empty = positive), + - `TemplateMemo` — affix-template-battery results (separate table: same key space, + different computation), + - `InProgress` (+ a separate `TemplateInProgress`, a Rust-side clarity improvement worth + adopting) — re-entry guards; a hit falls through to unmemoized expansion for that call. +- **`AnalysisStateKey`** (readonly struct) — `(Shape, Stratum, SyntacticFeatureStruct, + RealizationalFeatureStruct, NonHeadCount, multiset of per-rule unapplication counts)`. + Order-invariant by construction: the multiset collapses the k! orderings that cause the + 98.4% expansion redundancy. Hash uses cached frozen hashes + commutative XOR over the + multiset; equality compares full values (never trust a bare hash — entries store the full + key). The constructor **freezes shape/FSs defensively on read** (known landmine: + `AnalysisAffixTemplateRule.Apply` reassigns an unfrozen `SyntacticFeatureStruct` after + the Word is frozen; that setter has no `CheckFrozen()` guard). +- **`MemoEntry`** — `{ Results: IReadOnlyList, MruleTrailPrefixLength, + NonHeadPrefixLength }`. +- **`Word.ReplayOnto(query, mruleTrailPrefixLen, nonHeadPrefixLen)`** — the replay graft: + clone the stored result, replace the trail/non-head *prefix* with the query node's own, + keep the suffix, re-freeze. Sound because everything strictly inside a memoized subtree + is a deterministic function of the key fields alone; only the two ordered structures the + key collapses to counts need grafting. +- **`MemoizedCombinationRuleCascade`** (mrule cascade) and an `ApplyTemplateBattery` + hook in `AnalysisStratumRule` (template battery): both do key → `TryGetValue` → replay + on hit / compute + `TryAdd` on miss. +- **Tracing bypasses the memo entirely** (`ParseWord` only installs `AnalysisScope` when + `!IsTracing`): traces must stay byte-identical to the unmemoized engine. + +**Design rules locked in from the research (each maps to a specific finding):** + +1. **No pruning gate may be coupled to memo presence.** The prototype bundled the Phase-5 + `HasReachableRoot` lexical gate into the memo-on path; Rust deliberately removed the + coupling to preserve `memo-on == memo-off` as a true invariant. We port the memo only — + no Phase 5, no gating — so the toggle is a pure optimization knob. +2. **Key-completeness is the primary correctness risk** (packrat/Ford; Bazel cache + doctrine). The port includes a written audit: every field read by any + `MorphologicalRules/Analysis*.cs` rule is either in the key or provably irrelevant + (`IsPartial` and `_isLastAppliedRuleFinal` are grep-verified unread). This audit lives + as a code comment on `AnalysisStateKey` and must be re-run when analysis rules change. +3. **Trail push and multiset increment are one atomic unit.** `record`-style bookkeeping + pairing (`_mruleApps.Add` + unapplied-count increment) must never drift; replay's + prefix/suffix split depends on multiset-equality ⇒ prefix-length-equality. +4. **Budget interaction (future):** if complexity-cap ever lands, a budget-interrupted + subtree must never be written to the memo ("exhausted subtree is not memoized" — the + PanGloss ground rule). Not needed now (master has no budgets) but recorded as a standing + invariant next to the cap constant. +5. **Per-parse lifetime, pre-sized where cheap** — no cross-word or process-global caching. + (A word→analysis-set cache above the engine is sound and orthogonal — explicitly out of + scope here; see fst-advisor `HYBRID_FST_FEASIBILITY.md` §6.) + +## 3. What master is missing (the port deltas) + +Master's HC tree is unchanged since the RUSTIFY fork point (`78350670`), so the substrate +the memo needs — `Word._mruleApps`/`_nonHeadApps`/`Clone`/`Freeze`, +`Shape/FeatureStruct.GetFrozenHashCode/ValueEquals`, `CombinationRuleCascade` — exists +**byte-identical** on master. Deltas: + +| Delta | Adaptation | +|---|---| +| `TOffset` is `ShapeNode` on master, `int` on RUSTIFY | Mechanical: port against `IRule` etc. | +| **Master has no runtime sequential/parallel toggle** — cascade choice is `#if SINGLE_THREADED`, which no csproj defines, so the shipped library always runs the parallel cascade and the memo path would be unreachable | **Prerequisite work** (the only genuinely new work): add a runtime knob to `Morpher` (RUSTIFY's `maxDegreeOfParallelism` ctor param is the reference design). Default preserves current parallel behavior. Scoped to the analysis side only — `SynthesisStratumRule` is untouched; synthesis has no equivalent memo. | +| No COW `Shape`/`FeatureStruct` (RUSTIFY-only) | Omit Phase 10a-1/7b polish. `ReplayOnto` does a real deep clone on master — slower than the prototype, never wrong. Expect somewhat less than the prototype's exact numbers; measure. | +| `InstrumentedRule`/rule-stats infra, `TrailDirectedRuleCascade` (Phase 0/1), `hc batch` command | Not load-bearing for the memo. Skip from the shipped diff; a slim corpus-signature harness is built as test-only tooling instead. | +| Master's `.Distinct(FreezableEqualityComparer)` in `AnalysisStratumRule` | Keep (harmless with the memo's deduped output; removal is a separate evaluation). | + +## 4. Implementation structure + +Built and gated in five logical stages (not five separate PR commits — the shipped diff +is squashed to one): + +**Stage 1 — prerequisite: runtime sequential-cascade toggle.** +`Morpher` gains `MaxDegreeOfParallelism` (ctor param; default = current parallel +behavior). `AnalysisStratumRule` selects sequential vs parallel cascade at runtime +instead of `#if SINGLE_THREADED`. No memoization yet. Gate: full unit suite green; +behavior byte-identical at default; a test pins sequential == parallel output on an +existing grammar. + +**Stage 2 — key + scope + replay primitives, with their unit tests.** +New `AnalysisStateKey`, `AnalysisScope`; additive `Word` members (`ReplayOnto`, +`UnappliedRuleCounts`, `MorphologicalRuleTrailLength`, `AnalysisScope` property). +Nothing consumes them yet. Tests ported from the archive + PanGloss `hc-memo` unit +battery: key order-invariance over permuted unapplication sequences, hash consistency, +sensitivity to non-head count and differing multisets, replay prefix/suffix graft, +in-flight guard semantics. The key-completeness audit comment lands here. + +**Stage 3 — mrule-cascade memo (positive + nogood).** +`MemoizedCombinationRuleCascade` wired into `AnalysisStratumRule`'s Unordered sequential +path; `ParseWord` installs `AnalysisScope` when sequential and `!IsTracing`. Tests: +hit-counter-guarded equivalence test (compounding grammar, from archive `MorpherTests`), +plus the **new adversarial fixtures the prototype never had** (§5, items a-b). + +**Stage 4 — template-battery memo (the 5×).** +`TemplateMemo` + `ApplyTemplateBattery` in `AnalysisStratumRule`. Tests: the +affix-template equivalence test (two commuting prefix rules, hit-counter-guarded), and +the memo-on/off gate exercising both tables in one parse (PanGloss `memo_gate.rs` shape). + +**Stage 5 — corpus verification harness + benchmark evidence.** +Test-only `[Explicit]` corpus runner in the HC test project (modeled on +`FstSenaBenchmark.cs`: env-var driven, per-word watchdog), comparing canonical +analysis-set signatures against uncommitted local grammars. Results recorded in §5/§6. + +Ship the memo **on by default whenever the cascade runs sequentially** (it is a pure +optimization once the invariant tests pass); the *default cascade mode* stays parallel in +this PR — flipping the library default to sequential+memo (which the corpus numbers say is +faster anyway) is proposed as a follow-up PR with its own benchmark evidence, so the +behavior-default change and the mechanism land as separately revertable units. + +## 5. Verification plan (stronger than the prototype's) + +Per-commit unit gates as above, plus: + +- **Memo-on/off analysis-set equality as the standing acceptance gate** (the metamorphic + relation: equal keys ⇒ equal result sets) — **not byte-for-byte object equality.** The + gate compares canonical signature sets (sorted `join("+", morphemeIds)+":"+rootIndex`), + because `ReplayOnto`'s grafted `Word` is not guaranteed field-for-field identical to a + freshly-computed one (e.g. `ShapeNode`/annotation object identity differs) even when it + represents the same analysis. Every equivalence test asserts non-vacuously (memo hit + counters > 0, both tables non-empty where applicable). +- **(a) SelfOpaquing ≥2-iteration fixture — the confirmed-bug shape (resolved: not reproduced, + wired in as a standing regression).** PanGloss's rust conformance suite documents a + confirmed C#-oracle bug in this exact code path (a `RewriteApplicationMode.Simultaneous` + epenthesis rule against root 19's `"b+ubu"`; `AnalysisRewriteRule` compiles this shape as + `ReapplyType.SelfOpaquing`, a repeat-until-fixpoint loop). Before trusting stage 3, this + was reconstructed directly against `parse-optimization-archive`'s own `AnalysisScope` + using the real `RewriteRuleTests.EpenthesisRules` rule definition: parsing `"buibui"` + gave the same result (1 parse) under every tracing/call-order combination tried + (non-traced, traced, repeated, order-swapped) — no divergence. The real minimal fixture + (`conformance/rewrite/simultaneous-epenthesis/`) is not present in this checkout to test + against directly (its own Rust test is `#[ignore]`'d for the same reason). Per the + advisor consult: this does not block stage 3 — the general memo-on/off analysis-set + equality gate is strictly stronger than one fixture, and PanGloss's own Rust analysis + reached the identical conclusion for their side (memo-sound on this shape; the ≥2-iteration + loop×memo interaction specifically "remains untested" because no available fixture drives + the loop past one iteration). **Shipped:** `MorpherTests.ParseWord_MemoOnMatchesMemoOff_ + ForSelfOpaquingSimultaneousEpenthesis` wires this exact fixture in as a standing + regression case (memo-on/off equality plus a pinned-value assertion), and the + ≥2-iteration gap is recorded here verbatim as an open, honest limitation — not silently + dropped. +- **(b) Trail-order-observable grammar — closed at the right layer, not through `ParseWord`.** + `Morpher.ParseWord` does not return raw analysis-cascade words — it feeds them through a + full **synthesis** re-derivation (`Synthesize`/`LexicalLookup`/`PermuteRules`), which + re-explores rule orderings on its own, so a `ParseWord`-level probe is the wrong + instrument for this: an attempt via a disabled-merge grammar variant returned 0 results + (most likely `MergeEquivalentAnalyses=false` breaking that synthesis path some other way, + not evidence about trail-order observability one way or the other — not chased further, + since it isn't the right layer regardless). **What the actual safety net is:** the + standing memo-on/off analysis-*set*-equality gate catches the class of bug that matters + — a dropped or spuriously-added analysis, exactly the PanGloss 0-vs-1 shape — because a + dropped analysis never reaches synthesis to be re-normalized away. Internal analysis + trail order, by contrast, is plausibly a don't-care for the returned output set, which + is also why it doesn't need a `ParseWord`-level test. The graft primitive itself IS + pinned by a real red/green test, at the layer where it's actually observable: + `AnalysisStateKeyTests.ReplayOnto_Grafts*` (stage 2, direct construction) and + `MemoizedCombinationRuleCascadeTests.Apply_PositiveReplayMatchesUnmemoizedResultSet_ + IncludingTrailOrder` (stage 3, a synthetic 3-rule cascade run directly against + `MemoizedCombinationRuleCascade` — bypassing `Morpher` so a real commuting-order replay + can be forced — comparing the memoized run's result set against an unmemoized run + *including* `MorphemesInApplicationOrder`, i.e. trail order). Verified this actually + discriminates: temporarily breaking `ReplayOnto` to skip the prefix graft made both this + test and the stage-2 unit tests fail, confirming they are not vacuous. +- **Corpus gates, in the fst-advisor order:** Indonesian 121/121 first (cheap iteration); + guarded Sena slice (first 60 words, 5s/word watchdog); then the Sena 312-word heavy set. + Compare sequential-memo (`maxDegreeOfParallelism: 1`, the only sequential configuration + that exists post-stage-3 — the memo is unconditional whenever the cascade is sequential, + so "sequential-unmemoized" is no longer a reachable end-to-end configuration; that + comparison is instead covered at the unit level, see §5(b)) against parallel-unmemoized + master default — the actual user-visible claim. Must be **analysis-set identical** — same + canonical signature set, not byte-identical objects — including negative/no-parse words + (nogoods cache no-parses too — both sides must produce the empty set). Amharic + (machine-local, gitignored) as the alphabet-stress smoke test; never commit real-grammar + fixtures (privacy constraint). +- **Reporting** (standing requirement): every measured claim ships with p50/p95 per-word + latency, aggregate wall, and hit/miss counts (`DiagMemoHits`/`DiagNogoodHits`, split — + not just totals) — and stage 5 must assert `DiagMemoHits > 0` somewhere in the corpus + run, so the positive-replay path can't ship having never actually fired end-to-end. + +**Stage 5 result (local uncommitted grammar — Sena, both slices):** ran +`MemoCorpusVerification` against the local `samples/data/sena-hc.xml` / +`sena-words.txt` (never committed — grammar-privacy constraint). + +| Slice | Timeout | Compared | Timed out | No-parse (both) | Divergences | Mrule memo (pos/nogood) | Template memo (pos/nogood) | +|---|---|---|---|---|---|---|---| +| First 60 | 5s | 54 | 6 | 11 | **0** | 8,652 / 59,156 | 10,481 / 3 | +| Full 312 | 8s | 252 | 60 | 87 | **0** | 201,051 / 1,692,696 | 286,581 / 29 | + +Zero divergences across both runs, hundreds of thousands of memo hits on both +tables (not vacuous), on real analysis-rule content the toy-grammar unit tests +(stages 2-4) structurally cannot reach — this is the empirical confirmation +of `AnalysisStateKey`'s key-completeness audit. The full-312 run's timeouts +(60 words, 8s/side budget) are the same known pathological Bantu template +words the archive prototype measured; p50 678.7ms / p95 +7,997.2ms per word (memo-on + memo-off combined) on the successfully-compared +words. Full per-word timing table is in the run's `TestContext` output, not +committed (never commit real-word signature data). + +**Indonesian (121/121, PanGloss's local `indonesian-hc.xml`/`-words.txt`):** +**0 divergences**, 0 timeouts, p50 37.2ms / p95 346.0ms. Mrule memo: 64 +positive / 157 nogood. Template memo: **0 positive / 410 nogood** — Indonesian +has no Bantu-style affix-template redundancy, so the template table only ever +proves subtrees empty here, never replays; matches §6's own prediction that +typical (non-template) grammars see little upside. Per-heavy-word timing on +the 10 slowest words shows memo-on (sequential) is actually *slower* than +memo-off (parallel default) by ~10-15% on this grammar — expected and +consistent with §6: the memo only pays for itself where order-variant +redundancy exists, and sequential-single-core loses to parallel-multi-core +when it doesn't. This is the honest "no regression in the wrong direction on +non-Bantu grammars" finding, not a null result — the comparison being measured +here is user-visible (sequential-memo vs parallel-unmemoized default), not the +memo mechanism isolated from single- vs multi-threading (see §5's note on why +"sequential-unmemoized" isn't independently constructible once the mrule-cascade +memo is wired in). + +**Amharic (machine-local, gitignored, alphabet-stress smoke test):** 30 words, +8s/side timeout. **0 divergences** on the 6 words that finished both sides +within budget (10 timed out — Amharic's Ge'ez-script grammar is known to be +build/parse-heavy; 14 had no parse on either side). Mrule memo: 14 positive / +113 nogood. Template memo: 41 positive / 2 nogood — both fired. Confirms the +key/replay machinery is script-agnostic (Ge'ez abugida segments, not Latin +transliteration) with no divergence. No Amharic word forms appear in this doc +or any committed file, per the standing grammar-privacy constraint. + +**Summary across all three real grammars tested: zero divergences, in every +case with both memo tables exercised non-vacuously.** This is the strongest +available evidence for `AnalysisStateKey`'s key-completeness audit short of +running the entire, much larger production corpora. + +**Critical follow-up (advisor-flagged): the aggregate corpus runs above prove +nothing about the heavy words, because they timed out and were excluded from +the equality gate.** 60 of the full-312 Sena run's words never got compared — +those are exactly the template-heavy words the memo exists for, and the ones +where a key-completeness miss would actually surface. A short timeout also +means the only completed timings showed memo-on *slower* (Indonesian, +Amharic) — sequential-memo vs parallel-unmemoized, not evidence of the actual +speedup goal. Closed with a targeted, longer-timeout run on heavy word **H1** +(known-pathological, present in the Sena heavy-word set) at 240s: + +- **Soundness on a heavy word, actually executed:** memo-on vs memo-off — + analysis-set identical (0 divergences). Mrule memo: 29,736 positive / + 88,426 nogood hits. Template memo: 37,512 positive / 0 nogood hits — this + is the key-completeness confirmation the excluded corpus runs above could + not provide. +- **User-visible timing:** memo-on (sequential+memo) **22.1s** vs memo-off + (today's shipped parallel default) **136.1s** — **6.2×**. +- **Isolated memo contribution** (mutation-tested: temporarily disabled the + `AnalysisScope` install in `Morpher.ParseWord`, reverted after — same + technique as the ReplayOnto mutation test in §5(b)): sequential-no-memo + **138.0s** vs parallel-no-memo (unchanged) **141.5s** — parallelism alone + contributes essentially nothing on this word (~2%). Compared against + sequential-**memo**'s 22.1s: the memo mechanism itself is responsible for + essentially the entire **~6.3× speedup**, not a threading artifact. + +This exceeds the plan's own target (≥3× on heavy-word-class words) despite +master lacking the prototype's copy-on-write `Shape`/`FeatureStruct` — the +`ReplayOnto` deep-clone tax feared in §6 below did not dominate here. + +**Second heavy word, H2 — soundness inconclusive, not a clean win.** Two +independent runs at a 400s per-side timeout both had the memo-on +(sequential+memo) side fail to finish within 400s — i.e. this word is *not* +analysis-set-verified at this depth; the "Passed" result in the first run +reflects the harness's by-design exclusion of timed-out words from the +divergence check, not a completed comparison. This is a materially +different outcome from H1 and is reported honestly rather than folded into +the "0 divergences" summary above. Notably, the archive prototype's own +measurement (§1's table above) had H2 at 102.7s → 26.9s with the template +memo — master's ported memo-on run did not complete this word within 400s, +~15× slower than the archive's memoized figure. Plausible explanations +(unconfirmed): master's deep-clone `ReplayOnto` (vs the archive's +copy-on-write `Shape`/`FeatureStruct`) may cost materially more on this +word's particular redundancy shape than it did on H1; or H2 exercises a +rule-graph region where the memo's hit rate is lower and raw expansion +dominates. Not chased further with a longer timeout, per the same +bounded-effort judgment applied to the other unmeasured heavy words below — +documented as unmeasured/slower-than-expected, a concrete follow-up, not a +blocker for this PR (soundness is unaffected either way: an incomplete run +makes no claim, positive or negative, about key-completeness for this +word). + +Remaining honest gap: H1 is the one heavy word with a completed, positive +measurement; H2 is a heavy word with an *incomplete* one (see above); the +other 58 timed-out full-312-run words (and Indonesian/Amharic's timed-out +words) are unmeasured at this depth — a natural follow-up, not a blocker, +given the confirmed pattern (soundness holds, memo dominates the win) on +the one word measured to completion. + +**Aggregate corpus evidence: count-based and wall-clock, reported +separately (see the harness's own two-line report format).** A Sena +sub-corpus (words 1-70, 200s per-side timeout — long enough for the one +tractable heavy word in this range to complete on both sides, per §5's H1 +measurement above) gives, of the 69 words that completed within budget: +**53/69 (77%) faster under memo, 16/69 (23%) slower** — count-based, this +is the "yes, some words are slower" half of the picture. **Wall-clock: +memo-on total 114.6s vs memo-off total 277.8s across those same 69 +words — 2.42× faster in aggregate** — this is the "but a lot are faster, +and it nets out strongly positive" half. The one word that timed out at +200s (a second tractable-but-slow heavy word, distinct from both H1 and +H2) is excluded from both numbers. The harness's timeout wraps both the +memo-on and memo-off calls in one try block per word, so it cannot record +which side actually timed out for this word — if it was memo-on (the more +likely case here, since it's attempted first and this class of word is +exactly where the memo's own cost is least understood, see the H2 +discussion below), the word's true memo-off time is genuinely unmeasured, +not "presumably at least as slow." **This 2.42× figure should be read as +this slice's measured result, not asserted as a guaranteed lower bound** +in either direction — a longer timeout is the only way to actually find +out. Both memo tables fired heavily and non-vacuously on this slice +(mrule: tens of thousands of positive/nogood hits; template: tens of +thousands positive, near-zero nogood). + +**Is the typical-word slowdown "just" losing a thread? Mostly, not +purely.** Isolated on Indonesian (no template redundancy — the cleanest +instrument for this, since the template memo never replays there) via +three repeated runs per configuration to average out run-to-run JIT/GC +noise (single-run absolute times varied by as much as 60% between +otherwise-identical runs of the unchanged parallel baseline — only +within-run memo-on-vs-memo-off ratios are trustworthy here, not +across-run absolute milliseconds): + +| Configuration vs parallel-default | Aggregate ratio (3 reps) | Implied slowdown (1/ratio − 1) | +|---|---|---| +| sequential + memo (shipped opt-in) | 0.80×, 0.80×, 0.78× | ~25-28% | +| sequential, memo forced off (mutation-tested) | 0.69×, 0.80×, 0.68× | ~25-47% | + +The sequential-*no-memo* slowdown is consistently at or above the +sequential-*memo* slowdown, never below it. So most of the typical-word +regression genuinely is the lost thread (both configurations are slower +than parallel-default, by a similar order of magnitude) — but it is not +*purely* that: even on a grammar where the template memo never once +replays, the mrule-cascade's own cache still fires (64 positive / 157 +nogood hits on this corpus) and measurably claws back some of the +thread-loss penalty rather than adding to it. Three reps each is enough to +see the direction of the effect, not enough to state a precise recovered +percentage with confidence — a larger repeated-trial run would sharpen +this if the exact number ever matters for the default-flip decision. + +**Is the H2 shortfall actually the deep-clone tax?** Plausible, not +confirmed — resist stating it as established. The one piece of direct +evidence is that H2's memo fired heavily before timing out (tens of +thousands of positive hits on both tables, §5 above) — the memo is +clearly not idle on this word, which favors "replay/clone cost per hit is +what's expensive" over "the memo doesn't engage." But that is a +directional tell, not a measurement: distinguishing "expensive replay" from +"just a lot more raw expansion than H1" requires profiling where memo-on +wall time actually goes (`Clone`/`Freeze`/`ReplayOnto` vs raw rule +expansion) on H2 specifically. Scoped as a follow-up, not undertaken here. + +## 6. Expected outcome and honest caveats + +- Pathological template-heavy words: prototype showed ~5× (with COW clones). Master's + deep-clone `ReplayOnto` was expected to give back some of that; **measured directly on + heavy word H1 (§5 addendum): ~6.3× isolated memo contribution, ~6.2× on the + user-visible sequential-memo-vs-parallel-default comparison — met and exceeded the + ≥3× target**, meaning the feared `ReplayOnto` deep-clone tax did not dominate on this + word. The ≥50% Sena-heavy-set aggregate target remains unmeasured (that run's heavy + words timed out before completing the equality gate, §5 addendum) — a natural + follow-up. A second heavy word, H2, did NOT complete within a 400s memo-on budget (§5 + addendum) — the first concrete (if inconclusive) counter-data-point to the + deep-clone-tax question, since the archive's COW-based prototype handled this same + word in 26.9s. If a future heavy-set measurement shows replay cost dominating after + all, this is the evidence that would motivate porting `Shape` COW as its own + follow-up — do not fold it into this PR pre-emptively. +- Typical words (Indonesian-class) get measurably *slower* under sequential+memo + (~25-28% in aggregate, §5 addendum) — mostly the cost of losing a thread, not the memo + itself (isolated via a sequential-no-memo comparison, same addendum). The corpus gate + proves "no divergence" there, not "no regression" — this is exactly why the memo ships + off by default rather than flipping the library's default cascade mode. +- The memo is inert for `Ordered`-strata-only grammars and for `ParallelCombinationRuleCascade` + (no natural subtree-completion moment in its breadth-first walk — same reason the + prototype left it unmemoized). + +## 7. Out of scope (deliberately) + +Phase 0/1 instrumentation and synthesis-side `TrailDirectedRuleCascade`; Phase 4/5 gates +(measured inert or unsound-coupled); RUSTIFY COW polish (7b/10a); packed forest (9c); +word-level result caching above the engine; any interned-ID key redesign (`rust-conversion.md` +§6.3 is an unshipped plan — needs its own measurement if ever wanted). diff --git a/src/SIL.Machine.Morphology.HermitCrab/AnalysisScope.cs b/src/SIL.Machine.Morphology.HermitCrab/AnalysisScope.cs new file mode 100644 index 000000000..73d6cc99c --- /dev/null +++ b/src/SIL.Machine.Morphology.HermitCrab/AnalysisScope.cs @@ -0,0 +1,72 @@ +using System.Collections.Concurrent; +using System.Collections.Generic; + +namespace SIL.Machine.Morphology.HermitCrab +{ + /// + /// Per-parse cache carrier threaded through clones exactly like + /// -- reference-shared, excluded from Word.FreezeImpl/ + /// Word.ValueEquals so existing dedup semantics are unchanged. Holds the analysis-cascade memo + /// (memoization.md) -- see . + /// One instance per call: entries are state facts + /// about a specific parse (a state key does not encode the target surface word), so sharing this + /// across concurrent parses of different words would be unsound without also scoping the key to the + /// word -- that cross-word extension is explicitly out of scope. + /// Thread-safe because a single word's analysis can itself run in parallel + /// (ParallelCombinationRuleCascade, used when is not + /// 1) -- though today only the sequential cascade actually reads/writes this. + /// + internal sealed class AnalysisScope + { + // OOM guard: a positive memo holds actual Word lists (not just a boolean like the nogood case), so + // it is the one that can grow unboundedly on a pathological word. Past the cap, new subtrees are + // simply not memoized -- correctness is unaffected, only the hit rate degrades. No corpus word + // seen so far has come close to this cap; deliberately untested against an actual overflow. + private const int MaxMemoEntries = 100_000; + + public ConcurrentDictionary Memo { get; } = + new ConcurrentDictionary(); + + // Second table, same key space, different computation: the affix-template battery result for a + // state (AnalysisStratumRule.ApplyTemplateBattery), as opposed to the mrule-cascade subtree result + // above. Kept separate because a state can be memoized in one table but not (yet) the other. + public ConcurrentDictionary TemplateMemo { get; } = + new ConcurrentDictionary(); + + // Keys currently under expansion on some call stack -- guards the in-flight re-entry case (a + // multiApp cascade can reach the same state again before its own first expansion has completed, + // e.g. via a self-loop). A hit here must fall through to plain, unmemoized expansion rather than + // read a nonexistent/partial entry or deadlock; see MemoizedCombinationRuleCascade.ApplyRules. The + // template battery needs no equivalent guard: ApplyTemplateBattery's call is eager and + // self-contained (no template<->mrule mutual recursion happens inside it). + public ConcurrentDictionary InProgress { get; } = + new ConcurrentDictionary(); + + public bool HasMemoCapacity => Memo.Count < MaxMemoEntries; + + public bool HasTemplateMemoCapacity => TemplateMemo.Count < MaxMemoEntries; + } + + /// + /// A memoized analysis-cascade subtree or template-battery result (memoization.md). + /// empty = the "nogood" case (subtree/battery proved to yield nothing); non-empty = the positive case, + /// replayable onto a differently-ordered arrival at the same via + /// , using / + /// to split each stored result's trail/non-heads into the (discarded, replaced) prefix and the (kept) + /// subtree-local suffix. There is no "budget exhausted / incomplete" flag: this engine has no step/time + /// budget infrastructure, so every recorded subtree was explored to full completion. + /// + internal sealed class MemoEntry + { + public MemoEntry(IReadOnlyList results, int mruleTrailPrefixLength, int nonHeadPrefixLength) + { + Results = results; + MruleTrailPrefixLength = mruleTrailPrefixLength; + NonHeadPrefixLength = nonHeadPrefixLength; + } + + public IReadOnlyList Results { get; } + public int MruleTrailPrefixLength { get; } + public int NonHeadPrefixLength { get; } + } +} diff --git a/src/SIL.Machine.Morphology.HermitCrab/AnalysisStateKey.cs b/src/SIL.Machine.Morphology.HermitCrab/AnalysisStateKey.cs new file mode 100644 index 000000000..ca19fef0b --- /dev/null +++ b/src/SIL.Machine.Morphology.HermitCrab/AnalysisStateKey.cs @@ -0,0 +1,110 @@ +using System; +using System.Collections.Generic; +using SIL.Machine.Annotations; +using SIL.Machine.FeatureModel; + +namespace SIL.Machine.Morphology.HermitCrab +{ + /// + /// Order-independent identity of an analysis-cascade node, used by the memo cache in + /// 's Unordered-mode morphological rule cascade (memoization.md). Two + /// Words with an equal key are guaranteed to make identical decisions in every analysis-side + /// morphological rule that cascade can invoke -- verified by inspecting what each one reads from its + /// input (key-completeness audit, re-run whenever an Analysis*.cs rule changes): + /// + /// : Shape (FST pattern match) and + /// (unifiability gate) plus a per-rule unapplication count. + /// : adds + /// (MaxStemCount gate) -- it never reads the non-heads' own content, only the count. + /// : adds + /// . + /// + /// but never the ORDER those rules were unapplied in -- exactly the redundancy this key collapses. + /// Deliberately excludes fields Word.ValueEquals includes for a different purpose (result + /// dedup): the unapplication trail as an ordered SEQUENCE (replaced here by an order-independent + /// multiset) and _isLastAppliedRuleFinal/IsPartial, which are not read by any + /// analysis-side rule (grep-verified against every file matching MorphologicalRules/Analysis*.cs + /// and PhonologicalRules/Analysis*.cs). + /// + internal readonly struct AnalysisStateKey : IEquatable + { + private readonly Shape _shape; + private readonly Stratum _stratum; + private readonly FeatureStruct _syntacticFS; + private readonly FeatureStruct _realizationalFS; + private readonly int _nonHeadCount; + private readonly IReadOnlyDictionary _ruleCounts; + private readonly int _hashCode; + + public AnalysisStateKey(Word word) + { + _shape = word.Shape; + _stratum = word.Stratum; + _syntacticFS = word.SyntacticFeatureStruct; + _realizationalFS = word.RealizationalFeatureStruct; + _nonHeadCount = word.NonHeadCount; + _ruleCounts = word.UnappliedRuleCounts; + + // Defensive: AnalysisAffixTemplateRule.Apply reassigns SyntacticFeatureStruct to a fresh, + // unfrozen clone AFTER the owning Word is already frozen (no CheckFrozen() guard on that + // setter). Freeze() is idempotent and safe to call again here. + _shape.Freeze(); + _syntacticFS.Freeze(); + _realizationalFS.Freeze(); + + int hash = 17; + hash = hash * 31 + _shape.GetFrozenHashCode(); + hash = hash * 31 + (_stratum?.GetHashCode() ?? 0); + hash = hash * 31 + _syntacticFS.GetFrozenHashCode(); + hash = hash * 31 + _realizationalFS.GetFrozenHashCode(); + hash = hash * 31 + _nonHeadCount; + if (_ruleCounts != null) + { + // XOR, not the usual *31 rolling combine: the multiset is unordered, so the combination + // must be commutative -- two dictionaries with the same entries built up in different + // unapplication orders must hash identically. + int multisetHash = 0; + foreach (KeyValuePair kvp in _ruleCounts) + multisetHash ^= (kvp.Key.GetHashCode() * 397) ^ kvp.Value; + hash = hash * 31 + multisetHash; + } + _hashCode = hash; + } + + public override int GetHashCode() => _hashCode; + + public override bool Equals(object obj) => obj is AnalysisStateKey other && Equals(other); + + public bool Equals(AnalysisStateKey other) + { + if (_hashCode != other._hashCode) + return false; + if (_nonHeadCount != other._nonHeadCount || !ReferenceEquals(_stratum, other._stratum)) + return false; + if (!_shape.ValueEquals(other._shape)) + return false; + if (!_syntacticFS.ValueEquals(other._syntacticFS) || !_realizationalFS.ValueEquals(other._realizationalFS)) + return false; + return RuleCountsEqual(_ruleCounts, other._ruleCounts); + } + + private static bool RuleCountsEqual( + IReadOnlyDictionary a, + IReadOnlyDictionary b + ) + { + int aCount = a?.Count ?? 0; + int bCount = b?.Count ?? 0; + if (aCount != bCount) + return false; + if (aCount == 0) + return true; + foreach (KeyValuePair kvp in a) + { + if (!b.TryGetValue(kvp.Key, out int otherCount) || otherCount != kvp.Value) + return false; + } + return true; + } + } +} diff --git a/src/SIL.Machine.Morphology.HermitCrab/AnalysisStratumRule.cs b/src/SIL.Machine.Morphology.HermitCrab/AnalysisStratumRule.cs index 36d9557ad..e1414f8f6 100644 --- a/src/SIL.Machine.Morphology.HermitCrab/AnalysisStratumRule.cs +++ b/src/SIL.Machine.Morphology.HermitCrab/AnalysisStratumRule.cs @@ -46,19 +46,17 @@ public AnalysisStratumRule(Morpher morpher, Stratum stratum) ); break; case MorphologicalRuleOrder.Unordered: -#if SINGLE_THREADED - _mrulesRule = new CombinationRuleCascade( - mrules, - true, - FreezableEqualityComparer.Default - ); -#else - _mrulesRule = new ParallelCombinationRuleCascade( - mrules, - true, - FreezableEqualityComparer.Default - ); -#endif + // Sequential (and memoized, see MemoizedCombinationRuleCascade) when the caller caps + // within-word parallelism; parallel cascade otherwise. + _mrulesRule = + morpher.MaxDegreeOfParallelism == 1 + ? (IRule) + new MemoizedCombinationRuleCascade(mrules, FreezableEqualityComparer.Default) + : new ParallelCombinationRuleCascade( + mrules, + true, + FreezableEqualityComparer.Default + ); break; } } @@ -166,9 +164,51 @@ private IEnumerable ApplyMorphologicalRules(Word input) } } + // Test/reporting hooks (memoization.md's standing hit/miss-count requirement), mirroring + // MemoizedCombinationRuleCascade's DiagMemoHits/DiagNogoodHits split. + internal static long DiagTemplateMemoHits; + internal static long DiagTemplateNogoodHits; + + // Runs the affix-template battery for `input`, memoized by AnalysisStateKey (memoization.md), + // same replay mechanism as MemoizedCombinationRuleCascade -- see AnalysisScope.InProgress for + // why no re-entry guard is needed here. Measured motivation (an archive prototype benchmark on + // a Bantu-template grammar): this battery accounted for the large majority of parse wall time + // once the mrule cascade's own memo had already shrunk its own share to near-nothing. + private IEnumerable ApplyTemplateBattery(Word input) + { + AnalysisScope scope = input.AnalysisScope; + if (scope == null || _morpher.MaxDegreeOfParallelism != 1) + return _templatesRule.Apply(input); + + var key = new AnalysisStateKey(input); + if (scope.TemplateMemo.TryGetValue(key, out MemoEntry entry)) + { + if (entry.Results.Count == 0) + { + DiagTemplateNogoodHits++; + return Enumerable.Empty(); + } + var replayed = new List(entry.Results.Count); + foreach (Word stored in entry.Results) + replayed.Add(stored.ReplayOnto(input, entry.MruleTrailPrefixLength, entry.NonHeadPrefixLength)); + DiagTemplateMemoHits++; + return replayed; + } + + var results = new List(_templatesRule.Apply(input)); + if (scope.HasTemplateMemoCapacity) + { + scope.TemplateMemo.TryAdd( + key, + new MemoEntry(results, input.MorphologicalRuleTrailLength, input.NonHeadCount) + ); + } + return results; + } + private IEnumerable ApplyTemplates(Word input) { - foreach (Word tempOutWord in _templatesRule.Apply(input).Distinct(FreezableEqualityComparer.Default)) + foreach (Word tempOutWord in ApplyTemplateBattery(input).Distinct(FreezableEqualityComparer.Default)) { switch (_stratum.MorphologicalRuleOrder) { diff --git a/src/SIL.Machine.Morphology.HermitCrab/MemoizedCombinationRuleCascade.cs b/src/SIL.Machine.Morphology.HermitCrab/MemoizedCombinationRuleCascade.cs new file mode 100644 index 000000000..bad620f3a --- /dev/null +++ b/src/SIL.Machine.Morphology.HermitCrab/MemoizedCombinationRuleCascade.cs @@ -0,0 +1,122 @@ +using System.Collections.Generic; +using SIL.Machine.Annotations; +using SIL.Machine.Rules; + +namespace SIL.Machine.Morphology.HermitCrab +{ + /// + /// Drop-in replacement for the sequential on + /// Unordered-order analysis strata (memoization.md). Before expanding a node, checks whether an + /// earlier expansion elsewhere in the same word's analysis -- reached via a different unapplication + /// order, but with an equal -- already searched this exact state: + /// + /// proved empty (the "nogood" case) -> skip straight to "no results"; + /// produced results (the positive case) -> replay them () instead + /// of re-searching: clone each stored result and graft the CURRENT arrival's own trail/non-head + /// prefix onto the stored subtree-local suffix (see and + /// for why only the prefix -- never the suffix -- needs replacing). + /// + /// Scoped to the sequential cascade only. The parallel cascade + /// (ParallelCombinationRuleCascade, used when is + /// not 1) is a level-by-level breadth-first walk with no natural "this subtree is fully expanded" + /// moment to hang a memo write on, so it is left unmemoized -- callers that want this optimization + /// should construct their with maxDegreeOfParallelism: 1. + /// + internal class MemoizedCombinationRuleCascade : RuleCascade + { + // Test/reporting hooks (memoization.md's standing hit/miss-count requirement): DiagMemoHits counts + // positive replays (a stored non-empty subtree grafted onto a new arrival); DiagNogoodHits counts + // nogood hits (a stored EMPTY subtree short-circuited without any replay work). The equivalence + // tests that cover the replay path assert DiagMemoHits is nonzero so they can never go vacuous -- + // a memo that silently stopped firing would otherwise look exactly like a passing test. + internal static long DiagMemoHits; + internal static long DiagNogoodHits; + + public MemoizedCombinationRuleCascade( + IEnumerable> rules, + IEqualityComparer comparer + ) + : base(rules, true, comparer) { } + + public override IEnumerable Apply(Word input) + { + var output = new HashSet(Comparer); + ApplyRules(input, output); + return output; + } + + // Returns every result produced strictly within the subtree rooted at `input` (i.e. by applying + // one or more rules starting from `input`, at any depth) -- NOT including `input` itself. This is + // both the return value callers use and the value memoized against `input`'s AnalysisStateKey + // once the subtree finishes, so a later differently-ordered arrival at the same state can replay + // it via Word.ReplayOnto instead of re-searching. + private List ApplyRules(Word input, HashSet output) + { + AnalysisScope scope = input.AnalysisScope; + // See Word.AnalysisScope's doc for when this is null. + if (scope == null) + return ApplyRulesRaw(input, output); + + var key = new AnalysisStateKey(input); + + if (scope.Memo.TryGetValue(key, out MemoEntry entry)) + { + if (entry.Results.Count == 0) + { + DiagNogoodHits++; + return new List(); + } + var replayed = new List(entry.Results.Count); + foreach (Word storedResult in entry.Results) + { + Word replay = storedResult.ReplayOnto( + input, + entry.MruleTrailPrefixLength, + entry.NonHeadPrefixLength + ); + output.Add(replay); + replayed.Add(replay); + } + DiagMemoHits++; + return replayed; + } + + // In-flight re-entry guard, see AnalysisScope.InProgress. + if (!scope.InProgress.TryAdd(key, 0)) + return ApplyRulesRaw(input, output); + + List results; + try + { + results = ApplyRulesRaw(input, output); + } + finally + { + scope.InProgress.TryRemove(key, out _); + } + + // Past the cap, keep searching correctly, just stop growing the table (AnalysisScope.HasMemoCapacity). + if (scope.HasMemoCapacity) + scope.Memo.TryAdd(key, new MemoEntry(results, input.MorphologicalRuleTrailLength, input.NonHeadCount)); + + return results; + } + + private List ApplyRulesRaw(Word input, HashSet output) + { + var local = new List(); + for (int i = 0; i < Rules.Count; i++) + { + foreach (Word result in ApplyRule(Rules[i], i, input)) + { + local.Add(result); + output.Add(result); + // avoid infinite loop -- same guard CombinationRuleCascade uses + if (!Comparer.Equals(input, result)) + local.AddRange(ApplyRules(result, output)); + } + } + return local; + } + } +} diff --git a/src/SIL.Machine.Morphology.HermitCrab/Morpher.cs b/src/SIL.Machine.Morphology.HermitCrab/Morpher.cs index e4fd1879d..4f12f144a 100644 --- a/src/SIL.Machine.Morphology.HermitCrab/Morpher.cs +++ b/src/SIL.Machine.Morphology.HermitCrab/Morpher.cs @@ -28,10 +28,13 @@ public class Morpher : IMorphologicalAnalyzer, IMorphologicalGenerator private readonly ReadOnlyObservableCollection _morphemes; private readonly IList _lexicalPatterns = new List(); - public Morpher(ITraceManager traceManager, Language lang) + public Morpher(ITraceManager traceManager, Language lang, int maxDegreeOfParallelism = 0) { _lang = lang; _traceManager = traceManager; + // Must be set before CompileAnalysisRule: AnalysisStratumRule picks a sequential vs. parallel + // cascade for Unordered-order analysis strata at construction time based on this value. + MaxDegreeOfParallelism = maxDegreeOfParallelism; _allomorphTries = new Dictionary(); var morphemes = new ObservableList(); foreach (Stratum stratum in _lang.Strata) @@ -84,6 +87,15 @@ public ITraceManager TraceManager /// public bool MergeEquivalentAnalyses { get; set; } + /// + /// Caps parallelism used within a single parse's Unordered-order analysis-rule cascade. A value of + /// 1 selects the sequential cascade, which is also the only one eligible for the analysis-cascade + /// memo (see ); any other value (default 0) keeps the + /// existing parallel cascade. Set via the constructor: it influences how the analysis rules are + /// compiled. + /// + public int MaxDegreeOfParallelism { get; } + public Func LexEntrySelector { get; set; } public Func RuleSelector { get; set; } @@ -115,6 +127,11 @@ public IEnumerable ParseWord(string word, out object trace, bool guessRoot Shape shape = _lang.SurfaceStratum.CharacterDefinitionTable.Segment(word); var input = new Word(_lang.SurfaceStratum, shape); + // Only the sequential cascade reads AnalysisScope (see MemoizedCombinationRuleCascade); + // skip the allocation on the parallel path, and while tracing (tracing must stay + // byte-identical to the unmemoized engine). + if (!_traceManager.IsTracing && MaxDegreeOfParallelism == 1) + input.AnalysisScope = new AnalysisScope(); input.Freeze(); if (_traceManager.IsTracing) _traceManager.AnalyzeWord(_lang, input); diff --git a/src/SIL.Machine.Morphology.HermitCrab/Word.cs b/src/SIL.Machine.Morphology.HermitCrab/Word.cs index 9b29429e9..c98946fba 100644 --- a/src/SIL.Machine.Morphology.HermitCrab/Word.cs +++ b/src/SIL.Machine.Morphology.HermitCrab/Word.cs @@ -90,6 +90,7 @@ protected Word(Word word) _isLastAppliedRuleFinal = word._isLastAppliedRuleFinal; _isPartial = word._isPartial; CurrentTrace = word.CurrentTrace; + AnalysisScope = word.AnalysisScope; _disjunctiveAllomorphIndices = word._disjunctiveAllomorphIndices.ToDictionary( kvp => kvp.Key, kvp => new HashSet(kvp.Value) @@ -212,6 +213,16 @@ public IEnumerable MorphemesInApplicationOrder public object CurrentTrace { get; set; } + /// + /// Carrier for the analysis-cascade memo (memoization.md). Reference-shared like + /// , deliberately excluded from FreezeImpl and + /// ValueEquals so existing dedup semantics are unchanged. Null for words never routed + /// through (e.g. words built directly by + /// rule-level unit tests) or while tracing, in which case the cascade that reads this must fall + /// back to unmemoized behavior rather than throw. + /// + internal AnalysisScope AnalysisScope { get; set; } + public bool IsPartial { get { return _isPartial; } @@ -338,6 +349,12 @@ internal int GetUnapplicationCount(IMorphologicalRule mrule) return numUnapplies; } + /// + /// The full per-rule unapplication-count multiset backing , for + /// (order-independent analysis-cascade memoization, memoization.md). + /// + internal IReadOnlyDictionary UnappliedRuleCounts => _mrulesUnapplied; + /// /// Notifies this word synthesis that the specified morphological rule has applied. /// @@ -392,6 +409,14 @@ internal int NonHeadCount get { return _nonHeadApps.Count; } } + /// + /// Length of the morphological-rule trail so far -- _mruleApps.Count. Recorded alongside + /// at the point an 's subtree is + /// memoized (memoization.md), so a later differently-ordered arrival at the same key knows where + /// its own trail ends and the memoized subtree's suffix begins -- see . + /// + internal int MorphologicalRuleTrailLength => _mruleApps.Count; + internal void NonHeadUnapplied(Word nonHead) { CheckFrozen(); @@ -450,6 +475,52 @@ internal IList ExpandAlternatives() return alternatives; } + /// + /// Re-parents a Word computed while exploring the subtree below some analysis-cascade node N onto + /// -- a different Word that reached the same + /// as N via a different morphological-rule unapplication order + /// (memoization.md's positive memo; see ). Everything + /// computed strictly WITHIN the subtree -- deeper shape/feature edits, and any rules or non-heads + /// unapplied below N -- is a deterministic function of N's content alone (Shape, both + /// FeatureStructs, the rule-unapplication multiset, and non-head count all match between N and + /// by definition of an equal key), so it is kept as-is from `this`. + /// Only the two ORDERED structures the key deliberately summarizes as counts/multisets -- the + /// morphological-rule trail and the non-head list -- have their PREFIX (whatever was accumulated + /// before reaching N) replaced with 's own actual prefix, since arrival + /// order can only ever affect that part. + /// + /// The word that hit the memo -- its trail/non-heads become the new prefix. + /// + /// _mruleApps.Count of N at the moment its subtree was memoized -- everything in `this`'s + /// trail from this index on is the subtree-local suffix to keep. + /// + /// Same, for _nonHeadApps. + internal Word ReplayOnto(Word queryNode, int mruleTrailPrefixLength, int nonHeadPrefixLength) + { + Word clone = Clone(); + + List mruleSuffix = clone._mruleApps.GetRange( + mruleTrailPrefixLength, + clone._mruleApps.Count - mruleTrailPrefixLength + ); + clone._mruleApps.Clear(); + clone._mruleApps.AddRange(queryNode._mruleApps); + clone._mruleApps.AddRange(mruleSuffix); + clone._mruleAppIndex = clone._mruleApps.Count - 1; + + List nonHeadSuffix = clone._nonHeadApps.GetRange( + nonHeadPrefixLength, + clone._nonHeadApps.Count - nonHeadPrefixLength + ); + clone._nonHeadApps.Clear(); + clone._nonHeadApps.AddRange(queryNode._nonHeadApps.CloneItems()); + clone._nonHeadApps.AddRange(nonHeadSuffix); + clone._nonHeadAppIndex = clone._nonHeadApps.Count - 1; + + clone.Freeze(); + return clone; + } + public Allomorph GetAllomorph(Annotation morph) { var alloID = (string)morph.FeatureStruct.GetValue(HCFeatureSystem.Allomorph); diff --git a/tests/SIL.Machine.Morphology.HermitCrab.Tests/AnalysisStateKeyTests.cs b/tests/SIL.Machine.Morphology.HermitCrab.Tests/AnalysisStateKeyTests.cs new file mode 100644 index 000000000..29469988f --- /dev/null +++ b/tests/SIL.Machine.Morphology.HermitCrab.Tests/AnalysisStateKeyTests.cs @@ -0,0 +1,148 @@ +using NUnit.Framework; +using SIL.Machine.FeatureModel; +using SIL.Machine.Morphology.HermitCrab.MorphologicalRules; + +namespace SIL.Machine.Morphology.HermitCrab; + +// Unit battery for the analysis-cascade memo's primitives (memoization.md Stage 2): AnalysisStateKey's +// order-invariance over the rule-unapplication multiset, its sensitivity to every other field the key +// captures, and Word.ReplayOnto's prefix/suffix graft. Nothing in production code consumes these yet +// (that is Stage 3) -- these tests pin the primitives' own contract in isolation, independent of any +// particular cascade wiring. +[TestFixture] +public class AnalysisStateKeyTests : HermitCrabTestBase +{ + [Test] + public void Equals_IsInvariantOverUnapplicationOrder_ForEqualMultisets() + { + var ruleA = new AffixProcessRule { Name = "ruleA" }; + var ruleB = new AffixProcessRule { Name = "ruleB" }; + + // Same multiset {ruleA: 2, ruleB: 1}, built up in two genuinely different orders -- + // wordY touches ruleB first, unlike wordX, so this also exercises different backing + // dictionary insertion order, not just a different position for a repeated rule. + Word wordX = NewTestWord(); + wordX.MorphologicalRuleUnapplied(ruleA); + wordX.MorphologicalRuleUnapplied(ruleB); + wordX.MorphologicalRuleUnapplied(ruleA); + wordX.Freeze(); + + Word wordY = NewTestWord(); + wordY.MorphologicalRuleUnapplied(ruleB); + wordY.MorphologicalRuleUnapplied(ruleA); + wordY.MorphologicalRuleUnapplied(ruleA); + wordY.Freeze(); + + var keyX = new AnalysisStateKey(wordX); + var keyY = new AnalysisStateKey(wordY); + + Assert.That(keyX.GetHashCode(), Is.EqualTo(keyY.GetHashCode())); + Assert.That(keyX.Equals(keyY), Is.True); + } + + [Test] + public void Equals_False_WhenUnapplicationMultisetsDiffer() + { + var ruleA = new AffixProcessRule { Name = "ruleA" }; + + Word wordX = NewTestWord(); + wordX.MorphologicalRuleUnapplied(ruleA); + wordX.Freeze(); + + Word wordY = NewTestWord(); + wordY.MorphologicalRuleUnapplied(ruleA); + wordY.MorphologicalRuleUnapplied(ruleA); + wordY.Freeze(); + + Assert.That(new AnalysisStateKey(wordX).Equals(new AnalysisStateKey(wordY)), Is.False); + } + + [Test] + public void Equals_False_WhenNonHeadCountDiffers() + { + Word wordX = NewTestWord(); + wordX.Freeze(); + + Word wordY = NewTestWord(); + Word nonHead = NewTestWord(); + nonHead.Freeze(); + wordY.NonHeadUnapplied(nonHead); + wordY.Freeze(); + + Assert.That(new AnalysisStateKey(wordX).Equals(new AnalysisStateKey(wordY)), Is.False); + } + + [Test] + public void Equals_False_WhenSyntacticFeatureStructDiffers() + { + Word wordX = NewTestWord(); + wordX.SyntacticFeatureStruct = FeatureStruct.New(Language.SyntacticFeatureSystem).Symbol("V").Value; + wordX.Freeze(); + + Word wordY = NewTestWord(); + wordY.SyntacticFeatureStruct = FeatureStruct.New(Language.SyntacticFeatureSystem).Symbol("N").Value; + wordY.Freeze(); + + Assert.That(new AnalysisStateKey(wordX).Equals(new AnalysisStateKey(wordY)), Is.False); + } + + [Test] + public void ReplayOnto_GraftsQueryPrefixOntoStoredSuffix_ForMruleTrail() + { + var ruleA = new AffixProcessRule { Name = "ruleA" }; + var ruleB = new AffixProcessRule { Name = "ruleB" }; + var ruleC = new AffixProcessRule { Name = "ruleC" }; + + // The memoized node's own trail was [ruleA, ruleB] at the moment its subtree was recorded, with + // ruleA as the length-1 prefix (whatever led to this state) and ruleB as the subtree-local suffix + // that must survive the graft. + Word memoized = NewTestWord(); + memoized.MorphologicalRuleUnapplied(ruleA); + memoized.MorphologicalRuleUnapplied(ruleB); + memoized.Freeze(); + + // A different arrival at the same AnalysisStateKey, via a different prefix: [ruleC]. + Word query = NewTestWord(); + query.MorphologicalRuleUnapplied(ruleC); + query.Freeze(); + + Word replayed = memoized.ReplayOnto(query, mruleTrailPrefixLength: 1, nonHeadPrefixLength: 0); + + Assert.That(replayed.MorphologicalRules, Is.EqualTo(new IMorphologicalRule[] { ruleC, ruleB })); + } + + [Test] + public void ReplayOnto_GraftsQueryPrefixOntoStoredSuffix_ForNonHeads() + { + // The memoized node had already unapplied one non-head (its own prefix, at the memo point) before + // its subtree unapplied a second one -- that second one is the subtree-local suffix to keep. The + // two non-heads are built from distinct lexical entries (32 vs 33) so a wrong-prefix graft (e.g. + // one that kept storedNonHead instead of subtreeNonHead, or reversed the GetRange window) is + // actually distinguishable via RootAllomorph identity, not just NonHeadCount. + Word storedNonHead = NewTestWord("32"); + storedNonHead.Freeze(); + Word subtreeNonHead = NewTestWord("33"); + subtreeNonHead.Freeze(); + Word memoized = NewTestWord("32"); + memoized.NonHeadUnapplied(storedNonHead); + memoized.NonHeadUnapplied(subtreeNonHead); + memoized.Freeze(); + + // Query reached the same key with a different (empty) non-head prefix. + Word query = NewTestWord("32"); + query.Freeze(); + + Word replayed = memoized.ReplayOnto(query, mruleTrailPrefixLength: 0, nonHeadPrefixLength: 1); + + // Query's (empty) prefix + the memoized subtree's suffix (subtreeNonHead) = 1 non-head. + Assert.That(replayed.NonHeadCount, Is.EqualTo(1)); + Assert.That(replayed.CurrentNonHead.RootAllomorph, Is.SameAs(subtreeNonHead.RootAllomorph)); + Assert.That(replayed.CurrentNonHead.RootAllomorph, Is.Not.SameAs(storedNonHead.RootAllomorph)); + } + + private Word NewTestWord(string entryId = "32") + { + var word = new Word(Entries[entryId].PrimaryAllomorph, FeatureStruct.New().Value) { Stratum = Morphophonemic }; + return word; + } +} diff --git a/tests/SIL.Machine.Morphology.HermitCrab.Tests/MemoCorpusVerification.cs b/tests/SIL.Machine.Morphology.HermitCrab.Tests/MemoCorpusVerification.cs new file mode 100644 index 000000000..be195a77f --- /dev/null +++ b/tests/SIL.Machine.Morphology.HermitCrab.Tests/MemoCorpusVerification.cs @@ -0,0 +1,234 @@ +using System.Diagnostics; +using NUnit.Framework; + +namespace SIL.Machine.Morphology.HermitCrab; + +/// +/// Stage 5 of memoization.md: the real-data verification the toy-grammar unit tests (stages 2-4) +/// cannot provide. The mrule/template unit tests use 2-3-rule synthetic grammars specifically because +/// they let a specific redundant-order scenario be forced deterministically -- but they cannot exercise +/// whether 's key-completeness audit actually holds against the FULL +/// analysis-side rule set a real grammar invokes (AnalysisAffixProcessRule, +/// AnalysisCompoundingRule, AnalysisRealizationalAffixProcessRule, and every phonological +/// rule feeding into the states those rules read). If the key is missing a field some real rule +/// consults, it manifests as a divergence on some real word, nowhere else -- this harness is that check. +/// +/// [Explicit] and env-var driven, modeled on FstSenaBenchmark.cs's convention (this repo never commits +/// real morphological grammars or word lists -- see the standing grammar-privacy constraint -- so this +/// test ships with zero embedded grammar/word content and writes no output files, only TestContext +/// lines, to avoid leaking derived corpus data such as signature dumps into a committed path): +/// $env:HC_MEMO_GRAMMAR = "...\sena-hc.xml" +/// $env:HC_MEMO_WORDS = "...\sena-words.txt" +/// $env:HC_MEMO_MAX_WORDS = "60" # optional, default 60 +/// $env:HC_MEMO_TIMEOUT_MS = "5000" # optional, default 5000 (per-word watchdog) +/// dotnet test --filter "FullyQualifiedName~MemoCorpusVerification" +/// +/// Deliberately NOT the archive's 900-line parallel-benchmark harness (16-way scheduling, GC heap-limit +/// watchdog): per design, the only new runtime configuration this port introduces is single-threaded +/// (maxDegreeOfParallelism: 1), so there is no new concurrency to stress-test here -- just a +/// per-word timeout so one pathological word can't hang the whole run. +/// +[TestFixture] +[Explicit("Manual corpus verification against a local, uncommitted real grammar; not part of CI.")] +public class MemoCorpusVerification +{ + [Test] + public void MemoOnMatchesMemoOff_AnalysisSetIdentical_OnRealCorpus() + { + (Language language, List words) = Load(); + + var memoOff = new Morpher(new TraceManager(), language); + var memoOn = new Morpher(new TraceManager(), language, maxDegreeOfParallelism: 1); + int timeoutMs = int.TryParse(Environment.GetEnvironmentVariable("HC_MEMO_TIMEOUT_MS"), out int t) ? t : 5000; + + long mruleHitsBefore = MemoizedCombinationRuleCascade.DiagMemoHits; + long mruleNogoodsBefore = MemoizedCombinationRuleCascade.DiagNogoodHits; + long templateHitsBefore = AnalysisStratumRule.DiagTemplateMemoHits; + long templateNogoodsBefore = AnalysisStratumRule.DiagTemplateNogoodHits; + + var elapsedMsPerWord = new List(); + var perWordTimes = new List<(string Word, double OnMs, double OffMs)>(); + var divergences = new List(); + var timedOut = new List(); + int noParseBoth = 0; + + foreach (string word in words) + { + List onSignatures; + List offSignatures; + double onMs; + double offMs; + try + { + var swOn = Stopwatch.StartNew(); + onSignatures = RunWithTimeout(() => Signatures(memoOn, word), timeoutMs); + swOn.Stop(); + onMs = swOn.Elapsed.TotalMilliseconds; + + var swOff = Stopwatch.StartNew(); + offSignatures = RunWithTimeout(() => Signatures(memoOff, word), timeoutMs); + swOff.Stop(); + offMs = swOff.Elapsed.TotalMilliseconds; + } + catch (TimeoutException) + { + timedOut.Add(word); + continue; + } + elapsedMsPerWord.Add(onMs + offMs); + perWordTimes.Add((word, onMs, offMs)); + + if (onSignatures.Count == 0 && offSignatures.Count == 0) + noParseBoth++; + + if (!onSignatures.SequenceEqual(offSignatures)) + { + divergences.Add( + $"{word}: memo-on={{{string.Join(",", onSignatures)}}} vs " + + $"memo-off={{{string.Join(",", offSignatures)}}}" + ); + } + } + + elapsedMsPerWord.Sort(); + double p50 = Percentile(elapsedMsPerWord, 0.50); + double p95 = Percentile(elapsedMsPerWord, 0.95); + double totalMs = elapsedMsPerWord.Sum(); + + TestContext.Out.WriteLine($"words attempted: {words.Count}, timed out (>{timeoutMs}ms): {timedOut.Count}"); + TestContext.Out.WriteLine($"words with no parse on both sides: {noParseBoth}"); + TestContext.Out.WriteLine($"aggregate wall: {totalMs:F1} ms, p50: {p50:F1} ms, p95: {p95:F1} ms"); + + // Count-based vs wall-clock aggregates, reported SEPARATELY on purpose: a corpus is typically + // bimodal (many cheap words the memo makes slightly slower by losing a thread; a few pathological + // words the memo makes drastically faster) -- collapsing that into one ratio hides which regime a + // reader is in. "Most words are a bit slower" and "the corpus finishes much faster in total" are + // both true simultaneously and neither contradicts the other. + int fasterCount = perWordTimes.Count(x => x.OnMs < x.OffMs); + int slowerCount = perWordTimes.Count(x => x.OnMs > x.OffMs); + int tiedCount = perWordTimes.Count - fasterCount - slowerCount; + double totalOnMs = perWordTimes.Sum(x => x.OnMs); + double totalOffMs = perWordTimes.Sum(x => x.OffMs); + TestContext.Out.WriteLine( + $"count-based: {fasterCount}/{perWordTimes.Count} words faster under memo, " + + $"{slowerCount}/{perWordTimes.Count} slower, {tiedCount} tied" + ); + TestContext.Out.WriteLine( + $"wall-clock: memo-on total {totalOnMs:F1} ms vs memo-off total {totalOffMs:F1} ms " + + $"({(totalOnMs > 0 ? totalOffMs / totalOnMs : 0):F2}x)" + ); + if (timedOut.Count > 0) + { + // NOT a guaranteed lower bound in either direction: the try block above wraps BOTH the + // memo-on and memo-off calls, so a timeout could come from either side -- this harness + // never records which one actually timed out. If it was memo-on, the word's true + // memo-off time is genuinely unmeasured (could be faster OR slower than the ratio above + // implies); only a longer timeout actually resolves it. Report the fact, don't imply a + // direction the data doesn't support. + TestContext.Out.WriteLine( + $"(the {timedOut.Count} timed-out word(s) above are excluded from both aggregates; " + + "re-run with a higher HC_MEMO_TIMEOUT_MS to actually measure them)" + ); + } + // Per-heavy-word attribution (memoization.md's own methodological rule: an aggregate can be + // dominated by cheap words while hiding what pathological words actually do -- report both). + // memo-on is sequential+memo; memo-off is the untouched parallel default, so this is the + // user-visible claim (sequential-memo vs today's shipped behavior), not an isolated measurement + // of the memo mechanism's own contribution in isolation from single- vs multi-threading. + TestContext.Out.WriteLine("heaviest words (by memo-off time), memo-on vs memo-off:"); + foreach ((string w, double onMs2, double offMs2) in perWordTimes.OrderByDescending(x => x.OffMs).Take(10)) + TestContext.Out.WriteLine($" {w}: memo-on {onMs2:F1} ms, memo-off {offMs2:F1} ms"); + TestContext.Out.WriteLine( + $"mrule memo -- positive hits: {MemoizedCombinationRuleCascade.DiagMemoHits - mruleHitsBefore}, " + + $"nogood hits: {MemoizedCombinationRuleCascade.DiagNogoodHits - mruleNogoodsBefore}" + ); + TestContext.Out.WriteLine( + $"template memo -- positive hits: {AnalysisStratumRule.DiagTemplateMemoHits - templateHitsBefore}, " + + $"nogood hits: {AnalysisStratumRule.DiagTemplateNogoodHits - templateNogoodsBefore}" + ); + if (timedOut.Count > 0) + { + // Named, not just counted: a timed-out word is EXCLUDED from the equality gate above, so + // "0 divergences" says nothing about it. These are exactly the candidates for a follow-up + // run with a longer HC_MEMO_TIMEOUT_MS (see memoization.md §5's addendum on why this + // mattered -- the heavy words are precisely the ones the memo, and the key-completeness + // audit, most need to be checked against). + TestContext.Out.WriteLine( + $"timed-out words (excluded from the equality gate above -- re-run with a higher " + + $"HC_MEMO_TIMEOUT_MS to actually check these): {string.Join(", ", timedOut)}" + ); + } + + Assert.That( + divergences, + Is.Empty, + $"{divergences.Count} word(s) diverged between memo-on and memo-off " + + $"(showing up to 10): {string.Join(" | ", divergences.Take(10))}" + ); + Assert.That( + MemoizedCombinationRuleCascade.DiagMemoHits + AnalysisStratumRule.DiagTemplateMemoHits, + Is.GreaterThan(mruleHitsBefore + templateHitsBefore), + "the positive replay path must actually have fired somewhere in this corpus -- otherwise " + + "this run cannot distinguish a working memo from a no-op one" + ); + } + + private static List Signatures(Morpher morpher, string word) + { + try + { + return morpher + .ParseWord(word) + .Select(MorpherTests.WordAnalysisSignature) + .OrderBy(s => s, StringComparer.Ordinal) + .ToList(); + } + catch (InvalidShapeException) + { + // Matches Morpher.AnalyzeWord's own handling: a word list drawn from real text can contain + // strings the grammar's character table doesn't cover (e.g. punctuation) -- not a memo + // concern, both sides would throw identically. + return new List(); + } + } + + // Does NOT cancel `action` on timeout -- Morpher.ParseWord has no cooperative-cancellation + // hook, so a timed-out word's Task keeps running in the background. This can inflate the + // hit/nogood counters and per-word timings reported below with work from an abandoned prior + // word, and piled-up orphaned tasks from several timed-out words in a row can starve the + // thread pool. Acceptable for this [Explicit], never-in-CI diagnostic harness (the equality + // gate itself is unaffected -- a timed-out word is excluded from it either way, see the + // caller), but do not read the reported hit counts/timings as precise when timeouts occurred. + private static T RunWithTimeout(Func action, int timeoutMs) + { + Task task = Task.Run(action); + if (!task.Wait(timeoutMs)) + throw new TimeoutException(); + return task.Result; + } + + private static double Percentile(List sortedValues, double fraction) + { + if (sortedValues.Count == 0) + return 0; + int index = (int)Math.Ceiling(fraction * sortedValues.Count) - 1; + return sortedValues[Math.Clamp(index, 0, sortedValues.Count - 1)]; + } + + private static (Language, List) Load() + { + string? grammarPath = Environment.GetEnvironmentVariable("HC_MEMO_GRAMMAR"); + string? wordsPath = Environment.GetEnvironmentVariable("HC_MEMO_WORDS"); + if (string.IsNullOrEmpty(grammarPath) || string.IsNullOrEmpty(wordsPath)) + Assert.Ignore("set HC_MEMO_GRAMMAR and HC_MEMO_WORDS"); + + int maxWords = int.TryParse(Environment.GetEnvironmentVariable("HC_MEMO_MAX_WORDS"), out int mw) ? mw : 60; + Language language = XmlLanguageLoader.Load(grammarPath!); + List words = File.ReadAllLines(wordsPath!) + .Select(w => w.Trim()) + .Where(w => w.Length > 0) + .Take(maxWords) + .ToList(); + return (language, words); + } +} diff --git a/tests/SIL.Machine.Morphology.HermitCrab.Tests/MemoizedCombinationRuleCascadeTests.cs b/tests/SIL.Machine.Morphology.HermitCrab.Tests/MemoizedCombinationRuleCascadeTests.cs new file mode 100644 index 000000000..2caf0b61f --- /dev/null +++ b/tests/SIL.Machine.Morphology.HermitCrab.Tests/MemoizedCombinationRuleCascadeTests.cs @@ -0,0 +1,189 @@ +using NUnit.Framework; +using SIL.Machine.Annotations; +using SIL.Machine.FeatureModel; +using SIL.Machine.Morphology.HermitCrab.MorphologicalRules; +using SIL.Machine.Rules; +using SIL.ObjectModel; + +namespace SIL.Machine.Morphology.HermitCrab; + +// Stage 3 of memoization.md: MemoizedCombinationRuleCascade exercised directly, bypassing +// Morpher/AnalysisStratumRule entirely, so the test can force the exact commuting-order redundancy the +// memo targets without depending on whether a particular morphology grammar's search happens to revisit +// a PRODUCTIVE (not just nogood) state -- MorpherTests' end-to-end compounding grammar, checked via the +// real Morpher pipeline, only ever hits the nogood table on this small a grammar (see its own hit-count +// diagnostic output), so this is the test that pins the POSITIVE replay path non-vacuously. +[TestFixture] +public class MemoizedCombinationRuleCascadeTests : HermitCrabTestBase +{ + [Test] + public void Apply_ReplaysPositiveHit_WhenTwoOrdersReachTheSameKey() + { + var ruleA = new AffixProcessRule { Name = "ruleA" }; + var ruleB = new AffixProcessRule { Name = "ruleB" }; + var ruleC = new AffixProcessRule { Name = "ruleC" }; + + // Each fake rule unapplies its own IMorphologicalRule at most once, so from the initial word, + // trying ruleA-then-ruleB-then-ruleC and ruleB-then-ruleA-then-ruleC reach the SAME + // AnalysisStateKey after the first two steps (multiset {ruleA:1, ruleB:1}) via different orders + // -- exactly the redundancy AnalysisStateKey collapses. Only from that shared state can ruleC + // still apply, so the shared node's own subtree is POSITIVE (one result), not a nogood. + var cascade = new MemoizedCombinationRuleCascade( + new IRule[] + { + new SingleUseUnapplyRule(ruleA), + new SingleUseUnapplyRule(ruleB), + new SingleUseUnapplyRule(ruleC), + }, + FreezableEqualityComparer.Default + ); + + Word initial = NewTestWord(); + initial.AnalysisScope = new AnalysisScope(); + initial.Freeze(); + + long hitsBefore = MemoizedCombinationRuleCascade.DiagMemoHits; + List results = new List(cascade.Apply(initial)); + + // Every leaf where all 3 rules have been unapplied, in whichever of the two orders explored the + // shared {ruleA,ruleB} state first vs via replay, must appear -- and the replay path must have + // actually fired (memoization.md's non-vacuousness requirement). + Assert.That( + results, + Has.Some.Matches(w => + w.GetUnapplicationCount(ruleA) == 1 + && w.GetUnapplicationCount(ruleB) == 1 + && w.GetUnapplicationCount(ruleC) == 1 + ) + ); + Assert.That( + MemoizedCombinationRuleCascade.DiagMemoHits, + Is.GreaterThan(hitsBefore), + "this test's whole point is to force a positive replay -- it must not go vacuous" + ); + } + + [Test] + public void Apply_PositiveReplayMatchesUnmemoizedResultSet_IncludingTrailOrder() + { + // The previous test proves a replay FIRES; this one proves it produces the RIGHT result. A count + // assertion (3 rules applied) would pass even if ReplayOnto grafted the wrong prefix, since counts + // are order-invariant -- comparing MorphemesInApplicationOrder (which walks the trail ReplayOnto + // rewrites, see WordAnalysisSignature's own doc comment in MorpherTests.cs) catches a wrong-prefix + // graft that silently collapses [ruleB,ruleA,ruleC] into a duplicate of [ruleA,ruleB,ruleC]. + var ruleA = new AffixProcessRule { Id = "RULE_A", Name = "ruleA" }; + var ruleB = new AffixProcessRule { Id = "RULE_B", Name = "ruleB" }; + var ruleC = new AffixProcessRule { Id = "RULE_C", Name = "ruleC" }; + IRule[] rules = + { + new SingleUseUnapplyRule(ruleA), + new SingleUseUnapplyRule(ruleB), + new SingleUseUnapplyRule(ruleC), + }; + + Word memoized = NewTestWord(); + memoized.AnalysisScope = new AnalysisScope(); + memoized.Freeze(); + + Word unmemoized = NewTestWord(); + // AnalysisScope left null -- exercises MemoizedCombinationRuleCascade's own unmemoized fallback + // (ApplyRulesRaw), the same path a tracing parse takes. + unmemoized.Freeze(); + + var memoizedCascade = new MemoizedCombinationRuleCascade(rules, FreezableEqualityComparer.Default); + var unmemoizedCascade = new MemoizedCombinationRuleCascade(rules, FreezableEqualityComparer.Default); + + long hitsBefore = MemoizedCombinationRuleCascade.DiagMemoHits; + List memoizedSignatures = memoizedCascade + .Apply(memoized) + .Select(TrailSignature) + .OrderBy(s => s, StringComparer.Ordinal) + .ToList(); + List unmemoizedSignatures = unmemoizedCascade + .Apply(unmemoized) + .Select(TrailSignature) + .OrderBy(s => s, StringComparer.Ordinal) + .ToList(); + + Assert.That( + memoizedSignatures, + Is.EqualTo(unmemoizedSignatures), + "a positive replay must reproduce exactly the unmemoized result set, INCLUDING trail order" + ); + Assert.That( + MemoizedCombinationRuleCascade.DiagMemoHits, + Is.GreaterThan(hitsBefore), + "this test's whole point is to compare a real replay against the unmemoized result -- it " + + "must not go vacuous" + ); + } + + [Test] + public void Apply_FallsBackToUnmemoizedExpansion_WhenKeyIsAlreadyInProgress() + { + // Pins the in-flight re-entry guard (memoization.md's design rule 3 / the InProgress table): + // a multiApp cascade can reach the same AnalysisStateKey again while its own first expansion is + // still on the call stack (e.g. via a self-loop elsewhere in a real grammar's rule graph). + // Rather than construct that reentrancy organically -- the key is monotonic in rule-application + // count for straightforward single-use rules, so a real cyclic re-arrival is hard to force here + // -- this simulates the in-flight state directly: pre-populate InProgress with the exact key + // `initial` will compute, then call Apply and confirm it falls through to a correct, unmemoized + // expansion (ApplyRulesRaw) instead of reading a nonexistent Memo entry or hanging. + var ruleA = new AffixProcessRule { Id = "RULE_A", Name = "ruleA" }; + var cascade = new MemoizedCombinationRuleCascade( + new IRule[] { new SingleUseUnapplyRule(ruleA) }, + FreezableEqualityComparer.Default + ); + + Word initial = NewTestWord(); + var scope = new AnalysisScope(); + initial.AnalysisScope = scope; + initial.Freeze(); + + var key = new AnalysisStateKey(initial); + scope.InProgress.TryAdd(key, 0); + + long hitsBefore = MemoizedCombinationRuleCascade.DiagMemoHits; + List results = new List(cascade.Apply(initial)); + + Assert.That(results, Has.Some.Matches(w => w.GetUnapplicationCount(ruleA) == 1)); + Assert.That( + MemoizedCombinationRuleCascade.DiagMemoHits, + Is.EqualTo(hitsBefore), + "the in-flight fallback must not read/count a memo hit -- it never consults Memo at all" + ); + Assert.That( + scope.Memo.ContainsKey(key), + Is.False, + "the in-flight arrival's OWN key must never be written to Memo (deeper recursive calls for " + + "OTHER keys, reached via ApplyRulesRaw's normal recursion, may still memoize themselves)" + ); + } + + private static string TrailSignature(Word word) => + string.Join("+", word.MorphemesInApplicationOrder.Select(m => m.Id)); + + private Word NewTestWord() + { + return new Word(Entries["32"].PrimaryAllomorph, FeatureStruct.New().Value) { Stratum = Morphophonemic }; + } + + // Minimal stand-in for a compiled analysis morphological rule: unapplies `_rule` against the input + // exactly once (per input), independent of Shape/FeatureStruct pattern matching, so the cascade's + // own commuting-order redundancy can be exercised without compiling a real FST-backed rule. + private sealed class SingleUseUnapplyRule(IMorphologicalRule rule) : IRule + { + private readonly IMorphologicalRule _rule = rule; + + public IEnumerable Apply(Word input) + { + if (input.GetUnapplicationCount(_rule) > 0) + yield break; + + Word result = input.Clone(); + result.MorphologicalRuleUnapplied(_rule); + result.Freeze(); + yield return result; + } + } +} diff --git a/tests/SIL.Machine.Morphology.HermitCrab.Tests/MorpherTests.cs b/tests/SIL.Machine.Morphology.HermitCrab.Tests/MorpherTests.cs index eb8944ad0..de52d1cc0 100644 --- a/tests/SIL.Machine.Morphology.HermitCrab.Tests/MorpherTests.cs +++ b/tests/SIL.Machine.Morphology.HermitCrab.Tests/MorpherTests.cs @@ -454,4 +454,302 @@ IList GetNodes(string pattern) Shape shape = new Segments(Table2, pattern, true).Shape; return shape.GetNodes(shape.Range).ToList(); } + + [Test] + public void ParseWord_SingleThreaded_MatchesParallel_WithCompounding() + { + // Stage 1 of memoization.md: pins the new runtime MaxDegreeOfParallelism toggle to be a pure + // no-op on results before any memoization is wired up. Compounding specifically (not just plain + // affixes) because it's the mechanism the eventual analysis-cascade memo cares about: an affix + // rule that commutes with a compounding rule -- both peers in the same Unordered + // MorphologicalRules cascade -- can revisit an equal AnalysisStateKey via different arrival + // orders, so this grammar is the one later commits' equivalence tests build on. + var any = FeatureStruct.New().Symbol(HCFeatureSystem.Segment).Value; + var crule = new CompoundingRule { Name = "rule1" }; + Allophonic.MorphologicalRules.Add(crule); + crule.Subrules.Add( + new CompoundingSubrule + { + HeadLhs = { Pattern.New("head").Annotation(any).OneOrMore.Value }, + NonHeadLhs = { Pattern.New("nonHead").Annotation(any).OneOrMore.Value }, + Rhs = { new CopyFromInput("head"), new InsertSegments(Table3, "+"), new CopyFromInput("nonHead") }, + } + ); + + var prefix = new AffixProcessRule + { + Id = "PREFIX", + Name = "prefix", + Gloss = "PAST", + RequiredSyntacticFeatureStruct = FeatureStruct.New(Language.SyntacticFeatureSystem).Symbol("V").Value, + OutSyntacticFeatureStruct = FeatureStruct + .New(Language.SyntacticFeatureSystem) + .Feature(Head) + .EqualTo(head => head.Feature("tense").EqualTo("past")) + .Value, + }; + Allophonic.MorphologicalRules.Insert(0, prefix); + prefix.Allomorphs.Add( + new AffixProcessAllomorph + { + Lhs = { Pattern.New("1").Annotation(any).OneOrMore.Value }, + Rhs = { new InsertSegments(Table3, "di+"), new CopyFromInput("1") }, + } + ); + + var parallel = new Morpher(TraceManager, Language); + var singleThreaded = new Morpher(TraceManager, Language, maxDegreeOfParallelism: 1); + + foreach (string word in new[] { "pʰutdidat", "pʰutdat" }) + { + List singleResult = singleThreaded.ParseWord(word).ToList(); + List parallelResult = parallel.ParseWord(word).ToList(); + Assert.That( + singleResult.Select(WordAnalysisSignature).OrderBy(s => s, StringComparer.Ordinal), + Is.EqualTo(parallelResult.Select(WordAnalysisSignature).OrderBy(s => s, StringComparer.Ordinal)), + $"single-threaded parse of '{word}' must match the parallel parse" + ); + } + } + + [Test] + public void ParseWord_MemoOnMatchesMemoOff_HitCounterGuarded_WithCompounding() + { + // Stage 3 of memoization.md: the mrule-cascade memo is now actually wired in for + // maxDegreeOfParallelism: 1. This is the standing acceptance gate (memoization.md §5) -- + // analysis-set (signature) equality between the memoized single-threaded cascade and the + // unmemoized parallel default -- made non-vacuous by asserting the memo's hit counter actually + // moved, so a memo that silently stopped firing couldn't pass this test by accident. + var any = FeatureStruct.New().Symbol(HCFeatureSystem.Segment).Value; + var crule = new CompoundingRule { Name = "rule1" }; + Allophonic.MorphologicalRules.Add(crule); + crule.Subrules.Add( + new CompoundingSubrule + { + HeadLhs = { Pattern.New("head").Annotation(any).OneOrMore.Value }, + NonHeadLhs = { Pattern.New("nonHead").Annotation(any).OneOrMore.Value }, + Rhs = { new CopyFromInput("head"), new InsertSegments(Table3, "+"), new CopyFromInput("nonHead") }, + } + ); + + var prefix = new AffixProcessRule + { + Id = "PREFIX", + Name = "prefix", + Gloss = "PAST", + RequiredSyntacticFeatureStruct = FeatureStruct.New(Language.SyntacticFeatureSystem).Symbol("V").Value, + OutSyntacticFeatureStruct = FeatureStruct + .New(Language.SyntacticFeatureSystem) + .Feature(Head) + .EqualTo(head => head.Feature("tense").EqualTo("past")) + .Value, + }; + Allophonic.MorphologicalRules.Insert(0, prefix); + prefix.Allomorphs.Add( + new AffixProcessAllomorph + { + Lhs = { Pattern.New("1").Annotation(any).OneOrMore.Value }, + Rhs = { new InsertSegments(Table3, "di+"), new CopyFromInput("1") }, + } + ); + + var memoOff = new Morpher(TraceManager, Language); + var memoOn = new Morpher(TraceManager, Language, maxDegreeOfParallelism: 1); + + long hitsBefore = MemoizedCombinationRuleCascade.DiagMemoHits; + long nogoodHitsBefore = MemoizedCombinationRuleCascade.DiagNogoodHits; + foreach (string word in new[] { "pʰutdidat", "pʰutdat" }) + { + List onResult = memoOn.ParseWord(word).ToList(); + List offResult = memoOff.ParseWord(word).ToList(); + Assert.That( + onResult.Select(WordAnalysisSignature).OrderBy(s => s, StringComparer.Ordinal), + Is.EqualTo(offResult.Select(WordAnalysisSignature).OrderBy(s => s, StringComparer.Ordinal)), + $"memo-on parse of '{word}' must be analysis-set identical to memo-off" + ); + } + TestContext.Out.WriteLine( + $"positive hits: {MemoizedCombinationRuleCascade.DiagMemoHits - hitsBefore}, " + + $"nogood hits: {MemoizedCombinationRuleCascade.DiagNogoodHits - nogoodHitsBefore}" + ); + Assert.That( + MemoizedCombinationRuleCascade.DiagMemoHits + MemoizedCombinationRuleCascade.DiagNogoodHits, + Is.GreaterThan(hitsBefore + nogoodHitsBefore), + "the memo must actually have hit (positive or nogood) at least once on this grammar -- " + + "otherwise this test cannot distinguish a working memo from a no-op one" + ); + } + + [Test] + public void ParseWord_MemoOnMatchesMemoOff_ForSelfOpaquingSimultaneousEpenthesis() + { + // Diagnostic pulled forward per memoization.md §5(a): PanGloss's rust conformance suite documents + // a confirmed C#-oracle nogood-cache bug on this exact fixture shape (a Simultaneous-mode + // epenthesis rule, which AnalysisRewriteRule compiles with ReapplyType.SelfOpaquing -- a + // repeat-until-fixpoint loop -- against root 19's boundary-bearing "b+ubu"). A direct + // reconstruction attempt against this prototype's own AnalysisScope (parse-optimization-archive) + // did not reproduce a divergence for "buibui" under any tracing/order combination tried, and the + // real minimal fixture (rust conformance/rewrite/simultaneous-epenthesis) is not present in this + // checkout to test against directly. Wiring this in as a standing regression case is the + // practical mitigation: the general memo-on/off equality gate (this test) covers it going + // forward, and the ≥2-self-opaquing-iteration case remains an open, documented gap (PanGloss's + // own conclusion too -- no available fixture drives the loop past one iteration). + var highVowel = FeatureStruct + .New(Language.PhonologicalFeatureSystem) + .Symbol(HCFeatureSystem.Segment) + .Symbol("cons-") + .Symbol("voc+") + .Symbol("high+") + .Value; + var highFrontUnrndVowel = FeatureStruct + .New(Language.PhonologicalFeatureSystem) + .Symbol(HCFeatureSystem.Segment) + .Symbol("cons-") + .Symbol("voc+") + .Symbol("high+") + .Symbol("back-") + .Symbol("round-") + .Value; + + var rule4 = new RewriteRule { Name = "rule4", ApplicationMode = RewriteApplicationMode.Simultaneous }; + Allophonic.PhonologicalRules.Add(rule4); + rule4.Subrules.Add( + new RewriteSubrule + { + Rhs = Pattern.New().Annotation(highFrontUnrndVowel).Value, + LeftEnvironment = Pattern.New().Annotation(highVowel).Value, + } + ); + + var memoOff = new Morpher(TraceManager, Language); + var memoOn = new Morpher(TraceManager, Language, maxDegreeOfParallelism: 1); + + foreach (string word in new[] { "buibui", "bubu", "bibu" }) + { + List onResult = memoOn.ParseWord(word).ToList(); + List offResult = memoOff.ParseWord(word).ToList(); + Assert.That( + onResult.Select(WordAnalysisSignature).OrderBy(s => s, StringComparer.Ordinal), + Is.EqualTo(offResult.Select(WordAnalysisSignature).OrderBy(s => s, StringComparer.Ordinal)), + $"memo-on parse of '{word}' must be analysis-set identical to memo-off" + ); + } + // The traced/correct oracle value for "buibui" (PanGloss's rust conformance fixture): root 19, + // one epenthesized result. Pinned directly, not just compared on-vs-off, so a bug that happened + // to affect both sides identically (e.g. both wrongly returning empty) would still be caught. + Assert.That(memoOn.ParseWord("buibui").Count(), Is.EqualTo(1)); + } + + [Test] + public void ParseWord_MemoOnMatchesMemoOff_HitCounterGuarded_WithAffixTemplate() + { + // Stage 4 of memoization.md: exercises the template-battery memo + // (AnalysisStratumRule.ApplyTemplateBattery) specifically -- TWO free prefix rules that commute + // with each other and with a template slot suffix. Unapplying di-then-gu vs gu-then-di reaches + // the same AnalysisStateKey (same shape, same rule MULTISET) via a different trail ORDER, so the + // second arrival replays the first arrival's stored template outputs with its own trail prefix + // grafted on (Word.ReplayOnto). One commuting prefix would NOT be enough -- a single rule can + // only unapply once, so no key would ever be re-arrived at and the memo would never fire. + var any = FeatureStruct.New().Symbol(HCFeatureSystem.Segment).Value; + + var edSuffix = new AffixProcessRule + { + Id = "TPAST", + Name = "template_ed_suffix", + Gloss = "PAST", + RequiredSyntacticFeatureStruct = FeatureStruct.New(Language.SyntacticFeatureSystem).Symbol("V").Value, + }; + edSuffix.Allomorphs.Add( + new AffixProcessAllomorph + { + Lhs = { Pattern.New("1").Annotation(any).OneOrMore.Value }, + Rhs = { new CopyFromInput("1"), new InsertSegments(Table3, "+d") }, + } + ); + var verbTemplate = new AffixTemplate + { + Name = "verb_template", + RequiredSyntacticFeatureStruct = FeatureStruct.New(Language.SyntacticFeatureSystem).Symbol("V").Value, + }; + verbTemplate.Slots.Add(new AffixTemplateSlot(edSuffix) { Optional = true }); + Morphophonemic.AffixTemplates.Add(verbTemplate); + + var diPrefix = new AffixProcessRule + { + Id = "TDI", + Name = "template_di_prefix", + Gloss = "DI", + RequiredSyntacticFeatureStruct = FeatureStruct.New(Language.SyntacticFeatureSystem).Symbol("V").Value, + }; + diPrefix.Allomorphs.Add( + new AffixProcessAllomorph + { + Lhs = { Pattern.New("1").Annotation(any).OneOrMore.Value }, + Rhs = { new InsertSegments(Table3, "di+"), new CopyFromInput("1") }, + } + ); + Morphophonemic.MorphologicalRules.Add(diPrefix); + + var guPrefix = new AffixProcessRule + { + Id = "TGU", + Name = "template_gu_prefix", + Gloss = "GU", + RequiredSyntacticFeatureStruct = FeatureStruct.New(Language.SyntacticFeatureSystem).Symbol("V").Value, + }; + guPrefix.Allomorphs.Add( + new AffixProcessAllomorph + { + Lhs = { Pattern.New("1").Annotation(any).OneOrMore.Value }, + Rhs = { new InsertSegments(Table3, "gu+"), new CopyFromInput("1") }, + } + ); + Morphophonemic.MorphologicalRules.Add(guPrefix); + + var memoOff = new Morpher(TraceManager, Language); + var memoOn = new Morpher(TraceManager, Language, maxDegreeOfParallelism: 1); + + long templateHitsBefore = AnalysisStratumRule.DiagTemplateMemoHits; + long templateNogoodHitsBefore = AnalysisStratumRule.DiagTemplateNogoodHits; + foreach (string word in new[] { "digusagd", "disagd", "gusagd", "sagd", "sag" }) + { + List onResult = memoOn.ParseWord(word).ToList(); + List offResult = memoOff.ParseWord(word).ToList(); + Assert.That( + onResult.Select(WordAnalysisSignature).OrderBy(s => s, StringComparer.Ordinal), + Is.EqualTo(offResult.Select(WordAnalysisSignature).OrderBy(s => s, StringComparer.Ordinal)), + $"memo-on parse of '{word}' must be analysis-set identical to memo-off" + ); + } + TestContext.Out.WriteLine( + $"template positive hits: {AnalysisStratumRule.DiagTemplateMemoHits - templateHitsBefore}, " + + $"template nogood hits: {AnalysisStratumRule.DiagTemplateNogoodHits - templateNogoodHitsBefore}" + ); + // Guards against this test going vacuous: the template memo's replay path must actually fire + // for this grammar. (As with the mrule-cascade memo, the ReplayOnto graft's effect on final + // signatures is unobservable through ParseWord's synthesis round-trip -- see memoization.md §5(b) + // -- so this counter, not the equivalence assertions above, is what proves the memoized path is + // actually exercised here, not silently bypassed.) + Assert.That( + AnalysisStratumRule.DiagTemplateMemoHits + AnalysisStratumRule.DiagTemplateNogoodHits, + Is.GreaterThan(templateHitsBefore + templateNogoodHitsBefore), + "the template memo must actually have hit (positive or nogood) at least once on this " + + "grammar -- otherwise this test cannot distinguish a working memo from a no-op one" + ); + } + + // Canonical analysis-set signature (memoization.md §5: gates compare this, never byte/object + // equality -- a memo-replayed Word is not guaranteed field-for-field identical to a freshly-computed + // one). AllomorphsInMorphOrder alone would not catch a broken trail/non-head graft (it walks Shape + // annotations, which Word.ReplayOnto never touches) -- MorphemesInApplicationOrder walks + // _mruleApps/_nonHeadApps directly, which is exactly what ReplayOnto rewrites. Root index is included + // so two analyses with the same morpheme sequence but different lexical roots can't collide. + internal static string WordAnalysisSignature(Word word) + { + return string.Join("+", word.AllomorphsInMorphOrder.Select(a => a.Morpheme.Id)) + + "|" + + string.Join("+", word.MorphemesInApplicationOrder.Select(m => m.Id)) + + "|root=" + + word.RootAllomorph.Morpheme.Id; + } }