Prune reduplication matches whose copies cannot agree - #519
johnml1135 wants to merge 4 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #519 +/- ##
==========================================
+ Coverage 74.07% 74.14% +0.06%
==========================================
Files 456 457 +1
Lines 38164 38236 +72
Branches 5228 5242 +14
==========================================
+ Hits 28270 28349 +79
+ Misses 8734 8728 -6
+ Partials 1160 1159 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…not agree Port of sillsdev/machine#519. Morpher::with_prune_disagreeing_copies (off by default) skips an affix-process analysis match when two copies of an unmodified input part differ in length or fail to unify. Copies with an optional node, a failed or zero-width capture, or a modified part are kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Matches sillsdev/machine#519, which now defaults PruneDisagreeingCopies on. with_prune_disagreeing_copies(false) restores the old search. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Pushed 2e1eb06: The same change in PanGloss (Rust) was measured against v0.4.0 on words both builds finish, with identical analyses:
All runs used 🤖 Generated with Claude Code |
When unapplying an affix process rule that copies a part more than once, Morpher.PruneDisagreeingCopies (off by default) skips matches whose copies cannot unify segment by segment. Synthesis writes every copy from the same input, so such a match never survives synthesis. Copies containing an optional node or modified by the rule are always kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A self-feeding deletion strips two segments from the second copy only; with DeletionReapplications 0 analysis restores one. Forward generation defines the expected parse, and pruning must return the unpruned result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…letion bug 049 records the reduplication copy-agreement prune ported from sillsdev/machine#519, its soundness argument, and the v0.3.3/v0.4.0 parity and timing evidence. 050 records sillsdev/machine#520, reproduced in both engines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2e1eb06 to
d73fc10
Compare
Rules that copy no part twice no longer rescan their captured parts on every unapplication. Mirrors PanGloss 77aca87f, which measured 0.62-0.64x of the prior time on Aweti with identical analyses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
I don't understand why MultiplePatternRule doesn't already filter this, given that PatternRule does. What is synthesis doing different from analysis? I noticed that AnalysisAffixProcessRule doesn't propagate the SyntacticFeatureStruct through during Apply the way that SynthesisAffixProcessRule does. Is this the issue? Or is the issue that we don't have the stem features during analysis? If we changed things so that analysis filters the way that synthesis does, that would be a more general solution. |
Quick summary
HermitCrab now drops reduplication analyses whose copies provably disagree, since synthesis would reject them anyway.
Anything it cannot judge, such as an unapplied deletion inside a copy, is kept.
Synthesis is untouched; the new
Morpher.PruneDisagreeingCopiesoption defaults to on.Where to look
AnalysisMorphologicalTransform.HasDisagreeingCopies-- the invariant: an undecidable copy is always kept.CopyAgreementPruneTests-- split selection, on/off pruning, on-by-default, and a cap-limited-deletioncase where forward generation defines the expected parse.
AffixProcessRuleTests.ReduplicationRules/ModifyFromInputRules-- run with pruning both off and on.Deliberately not included
parity sweep. This session only reran the unit/integration suite; prior figures are kept below the
rule, labelled as prior measurements.
Validation
dotnet build-- Build succeeded, 0 Warning(s), 0 Error(s).dotnet test tests/SIL.Machine.Morphology.HermitCrab.Tests --no-build-- Passed: 114, Failed: 0, Skipped: 0.dotnet csharpier check .-- Checked 721 files, clean.pwsh -NoProfile -File scripts/comment-hygiene.ps1 -BaseRef origin/master-- clean.Reading this a year from now
A full-copy reduplication rule (
CopyFromInputof the same part twice) is unapplied by matching eachcopy as an independent capture; only the first capture rebuilds the base. So every split where the
inserted material lines up becomes its own analysis, and synthesis throws the disagreeing ones away much
later. This change moves that rejection earlier, into analysis, for matches that can be judged.
Synthesis writes every copy of a part from the same input, and anything that later changes one copy is
unapplied before this rule runs, so a match whose copies are proven to disagree segment by segment can
never survive synthesis. A copy containing an optional node (an unapplied deletion), or a part the rule
modifies, cannot be judged and is never reported as disagreeing -- it is kept, at the cost of the
candidates the prune would otherwise have removed.
Decisions, and why
Consistent/Inconsistent/Undecidableenum. Nocaller or test distinguished
ConsistentfromUndecidable--CopyAgreementPatternRuleonly everchecked for
Inconsistent, and both other states took the same keep path -- so it was collapsed to asingle
HasDisagreeingCopiesbool, removing the enum and the now-unreachablebranching that tracked the discarded state.
AnalysisMorphologicalTransform.GetCapturedNodesreuses the optional-node walk behindMorphologicalOutputAction.GetSkippedOptionalNodesthrough a new internal static helper; the protectedmethod keeps its signature.
Morpher.PruneDisagreeingCopies's summary no longer promises a specific memory/timing outcome; thecontract is what the option controls (skip provably-disagreeing copies) and its default, not a
particular grammar's measured speedup.
CopyAgreementPatternRule's class-level summary was removed: it only restated the class and propertynames, the class is internal with no external reader, and no sibling rule class in this file's
directory carries one.
Prior measurements (reported on an earlier revision of this branch, not reverified this session)
Grammars without full-copy rules (Sena, Amharic) never enter the new code path. The unpruned parse of
Aweti index 182 never finishes, so its 2 analyses rest on the argument above plus the parity sweep, not a
direct comparison. Mbugwe gains little from this change alone (the prune fires on 161 words but removes
0.2% of candidates).
🤖 Generated with Claude Code
This change is