LT-21834: Update libpalaso to the build with the abandoned mutex fix - #1164
Merged
Merged
Conversation
A GlobalMutex whose owner died without releasing it was left abandoned, and every later attempt to take it threw. FLEx hit that when opening a project, and because the failing caller abandoned the mutex again, the crash repeated on every launch until the machine was rebooted. The fix is in libpalaso, so nothing in FieldWorks changes but the version pin. The pin moves all fourteen libpalaso packages at once, so twelve upstream commits come with it rather than one. Eleven are bug fixes, resource-leak fixes and test repairs. The twelfth is a declared breaking change: ImageCropper.GetCroppedImage now returns a bitmap whose RawFormat is MemoryBmp rather than Jpeg. PicturePropertiesDialog is the only place FieldWorks reaches that code, and it saves through PalasoImage.Save, which chooses the encoder from the file extension, so it is on the safe side of the change. beta0043 exists but only adds a CHANGELOG section. beta0042 is pinned because its package provenance points at the fix's merge commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1164 +/- ##
==========================================
- Coverage 39.02% 39.02% -0.01%
==========================================
Files 1522 1522
Lines 352937 352937
Branches 40726 40726
==========================================
- Hits 137741 137733 -8
- Misses 185893 185900 +7
- Partials 29303 29304 +1 🚀 New features to boost your workflow:
|
thejambi
marked this pull request as ready for review
September 28, 2026 21:07
mark-sil
approved these changes
Sep 29, 2026
mark-sil
left a comment
Contributor
There was a problem hiding this comment.
@mark-sil reviewed 1 file and all commit messages.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on thejambi).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Start here:
Build/SilVersions.props. The whole diff is one line; review what that line pulls in, not the line.Moves
SilLibPalasoVersionfrom18.0.0-beta0030to18.0.0-beta0042, the first libpalaso build carrying the LT-21834 fix (libpalaso#1544). A mutex abandoned by a crashed process made FLEx crash on every later project open;GlobalMutexnow recovers from the abandonment instead. It also fixes LT-21467, open since 2023: re-entering Crop in Picture Properties failed with a GDI+ error. No FieldWorks code changes.What else comes with it? The pin moves all fourteen libpalaso packages, so twelve upstream commits land, not one. Eight are test repairs, CI plumbing, or fixes in code FieldWorks does not call. Two are fixes in code FLEx reaches: #1535 (writing system
Replaceafter an interrupted update) and #1536 (ImageCropperdisposal). One is a declared breaking change that FLEx also reaches.Where to look
RawFormatMemoryBmprather thanJpeg.PicturePropertiesDialogsaves throughPalasoImage.Save, which picks the encoder from the file extension. Verified by hand: a crop saved as.jpgis a genuine, complete JPEG.FwNewLangProjectModel_CanFinish_FalseIfNoneCompleteis flaky, in the writing-system code this fix touches. A control run shows it is already flaky onmain: it failed 2 of 3 full runs without the bump and 1 of 3 with it.Not here
18.0.0-beta0043, a CHANGELOG-only rebuild.beta0042is pinned because its package provenance is the fix's merge commit.Verification
build.ps1 -CommentHygiene -TokenHygieneclean.test.ps1: 6317 tests, 6255 passed, 62 skipped, 0 failed.Next: approve, or tell me to take
beta0043instead. On merge, resolve LT-21467 with LT-21834.Reading this a year from now -- start here
This is a version-pin PR. The fix itself is in libpalaso, and the reasoning behind it lives on libpalaso#1544. What this record keeps is the FieldWorks side: which build to pin, why no FieldWorks code changed, how both bugs were reproduced, and the evidence behind the risk claims above. No working documents were written on this branch, so nothing was deleted from the tree.
Decisions, and why
No FieldWorks code change. The recovery has to happen where the mutex is acquired. The
GlobalMutexthat throws is private toGlobalWritingSystemRepository, so FieldWorks can only catch the exception after the fact, by which point the catching thread already owns the mutex and nothing it can reach will release it. See Paths not taken.beta0042, notbeta0043.beta0043adds a CHANGELOG section and nothing else (libpalaso#1548), so the two are functionally identical.beta0042's nuspec recordsrepository commit="8948e6c5…", the LT-21834 merge itself, which makes the pin self-describing.One bump, not a catch-up followed by the fix.
beta0041is built from5ecf001f, the commit immediately before the fix, so the other eleven commits could have landed separately. They were taken together because the full suite and both manual checks ran againstbeta0042, and a separate catch-up would repeat that validation without changing what ships.Paths not taken
Catching the exception in FieldWorks (#1141, closed). A catch in
FieldWorks.csturned the crash into an "Unable to Open Project" dialog, and retrying from it opened the project. Measured, it was worse than the crash. The thread that receivesAbandonedMutexExceptionowns the mutex, so the retry was a recursive acquisition that returned immediately. The mutex was then held for the whole session, blocking other SIL programs that share the writing system store, and abandoned again on exit, so the dialog returned on every launch.Reproducing LT-21467 -- for testers
Before the fix: "Sorry, something went wrong with the ImageToolbox", whose details show
A generic error occurred in GDI+thrown fromImageCropper.set_Image. Re-entering Crop hands the cropper an image backed by a stream disposed when it was made. With the fix, the cropper reopens with the image intact.On a developer machine, an installed FLEx reads the dev tree's configuration: every build writes
HKCU\SOFTWARE\SIL\FieldWorks\9\RootCodeDirandRootDataDir(setKeysInHKCU,Build/mkall.targets:279), andFwDirectoryFinder.GetDirectoryprefers them over the install path. The symptom is an XCore error namingAiAnalysisExportListener. For a clean before, back up that key and remove the two values; restore them, or rebuild, before running the branch build.Reproducing LT-21834 -- for testers
A named mutex exists only while some process holds a handle to it, so abandoning it takes two processes: one that keeps a handle open, standing in for a second SIL program, and one that acquires it and exits without releasing it.
Before the fix, FLEx dies before its main window appears, with the crash reporter showing
The wait completed due to an abandoned mutex.(AbandonedMutexException, raised fromSIL.Threading.GlobalMutex.Lock()underFieldWorks.EnsureDefaultCollationsPresent). The crash abandons the mutex again, so it repeats on every launch. With the fix, the project opens with no error.Run only one FLEx at a time. A second instance opening the same project meets FieldWorks' own project-in-use handling, which looks like a failure but is unrelated.
Surprising findings
kstidUnableToOpenLastProject("The last time FLEx started, it stopped responding when attempting to open the FieldWorks project: …") is raised atFieldWorks.cs:1434when the previous launch leftLoadingProcessIdset (:3081) and its process is gone. A successful load clears it (:3101), so a user leaving the crash loop sees the notice once, naming the project that last failed.PicturePropertiesDialog.ApplySaveFilecatches every exception fromPalasoImage.Saveand showsksErrorFileInUse("This file is probably open somewhere"), writing the real exception only to the log. Had#1530broken the save path, a tester would most likely have dismissed it as a file-locking problem.Preflight review details
Code Review Summary
Branch: LT-21834-update-libpalaso
Base: origin/main (30564cc)
Date: 2026-09-28
Review model: Claude Opus 5 and Claude Opus 5.5 (Claude Code)
Files changed: 1
Overview
Raises
SilLibPalasoVersionfrom18.0.0-beta0030to18.0.0-beta0042so thatFieldWorks picks up the fix for LT-21834, where an abandoned
GlobalMutexleftFLEx unable to open a project again until the machine was rebooted. The fix
itself is in libpalaso (#1544,
merged as
8948e6c5); no FieldWorks code change is needed, so the version pin isthe whole of the FieldWorks-side fix.
The pin is a single property consumed by
Directory.Packages.propsand therestore targets, so one line moves all fourteen libpalaso packages. That makes
the diff trivial and the risk surface large: twelve libpalaso commits come with
it, not one. The analysis below is therefore about the delta, not the diff.
The bump also fixes LT-21467, open since
2023: cropping in the Picture Properties dialog, switching to Get Image and
returning to Crop failed with a GDI+ error. The fix is libpalaso
#1530, which closed
libpalaso #1275, the issue filed against it; nothing had carried it into
FieldWorks until now.
One commit:
830daeedf. Branch is current withmainas of 2026-09-28.Contract/API Changes
None in FieldWorks. One in the dependency:
declared BREAKING CHANGE.
ImageCropper.GetCroppedImagenow returns thecropped bitmap directly, so its
RawFormatisMemoryBmprather thanJpeg.Callers that read
RawFormat, or that callImage.Save(path)and rely on theJPEG encoder being chosen implicitly, must now pass an explicit
ImageFormat.fixes and CI plumbing. Notably
#1535 fixes
Replacefailing forever after an interrupted writing system update, which is
complementary to the LT-21834 fix rather than incidental.
Findings
Critical - Must address before merge
None.
Important - Should address before merge
#1530breaking change is reachable from FLEx. (validatedduring review: author exercised Insert Picture → crop → Save As on the branch
build and reports it working; the saved file was then checked directly and is
a genuine, complete JPEG — see Required Validation)
PicturePropertiesDialoghostsImageToolboxControl(
Src/FwCoreDlgs/PicturePropertiesDialog.cs:21,.Designer.cs:45), which isthe Insert Picture flow. Reading the code, FieldWorks is on the safe side of
the change: it never reads
RawFormat; it saves throughimageToolbox.ImageInfo.Save(savePath)atPicturePropertiesDialog.cs:414,which is
PalasoImage.Saveand picks the encoder from the file extension, thepath the libpalaso changelog names as safe;
IsCropped(:68) compares imagesizes, not formats; and
FileFormatSupportsMetadata(:84) resolves throughMetadata.FileFormatSupportsMetadata(path), a file path rather thanRawFormat. No automated test covers the crop-and-save flow; the manualcheck below is what confirms the reading.
Minor - Consider
18.0.0-beta0043exists and was not taken. It is a CHANGELOG-onlyrebuild (
Add missing 17.0.0 section to CHANGELOG,#1548), so it is
functionally identical to
beta0042.beta0042was chosen deliberatelybecause its nuspec
repository commitis the LT-21834 merge itself, whichmakes the pin self-documenting. Worth a sentence in the PR so a reviewer does
not read it as an oversight.
deleted
AlloGenServiceTests/TestData/SharedSettings/andTestData/WritingSystemStore/, left over from an interrupted test run; onefile carried the developer's account name).
Required Validation / Evidence
Run on the current head, after merging
main:.\build.ps1 -CommentHygiene -TokenHygiene— exit 0, 0 warnings, 0 errors.comment-hygiene clean, token-hygiene clean (176 files scanned),
powershell-compat clean (5.1 and 7.0).
.\test.ps1 -CommentHygiene -TokenHygiene— 6317 tests, 6255 passed, 62skipped, 0 failed.
gitlint --ignore body-is-missing --commits origin/main..HEAD— exit 0.Output\Debug\SIL.WritingSystems.dllandSIL.Core.dllreport18.0.0-beta.42+Branch.master.Sha.8948e6c5e5a97a1c82b802ec9e87590078fb5a6d—the LT-21834 merge commit.
One pre-existing flaky test, investigated rather than waved through:
FwNewLangProjectModel_CanFinish_FalseIfNoneCompletefailed once withKeyNotFoundException : The writing system en was not found in this manager.Because materialising
enruns through the SLDR and global writing system storepaths that libpalaso #1544 rewrote, this could not be assumed unrelated. Control
experiment, reverting only the version pin and rebuilding:
beta0030(base)beta0042(this branch)The same test, the same exception, failing more often without the bump than with
it. It also passes in isolation (
-TestProject FwCoreDlgsTests -TestFilter FwNewLangProjectModel_CanFinish_FalseIfNoneComplete, 1/1). Pre-existing onmain, order- or parallelism-sensitive, not caused by this change. It is notdocumented as flaky anywhere in the repo and has no ticket.
Manual validation, on the branch build (
Output\Debug\FieldWorks.exe), 2026-09-28:LT-21834, reproduced and then fixed. The writing system store's mutex
(
C:_ProgramData_SIL_WritingSystemRepository_3) was left abandoned with asecond handle held open, using
Abandon-FwMutex.ps1. The abandoned state wasconfirmed from a separate process, whose
WaitOnethrewAbandonedMutexException, before either FLEx was started. Project:Lex Training Sample Project 1, one FLEx running at a time.Before — installed FLEx 9.3.9.1439 (
SIL.WritingSystems 18.0.0-beta.13).Crashed during startup, before any main window, with the crash reporter:
The same path as the report on the ticket. The thread that received the
exception owned the mutex and died with it, so the crash left it abandoned
again — the repeating half of the bug, and the state the next step starts
from.
After — branch build. The Open Project dialog first showed "The last time
FLEx started, it stopped responding when attempting to open the FieldWorks
project: Lex Training Sample Project 1". That is
kstidUnableToOpenLastProject, raised atFieldWorks.cs:1434when theprevious launch recorded
StartupStatus.Failedin the registry settings bothbuilds share: it reports step one's crash, not anything in this change. The
project then opened normally, with no error.
A user upgrading out of the crash loop should expect that notice once, on the
first launch after the upgrade, naming the project that last failed.
#1530crop path. Author inserted a JPEG through Insert Picture, croppedit and saved it, and reports it working. The saved file,
LinkedFiles\Pictures\LT-21834-crop-test_croppped.jpg, was then inspecteddirectly:
FF D8 FF E0— JPEG (JFIF)FF D9— end-of-image marker present, file completeRawFormatread back from diskJpegThe risk was a bitmap written under a
.jpgname, since the in-memory crop isnow
MemoryBmp.PalasoImage.Savechose the encoder from the extension, asthe code reading predicted.
The mutex holder was released with
Abandon-FwMutex.ps1 -Stopafterwards.LT-21467, reproduced and then fixed. Steps: Insert Picture on a sense,
Get Image with a JPEG, Crop with one edge dragged, Get Image, Crop again.
Before — installed FLEx 9.3.9. "Sorry, something went wrong with the
ImageToolbox", with the details:
Re-entering Crop hands the cropper an image backed by a stream that was
disposed when it was created, and saving it throws — the mechanism #1530
describes.
After — branch build. The same steps return to the cropper with the image
intact and no error. Author reports it working.
For a clean before, the installed FLEx was run with the two
HKCU\SOFTWARE\SIL\FieldWorks\9valuesRootCodeDirandRootDataDirremoved. Every build writes them (
setKeysInHKCU,Build/mkall.targets:279),pointing at
DistFiles, andFwDirectoryFinder.GetDirectoryprefers themover the machine-wide install path, so on a developer machine the installed
FLEx otherwise reads
main's configuration over 9.3.9 binaries. With thempresent it reported
AiAnalysisExportListener, a classmain'sMain.xmldeclares and 9.3.9 lacks; with them removed it opened normally. The key was
backed up first and restored afterwards, matching the backup.
Positive Observations
conditional compilation, no shims.
Manage-LocalLibraries.ps1was used rather than hand-editing the props file,so the 15 stale package folders were cleared as part of the change. That
avoids the LT-22728 trap where NuGet keeps serving an already-extracted
package after its
.nupkgis gone.trusted as "twelve beta bumps", which is what surfaced the
#1530breakingchange.
Interview Notes
user-visible (FLEx failing to reopen a project), so it belongs in a tester's
queue; no new ticket needed.
#1530validation: author chose to verify the picture crop flow manuallyrather than ship on code reading alone or have the agent drive the UI.
Performed on 2026-09-28 and reported good; the output file was independently
verified.
was started while the branch build still had the project open, so it met
FieldWorks' own project-in-use handling rather than the mutex. Rerun with one
FLEx at a time: reproduced on 9.3.9, fixed on the branch. Author supplied the
crash report text and the dialog text verbatim.
filed with a link to this FLEx ticket, and the ticket was still open. Author
ran the ticket's steps on both builds and reports reproduction on 9.3.9 and a
clean pass on the branch.
In-Review Quality Check
Two directories of leftover test output were deleted. No source was modified
during review; the branch remains a one-line version pin.
Suggested Review Focus
catch-up be separated?
18.0.0-beta0041is built from5ecf001f, thecommit immediately before the fix, so it carries the other eleven
without it. A separate catch-up to
beta0041is possible, at the cost ofrepeating this validation against it.
FwNewLangProjectModel_CanFinish_FalseIfNoneCompleteis flaky onmaintoday. Worth a ticket of its own?
reaches testers.
🤖 Generated with Claude Code
This change is