BUG: Release ImageIOFactory mutex before probing candidate IOs - #6831
Conversation
|
| // access to the same file. Release the lock as soon as | ||
| // possibleImageIO is populated so independent reads/writes on | ||
| // different files do not serialize on each other. | ||
| const std::lock_guard<std::mutex> lockGuard(createImageIOMutex); |
There was a problem hiding this comment.
Concurrent probe coverage is missing
CreateImageIO now deliberately releases createImageIOMutex before CanReadFile and CanWriteFile, but no registered test runs independent factory calls concurrently and detects serialized probes. A future lock-scope regression could therefore restore process-wide probe serialization while ordinary IO tests still pass. Add a controllable ImageIO test double that records simultaneous read and write probe entry, then assert concurrent CreateImageIO calls for independent paths reach the probe concurrently.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Artifacts
ImageIO factory probe-lock coverage inspection script
- The executed Bash script compares the parent and reviewed revision's mutex/probe placement and ImageBase test coverage, ending with the test-tree status.
ImageIO factory probe lock before the change
- Running the inspection script against `HEAD^` shows the mutex before both read and write probe calls, with no relevant registered or source-level ImageBase test coverage.
ImageIO factory probe lock after the change
- Running the same inspection against `HEAD` shows the mutex block ends before the read/write probes and confirms the reviewed commit contains no ImageBase test-tree change.
Standalone compilation attempt without configured ITK build
- A real C++ syntax-only invocation of the changed source fails because generated `ITKIOImageBaseExport.h` is absent, confirming why the ITK runtime path could not be executed.
Working tree after evidence capture
- The captured `git status --short` records the evidence-only worktree state after validation, with no repository source files edited.
8a02f1c to
809d75d
Compare
CreateImageIO() held createImageIOMutex for its entire body, including the CanReadFile()/CanWriteFile() probing loop over every registered IO. Several IO CanReadFile/CanWriteFile implementations open and parse the file, so this serialized every ReadImage/WriteImage call in the process, even for unrelated files, across all image formats. Under concurrent reads of many distinct files from many threads, this collapses effective throughput and can look indistinguishable from a hang once enough concurrent demand builds up. Narrow the lock to only the ObjectFactoryBase::CreateAllInstance call that builds the candidate IO list; release it before probing files, so independent reads/writes on different files no longer serialize. This mirrors the concurrency portion of b5def87 (ENH: Two-phase ImageIOFactory dispatch via extension check, main), backported here as a minimal, behavior-preserving fix for the release branch without the extension-dispatch API changes.
809d75d to
cf983db
Compare
Summary
ImageIOFactory::CreateImageIO()holdscreateImageIOMutexfor its entire body, including theCanReadFile()/CanWriteFile()probing loop over every registeredImageIOBase. Several IOCanReadFile/CanWriteFileimplementations (e.g.GDCMImageIO) open and parse the file, so this serializes everyReadImage/WriteImagecall in the process — across all images, not just concurrent access to the same file.Under concurrent reads of many distinct files from many threads, this collapses effective throughput. It can look indistinguishable from a hang once concurrent demand is high enough, since a large number of threads all queue up on this single mutex while only one thread makes progress at a time, each turn potentially doing blocking disk I/O.
This is the same underlying issue behind the intermittent hang in SimpleITK's
Wrapping/Python/tests/ConcurrentImageRead.py(labeledUNSTABLEon macOS). Confirmed withlldb: at the moment of the "hang," 70 of 72 worker threads were blocked instd::mutex::lock()inside this function, while the lock holder was insideGDCMImageIO::CanReadFile()'sfopen().Fix
Narrow the lock to only the
ObjectFactoryBase::CreateAllInstancecall that builds the candidate IO list; release it before probing files, so independent reads/writes on different files no longer serialize.This mirrors the concurrency portion of b5def87 ("ENH: Two-phase ImageIOFactory dispatch via extension check",
main), backported here as a minimal, behavior-preserving fix for therelease-5.4branch — without the extension-dispatch API changes from that commit, which aren't appropriate for a stable release branch.Test plan
release-5.4and existingitkImageIOFactory/related tests still passstd::thread-based stress test reading many distinct files concurrently), mirroring SimpleITK'sConcurrentImageRead.py