Description
Disposing a NamedPipeServerStream opened with PipeOptions.Asynchronous while an overlapped operation (a WaitForConnectionAsync(token) accept or a ReadAsync(token)) is still in flight can leave a completion to be delivered after the overlapped's GCHandle has been freed. The portable managed IO-completion poller then throws InvalidOperationException("Handle is not initialized.") on a thread-pool thread. The throw is unhandled and the process dies.
The same race also manifests, less often, as a lost completion — the accept/read never completes and the consumer deadlocks waiting for it.
Reproduction Steps
Attached: PipeIocpRepro.zip — a self-contained MSTest project (740 lines of code, only System.IO.Pipes plus MSTest packages).
The crash is intermittent (~1–5 crashes per 60 runs) and does not reproduce as a plain dotnet run console app. It needs a VSTest test host on x64 running the whole test assembly repeatedly.
dotnet build -c Release -p:Platform=x64
# 2-core affinity substantially raises the hit rate.
$vstest = "<VS>\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"
$dll = "bin\x64\Release\net10.0-windows10.0.18362.0\PipeIocpRepro.dll"
for ($i = 1; $i -le 60; $i++) {
$q = Start-Process $vstest -ArgumentList @($dll, "/Platform:x64") -PassThru -NoNewWindow
$q.ProcessorAffinity = [IntPtr]3
$q.WaitForExit()
if ($q.ExitCode -ne 0) { Write-Host "run $i CRASHED" }
}Why it needs a mixed workload — not a one-liner. The churn test alone doesn't crash. The equivalent code as a plain console app doesn't crash under millions of cycles. The race surfaces only when the create/connect/dispose churn runs alongside other pipe I/O in the same process — a multi-threaded 8×1000 connect storm and 133 KB message exchanges keep completions flowing on the poller while the churn supplies the disposed-with-pending-overlapped that a late completion lands on. This matches the timing-window nature of the race and was confirmed by bisecting: removing either the churn or the co-workload eliminates the crash.
PipeIocpRepro.zip
Expected behavior
All 60 runs succeed. Disposing an async pipe with pending overlapped ops must cancel them cleanly without leaking a completion to freed memory.
Actual behavior
1–5 runs of 60 abort with Test host process crashed : Unhandled exception. System.InvalidOperationException: Handle is not initialized.
Stack (captured heap dump):
Unhandled exception. System.InvalidOperationException: Handle is not initialized.
at System.Threading.Overlapped.GetOverlappedFromNative(NativeOverlapped* pNativeOverlapped)
at System.Threading.IOCompletionCallbackHelper.PerformSingleIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pNativeOverlapped)
at System.Threading.PortableThreadPool.IOCompletionPoller.Event.Invoke()
at System.Threading.ThreadPoolTypedWorkItemQueue`2.System.Threading.IThreadPoolWorkItem.Execute()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
at System.Threading.Thread.StartCallback()
The disposal cancels the pending overlapped op via PipeCompletionSource.Cancel / CancelIoEx; the pipe handle and the overlapped's GCHandle are released. If the kernel delivers a completion after that release (the normal race between cancellation and completion), the managed poller processes an entry whose NativeOverlapped* points at a freed GCHandle, and Overlapped.GetOverlappedFromNative throws.
Regression?
Regressed since .NET 9. The portable managed IO-completion poller became the Windows default in .NET 9 (#64834). The native-IOCP fallback and its opt-out env var DOTNET_ThreadPool_UsePortableThreadPoolForIO were removed in .NET 8 (#71719), so there is no supported switch back on .NET 9/10.
Not reproducible on .NET 8. Reproduced on both .NET 9 and .NET 10.
Known Workarounds
Measured on the attached reproducer, VSTest x64, 2-core affinity.
| Workaround |
Result |
Notes |
DOTNET_ThreadPool_UseWindowsThreadPool=1 |
1 crash / 50 |
still crashes; the freed-overlapped race also exists on the Win32 thread pool path (#72644, #48402) |
DOTNET_TieredCompilation=0 |
3 crashes / 80 |
no effect; belongs to #132627, a separate JIT bug |
DOTNET_ThreadPool_UsePortableThreadPoolForIO=0 |
no-op |
the switch was removed in #71719 |
| Draining pending ops before Dispose (application side) |
3 crashes / 60 |
same rate as baseline |
No supported configuration eliminates the crash.
Configuration
- .NET 10, SDK 10.0.400, TFM
net10.0-windows10.0.18362.0
- Windows 11 (also observed on Windows Server 2022 CI agents)
- x64
- Reproduced on .NET 9 as well. Not reproducible on .NET 8.
Other information
Related issues:
#72644 "'overlapped' has already been freed" — same family, milestone Future, unfixed.#76060, #48402 — same family, unfixed.#40674 "Disposing NamedPipeServerStream does not cancel WaitForConnectionAsync" — fixed via #52825; the runtime team's position is that dispose must cancel pending ops cleanly, i.e. this is a runtime responsibility.#39902 "PipeCompletionSource.Cancel throws ObjectDisposedException during shutdown" — related dispose/cancel race.#116768 / #116830 — same poller stack (IOCompletionPoller → freed overlapped), FileSystemWatcher sibling, fixed.- aspnetcore
#54251 (VTS double-completion) — fixed in HTTP.sys layer via #54437.
A sanitized heap crash dump (55 MB) is available privately on request via any secure channel the team prefers.
Description
Disposing a
NamedPipeServerStreamopened withPipeOptions.Asynchronouswhile an overlapped operation (aWaitForConnectionAsync(token)accept or aReadAsync(token)) is still in flight can leave a completion to be delivered after the overlapped'sGCHandlehas been freed. The portable managed IO-completion poller then throwsInvalidOperationException("Handle is not initialized.")on a thread-pool thread. The throw is unhandled and the process dies.The same race also manifests, less often, as a lost completion — the accept/read never completes and the consumer deadlocks waiting for it.
Reproduction Steps
Attached:
PipeIocpRepro.zip— a self-contained MSTest project (740 lines of code, onlySystem.IO.Pipesplus MSTest packages).The crash is intermittent (~1–5 crashes per 60 runs) and does not reproduce as a plain
dotnet runconsole app. It needs a VSTest test host on x64 running the whole test assembly repeatedly.Why it needs a mixed workload — not a one-liner. The churn test alone doesn't crash. The equivalent code as a plain console app doesn't crash under millions of cycles. The race surfaces only when the create/connect/dispose churn runs alongside other pipe I/O in the same process — a multi-threaded 8×1000 connect storm and 133 KB message exchanges keep completions flowing on the poller while the churn supplies the disposed-with-pending-overlapped that a late completion lands on. This matches the timing-window nature of the race and was confirmed by bisecting: removing either the churn or the co-workload eliminates the crash.
PipeIocpRepro.zip
Expected behavior
All 60 runs succeed. Disposing an async pipe with pending overlapped ops must cancel them cleanly without leaking a completion to freed memory.
Actual behavior
1–5 runs of 60 abort with
Test host process crashed : Unhandled exception. System.InvalidOperationException: Handle is not initialized.Stack (captured heap dump):
The disposal cancels the pending overlapped op via
PipeCompletionSource.Cancel/CancelIoEx; the pipe handle and the overlapped'sGCHandleare released. If the kernel delivers a completion after that release (the normal race between cancellation and completion), the managed poller processes an entry whoseNativeOverlapped*points at a freedGCHandle, andOverlapped.GetOverlappedFromNativethrows.Regression?
Regressed since .NET 9. The portable managed IO-completion poller became the Windows default in .NET 9 (#64834). The native-IOCP fallback and its opt-out env var
DOTNET_ThreadPool_UsePortableThreadPoolForIOwere removed in .NET 8 (#71719), so there is no supported switch back on .NET 9/10.Not reproducible on .NET 8. Reproduced on both .NET 9 and .NET 10.
Known Workarounds
Measured on the attached reproducer, VSTest x64, 2-core affinity.
DOTNET_ThreadPool_UseWindowsThreadPool=1DOTNET_TieredCompilation=0DOTNET_ThreadPool_UsePortableThreadPoolForIO=0No supported configuration eliminates the crash.
Configuration
net10.0-windows10.0.18362.0Other information
Related issues:
#72644"'overlapped' has already been freed" — same family, milestoneFuture, unfixed.#76060,#48402— same family, unfixed.#40674"Disposing NamedPipeServerStream does not cancel WaitForConnectionAsync" — fixed via#52825; the runtime team's position is that dispose must cancel pending ops cleanly, i.e. this is a runtime responsibility.#39902"PipeCompletionSource.Cancel throws ObjectDisposedException during shutdown" — related dispose/cancel race.#116768/#116830— same poller stack (IOCompletionPoller→ freed overlapped),FileSystemWatchersibling, fixed.#54251(VTS double-completion) — fixed in HTTP.sys layer via#54437.A sanitized heap crash dump (55 MB) is available privately on request via any secure channel the team prefers.