Skip to content

Fix orphaned processes on Windows when quickly navigating between commits - #5885

Open
stefanhaller wants to merge 4 commits into
prevent-stale-index-lock-windowsfrom
fix-windows-pty-orphans
Open

Fix orphaned processes on Windows when quickly navigating between commits#5885
stefanhaller wants to merge 4 commits into
prevent-stale-index-lock-windowsfrom
fix-windows-pty-orphans

Conversation

@stefanhaller

Copy link
Copy Markdown
Collaborator

Fixes #5879: with an external diff command configured, quickly navigating between commits on Windows accumulates orphaned git.exe/difft.exe/conhost.exe processes that keep computing their diffs in the background and persist after lazygit exits.

Stopping a pty task on Windows relied on ClosePseudoConsole, whose CTRL_CLOSE_EVENT only reaches clients attached to the pseudoconsole at that moment. Attachment happens asynchronously during child startup, so a task stopped within the first few milliseconds of its life — which is exactly what rapid navigation produces — misses the event entirely and survives, together with its whole process tree (git for Windows spawns through a two-level git.exe wrapper, so a single task has several attach windows).

The fix puts the child into a job object before it executes its first instruction (created suspended → assigned → resumed), so every descendant is in the job from the start. The pty teardown still closes the pseudoconsole first and gives clients that received the close event a moment (500ms) to exit through their own handlers — git's cleans up lock files — and then terminates whatever is left in the job. JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE doubles as a safety net: if lazygit exits without running the teardown, the OS closes the job handle and reaps the tree.

Validated with a harness that mimics lazygit's stop path with randomized 0–120ms stop delays: before the fix, 3 of 30 process trees survived as permanent orphans per run; after it, three runs of 30 all came back with zero orphans, with the job kill catching exactly the children that missed the close event (6 of 90) while the rest still exited gracefully through the event as before.

@stefanhaller stefanhaller added the bug Something isn't working label Aug 2, 2026
@stefanhaller
stefanhaller force-pushed the fix-windows-pty-orphans branch 3 times, most recently from 323b0e5 to 2528e42 Compare August 3, 2026 17:13
@stefanhaller
stefanhaller changed the base branch from master to prevent-stale-index-lock-windows August 3, 2026 17:15
stefanhaller and others added 4 commits August 3, 2026 19:29
The task stop path terminates the still-running command by pulling its
*os.Process out of the Cmd interface and applying one global strategy
(TerminateProcessGracefully) to it. That shape can't accommodate the
upcoming fix for orphaned process trees on Windows: there, stopping a
pty task requires terminating the entire process tree via a job object
whose handle lives with the pty, not with the process. And the two Cmd
implementations genuinely need different strategies anyway: a
process-group kill (the likely future fix for #5675 on Unix) is only
safe for pty children, which run as session leaders, while plain
commands share lazygit's own process group.

So let each Cmd implementation decide how to terminate itself, and drop
GetProcess, which had no other callers. No change in behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stopping a pty task on Windows relies on ClosePseudoConsole, which
delivers CTRL_CLOSE_EVENT to the console's attached clients. But only
to those attached at that moment: when the user flicks quickly through
commits, a task is often stopped within the first few milliseconds of
its life, before the child has attached to the pseudoconsole. Such a
child misses the event and survives, running the entire diff to
completion in the background (spawning one external differ per changed
file) and keeping its conhost.exe alive; rapid navigation accumulates
these git/difft/conhost trees, and they outlive lazygit. Grandchildren
are affected too: git for Windows runs commands through a two-level
git.exe wrapper, so a single task has several attach windows, and a
grandchild spawned while the console is going down is orphaned even
when its parent got the event and exited.

Fix this by putting the child into a job object before it runs its
first instruction (created suspended, assigned, then resumed), so that
every descendant is in the job from the start; the teardown in Close
terminates the job right after initiating the pseudoconsole close.
There is no point in a grace period between the two: the close event
is not a graceful signal -- git and the common diff tools leave it to
the default handler, which calls ExitProcess at an arbitrary point --
so clients that received it are already dying, and the kill exists for
those that missed it. Killing at an arbitrary point cannot leak a
stale index.lock, because pty-rendered commands no longer take that
lock (see withPtyGitConfig in pkg/gui/pty.go).

The pseudoconsole close runs on its own goroutine because the kill
must not wait for it: on builds where ClosePseudoConsole blocks until
the console host exits (pre-24H2), the host keeps running as long as a
surviving client does, and that client only goes away through the job
kill; sequencing the kill after a blocking close would deadlock in
exactly the case the kill exists for.

KILL_ON_JOB_CLOSE doubles as a safety net: if lazygit exits without
running the teardown, the OS closes the job handle and reaps the tree.

In a harness that mimicked the stop path with randomized 0-120ms stop
delays, 3 of 30 process trees survived as orphans before this change;
none survive with it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…5879)

A pseudoconsole's conhost.exe is spawned by CreatePseudoConsole as a
child of lazygit, so it is not part of the job object that the pty
teardown kills. That is normally fine: a healthy conhost runs itself
down once the reference handle is closed and its clients are gone. But
conhost builds before the ConPTY overhaul that shipped with Windows 11
24H2 (confirmed on 23H2, build 22631) fail to complete the rundown
when a client attached after the close event was delivered and was
then killed -- the fate of exactly the clients the job kill exists for
-- and such a conhost lingers forever with no clients, at a rate of
about one per five fast commit navigations. These builds remain
widespread: all of Windows 10 (whose ESU tail runs into 2028, and
whose hardware often cannot run Windows 11 at all) plus pre-24H2
Windows 11 fleets.

Since Windows offers no way to obtain the conhost's pid or handle from
the HPCON, identify it by diffing lazygit's direct conhost children
around the CreatePseudoConsole call, serialized by a mutex so that two
concurrently starting ptys can't confuse each other's diff, and open a
handle immediately so that pid reuse is harmless. The teardown then
gives conhost a second to exit on its own before terminating it; on
healthy builds the wait succeeds and the reap never fires. If the
conhost can't be identified unambiguously, we simply don't reap, which
is no worse than before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pty teardown in Close runs on a background goroutine that doesn't
get to finish when lazygit is quitting: the process exits milliseconds
after the view buffer managers are closed. The job objects still cover
the clients -- KILL_ON_JOB_CLOSE reaps them when the process's handles
are rundown at exit -- but nothing reaps the conhost, so on Windows
builds whose conhost fails to run down on its own, quitting leaks one
conhost per live pty.

This is not a rare timing window: a diff longer than what has been
read keeps its git process (and thus its pty and conhost) running for
the entire time it is displayed, so that scrolling can read more.
Quitting while looking at a long diff is therefore the common case,
and with an external differ configured it leaks a conhost on affected
builds on almost every quit.

Fix this by having the gui's shutdown path wait synchronously for the
in-flight teardowns after closing the view buffer managers. A quit
signal makes the teardowns skip the conhost rundown wait -- the
conhost serves nothing once its clients are dead, and the exit must
not stall for its sake -- so the wait normally completes in
milliseconds, keeping quit as fast as before; a 2-second cap protects
the exit path even if a teardown wedges.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@stefanhaller
stefanhaller force-pushed the fix-windows-pty-orphans branch from 2528e42 to bc56716 Compare August 3, 2026 17:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Orphans on Windows

1 participant