Skip to content

agentHost: await Codex process shutdown before directory cleanup - #337070

Closed
Christof Marti (chrmarti) wants to merge 1 commit into
mainfrom
chrmarti/fix-codex-shutdown-cleanup
Closed

Christof Marti (chrmarti) wants to merge 1 commit into
mainfrom
chrmarti/fix-codex-shutdown-cleanup

Conversation

@chrmarti

@chrmarti Christof Marti (chrmarti) commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

Main-based adaptation of Dmitriy Vasyura (@dmitrivMS)'s #335992, including both the original fix and its review follow-up. This is not an independent implementation: the original commits are recorded in the commit message and Dmitriy Vasyura is credited as a co-author. The existing PR remains open and is not modified by this PR.

Investigated after the unrelated Windows / Electron failure on #337043: all 100 Codex E2E tests passed, but suite teardown failed to remove its isolated home. Current main stops the direct Codex child without awaiting its process tree and test teardown observes the agent-host parent's exit, which does not prove descendants have released filesystem handles. The job's aggregate error does not identify the exact locking process or nested filesystem errno, so that individual CI occurrence cannot be conclusively attributed from its logs alone.

Changes

  • Await bounded Codex process-tree shutdown, covering persistent, transient, cancelled, and late-ready startup paths.
  • Record owned Windows descendants before EOF and reap survivors, including when the parent exits cleanly. Reuse killTree for forced cleanup.
  • Release sandbox/proxy resources after shutdown and retain nested shutdown/filesystem diagnostics.
  • Give protocol drain, provider shutdown, and persistence flushing their own bounded phases; derive the test-server grace period from the complete budget.
  • Retain the original lifecycle/process tests and E2E troubleshooting documentation. No SDK upgrade, replay-fixture changes, test quarantine, or extended directory-removal retry budget.

The port preserves current main's newer ThrottlerByKey workspace-change handling; that import was the only textual conflict when applying the two original commits.

Validation

Local Windows, Electron 43.6.0, Codex 0.153.0:

  • npm run gulp compile-client: passed with 0 errors.
  • Seven affected unit-test suites: 291 passing, including delayed/ignored EOF, detached descendants, held-file diagnostics, startup races, and shutdown budgets.
  • ESLint on all 15 changed TypeScript files: passed.
  • Diff checks and commit hygiene: passed.
  • Before the adaptation, the 3 focused Codex replay tests passed on current main; the intermittent CI directory lock was not reproduced by that focused run.
  • Full Codex strict replay: 100 passing, 29 pending, including successful suite teardown (10 minutes).
  • A second focused strict replay covering common and Codex-specific suite lifetimes: 3 passing, teardown successful (29 seconds).

Azure pipeline 111 build 475538 is running for Windows, Linux, and macOS x64 with publishing/release disabled, pinned to commit 9b149aee4c1e9a4238e8140df309d4c552344392. Cross-platform results are pending; this remains a draft.

Adapt #335992 to current main, including its review follow-up for startup cleanup errors and shutdown budgets. Preserve the current throttled workspace-change handling while awaiting owned process-tree shutdown and retaining cleanup diagnostics.

Based on 6b488c7 and 072087b.

Co-authored-by: Dmitriy Vasyura <dmitriv@microsoft.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 21, 2026 15:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Cross-platform process lifecycle behavior remains high-risk and the described strict replay and Azure validations are still pending.

Review effort: Balanced
Findings: None

What changed in this PR

Hardens Agent Host teardown so Codex process trees fully stop before temporary resources are removed.

Changes:

  • Adds bounded, awaitable process-tree shutdown and cleanup.
  • Separates Agent Host shutdown into independently bounded phases.
  • Expands lifecycle tests and teardown diagnostics.
File Description
src/​vs/​base/​node/​processes.ts Adds bounded process-tree shutdown.
src/​vs/​platform/​agentHost/​node/​agentHostManagementService.ts Extends provider shutdown allowance.
src/​vs/​platform/​agentHost/​node/​agentHostServerMain.ts Reports shutdown failures through exit status.
src/​vs/​platform/​agentHost/​node/​agentHostService.ts Extends managed shutdown timeout.
src/​vs/​platform/​agentHost/​node/​agentHostShutdown.ts Introduces separate bounded shutdown phases.
src/​vs/​platform/​agentHost/​node/​codex/​codexAgent.ts Awaits connection and resource cleanup.
src/​vs/​platform/​agentHost/​node/​codex/​codexAppServerClient.ts Adds awaitable transport shutdown.
src/​vs/​platform/​agentHost/​test/​node/​agentHostShutdown.test.ts Tests shutdown phases and budgets.
src/​vs/​platform/​agentHost/​test/​node/​codex/​codexAppServerClient.test.ts Tests client shutdown behavior.
src/​vs/​platform/​agentHost/​test/​node/​codex/​codexAppServerProcess.test.ts Covers real process-tree cleanup scenarios.
src/​vs/​platform/​agentHost/​test/​node/​codex/​codexCreateChat.test.ts Updates lifecycle test transports.
src/​vs/​platform/​agentHost/​test/​node/​codex/​codexModelRefresh.test.ts Adds startup and shutdown race coverage.
src/​vs/​platform/​agentHost/​test/​node/​codex/​codexPrewarmEviction.test.ts Updates mock transport shutdown.
src/​vs/​platform/​agentHost/​test/​node/​e2e/​harness/​agentHostE2ETestHarness.ts Awaits server teardown before directory removal.
src/​vs/​platform/​agentHost/​test/​node/​e2e/​README.md Documents teardown behavior and diagnostics.
src/​vs/​platform/​agentHost/​test/​node/​serverIntegrationTestHelpers.ts Uses process-tree shutdown and captures output.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown
Contributor

Screenshot Changes

Base: 58d7d254 Current: 0ca20a16

Changed (1)

chat/agent-lifecycle/chatAgentLifecycle/RetainedFollowUpLatestTurn/Dark
Before After
before after

1 insignificant change(s) omitted (≤20 px, Δ≤2). See CI logs for details.

@chrmarti

Copy link
Copy Markdown
Collaborator Author

Closing this overly broad adaptation in favor of the minimal test-harness-only fix in #337080. The broader provider-lifecycle work remains in #335992; this PR is not taking it over.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants