Skip to content

fix(signal): keep honoring signals after the force-quit window, add a hard shutdown deadline - #1347

Open
Dumbris wants to merge 1 commit into
mainfrom
fix/force-quit-deadline
Open

Dumbris wants to merge 1 commit into
mainfrom
fix/force-quit-deadline

Conversation

@Dumbris

@Dumbris Dumbris commented Sep 22, 2026

Copy link
Copy Markdown
Member

What

runServer's signal handler was one-shot: it read one signal, armed a 10s force-quit timer, and returned when that timer won the select. signal.Notify stays registered for the life of the process, so once that goroutine was gone the Go runtime kept intercepting SIGINT/SIGTERM (their default disposition is disabled) and dropping them into a one-slot buffer nobody read.

If graceful shutdown exceeds 10s — the precise situation in which an operator is hammering Ctrl+C — the daemon could then only be killed with SIGKILL. On the normal path cancel() finishes shutdown well inside the window, so this is a hung-shutdown bug, not an "after 10 seconds it always breaks" bug.

How

The logic moves into runSignalHandler (cmd/mcpproxy/signal_handler.go) behind an injected signal channel, clock and exit func. After the first signal it arms both windows and hands the process's fate to one small goroutine that reads only channels:

  • a further signal → immediate exit, code 1 (unchanged behavior)
  • a new 60s hard deadline, measured from the first signal → exit with the new ExitCodeShutdownTimeout (6)
  • shutdown completing → runServer returns and the process exits 0

That goroutine never logs, deliberately: the failure that wedges shutdown (a full disk, a tray that stopped draining the core's stderr pipe) is the same failure that wedges the log sink, so a forced exit that had to get past a log write first would not be forced at all. The reason is announced up front instead, with both exit codes as fields. For the same reason the goroutine is started immediately after signal.Notify, before any logging. The 10s window is demoted to a one-shot reminder — its old "within 10 seconds" wording was a lie once the window never closed.

Decisions taken per the Zero Interruption Policy: 60s for the hard deadline (under systemd's 90s DefaultTimeoutStopSec) and 6 for the exit code (6-9 were free; 10-12 are the Spec 098 preflight band). classifyError / classifyStartupError are deliberately untouched — this path calls os.Exit directly and exit 6 is not a startup outcome. Downstream, the tray's process monitor and CoreError.fromExitCode both fall through to their general-error default, which is the desired "supervisor restarts it".

Behavior change worth naming: there was previously no upper bound on shutdown here — srv.Shutdown() blocks indefinitely — so a cleanup that legitimately takes longer than 60s is now cut short with exit 6. Accepted trade-off.

Spec 024 is unchanged: receivedSignal is still stored before cancel(), which a test pins, since the ctx.Done() branch type-asserts it unchecked.

Verified

  • TDD: TestRunSignalHandler_SignalAfterGraceWindowStillHonored fails against the old one-shot logic (the process was never asked to exit) and passes after. Every later hardening step has its own test that fails against the step before it.
  • 7 unit tests, go test -race ./cmd/mcpproxy/ -count=5, injected channels only — no real sleeps, no timing assertions.
  • Full cmd/mcpproxy package with -race; both golangci-lint v2 passes (bare and --build-tags server) clean for the touched files.
  • On a real daemon (throwaway build with the windows at 2s/6s and a shutdown that hangs forever): a SIGTERM sent after the window still exits 1; with no further signal the process exits 6 on time; a healthy single SIGTERM still exits 0.
  • Cross-model review: codex exec --model gpt-5.6-sol, 5 rounds, final verdict NO BLOCKING ISSUES. Rounds 1-4 each found a genuine gap in the deadline's independence from logging; all were accepted and fixed.

Docs: exit code 6 added to the list in CLAUDE.md and to the tray's error-classification list in docs/architecture.md. docs/cli-management-commands.md's "Exit Codes" table is a different (subcommand) table and is left alone.

🤖 Generated with Claude Code

… hard shutdown deadline

The signal handler goroutine was one-shot: it read one signal, armed a 10s
timer, and returned when that timer won the select. signal.Notify stays
registered for the life of the process, so once the goroutine was gone the Go
runtime kept intercepting SIGINT/SIGTERM (their default disposition is
disabled) and dropping them into a one-slot buffer nobody read. If graceful
shutdown ran longer than 10s - exactly when an operator is hammering Ctrl+C -
the daemon could only be killed with SIGKILL.

The logic moves into runSignalHandler (signal_handler.go), behind an injected
signal channel, clock and exit func so it is unit-testable. After the first
signal it arms both windows and hands the process's fate to a small goroutine
that reads only channels: a further signal exits immediately with code 1, and a
new 60s hard deadline, measured from the first signal, exits with the new
ExitCodeShutdownTimeout (6) so launchd / systemd / the tray's process monitor
can restart a wedged daemon. That goroutine never logs, on purpose: the failure
that wedges shutdown (a full disk, a tray that stopped draining the core's
stderr pipe) is the same failure that wedges the log sink, so a forced exit
that had to get past a log write first would not be forced at all. The reason
is announced up front instead, with both exit codes in the fields, and the
handler goroutine does the remaining reporting. For the same reason the
goroutine is now started before runServer's own log line: signal.Notify is
already registered by then.

The 10s window is demoted to a one-shot reminder; its old "within 10 seconds"
wording was a lie once the window never closed.

This is also the first upper bound on shutdown in this path - srv.Shutdown()
was previously unbounded - so a cleanup that legitimately takes longer than 60s
is now cut short with exit 6. Accepted trade-off; 60s sits under systemd's 90s
DefaultTimeoutStopSec.

Spec 024 is unchanged: receivedSignal is still stored before cancel(), which a
test pins, since runServer's ctx.Done() branch type-asserts it unchecked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying mcpproxy-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 4da584b
Status: ✅  Deploy successful!
Preview URL: https://8310e800.mcpproxy-docs.pages.dev
Branch Preview URL: https://fix-force-quit-deadline.mcpproxy-docs.pages.dev

View logs

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 77.55102% with 11 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
cmd/mcpproxy/signal_handler.go 84.44% 4 Missing and 3 partials ⚠️
cmd/mcpproxy/main.go 0.00% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

@github-actions

Copy link
Copy Markdown
Contributor

📦 Build Artifacts

Workflow Run: View Run
Branch: fix/force-quit-deadline

Available Artifacts

  • archive-darwin-amd64 (30 MB)
  • archive-darwin-arm64 (27 MB)
  • archive-linux-amd64 (18 MB)
  • archive-linux-arm64 (16 MB)
  • archive-windows-amd64 (30 MB)
  • archive-windows-arm64 (26 MB)
  • frontend-dist-pr (0 MB)
  • installer-dmg-darwin-amd64 (24 MB)
  • installer-dmg-darwin-arm64 (22 MB)
  • smart-mcp-proxymcpproxy-goKFAL1O.dockerbuild (0 MB)

How to Download

Option 1: GitHub Web UI (easiest)

  1. Go to the workflow run page linked above
  2. Scroll to the bottom "Artifacts" section
  3. Click on the artifact you want to download

Option 2: GitHub CLI

gh run download 35748535939 --repo smart-mcp-proxy/mcpproxy-go

Note: Artifacts expire in 14 days.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants