service: self-update orphans background server and hangs every client

[mwikala]

Summary

If the CLI auto-updates itself while a serve --service background server is running, npm replaces
the package directory out from under the live server. The server process survives but its HTTP
listener is gone. It keeps holding the exclusive flock on service-<hash>.json.lock, so every
serve --service spawned afterwards sees the lock held, assumes a healthy server already exists,
and exits 0 immediately — without binding, and without logging a reason.

The client then loops forever: spawn serve --service → it exits instantly → retry 30s later. The
TUI sits on "Starting background server" indefinitely.

It is not recoverable with kill, because the orphan has a SIGTERM handler whose graceful-shutdown
path is itself wedged — the signal is delivered, consumed, and nothing happens. Only kill -9 (or a
reboot, which is what I had been doing) clears it.

Environment

• Affected opencode version: 0.0.0-next-15718 (self-updated in place from
  0.0.0-next-15707)
• Current opencode version: 0.0.0-next-15747
• OS: Linux 7.1.3-arch1-3, x86_64
• Terminal: Ghostty (TERM=xterm-256color, COLORTERM=truecolor)
• Shell: /usr/bin/zsh
• Install/channel: npm (@opencode-ai/cli), next
• Node: v24.18.0
• Active plugins: @ex-machina/opencode-anthropic-auth@1.8.1, local sounds.ts, local
  worktrunk.ts

Reproduction

1. Have a serve --service background server running on version N.
2. Let the CLI auto-update itself to N+1 via npm while that server is still alive.
3. Run opencode2 again → hangs on "Starting background server..." forever.

Expected Behavior

The updater should stop or restart the running service before replacing its files. If a stale
service remains, a later client should recover from it or report a bounded startup error.

Actual Behavior

The old service process remains alive with the service flock but no listening socket. Every new
service process exits successfully without binding, and clients retry forever while displaying
"Starting background server".

Evidence

The running server's executable is deleted — it points into npm's temporary directory
(.cli-NKZ9SKJr), where npm moved the old package while replacing it:

$ ls -l /proc/8802/exe
/proc/8802/exe -> .../@opencode-ai/.cli-NKZ9SKJr/node_modules/@opencode-ai/cli-linux-x64/bin/opencode2 (deleted)

The service descriptor still advertises the pre-update version and a port nothing is bound to:

$ cat ~/.local/state/opencode/service-<hash>.json
{"id":"6ed1a76f-...","version":"0.0.0-next-15707","url":"http://127.0.0.1:4096","pid":8802,"password":"..."}

$ curl -m5 http://127.0.0.1:4096/
curl: (7) Failed to connect to 127.0.0.1:4096 after 0 ms: Could not connect to server

The orphan holds the flock (and SQLite locks) but has no listening socket — only connected
socketpairs remain, and it idles in do_epoll_wait. It was reparented to systemd --user (PPID 1),
so nothing supervises or restarts it:

$ lslocks -u | grep opencode
opencode2  8802  FLOCK  WRITE  ~/.local/state/opencode/service-<hash>.json.lock
opencode2  8802  POSIX  READ   ~/.local/share/opencode/opencode-next.db

Timeline from ~/.local/share/opencode/log/opencode.log (UTC). The server starts, is healthy (MCP
servers connect), the CLI updates itself 20s later, and the server never logs again:

13:44:01  cli starting  version=0.0.0-next-15707 args=["serve","--service"]   # pid 8802
13:44:02  mcp connected server=slite tools=38                    role=server  # server healthy
13:44:03  mcp connected server=context7 tools=2                  role=server
13:44:21  updated OpenCode from=0.0.0-next-15707 to=0.0.0-next-15718 method=npm   role=cli
          # ...no role=server lines ever again...
13:45:10  cli starting  args=["serve","--service"]
13:45:40  cli starting  args=["serve","--service"]
13:46:10  cli starting  args=["serve","--service"]
          # every 30s, forever — 253 attempts over the next ~2h

Each respawn exits immediately and silently. Reproduced by hand while the orphan held the lock:

$ opencode2 serve --service
EXIT=0    # instant, no output, no bind, nothing in the log beyond "cli starting"

kill has no effect — SIGTERM is caught (bit 15 set in SigCgt) and swallowed by a shutdown path
that never completes. After sending it, no signal is left pending and the process is unchanged:

$ grep -E 'State|SigIgn|SigCgt|SigPnd' /proc/8802/status
State:   S (sleeping)
SigPnd:  0000000000000000
SigCgt:  00000001200044ca      # bit 15 (SIGTERM) set → handler installed

kill -9 8802 releases the flock, and the next client starts a healthy server normally.

Root cause (best guess)

Three things compound:

1. The flock is treated as proof of liveness. serve --service acquires the lock or bails. It
   never checks that the descriptor's url is reachable, so a lock-holding process with a dead
   listener wedges every future start permanently. The descriptor still records version: 15707
   while the newly installed CLI is 15718 — that mismatch is already on disk and unused.

2. Self-update doesn't stop the running server. npm rename-swaps the package dir under the live
   process. The server loses its listener but stays alive (pending handles keep the loop open), so
   it never releases the flock, and since it's been reparented to systemd --user, nothing notices.

3. Graceful shutdown is wedged in this post-update state. The SIGTERM handler consumes the
   signal but never completes. The exact reason is unknown, but it makes the state survive
   everything short of SIGKILL.

Suggested fixes

• Before deferring to a lock holder, health-check the descriptor's url (and/or compare its
  version/pid against the current process). If it's unreachable or stale, terminate the stale
  service and reacquire the lock instead of exiting 0.
• Make serve --service log why it exited. A silent exit 0 on the unhappy path made a 30s respawn
  loop completely invisible — the only user-visible symptom is a spinner that never resolves.
• Have the updater signal the running server to shut down and confirm it exited before swapping
  files, rather than deleting the package underneath it.
• Bound the client's retry loop: after N failed spawns, surface an error instead of spinning forever.

Workaround

kill -9 $(cat ~/.local/state/opencode/service-*.json | grep -oP '"pid":\K[0-9]+')

Plain kill is not enough. No reboot required.

[github-actions]

This issue might be related to existing issues. Please check:

• fix(cli): TUI crashes with unhandled transport defect during post-update service restart #36688: Post-update service restart also causes TUI crashes and service lock races — covers the election/reconnect side of the same update flow, while this issue specifically describes the flock-holding orphan with a dead listener that makes recovery require kill -9.