Skip to content

QoL: Kill server with a single keyboard interrupt#8919

Open
lstein wants to merge 7 commits intoinvoke-ai:mainfrom
lstein:lstein/feature/nicer-shutdown
Open

QoL: Kill server with a single keyboard interrupt#8919
lstein wants to merge 7 commits intoinvoke-ai:mainfrom
lstein:lstein/feature/nicer-shutdown

Conversation

@lstein
Copy link
Collaborator

@lstein lstein commented Feb 27, 2026

Summary

This PR cleans up what happens when you ^C the running invokeai-web server.

Before

Two ^C needed, and terminal looks lilke this:

^C[2026-02-26 18:56:13,606]::[ModelInstallService]::INFO --> Installer thread 139253501650624 exiting                                                                                                         
Traceback (most recent call last):                                                                      
  File "/home/lstein/invokeai-lstein/.venv/bin/invokeai-web", line 12, in <module>                                                                                                                              
    sys.exit(run_app())                                                                                 
             ^^^^^^^^^                                                                                  
  File "/home/lstein/Projects/InvokeAI-lstein/invokeai/app/run_app.py", line 103, in run_app                                                                                                                    
    loop.run_until_complete(server.serve())                                                                                                                                                                     
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 678, in run_until_complete                                                            
    self.run_forever()                                                                                                                                                                                          
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 645, in run_forever                                                                   
    self._run_once()                                                                                                                                                                                            
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 1999, in _run_once                                                                    
    handle._run()                                                                                                                                                                                               
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/events.py", line 88, in _run                                                                                
    self._context.run(self._callback, *self._args)                                                                                                                                                              
  File "/home/lstein/invokeai-lstein/.venv/lib/python3.12/site-packages/uvicorn/server.py", line 70, in serve                                                                                                   
    with self.capture_signals():                                                                                                                                                                                
         ^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                 
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 144, in __exit__                                                                               
    next(self.gen)                                                                                                                                                                                              
  File "/home/lstein/invokeai-lstein/.venv/lib/python3.12/site-packages/uvicorn/server.py", line 331, in capture_signals                                                                                        
    signal.raise_signal(captured_signal)   
KeyboardInterrupt                                                                                                                                                                      
 ^C
 Exception ignored in: <module 'threading' from '/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/threading.py'>                                                            
Traceback (most recent call last):                                                                                                                                                                              
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/threading.py", line 1624, in _shutdown                                                                              
    lock.acquire()                                                                                                                                                                                              
KeyboardInterrupt:                                                                                                                                                                                              

After:

Only one ^C needed, and terminal shows this:

^C
[2026-02-26 19:35:26,931]::[ModelInstallService]::INFO --> Installer thread 127057625994944 exiting
[2026-02-26 19:35:26,956]::[InvokeAI]::INFO --> InvokeAI shutting down...

Related Issues / Discussions

QA Instructions

  1. Start the server, run a few generations, then send a keyboard interrupt with ^C.
  2. Observe the terminal log messages, and see if server stops without additional intervention.

Merge Plan

Simple merge.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

Copilot AI and others added 4 commits February 26, 2026 03:41
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
* Initial plan

* Handle KeyboardInterrupt in run_app to allow single Ctrl+C shutdown

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

* Force os._exit(0) on KeyboardInterrupt to avoid hanging on background threads

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
@github-actions github-actions bot added python PRs that change python files frontend PRs that change frontend files labels Feb 27, 2026
@lstein lstein added the v6.12.0 Intended for 6.12.0 release label Feb 27, 2026
@lstein lstein changed the title QOL: Kill server with a single keyboard interrupt QoL: Kill server with a single keyboard interrupt Feb 27, 2026
Copy link
Collaborator

@JPPhoto JPPhoto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit concerned about the brute-force exit here. What happens to all of the existing threads?

except KeyboardInterrupt:
logger.info("InvokeAI shutting down...")
# Force exit to avoid hanging on non-daemon background threads (e.g. model install service).
os._exit(0)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we make sure threads clean up instead of forcing an exit?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! Fixed in commit 36150df

When no model install or download is active:

[2026-02-28 10:47:05,987]::[InvokeAI]::INFO --> Invoke running on http://127.0.0.1:9090 (Press CTRL+C to quit)
^C[2026-02-28 10:47:10,681]::[ModelInstallService]::INFO --> Installer thread 127430597535424 exiting
[2026-02-28 10:47:10,700]::[InvokeAI]::INFO --> InvokeAI shutting down...

When a download or install is active:

[2026-02-28 10:44:52,983]::[DownloadQueueService]::INFO --> File download started: https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/resolve/main/text_encoder_2/model-00001-of-00002.safetensors
^C[2026-02-28 10:45:03,826]::[ModelInstallService]::WARNING --> Pausing job 0
[2026-02-28 10:45:03,826]::[ModelInstallService]::WARNING --> Pausing black-forest-labs/FLUX.1-Kontext-dev
[2026-02-28 10:45:03,827]::[DownloadQueueService]::WARNING --> Download paused: https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/resolve/main/text_encoder_2/model-00001-of-00002.safetensors
[2026-02-28 10:45:04,821]::[ModelInstallService]::INFO --> Installer thread 136404375795392 exiting
[2026-02-28 10:45:05,532]::[InvokeAI]::INFO --> InvokeAI shutting down...

Copy link
Collaborator

@JPPhoto JPPhoto Feb 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried after generating? I cannot get Invoke to exit gracefully; it hangs on "InvokeAI shutting down...".

[2026-02-28 10:38:22,186]::[InvokeAI]::INFO --> InvokeAI shutting down...
^CException ignored in: <module 'threading' from '/home/jsp/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/threading.py'>
Traceback (most recent call last):
  File "/home/jsp/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/threading.py", line 1624, in _shutdown
    lock.acquire()
KeyboardInterrupt:

Something is holding onto a lock.

* Initial plan

* Replace os._exit(0) with ApiDependencies.shutdown() on KeyboardInterrupt

Instead of immediately force-exiting the process on CTRL+C, call
ApiDependencies.shutdown() to gracefully stop the download and install
manager services, allowing active work to complete or cancel cleanly
before the process exits.

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

* Make stop() idempotent in download and model install services

When CTRL+C is pressed, uvicorn's graceful shutdown triggers the FastAPI
lifespan which calls ApiDependencies.shutdown(), then a KeyboardInterrupt
propagates from run_until_complete() hitting the except block which tries
to call ApiDependencies.shutdown() a second time.

Change both stop() methods to return silently (instead of raising) when
the service is not running. This handles:
- Double-shutdown: lifespan already stopped the services
- Early interrupt: services were never fully started

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
@github-actions github-actions bot added the services PRs that change app services label Feb 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

frontend PRs that change frontend files python PRs that change python files services PRs that change app services v6.12.0 Intended for 6.12.0 release

Projects

Status: 6.12.x

Development

Successfully merging this pull request may close these issues.

3 participants