feat(task-runner): stop running tasks when the process is interrupted - #837
Merged
Merged
Conversation
…run is interrupted - Listen for SIGINT and SIGTERM for the duration of a run; on either, stop the processes still going, remove the run directory, drop the listeners and re-raise the signal so the process exits as it would have - Each run registers its own listeners and removes only those, so parallel runs do not clobber each other - Document the behaviour under “Converting several chunks at once”
- Add LiveTasks to @lde/task-runner: while any runner has a task going, the process listens for SIGINT and SIGTERM, stops every task the way the runner does, and then ends the process as it would have without the listening – with the signal’s exit status, or however another listener ends it. Listeners are removed as the signal arrives, so a second one ends the process at once. - Bind every process NativeTaskRunner spawns and every container DockerTaskRunner starts to the process’s lifetime, so QLever’s index build and server are covered as well. - Have SparqlAnythingConverter remove its run directory on process exit instead of listening for signals itself. - Document the behaviour on the task-runner docs page and link to it from the runner and converter pages.
ddeboer
force-pushed
the
fix/converter-stops-on-signal
branch
from
September 10, 2026 11:51
fb67550 to
dbf0eed
Compare
- ChildTasks says whose tasks they are, and that they end with the parent; LiveTasks only said they were running. - Keep it out of the task-runner docs page: it is for implementing a runner, and a runner’s users never see it.
- Track the listening in a flag of its own: it was inferred from the map of runners with tasks going, which onSignal leaves populated, so after a signal the process survived nothing could register a listener again and every task started afterwards was unbound. - Stop a task that starts while the tasks a signal found are being stopped, and hold the process until that stop has settled: the sweep is a snapshot, so a task spawning just behind it was left running. - Listen in front of any listener the process already had, so what stop() does before its first await – sending a native task’s process group its SIGTERM – happens even when that listener ends the process at once. Say on the docs page that a container’s stop, being a request to the daemon, has no such guarantee. - Share the listening state between copies of this package in one process: it is a plain dependency, so version skew duplicates it, and two listeners each waiting for the other to end the process would hang a Ctrl-C. - Remove the converter’s run directory before dropping its ‘exit’ listener, so an interrupted run cannot end the process between the two and leave the directory behind. - Sort the directory listing the converter test asserts on; readdir order is the filesystem’s.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A task outlives the process that started it: a native task runs in a process group of its own, so a Ctrl-C never reaches it, and a container runs under the Docker daemon, which hears nothing of the process at all. Interrupting a SPARQL Anything conversion therefore left JVMs running and a
sparql-anything-*directory behind.@lde/task-runnernow hasChildTasks, for implementing a runner – a runner’s users never see it. While any runner has a task going, the process listens forSIGINTandSIGTERM, stops every task the way that runner’sstop()does, and then ends as it would have without the listening: with the signal’s exit status (130,143), or however a listener of the process’s own ends it. Nothing listens while no task is going, so an idle runner leaves the process’s signal handling as it found it.What that binding covers:
NativeTaskRunnerspawns and every containerDockerTaskRunnerstarts, so QLever’s index build and server come along too.stop()does before its firstawait– sending a native task’s process group itsSIGTERM– happens even when that listener ends the process at once. A container’s stop is a request to the daemon, so it carries no such guarantee; the docs page says so rather than overpromising.SparqlAnythingConverterdrops its own signal handling for this: its run directory goes onexit, synchronously, and that listener is only removed once the normal cleanup has actually finished – an interrupted run could otherwise end the process between the two and leave the directory behind.Fix #831