Skip to content

feat(task-runner): stop running tasks when the process is interrupted - #837

Merged
ddeboer merged 4 commits into
mainfrom
fix/converter-stops-on-signal
Sep 21, 2026
Merged

ddeboer merged 4 commits into
mainfrom
fix/converter-stops-on-signal

Conversation

@ddeboer

@ddeboer ddeboer commented Sep 8, 2026 •

Copy link
Copy Markdown
Member

A task outlives the process that started it: a native task runs in a process group of its own, so a Ctrl-C never reaches it, and a container runs under the Docker daemon, which hears nothing of the process at all. Interrupting a SPARQL Anything conversion therefore left JVMs running and a sparql-anything-* directory behind.

@lde/task-runner now has ChildTasks, for implementing a runner – a runner’s users never see it. While any runner has a task going, the process listens for SIGINT and SIGTERM, stops every task the way that runner’s stop() does, and then ends as it would have without the listening: with the signal’s exit status (130, 143), or however a listener of the process’s own ends it. Nothing listens while no task is going, so an idle runner leaves the process’s signal handling as it found it.

What that binding covers:

  • Every process NativeTaskRunner spawns and every container DockerTaskRunner starts, so QLever’s index build and server come along too.
  • The listening goes in front of any listener the process already had, so what stop() does before its first await – sending a native task’s process group its SIGTERM – happens even when that listener ends the process at once. A container’s stop is a request to the daemon, so it carries no such guarantee; the docs page says so rather than overpromising.
  • A task that starts while the tasks a signal found are being stopped is stopped too, and the process waits for it – the sweep is a snapshot, and a chunk spawning just behind it would otherwise be missed.
  • A signal the process survives leaves the listening ready again, so tasks started afterwards are bound just the same.
  • The state is shared between copies of this package in one process. It is a plain dependency, so version skew can duplicate it, and two listeners each waiting for the other to end the process would hang a Ctrl-C.
  • A second signal while the tasks are being stopped ends the process at once, so a task that will not stop cannot hold it hostage.

SparqlAnythingConverter drops its own signal handling for this: its run directory goes on exit, synchronously, and that listener is only removed once the normal cleanup has actually finished – an interrupted run could otherwise end the process between the two and leave the directory behind.

Fix #831

@ddeboer ddeboer changed the title fix(sparql-anything): stop in-flight processes and clean up when the run is interrupted feat(task-runner): stop running tasks when the process is interrupted Sep 10, 2026
…run is interrupted

- Listen for SIGINT and SIGTERM for the duration of a run; on either, stop the processes
  still going, remove the run directory, drop the listeners and re-raise the signal so the
  process exits as it would have
- Each run registers its own listeners and removes only those, so parallel runs do not
  clobber each other
- Document the behaviour under “Converting several chunks at once”
- Add LiveTasks to @lde/task-runner: while any runner has a task going, the process
  listens for SIGINT and SIGTERM, stops every task the way the runner does, and then ends
  the process as it would have without the listening – with the signal’s exit status, or
  however another listener ends it. Listeners are removed as the signal arrives, so a
  second one ends the process at once.
- Bind every process NativeTaskRunner spawns and every container DockerTaskRunner starts
  to the process’s lifetime, so QLever’s index build and server are covered as well.
- Have SparqlAnythingConverter remove its run directory on process exit instead of
  listening for signals itself.
- Document the behaviour on the task-runner docs page and link to it from the runner and
  converter pages.
@ddeboer
ddeboer force-pushed the fix/converter-stops-on-signal branch from fb67550 to dbf0eed Compare September 10, 2026 11:51
- ChildTasks says whose tasks they are, and that they end with the parent; LiveTasks
  only said they were running.
- Keep it out of the task-runner docs page: it is for implementing a runner, and a
  runner’s users never see it.
- Track the listening in a flag of its own: it was inferred from the map of runners with
  tasks going, which onSignal leaves populated, so after a signal the process survived
  nothing could register a listener again and every task started afterwards was unbound.
- Stop a task that starts while the tasks a signal found are being stopped, and hold the
  process until that stop has settled: the sweep is a snapshot, so a task spawning just
  behind it was left running.
- Listen in front of any listener the process already had, so what stop() does before its
  first await – sending a native task’s process group its SIGTERM – happens even when that
  listener ends the process at once. Say on the docs page that a container’s stop, being a
  request to the daemon, has no such guarantee.
- Share the listening state between copies of this package in one process: it is a plain
  dependency, so version skew duplicates it, and two listeners each waiting for the other
  to end the process would hang a Ctrl-C.
- Remove the converter’s run directory before dropping its ‘exit’ listener, so an
  interrupted run cannot end the process between the two and leave the directory behind.
- Sort the directory listing the converter test asserts on; readdir order is the
  filesystem’s.
@ddeboer
ddeboer merged commit f6a21c4 into main Sep 21, 2026
4 checks passed
@ddeboer
ddeboer deleted the fix/converter-stops-on-signal branch September 21, 2026 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

task-runner-native: tasks outlive the process on SIGINT/SIGTERM

1 participant