Skip to content

Native async create_record: one linger thread for the process instead of one per executor (refs #406) - #417

Draft
tae898 wants to merge 1 commit into
mainfrom
perf/406-shared-linger
Draft

tae898 wants to merge 1 commit into
mainfrom
perf/406-shared-linger

Conversation

@tae898

@tae898 tae898 commented Nov 21, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

On the native wheel, NativeAsyncExecutor.create_record collects the records after the first 128 of an executor and a linger thread hands a batch over after 50 ms. Until now every executor started its own thread for that, with its first collected record (_start_linger), and kept it, blocked in Condition.wait(), until close(). This draft replaces it with one daemon thread for the whole process, as proposed in #406.

What I changed, all in src/arcadedb/_native/extras.py:

  • A module-level _Linger keeps the due times of the collected batches in a heap on one daemon thread, started at first use. The first record of a batch calls add(executor, due) (_start_linger); an executor starts no thread.
  • An entry whose batch was handed over earlier by a flush point (wait_completion, close, query, a change of type, 1,000 records) is dropped when its time comes, without a thread (_linger_pending, a read without the lock). An entry whose batch is still the executor's current one starts a short-lived thread that runs _linger_fire, under the executor's _buf_lock, exactly the hand-over the old thread made. It is a separate thread because a hand-over can block in the engine (a full queue, back pressure): with a thread for each executor a blocked hand-over held back only its own executor, and I did not want the shared thread to change that.
  • os.register_at_fork(after_in_child=...) gives a forked child a fresh _Linger, since the child has none of the parent's threads and may inherit the lock held. This does not make the native library usable in a forked child; it only keeps the new lock from being a new way to hang.

What stays the same for a caller:

  • Ordering. The engine queue still sees the calls in the order they were made: every hand-over (flush point or linger) swaps the buffer out and hands it over while holding the executor's _buf_lock, and a create_record that arrives meanwhile waits for that lock and goes after it. Lock order is one: an executor's _buf_lock, then the _Linger lock. The timer thread never holds its lock while it touches an executor.
  • The 50 ms bound, the error that a linger hand-over meets (kept in _linger_error, raised by the next call that hands records over), per-record callbacks and error callbacks, close() flushing first and stopping the linger of that executor, _ASYNC_DIRECT = 128 and _ASYNC_FRAME_MIN = 64.
  • Durability and read-your-writes are those of the async API before: a collected record is not in the engine queue until a hand-over, and only wait_completion() on that executor covers it. This is Native: async create_record hands records over as columnar frames with per-record callbacks (refs #388, #315) #393's behaviour for records beyond the 128th and I did not change it. It does not touch commits or indexes, so I do not expect it to change the intermittent miss of Windows Python 3.14 parity: test_vector_read_your_writes missed 1 of 200 committed vectors once on main (intermittent) #409 (a sync-path commit and search); I have no evidence either way.
  • At interpreter exit the thread is a daemon, as before, so a batch that no flush point has seen is lost as before.

One thing is different on purpose: an executor that is dropped without close() is no longer kept alive by a thread blocked for good. Its heap entry keeps it for at most the linger time, so a dropped executor still hands its batch over after 50 ms.

Related issues

Refs #406

Performance changes

Not measured yet, so this PR makes no speed claim. What the code shows: the thread start (Thread and Event objects, the _started handshake) and the blocked thread that each executor left behind are gone from the caller's path, and the copy of the properties and the lock for each collected record stay. Whether that explains the 198 to 388 us per executor in the band of 129 to 191 records is not established: #406 measured a bare thread start at about 30 us, and the split of the cost (thread start, condition and lock traffic, the copy, the first hand-over) has not been made. I also did not change _ASYNC_DIRECT; whether it can fall or go with the shared thread needs that measurement, not a guess.

The A/B is a script outside the repository: the published 26.11.1.post4 native wheel with only _native/extras.py swapped between the arms (the merge base, this branch, and the merge base again as control), the micro of #406, interleaved rounds, paired ratios with a bootstrap interval. I will post the numbers on #406 and put them here before this leaves draft.

Docs

docs/modules/ROOT/pages/how-to/async-executor.adoc said that a thread of the executor hands a batch over and that a shared thread "would remove this cost". It now says what post4 does and what this change does, and that the figures on the page are post4's and the shared thread has not been timed.

Tests

tests/test_native_async_create_batch_unit.py (no JVM, no native library: the library call is a recorder, so these count crossings and threads, not seconds). The two tests that named the removed _linger_thread now check the process-wide thread. New: an executor starts no thread of its own (20 executors, the thread count does not move), 10 executors are handed over in time by the one thread, a stale entry starts no hand-over thread, the entry of an earlier batch does not cut a later batch short, the order of the calls holds across linger hand-overs and flush points, a hand-over that blocks holds back no other executor, _linger_after_fork starts afresh, and close() leaves nothing for the linger thread to hand over.

Checked so far: compile() and pyflakes on both files (one older global _upcall_ref warning that is not from this change), uvx pre-commit run --files on the three files (black reformatted my lines, the run after it is clean), bandit clean on both. I have not run the unit file under pytest yet (a draft skips the platform jobs, so CI will not either), so the box below stays open. Instead I ran the same cases as a plain script against the fake library, one real os.fork with a child that creates and hands over a batch, and a 203-record run of the #406 micro against the real native library of the post4 wheel with this branch's extras.py laid over it (203 of 203 records stored). I will run the unit file before marking this ready.

Not tested: a real fork on Windows (no os.fork; register_at_fork is guarded), the interaction with a running engine under back pressure (the blocked hand-over is simulated), and anything about speed.

Checklist

  • A regression test fails on the old code and passes on the new
  • The test suite that the change can affect passes
  • uvx pre-commit run --files <changed files> passes

… of one per executor (refs #406)

The collected path of NativeAsyncExecutor.create_record started a Python thread with the first collected record of every executor (_start_linger)
and kept it, blocked in Condition.wait(), until close(); an executor that is never closed also kept its thread and itself alive.

Now a module-level _Linger keeps the due times of the batches (a heap of (due, serial, executor)) on one daemon thread that is started at first use.
The first record of a batch registers the batch with add(); a flush point that hands the batch over earlier leaves the entry to expire, and an
expired entry whose batch is gone or replaced is dropped without a thread (_linger_pending, read without the lock). A batch that is still due is
handed over by _linger_fire on a short-lived thread, not on the timer thread: the hand-over can block in the engine (a full queue, back pressure),
and one executor's blocked hand-over must not hold back the batches of the others, as it could not when each executor had its own thread.

What stays: the hand-over runs under the executor's _buf_lock exactly as before (so the engine queue sees the calls in the order they were made, and a
caller's next create_record waits for it), the 50 ms bound, the error kept in _linger_error and raised by the next call that hands records over,
close() flushing first and stopping the linger of that executor, _ASYNC_DIRECT = 128 and _ASYNC_FRAME_MIN = 64 (they need a measurement, not a guess).
Lock order is one: an executor's _buf_lock, then the _Linger lock; the timer thread never holds its lock while it touches an executor.
Fork: os.register_at_fork(after_in_child) starts a fresh _Linger, since the child has no thread of the parent and may inherit its lock held.
This does not make the native library usable in a forked child; it only keeps the new lock and thread from being a new way to hang.

Not measured: whether this removes the 198 to 388 us that #406 saw for executors of 129 to 191 records. The code shows that the thread start,
the Thread and Event objects and the leaked thread per executor are gone from the caller's path, and that the copy of the properties and the lock per
collected record stay; the split of the measured cost is not established. The A/B script is outside the repository.

The two tests that named the removed _linger_thread now check the process-wide thread; new tests cover: no thread per executor, many executors
handed over in time, a stale entry starting no hand-over thread, the entry of an earlier batch not cutting a later one short, the order of the calls
across linger hand-overs and flush points, a blocked hand-over holding back no other executor, the fork reset, and close.
@tae898 tae898 added concurrency Threads, transactions and conflicts native The arcadedb package (native library build) performance Speed or memory of the bindings, with a measurement python Pull requests that update python code labels Nov 21, 2026
@tae898 tae898 self-assigned this Nov 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

concurrency Threads, transactions and conflicts native The arcadedb package (native library build) performance Speed or memory of the bindings, with a measurement python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant