fix(lancedb): deadline must cover table-handle resolution

[gloryfromca]

Follow-up to #384. A 1h high-rate soak run (run9) reproduced the stall #384 was supposed to close, on a different table.

What happened

episode went 13 minutes without a prune — retained versions climbing 63 → 66 — while foresight and atomic_fact both collapsed to 1 in the same window. Zero failure, timeout, or conflict logs. Its last successful prune was logged at 11:41:59 and the prune-staleness clock matched to the second, so the beat simply stopped happening.

Why #384 didn't cover it

The deadline covered the critical section but not the await immediately ahead of it:

table = await self._table()                      # outside the deadline
async with self._locked(_PRUNE_TIMEOUT_SECONDS, "prune"):
    ...

A hang while resolving the table handle never returns. The maintenance scheduler runs one task per kind and skips a kind whose task is still in flight (a LanceDB table takes one writer), so that kind stops being maintained — permanently, and silently, because nothing failed.

Fix

• Handle resolution moved inside the deadline for all seven locked operations (add / upsert / update / delete / delete_by_md_path / prune / rebuild_indexes).
• The lock-free compaction beat gets its own _deadline. It takes no lock so it cannot block writers, but it can still park a kind by never returning.
• Last-resort bound in the scheduler: both beats now run under _MAINTENANCE_TASK_TIMEOUT_SECONDS (180s), so an await I have not thought of costs one cadence instead of forever. Generous against prune's own 60s budget, so it never fires on a healthy beat.

Test

A repo whose _table_lookup never resolves must make prune, optimize and add all raise VectorStoreBusyError and leave the write lock free. Without this change that test parks instead of failing.

make lint clean, 1896 unit + 182 integration passing.

Also verified in run9 (this is what found the bug)

The three fixes that shipped in #384 all landed as intended, measured against run8 on identical load:

	run8 (2h)	run9 (1h)
query-vector width rejections	—	7, each failing fast
unhandled exceptions	11	0
searches ≥10s	11	0
server.log	100 MB	4 MB
write-lock deadline warnings	2	0

The new slow-hold instrumentation also corrected an earlier measurement of mine: I had timed prune at 31ms on an offline copy and used that to rule out "slow prune holds the lock". Under real concurrency it holds for 1.2–1.9s — 40-60x that — which puts that hypothesis back on the table for the 16s stall seen in run8.

🤖 Generated with Claude Code

[gloryfromca]

Folded into the 1.2.2 release PR (#386) so the release does not ship with this stall. Same commit, rebased onto main.