fix(diskann): tombstone deletes with vector preservation, local graph repair, and delete_deferred escape hatch

[ohdearquant]

Summary

• Preserve deleted vectors and track deletion state in a tombstone bitset so distance calculations remain finite and deterministic.
• Filter tombstoned candidates before exact re-ranking so deleted IDs cannot appear in search results.
• Repair the graph locally on normal deletes, while exposing delete_deferred() for callers that prefer tombstone-only deletion and later batch cleanup.
• Persist tombstones and report the live vector count consistently across save and load.

Fixes #679

Design

Deletion now marks the internal vector position in a packed tombstone set and removes the external ID from the live reverse map. The original vector values remain unchanged. This removes the prior NaN distance hazard and lets tombstoned nodes remain valid routing waypoints when deletion is deferred. Search filters tombstones before exact re-ranking and uses a total floating-point ordering.

The default delete() path performs local graph repair. Because this crate does not maintain reverse adjacency, it finds the deleted node's in-neighbors by scanning the bounded-degree adjacency in O(n * degree) time per deletion. For each live in-neighbor, it combines the node's existing candidates with the deleted node's live out-neighbors, removes deleted candidates, and applies the existing alpha robust-prune selection with the configured maximum degree. The deleted node is then removed from all adjacency. Tombstoned sources and candidates are short-circuited during the scan.

delete_deferred() is the low-latency escape hatch. It records the tombstone without modifying graph edges, allowing callers to defer repair or rebuild work. Unknown IDs and repeated deletes return DiskAnnError::NotFound.

Measurements

Recall is recall@10 against brute-force ground truth over surviving vectors after seeded random deletion of 20% of the index. Main is unmeasurable because deleted IDs can be returned, invalidating survivor-only recall.

Vectors	Main	Tombstone only	Tombstone + repair	Fresh rebuild
20,000	Unmeasurable	0.9940	0.9960	0.9960
100,000	Unmeasurable	0.9680	0.9760	0.9820

Vectors	Tombstone-only median delete	Repair median delete
20,000	0.875 us	670.917 us
100,000	1.166 us	1,392.917 us

Measured on an Apple M2 Max, single-threaded, with the release profile. The raw machine-readable output is produced by bench_delete_recall.

Validation

• cargo fmt -p ruvector-diskann
• cargo clippy -p ruvector-diskann --all-targets -- -D warnings
• cargo test -p ruvector-diskann

WASM

The implementation uses only the Rust standard library, adds no dependencies, and remains compatible with the crate's WASM builds.

[ohdearquant]

CI note — the red checks on this PR are not caused by this diff:

• Clippy (deny warnings): pre-existing failure in crates/ruvector-rabitq/src/persist.rs:194 (useless_borrows_in_formatting on MAGIC, &magic), present on main — unrelated to this branch.
• dependency-review (PRs only): errors with "Dependency review is not supported on this repository. Please ensure that Dependency graph is enabled" — a repo Settings toggle, fails on every PR.
• Tests (core-and-rest): this did not fail an assertion — the job hit the 4-hour runner limit (wall-clock 4h0m18s) and was killed. The other test shards (core-and-rest-heavy, -wasm, ml-research-*, vector-index, etc.) all pass. Re-running the job clears it; I've kicked a re-run.

The DiskANN build jobs and remaining test shards for this change all pass.