Skip to content

Combine race-safe SQL namespaces with current Rustice runtime - #143

Merged
osipovartem merged 5 commits into
rustice/footer-hit-pre-touchfrom
fix/create-schema-runtime-integration
Oct 31, 2026
Merged

osipovartem merged 5 commits into
rustice/footer-hit-pre-touchfrom
fix/create-schema-runtime-integration

Conversation

@osipovartem

@osipovartem osipovartem commented Oct 31, 2026 •

Copy link
Copy Markdown

Summary

Combine the race-safe SQL namespace implementation from #136 with the actual
Rustice runtime branch, based on 5d7b6f6d (rustice/footer-hit-pre-touch).
Do not replace the newer runtime with the older SQL-only lineage.

  • Preserve namespace worker locks, footer pre-touch, pruning, write cleanup and
    the other runtime fixes. The Catalog API and runtime dependency versions are unchanged.
  • Store namespace identities separately from properties and classify only the
    namespace identity uniqueness violation as a typed commit conflict.
  • Add the SQL catalog's futures-timer dependency edge for bounded deadlock
    retry; do not upgrade package versions or replace the runtime lineage.
  • Force concurrent CREATE past both prechecks through independent SQLite,
    PostgreSQL and MySQL pools. Cover empty/empty, empty/nonempty and disjoint
    properties; require one success, one typed conflict, one identity row and
    winner-only properties visible through both catalogs.
  • Replace the legacy DROP test's timing heuristic with a prewarmed,
    cfg(test)-only pause after the missing-identity probe. Commit promotion and
    removal of the last legacy property before resuming DROP's locking read.
  • Stop older writable catalog instances during rollout. Legacy property-only
    namespaces remain readable, but mixed-version writes and downgrade are unsafe;
    initialization requires schema DDL privileges.

The latest test instrumentation adds no production SQL, scan-path work, runtime
configuration or public API. It does not solve remote completion-unknown recovery,
provide a PostgreSQL-backed Rustice daemon route, or qualify the benchmark metastore.

Local Qualification: 2026-10-31

Rust 1.99.0, offline/locked, opt-level-1 CI test profile, one shared target and
serial test execution. PostgreSQL 11-alpine, MySQL 8.1 and LocalStack 4.5 run as
local testcontainers; no Snowflake or cloud account is used.
Final strengthened test revision: 9fd801043972e4b03a4a3fd32f3704122c41b82b.

Check Result
Exact original combined pin f59168c0 24 passed, zero failures/ignored; includes all three formerly unqualified Docker tests
Strengthened complete SQL catalog suite 26 passed, zero failures/ignored
Negative control: remove namespace identity PK All three CREATE race tests fail with two successful CREATEs, not a timeout
Negative control: remove post-property-miss identity recheck Both PostgreSQL/MySQL legacy DROP tests fail because the namespace remains after DROP
Restore source after both controls Byte-for-byte restoration; SHA-256 6ad9e60e0db478f9f8d820fce9b80b957d73f0ede838efb717f9267dbc538942; complete 26-test suite passes again

A separate read-only reviewer examined correctness, regressions, performance,
API compatibility and test strength. Both earlier test-coverage findings are
addressed; the reviewer also checks the mutant failures and exact restoration.
Final independent review approved this exact fork revision after the downstream
qualification below. That approval does not make Rustice #954 merge-ready.

Downstream Qualification: 2026-10-31 UTC

All six Iceberg dependencies in Rustice draft #954 select 9fd80104. The
frozen combined candidate 59efc9d7,
based on main f2c4bf32, includes both dependent leaf stacks, not just #954.

Check Result
Six-package unit suites 3308 passed: catalog 303, executor 1459, functions 1220, REST 305, auth 7, daemon 14; zero failures
Strict Bronze 4295/4295 eligible records across 545/545 files; unchanged baseline, zero regressions or changed IDs
Stock connector 4.8.0 JSON/Arrow wire 604/604 assertions; separate comparison preserves all 580 required named checks, without baseline update
Catalog restart 14 daemon starts, typed rows/metadata and namespace lifecycle/isolation pass in both formats
PUBLIC restart 6 daemon starts; absence after DROP, explicit recreate/drop, NULL context and typed surviving rows pass
Cleared-database acceptance 4 daemon starts, 145/145 checks per format; exact DML readback, SEARCH_PATH, view binding and metadata survive restart

The same frozen default-feature daemon served all acceptance cases. Its
SHA-256 is b25e6aae72f22c7e2b6208033419e982a897e5e0dccc102004e59c2f63c59d57.
It is built separately from the test-helper-enabled SQL harness, using Rust
1.99.0 and the opt-level-1 CI profile, not a release performance build.

The final wire qualification adapter verified actual listener PID, endpoint,
executable path and SHA for all seven smoke starts, and atomically persisted
receipts before SQL. Five controls pass normally and under Python -O, including
an unrelated healthy listener, interrupted startup and receipt-write failure.
The exact adapter and final receipts received separate read-only approval.
Earlier smoke observations are retained but are not the primary qualification;
the original upstream smoke source was not changed by this local adapter.

No expectations or baseline were changed by the repin/qualification. The older
CREATE SCHEMA feature in the combined Rustice stack intentionally corrects one
fixture's post-DROP current schema to PUBLIC, as live-calibrated in Rustice #953.

Scope Limits

Keep Rustice #954's remote in-doubt recovery, broader backend-lifetime and
release performance gates explicit. No hosted CI, new full live Snowflake audit,
SPCS deployment, PostgreSQL daemon support or benchmark qualification is claimed.

Force two independent SQLite pools past missing-namespace prechecks before identity insertion. Cover empty, mixed empty/nonempty, and disjoint properties with exactly one success and one typed conflict; verify identity count and winner-only properties from both catalogs.

Document the requirement to stop older writable catalog versions during rollout. The synchronization hook is test-only; all runtime blobs outside the SQL catalog remain unchanged from 5d7b6f6.
@osipovartem
osipovartem marked this pull request as ready for review October 31, 2026 21:10
@osipovartem
osipovartem merged commit 75d36f9 into rustice/footer-hit-pre-touch Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant