Feature/calibration driver - #35
Open
Tinitto wants to merge 10 commits into
Open
Conversation
Reviews the draft against qpi-driver, qpi-ui and the SDKs, and fixes what did not survive contact with them: - Dispatch: the RFC pushed CalibrateDispatch onto "the driver's NNG PUSH socket". That socket is a local in runDriverDispatcher and is fed only by scheduler.FetchNextJob; no HTTP handler can reach it. Replaced with a calibration_requests queue collection the dispatcher polls, which inherits the job path's durability and requeue-on-send-failure. - Persistence: handleCryostatReading appends to the shared `events` log and creates no per-type collection, so calibration_results was a departure from the pattern it cited, not an instance of it. Deferred to the events log for v1; recorded the model/migration/config/rules work for when it is earned. - Operation blast radius: an operation is closed across all three SDKs and the server (RFC 0003 §13.1), with tests asserting it. Listed every required change, including AllEventTypes, which driver registration validates against. - Citations: 1109.6887 is PRA 2012, not PRL 2011 (that is 1009.3639); rows 15 and 16 both pointed at the interleaved-RB paper. - Resolved the pyproject contradiction, dropped the test-py-calibrate target (test-py runs one env per extra; tier-1 needs no scheduler), and renamed monitor_interval so a calibrate option stops naming another operation. Adds the Decisions rationale and rejected alternatives, a Security section (the device YAML is a trust boundary: a tuner writes what every job then reads), and an implementation-plan pointer, per the conventions in docs/rfcs/README.md.
Takes the reviewed RFC 0004 over the branch's original copy; the former is the latter plus the corrections in 9c92ac2.
Replaces the scaffold on feature/calibration-driver with working routines, per RFC 0004. What was there reported success in 22 microseconds having run nothing, and destroyed the device config doing it. Routines - One backend-agnostic routine set replaces 32 per-scheduler stub files. Both schedulers expose the same gate vocabulary, so a routine composed against a SchedulerBackend runs under either; two copies would be two things to keep in step. tuners/base/device.py absorbs the shapes that do differ — qcodes callables against pydantic fields, and motzoi against beta. - All 16 routines now build real swept schedules and compile against the dummy cluster. Previously Rabi was a single fixed Rxy(90) with no sweep, T1 was a bare Measure, apply() was `pass` everywhere, and the qblox tuner passed None where a compiler was expected. - Fine amplitude gained the pi/2 pre-rotation and two calibration points. Without the former the sign of the error is unrecoverable at integer n; without the latter only the product of contrast and error is. Correctness of failure - A missing or empty calibration.yml no longer means "calibrate nothing and report success": absent routines default to enabled, an unknown routine name is a startup error, and a run with nothing to do reports failed. - Fits raise instead of returning zeros, and every fitted device parameter is range-checked against the sweep that produced it. A fit that returns zeros on failure gets written to the device as though it were a measurement. - The Clifford group is generated by exact matrix composition and closed under inversion, replacing a table with 8 of 24 entries and a "dummy inverse". RB measures nothing if the recovery gate does not actually invert the sequence. Write-back (RFC 0004 §10) - Writes the schema load_quantum_device reads, via safe_dump, verified to read back before anything is replaced, with the previous file kept beside it. It was writing a compilation config through yaml.dump, producing python tags the loader's safe_load refuses — one calibration bricked the QPU. - A failed run no longer writes at all. Driver - CalibrationResult is emitted flat, matching CalibrationResultPayload. Nested under "results" it unmarshalled to an all-zero record without erroring. - check_fidelity returns a report, so the drift path no longer dies on AttributeError every tick; the thresholds are now actually compared and a drop queues a partial recalibration. - recalibrate() honours its qubits argument rather than running the full DAG. - A second calibration is refused rather than queued behind a multi-hour run. Also: discovery resolves a Tuner named by import path, and the shipped example config names routines that exist. Deletes generate_quantify_tuner.py, a codegen script with a hardcoded home directory that shipped inside the package. Tests: 359 passing, 99% coverage on the gated modules against a 96% floor.
…tion The operation existed in the Python SDK and the server but nowhere else, and nothing could trigger a calibration: CalibrateDispatchPayload was declared and never constructed, with no route, no queue and no dispatcher change. Dispatch (RFC 0004 §6.8) - A `calibration_requests` collection the driver's dispatcher polls, rather than a socket write from an HTTP handler — the PUSH socket is a local inside runDriverDispatcher and nothing else can reach it. Queueing through the database is also what makes a request survive a restart, requeue on a send error as a job does, and run when an offline driver reconnects. - POST /api/op/calibrate/dispatch, admin-only like the /api/op/* routes around it: a calibration takes a QPU out of service for hours and rewrites the parameters every later job runs against (RFC 0004 §10). - Calibrations are offered before jobs, or a busy QPU would never calibrate — the state calibration exists to get it out of. One at a time per driver. Result handling - handleCalibrationResult validates mode and status, as handleCryostatReading validates its readings, and records the QPU as well as the driver. A blank payload was previously stored as a blank row. - The report closes out the queued request it answers, so the dispatcher can offer the next one. Cross-language parity (RFC 0003 §8, RFC 0004 §6.1) - `calibrate` added to the Go and TypeScript SDK operation enums and event types. Their closed-set tests failed on the addition, which is what those tests are for; all three now assert the same three operations. Also: the catalog's drift option is `drift_check_interval`, since `monitor` is already an operation; a new test pins the catalog's -o keys to the ones the Python builder reads, because an option nothing reads is a startup error and the two lists are kept in step by hand (RFC 0003 §9). Drops the vestigial `calibration_data` field from the dashboard's QPU type — no collection has it.
Records the two design changes the implementation forced, with reasons: - One shared routine set rather than one per scheduler (§6.2). Both expose the same gate vocabulary, so the seams that carry the real differences are a SchedulerBackend and a device-parameter shim, not sixteen duplicated files. - calibration_results is a collection after all (§6.8), reversing the earlier deferral to the events log. The dashboard history list is a query over months rather than a tail of recent events, and a calibration report's retention is different in kind from a cryostat reading's. Also rewrites §6.3/§6.4 to the interfaces that exist (check_fidelity returns a report; routines declare "updates" and "benchmark", and analyse raises rather than returning zeros), marks tier 3 as not implemented, and adds §11 saying plainly what is built and what is not — above all that no routine here has been run against physical hardware. Adds the tuners doc page the mkdocs nav already pointed at, and drops the test-py-calibrate target, which ran an import check and was not in test-py. The calibration tests live in the existing per-extra targets: tier 1 needs no scheduler, tier 2 needs the one it tests.
Admin-only tab showing what the chip currently looks like and letting an operator act on it: - Measured fidelity per target, each against the threshold that governs it — two-qubit for an edge, one-qubit for a qubit. Holding a CZ to the 1Q threshold would make the whole grid cry wolf. Worst protocol wins per target, as the driver's own drift check does. - Current parameters, grouped by qubit rather than by routine, because "what does q0 look like now" is the question an operator actually has. Values are formatted by kind: GHz for frequencies, µs/ns for times. - History, with per-run status, duration and error counts. - A trigger modal defaulting to the drift check rather than the full run: it is the common case, it is cheap, and it changes nothing, whereas a mis-clicked full calibration costs hours of QPU time. A partial run with no targets is refused in the form as well as at the endpoint, since the driver would otherwise widen it to a full one. - An in-flight banner driven by calibration_requests. A calibration takes hours, so the queued request is the only thing that can say "in progress" during the gap before a report exists. Registration now offers the two tuner kinds, and the sidebar and hash router know the tab — the E2E spec caught the missing hash allow-list entry, which is what it is there for. make test-e2e-dashboard gains SPEC=<glob> to run one spec: eight seconds against four minutes when iterating on a single tab. E2E: 116 passing, including 6 new.
The page is symlinked into docs/driver/, so relative links resolve from there rather than from the package — mkdocs strict mode caught both. Uses absolute GitHub URLs, as the other double-duty READMEs do. Also rewords the RFC's note about there being no dedicated calibration test target, which check_docs.py read as a claim that one exists.
The merged docs described an endpoint and an access rule that do not exist, and gave two CLI examples that would fail on startup. Corrected against the code: - `POST /api/calibrate` is `POST /api/op/calibrate/dispatch`. - `calibration_results` is authenticated-read, not public read, and `calibration_requests` was missing from the collections table entirely. - Both tuner examples omitted `calibration_config`, which a tuner cannot start without — a calibration with nothing to run would report success having measured nothing, so its absence is deliberately a startup error. Adds the usage that was missing: how to queue a calibration from the server and what the three modes cost, how to turn on drift monitoring, a Tuner Backends section in the Python SDK README, and a calibration section in the operations runbook covering the queue, the `.prev` backup as the way back from a bad calibration, and why `eventsRetention` does not touch calibration reports. The changelog entry now covers what actually shipped — the dispatch endpoint and its queue, drift monitoring, the dashboard tab, the SDK operation enums and the write-back guarantees — and states plainly that none of it has been run against physical hardware. make test-docs passes: 137 claims checked, site builds under strict mode.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements RFC 0004: Calibration Tuners, bringing automated parameter calibration and drift tracking for superconducting transmon qubits to QPI.
It introduces the
calibrateoperation across the full stack—from Python SDK calibration tuners (quantify_tuner,qblox_tuner) and backend-agnostic experiment routines to server-side event handling, atomic device file write-backs, and an admin-facing dashboard UI for QPU calibration status and manual triggers.What's Changed
1. Operation & Event Infrastructure
calibrateas a first-class operation alongsideprocessandmonitor.CalibrateDispatchandCalibrationResultevent types.POST /api/op/calibrate/dispatchendpoint inqpi-uifor triggering full, partial, or fidelity-check calibrations.calibration_resultsandcalibration_requestsserver storage collections.2. Python SDK & Tuners Framework (
qpi-driver/py)CalibrateDriver: Worker subprocess architecture handling calibration runs without blocking the main event loop.Tuner&CalibrationDAG: Topological DAG execution of routines configured viacalibration.yml.quantify-schedulerandqblox-schedulervia a unifiedSchedulerBackendinterface.quantify.device.ymlwith automatic backup (.prev) and verification before swapping files. Fits outside expected parameter bounds or failed acquisitions abort writeback to prevent corrupting hardware state.drift_check_interval) triggering targeted partial recalibrations when gate fidelity drops below configured thresholds.3. Admin Dashboard UI (
qpi-ui)calibration_requests.4. Testing & Verification
test_fitting.py): Validates parameter recovery from noisy synthetic data and verifies error-handling when fits fail.test_tuner_routines.py): Verifies schedule compilation for all routines against Quantify and Qblox dummy clusters.make test-docsclaims pass strict MkDocs verification.Documentation
qpi-driver/py/qpi_driver/tuners/README.mdand linked it in MkDocs navigation (docs/driver/tuners.md).qpi-driver/py/calibration.example.yml.calibrateCLI examples, queue mechanics, and rollback procedures.How to Test