diff --git a/ops/README.md b/ops/README.md index 0bf096b8..31ee26fd 100644 --- a/ops/README.md +++ b/ops/README.md @@ -106,7 +106,7 @@ lint is TS run directly under node): |---|---|---|---| | `public` | `packages/extension/skills` (repo checkout) | `--structural-only` | 20 | | `internal` | `~/.amico/vaults/armonissima/skills` | full (package cross-check) | 50 | -| `staging` | `~/.amico/server/opencode-project-staging/opencode-project/skills` | full (package cross-check) | 45 | +| `staging` | `~/.amico/server/opencode-project-staging/opencode-project/skills` | full (package cross-check) | 46 | Every real run appends ONE JSON line to the upgrade-receipts journal (`~/.amico/server/upgrade-receipts/upgrade-receipts.jsonl`, diff --git a/packages/extension/skills/altissimo/SKILL.md b/packages/extension/skills/altissimo/SKILL.md new file mode 100644 index 00000000..2b70ca2e --- /dev/null +++ b/packages/extension/skills/altissimo/SKILL.md @@ -0,0 +1,87 @@ +--- +name: altissimo +description: Altissimo.jl — the matrix-free NLP solver beneath Piccolissimo's spline problems. The ρ-schedule doctrine (default ladder, caps, the :demand policy and its measured limits), the churn signature, warm-start rules, and which knobs actually bind. Use when tuning, warm-starting, or diagnosing stalls and churn on Altissimo solves. +surface: entitled +entitlement: issimo +--- + +# Altissimo — solver tuning & diagnosis + +Altissimo is the augmented-Lagrangian NLP solver (trust-region + ceiling-bound +CG inner solves) that Piccolissimo's spline problems ride on the matrix-free +backend. It is CPU-native and array-generic. This skill is the measured +doctrine: every claim below is banked evidence from paired multi-seed +experiments — not theory. + +## The ρ schedule — the biggest single lever, and its honest limits + +Altissimo's penalty schedule ρ controls feasibility absorption AND CG +conditioning (κ ∝ ρ → CG iters ∝ √κ) — it is the largest performance knob in +the solver. The measured facts (10-seed paired batteries): + +| Recipe | Measured behavior | +|---|---| +| **Default ×10 ladder** | Works but churns: ρ = ρ_max by outer ~6, then the tail runs at max conditioning with feasibility parked ~1e-2 and the merit function churning in penalty terms. Descends anyway on most shapes. The safe default. | +| **Capped ρ (ρ_max ≲ 1e3)** | STARVES feasibility on every arm, cold and warm — inf_pr freezes at the x₀ violation level. Warm-start preservation under a cap does NOT transfer to cold absorption. If a cap is ever wanted, the (untested) value class is 1e4–1e5. | +| **`adaptive_ρ` + cap** | The measured WORST case: multipliers pinned at ‖λ‖ = 0 from x₀, total starvation. Never. | +| **`ρ_policy = :demand`** | Escalates ONLY on stale trailing inf_pr windows (feasibility progress, never dual norms); holds on progress rung-by-rung; `:default` is bit-identical. Oracle-measured 25–33% fewer HVPs on contrived carriers. **Validated MIXED on real problems**: the policy behaved as designed everywhere (strictly fewer escalation rungs, zero starvation), but the efficiency did not cash out in rollout fidelity on most real shapes at real budgets. Expect holds, not miracles; measure on YOUR shape. | + +Knob facts: `AltissimoOptions`' `ρ_max` default is 1e6 (the raw path's 1e8) — at +pinned options the era is floor-AND-ceiling. + +## The churn signature (know it when you see it) + +Warmup→AL-entry churn: early outers ride INFEASIBILITY (low J that is NOT a +solution), the ρ ladder then re-inflates J (0.15→75 unitary; 0.02→400 at K=4), +and the post-churn phase parks far from any convergence bar. It is a **policy +artifact** (cold duals + the default ladder), not a physics limit. Two +consequences: + +- **Never trust raw J in the churn window** — it literally rewards passing + through infeasible states. Score progress on **rollout-truth fidelity**, and + expect the early low-J readings to be lies. (Measured: an arm's raw J read 255 + while its rollout truth was 404.) +- **A plateau at smoke budget is budget, not physics** — verify at real budgets + on real problems before concluding anything. + +## Warm-starting Altissimo — the AL-entry laws + +- **Never carry mid-churn duals (λ0/μ0) without the matching ρ era.** Bare + dual handoff DIVERGES — measured f_val → −1.2e18; the λᵀc term dominates by + five orders of magnitude. The entry state must carry the PENALTY ERA, not just + the multipliers. +- **The entry state changes the PATH, not the PARKING.** All four measured entry + variants (cold zero-duals / x-only / duals-carried / duals+era-carried) park + at the SAME plateau. Warm-starting the optimizer entry is not a fidelity lever. +- **The de-facto resume is a cold restart** — resuming a solve from its own + iterate behaved identically to cold entry, everywhere measured. +- **State warm-starts ARE the lever**: seed knot states via + `set_state_guess!(qcp, states; respect_initial = true)` (Piccolissimo) — 9/10 + seeds better rollout fidelity at 2.5–26× fewer HVPs. **Never cap ρ under an + infeasible seed.** +- **λ0/μ0 API facts** (when you do pass duals): the kwargs take UNSCALED duals; + the result ALIASES optimizer buffers — copy before reusing. + +## What actually binds (stop turning dead knobs) + +- **Inner solves are TR-bound (79–100% of outers) and ceiling-bound; the + tolerance essentially never binds** (measured 0/707 `residual_met`). + Tolerance-side tricks and forcing schedules have no traction in this family. + The levers that matter: the ρ schedule and the preconditioner. +- **The assembled/probed Gauss–Newton preconditioner is DEAD on routed-Unitary + shapes** — measured 52–79× more matrix-vector products at 5× wall (a + regression, not a rescue). Do not reach for it there. +- **Dual resets are real and telemetered** — duals get eaten and reset on a + measurable fraction of long runs; expect resets in any long solve; don't + mistake a reset for a crash. + +## Comparing two configurations honestly + +- **Paired per-seed comparisons** — never compare across seeds; sign + consistency across the pair is the evidence. +- **Never conclude from the ordering of two capped-budget arms** — the end-value + ordering flips with runner numerics (measured: one draw flipped a 1.6× ratio + to 2.3×). Run each arm to its own convergence bar, or compare both at + multiple budgets. +- **Rollout truth is the only fidelity gate** — stored-terminal infidelity is + gameable through infeasible states. diff --git a/packages/extension/skills/piccolissimo/SKILL.md b/packages/extension/skills/piccolissimo/SKILL.md index 8b15d703..a2862fb9 100644 --- a/packages/extension/skills/piccolissimo/SKILL.md +++ b/packages/extension/skills/piccolissimo/SKILL.md @@ -1,6 +1,6 @@ --- name: piccolissimo -description: Piccolissimo.jl usage — the entitled fast-path tier for smooth-pulse problems. Spline-faithful integrators for Piccolo problems, adjoint robustness objectives, Gauss–Newton solver configuration, Magnus algorithm selection, and warm-start idioms. Use when authoring a solve.jl in a Piccolissimo-enabled environment. +description: Piccolissimo.jl usage — the entitled fast-path tier for smooth-pulse problems. Spline-faithful integrators for Piccolo problems, matrix-free density routing, full-channel (ECO) objectives, adjoint robustness objectives, Gauss–Newton solver configuration, Magnus algorithm selection, and warm-start idioms. Use when authoring a solve.jl in a Piccolissimo-enabled environment. surface: entitled entitlement: issimo --- @@ -59,6 +59,52 @@ and `use_ket_sensitivity` (ket-level sensitivity propagation). Some combinations error at construction rather than silently falling back — if a requested algorithm does not support the sensitivity mode you asked for, you will know immediately. +## Matrix-free density routing + +`DensityTrajectory` and `MultiDensityTrajectory` spline cells route through the +**matrix-free** backend — two-sided Lindbladian Duhamel sweeps, no dense Jacobian +assembled. Measured on the density state-transfer family: the routed solve reached +J = 0.66 in 35 s where the dense fallback spent 148 s reaching J = 152 (**4.2× +faster at far lower J**); routed MultiDensity J 302.62 → 0.20. Open-system +(Lindbladian) problems are therefore first-class — do NOT assume density means +"slow". + +- **Assert the routing, never assume it.** The problem knows whether it carries the + matrix-free kernels and carries zero dense knot blocks — assert both on the + constructed problem. A mis-routed cell silently pays the dense path. +- **Pulse-type coverage**: cubic, linear, and smooth spline cells route + (zero-order convenience constructors are linear-spline-class cells under the + hood — assert, don't assume); bang-bang constructs and routes too. +- **The bang-bang plateau is family-inherent**: piecewise-constant density + problems plateau far above the routed-cell J-ratio at ANY budget. That is + expressivity, not a routing defect — don't burn budget fighting it. + +## Open-system cells: two known upstream bugs (verify at authoring time) + +1. **The `LinearDissipator` rate field is inert on the density rollout path** + (Piccolo #337). The supported route is **√γ-prescaling** — put the rates on the + operators, pre-scaled — and *assert the planted dissipation as REALIZED* (e.g. + an analytic decay check $P_1(T) \approx e^{-\Gamma_1 T}$), because the naive + constructor silently produces a unitary rollout. +2. **The `DensityTrajectory` conversion discards the `CubicSplinePulse` endpoint + pins** (Piccolo #338) — MultiKet and MultiDensity pin them; density didn't. + Restore the hardware-readiness pins (zero value AND zero derivative at both + endpoints) via stock `EqualityConstraints` on the problem, or the solved pulse + loses its clean turn-on/off silently. + +## Full-channel optimization (ECO) + +`ChannelProcessInfidelityObjective` optimizes the **realized channel** of a cycle +on the `MultiDensity` substrate — the K-basis dual-basis process fidelity +$F_{\text{pro}} = \frac{1}{d^2}\sum_k \langle \mathcal{U}(\tau_k), \Lambda(\rho_k)\rangle_{HS}$. +Two authoring facts: it requires the matrix-free routing above, and the gradient +oracle is rollout-FD (AD through the Duhamel cells is not dual-admissible). Two +hard traps: the objective is wrapped in the $|1-F_{\text{pro}}|$ envelope (the bare +form free-falls on infeasible iterates), and $F_{\text{pro}} \ne F_{\text{avg}}$ — +they differ by $(1-F_{\text{pro}})/(d+1)$; never compare one against a +Pedersen-average bar. Team members: the full dual-basis math and the six measured +traps live in the internal `objectives` skill. + ## Robustness objectives Two public objectives make a pulse robust to parameter error, and one wrapper @@ -130,7 +176,9 @@ Magnus algorithms when stiffness, long gates, or large-$\|H\|$ regimes actually demand them (e.g. `MagnusGL4Alg(n_steps ≈ 50)` for a deep Rydberg blockade, where the default under-resolves and optimizer fidelity diverges from a fine re-rollout). For simple bilinear single-qubit gates the plain path usually converges faster -than the Magnus variants. +than the Magnus variants — measured, MagnusGL4 matched the standard path on the +transmon X gate only with added complexity, and a bare low-drive run lost two +nines. Reserve Magnus for the regimes that need it. For manual substep sizing of the fixed-step cells, two exported diagnostics do the arithmetic for you: `suggest_n_sub(H_drift, H_drives, bracket, coeff!, Ψ0, Δt; @@ -161,3 +209,27 @@ solve!(qcp; max_iter = 60) knot data when you have it. - `load_pulse` returns only the pulse object; bundle metadata (fidelity, gate name) with `JLD2.jldsave(...; pulse = ..., fidelity = ...)` at save time. + +**State warm-starts beat control-only**: seed the knot STATES on the Lie geodesic +$\exp(s_k \log U_{goal})$ with arbitrary controls via +`set_state_guess!(qcp, states; respect_initial = true)` — knot-1 is checked +loudly, du/s_du re-derived, free-phase θ transferred. Measured (10-seed paired): +better rollout fidelity on 9/10 seeds at **2.5–26× fewer inner-solve HVPs** vs +cold. Two hard rules from the same measurements: + +- **Never cap the penalty (ρ_max ≲ 1e3) under an infeasible seed** — feasibility + starves at the seed's violation level; `adaptive_ρ` + a cap is the measured + worst case. The default ladder is the safe recipe (see the `altissimo` skill + for the full ρ-schedule doctrine). +- **Gate on rollout truth only** — the stored-terminal infidelity is gameable + through infeasible states (measured: stored-E ≈ 1e-3 while rollout-E = 0.667). + +## Honesty rails + +- **The smoke-budget attractor**: a fidelity plateau measured at a tiny smoke + budget is a BUDGET artifact, not physics. Near-1 is routine at real budgets + for these model families — never conclude a fidelity ceiling without + real-budget evidence. +- **Real-problem surface**: performance and fidelity claims validate on real + platform problems at real budgets; standardized contrived fixtures are for + mechanism isolation only.