Solver options: fallback observability, tolerance ownership, the single-field FMG unlock, and multiplier_schur_pc verified (#478 #483 #484 #486) - #534
Conversation
…rds what degraded, where, and why (#484) Ten of the twelve places a solver silently degrades its preconditioner (the single-field FMG gate, missing hierarchy, transfer-build failures, guard skips, forced Galerkin, the rotated-path substitutions) left no queryable state — a warning at best, nothing at worst — so neither a user nor a test could ask "did I get what I asked for?". Every site now writes through one recorder (_record_pc_fallback) into a public read-only property, solver.pc_fallbacks, with a fixed reason vocabulary: unavailable / declined / build_failed / check_skipped / forced. The record is cleared whenever the preconditioner options re-resolve (the same staleness rule as _pc_resolved), and solve-time sites (custom_mg, rotated_bc) re-record each solve. Warnings are unchanged where they exist; the silent "auto" declines gain a record but deliberately NO new warning. rotated_bc mirrors its two degraded arms into the same record; its own ctx keys stay authoritative. Behaviourally inert: no option value, route, or default changes. tests/test_1022_pc_fallback_observability.py: every probe proven to fire and proven silent on a clean solve (empty record is the global negative control). Underworld development team with AI support from Claude Code
…erive tolerance margins through one documented mechanism (#483) The Stokes solve re-pushed snes_rtol and ksp_atol before every solve (_reassert_outer_tolerances), silently discarding a value the user set explicitly — the worst reachability middle ground: documented as settable, actually owned. The re-push now runs through the same recorded-ownership latch that made snes_max_it reachable (ruling D18), generalised to a dict (_resolve_owned_option/_push_owned_option, with _resolve_snes_max_it kept as a named delegate): the framework keeps asserting the tolerance-derived values until the user sets a key, after which their value is honoured across solves. A user who never touches the keys sees byte-identical behaviour (proved by the no-override arms). Second half: Stokes and Stokes_Constrained derived different option keys from `tolerance` through two unrelated hand-rolled setters. Both now apply a per-class table (_TOLERANCE_DERIVED_KEYS, the base table keeping its historical _INNER_RTOL_MARGIN name as an alias) through one base method (_derive_tolerance_margins), at SET time only. The difference in table contents is deliberate and now documented: the base class derives the inner fieldsplit margins; Constrained derives the outer ksp_rtol and the Eisenstat-Walker pins, because EW pinning owns its outer accuracy. Constrained.__init__'s duplicate EW writes route through the same helper. Both `tolerance` docstrings now state the ownership table (OWNED vs DERIVED-at-set-time keys, and the EW caveat on ksp_rtol). tests/test_1023_saddle_tolerance_ownership.py reads the LIVE PETSc objects after two solves (the second solve is the one that used to clobber), with no-override negative controls on both classes. Stakeholder tests that write these keys (0820, 1013, 1014x2) pass. Underworld development team with AI support from Claude Code
…tion-free custom-P transfers (#478) preconditioner="fmg" on a scalar/vector solver was declined to GAMG with a warning, because the NATIVE geometric-FMG path needs DMCreateInjection, which PETSc cannot reliably build for a single-field discretisation on a refined DMPlex (err62, #276). That gate locked geometric MG out of every scalar/vector solver even though the robust custom-P route (own prolongations + Galerkin RAP, no injection anywhere) was already in utilities.custom_mg. The explicit request is now honoured: the gate flags the reroute (_pc_single_field_geo_requested) and custom_mg.build_transfers grows a THIRD hierarchy source — requested-native, wrapping the mesh's own dm_hierarchy tail in _DMLevelView adapters — resolved after solver-set and mesh-owned so build_transfers stays the single "which hierarchy" owner (#471, composing with #530's explicit-gamg opt-out, which is untouched). Installation rides the existing auto_inject_custom_mg solve hooks and the shared multigrid_options bundle (#468/#515), so the options DB deliberately keeps GAMG as the safe degrade base until the live PC is configured at first solve. Build failure degrades to that base through the recorded barycentric -> RBF -> default ladder (#484). Contract line, stated in both docstrings: preconditioner="fmg" is a PREFERENCE (degrades readably); set_custom_fmg is a DEMAND (raises). "auto" is deliberately unchanged — flipping the single-field default is its own validation campaign per #478 — it keeps GAMG and records the decline as the migration probe. Found en route: a gmsh-imported BASE hierarchy level carries section-only coordinates (PetscContainer, no PetscFE), and DMCreateInterpolation from such a source silently returns a ZERO matrix; _DMLevelView therefore works on a clone and installs a P1 Lagrange coordinate FE when missing. tests/test_1020 becomes the unlock test (live PC "mg" over every hierarchy level on the exact #276 err62 geometries; degrade arm proven by monkeypatched builders; auto-unchanged arm; Stokes velocity block untouched; DB-vs-live honesty arm) and tests/parallel/ptest_1020_fmg_single_field_parallel.py proves route parity at np=2. Underworld development team with AI support from Claude Code
…nert, correct the drifted docstrings (#486) The issue measured four identical convergence rows across two decades of viscosity contrast and asked "verify or remove". Tracing PETSc's fieldsplit.c settles it without ambiguity: the flag swaps only the Pmat (h,h) block, and under Stokes_Constrained's own defaults (selfp + diag_use_amat) selfp assembles the Schur preconditioner from AMAT sub-blocks — the swapped block is provably never read, which is exactly what the issue measured. The flag IS read under pc_fieldsplit_schur_precondition='a11' (Sp = the grouped [p,h] Pmat block) and under a monolithic direct factorisation of the Pmat. Verdict: keep + instrument, not remove. Instrumentation: at the hh_pc selection in _setup_solver, an explicit opt-in that cannot reach the PC (not a11, diag_use_amat set, not a direct solve) records reason='declined' in pc_fallbacks AND warns — an explicit opt-in silently doing nothing is exactly the #477 class. Docstrings: multiplier_schur_pc now states its two live regimes and the inert one; the Constrained saddle_preconditioner claim that selfp uses "the 1/viscosity mass from constitutive_model.K" was drifted (selfp reads the Amat; the 1/mu mass participates only under a11) — corrected, with a TODO(BUG) on whether selfp should see Pmat blocks at all (a design question, deliberately not changed here). tests/test_1024_multiplier_schur_pc.py: matrix-level oracle (assembled Schur pre differs flag-on/off under a11 — immune to "both converge in 4 iterations"), the inertness codified as an exact-equality control pinning the PETSc semantics, regime-aware probe silence, no-flag silence, and the setter re-registration contract. Underworld development team with AI support from Claude Code
There was a problem hiding this comment.
Pull request overview
This PR improves the solver-options layer’s observability and reachability for PETSc preconditioner fallbacks and tolerance-related options, and unlocks an opt-in geometric MG route for single-field solvers by reusing the existing custom-P transfer machinery. It also verifies and instruments multiplier_schur_pc so that “opt-in but inert” regimes become detectable (and testable) instead of silently doing nothing.
Changes:
- Add a solver-wide, queryable
pc_fallbacksrecord and thread all major fallback/guard-skip sites through a single recorder. - Fix tolerance “ownership” so user-set
snes_rtol/ksp_atolsurvive repeated solves while still preserving framework-owned defaults when untouched. - Honor explicit
preconditioner="fmg"on single-field solvers by routing to custom-P geometric MG overmesh.dm_hierarchy(with readable degrade paths), and instrumentmultiplier_schur_pcreachability.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
src/underworld3/cython/petsc_generic_snes_solvers.pyx |
Introduces pc_fallbacks recording, single-field explicit-FMG reroute flagging, and owned-option latching for tolerances / max-it. |
src/underworld3/utilities/custom_mg.py |
Adds requested-native hierarchy support via _DMLevelView and records build/guard fallbacks for custom MG transfer installation. |
src/underworld3/utilities/rotated_bc.py |
Mirrors rotated-path substitutions into the solver-wide pc_fallbacks record. |
src/underworld3/systems/solvers.py |
Refactors constrained tolerance derivation through shared derived-key tables and corrects Schur-preconditioning docstring semantics. |
tests/test_1022_pc_fallback_observability.py |
New coverage ensuring all fallback sites produce readable records and stay silent on healthy paths. |
tests/test_1023_saddle_tolerance_ownership.py |
New regression coverage for tolerance ownership/override reachability across repeated solves. |
tests/test_1020_fmg_single_field_lockout.py |
Reworks the prior lockout regression into an explicit-FMG unlock test and degrade-path negative controls. |
tests/parallel/ptest_1020_fmg_single_field_parallel.py |
Adds MPI parity coverage for the single-field explicit-FMG unlock route. |
tests/test_1024_multiplier_schur_pc.py |
Adds matrix-level oracle tests for multiplier_schur_pc live vs inert regimes and fallback/warn instrumentation. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| silent swallow; it now records check_skipped. Driven directly: a solver | ||
| with no SNES yet is exactly the unreadable-operator state.""" | ||
| p = _poisson_on(_box(refinement=0), name="Tguard") | ||
| assert not hasattr(p, "snes") or p.snes is None or True # un-built solver |
Adversarial review — PR #534 (solver-options bundle: #478 #483 #484 #486)Reviewed independently at MERGE-BLOCKERSB1. After the fmg unlock installs, a later explicit B2. The #486 inertness probe reads the presence of Both fixes are localized, neither touches a default path; everything else we Verified claims (measured)
Minor findings (non-blocking)
Run log
|
…it choice; the inertness probe reads the flag's value Two defects from the #534 adversarial review, both violations of the bundle's own observability contract. (1) The explicit-fmg install cached its hierarchy in solver._custom_mg, which auto_inject re-installs unconditionally — so a later preconditioner="gamg" was unreachable with an EMPTY pc_fallbacks record. The cache is now marked auto_cached and the preconditioner setter drops it; a set_custom_fmg registration is a demand and is kept. Regression test with a verified negative control (fails with the marker disabled). (2) The multiplier_schur_pc inertness probe read hasName("pc_fieldsplit_diag_use_amat") — the key's presence — so diag_use_amat=false (the flag LIVE, Schur pre measured differing by rel-Frobenius 0.30) still warned "no effect" and recorded a decline. It now reads the bool value; regression test pins the flag=false silence. Underworld development team with AI support from Claude Code
|
Response commit 45acc08: both blockers fixed — (1) the fmg-install hierarchy cache is now marked Underworld development team with AI support from Claude Code |
adapt-on-top-faults: the rotated free-slip path now picks up an adapt child's mesh-owned MG tail automatically (custom_mg.build_transfers, the #467 fix) — the "FUNDAMENTAL, not a quick fix" gotcha row was describing a bug that is gone. Point the band-sizing section at the interface-alignment primitives that now exist (place_sheet / place_thin_volume / remove_embedded, #517-#526). nonlinear-solver: three stale claims corrected — rotated free-slip is no longer the exception to the automatic tail pickup (#467); the adapt tail is one MG level per DOUBLING of h, not per refinement generation (mg_coarsening_ratio=2.0, #515); refinement=0 still yields a hierarchy that starts at the base, so "no coarse grid" overstated it. Three capabilities that landed since the branch: preconditioner="gamg" is respected on adapt children (#530), single-field FMG (#478/#534), and solver.pc_fallbacks as the observability hook (#534). The supersession note no longer names the removed in-SNES ramp API. Underworld development team with AI support from Claude Code
…current; plasticity-solvers rewritten (#454) (#489) * skills: parallel adapt engines, band pinning, and FMG on adapt children adapt-on-top-faults - engines section: nvb vs edge_split, both parallel in 2-D and 3-D and bit-confluent; edge_split has no conforming closure so refinement cannot escape the marked region, and marks on the DIAMETER (the volume proxy reported the target met while the mesh was 3.2x coarser across the feature). - repair=True: gates on reducing the largest angle, NOT on Delaunay. Delaunay maximises the minimum angle while P1 depends on the maximum, and flipping a gmsh mesh toward Delaunay raised the 99th-percentile max angle. Worth it on a poor base (156 -> 115 degrees, slivers 3.84% -> 0.00%), marginal on a clean one, and it gives up bit-confluence, so it is opt-in. - relax on a mesh refined onto an interface makes things WORSE (+77% leak); pin_bands is the fix. - new section on sizing the band and representing the fault margin: the -2 Cov(eta, edot) leak metric, why a within-cell marking rule loses to the plain distance size field, why the optimal band width depends on which objective you pick, and what a step-edged margin buys and costs. - gotchas: Mesh(dm) takes the DM over (bare SIGSEGV if you keep using the old handle); Mesh(dm) without boundaries= loses the boundary enum; evaluate() "Total components 8 != 6" on a variable-heavy mesh. adaptive-meshing - PIN THE INTERFACE section for relax(pin_bands=...), including the signed-vs-unsigned distance rule and the pinned_labels merge trap. - cross-reference to nonlinear-solver for the FMG setup. nonlinear-solver - new section: FMG on an adapt-on-top child. The child carries its own graded custom-P tail and solvers pick it up automatically; the base must have refinement>=1; a base-only tail triples the V-cycle count; V-cycle counts are insensitive to element quality (a pass, not a failed measurement) so use GAMG as the quality probe; relax can trip #424 into the dense RBF fallback; repair invalidates the any-degree transfer but not the vertex prolongation. - cross-references to adapt-on-top-faults and adaptive-meshing. Underworld development team with AI support from Claude Code * skills: bring the two adapt/solver skills current with development adapt-on-top-faults: the rotated free-slip path now picks up an adapt child's mesh-owned MG tail automatically (custom_mg.build_transfers, the #467 fix) — the "FUNDAMENTAL, not a quick fix" gotcha row was describing a bug that is gone. Point the band-sizing section at the interface-alignment primitives that now exist (place_sheet / place_thin_volume / remove_embedded, #517-#526). nonlinear-solver: three stale claims corrected — rotated free-slip is no longer the exception to the automatic tail pickup (#467); the adapt tail is one MG level per DOUBLING of h, not per refinement generation (mg_coarsening_ratio=2.0, #515); refinement=0 still yields a hierarchy that starts at the base, so "no coarse grid" overstated it. Three capabilities that landed since the branch: preconditioner="gamg" is respected on adapt children (#530), single-field FMG (#478/#534), and solver.pc_fallbacks as the observability hook (#534). The supersession note no longer names the removed in-SNES ramp API. Underworld development team with AI support from Claude Code * skills: rewrite plasticity-solvers around what the yield campaigns measured Fixes #454. The skill taught the retired yield-homotopy doctrine in ~8 places, headlined by an enable method that no longer exists in src/ and an in-SNES delta-ramp that is separately proven to diverge. The doctrine rested on a unit-scaling error: re-measured on the corrected problem, the delta-march never succeeded where a direct hard-Min solve failed, and the ruling was to regularise the problem, not the solver. Rewritten around the evidence: Newton with the automatic Picard entry (Picard is an entry requirement, not an accelerator), rescue on failure OR stagnation, grid sequencing as the validated warm start, the per-model tangent table, and the #475 yield_mode / yield_smoother / yield_anchor substrate presented as a modelling choice with the multi-solve-only discipline for any delta march. yield_continuation is described honestly per open issue #473 (the cold-start guarantee does not hold on a Piecewise yield stress; the step control is effectively one-shot). Floors updated to the post-#475 semantics (viscosity_min_rounding). Kept: the Newton-confirmation check, the VEP-indefinite ruling, the SNESFAS ruling, the Picard-folklore footnote. Underworld development team with AI support from Claude Code * The nonlinear-solver recipe demotes the delta-march to a rescue The recipe's step 2 still presented solve(homotopy=True) as the default entry point ("one call - automatic"), which is the retracted doctrine in its post-API form: the evidence that recommended a delta-march first rested on a unit-scaling error, and the driver's cold-start guarantee is broken (issue #473). The step now escalates honestly - grid sequencing first, the delta-continuation as rescue of last resort with the #473 caveat stated - and the Layer-2 status note carries the same demotion. The delta-discipline itself (constant per solve, never in-SNES) is unchanged; plasticity-solvers holds the ruling and evidence. Underworld development team with AI support from Claude Code
Solver options: make fallbacks readable, tolerances honest, explicit geometric MG reachable, and multiplier_schur_pc truthful
Fixes #478 #483 #484 #486
Four related reachability/observability defects in the solver-options layer, landed as one commit per issue (recorder first — everything else writes into it).
#484 — every preconditioner fallback leaves a readable record. Ten of the twelve places a solver silently degrades its preconditioner (single-field FMG gate, missing hierarchy, transfer-build failures, guard skips, forced Galerkin, the rotated-path substitutions) left no queryable state — a warning at best, nothing at worst — so neither a user nor a test could ask "did I get what I asked for?". Every site now writes through one recorder into the public read-only
solver.pc_fallbacks(keyed by site, fixed reason vocabulary:unavailable/declined/build_failed/check_skipped/forced), cleared whenever the preconditioner options re-resolve. What deliberately did NOT change: any option value, route, or warning — existing warnings stay, the silent "auto" declines gain a record but no new noise, androtated_bc's own ctx report keys stay authoritative for that path's tests. A clean solve records nothing (that empty record is the suite's global negative control).#483 — user-set
snes_rtol/ksp_atolare honoured on the saddle-point solvers. Both keys were re-pushed by every solve, silently discarding an explicit user value — documented as settable, actually owned. The re-push now runs through the recorded-ownership latch that madesnes_max_itreachable (ruling D18), generalised to a dict: the framework keeps asserting the tolerance-derived values until the user sets a key, after which their value wins across solves. Alongside, the two hand-rolled tolerance-derivation code paths (Stokes vs Stokes_Constrained) collapse into one mechanism over per-class tables (_TOLERANCE_DERIVED_KEYS), with the real design difference — Constrained derives the outerksp_rtol+ Eisenstat-Walker pins instead of the inner fieldsplit margins, because EW pinning owns its outer accuracy — now documented in bothtolerancedocstrings as an ownership table. What did NOT change: a user who never touches the keys sees byte-identical behaviour (proved by no-override arms reading the live PETSc objects after two solves), the table contents are untouched, and the EW flags are still never re-asserted.#478 — explicit geometric MG on single-field solvers is unlocked (opt-in only).
preconditioner="fmg"on a scalar/vector solver was declined to GAMG with a warning, because native FMG needsDMCreateInjection, which PETSc cannot reliably build for a single-field discretisation on a refined DMPlex (err62, #276) — locking geometric MG out even though the injection-free custom-P route already existed. The explicit request is now honoured:custom_mg.build_transfersgrows a third hierarchy source (requested-native: the mesh's owndm_hierarchytail behind_DMLevelViewadapters), resolved after solver-set and mesh-owned so it stays the single "which hierarchy" owner and composes with #530's explicit-gamg opt-out. The options DB deliberately keeps GAMG as the degrade base until the live PC is configured at first solve; build failure degrades through the recorded barycentric→RBF→default ladder. Contract line stated in both docstrings:preconditioner="fmg"is a preference (degrades readably),set_custom_fmgis a demand (raises). What did NOT change:"auto"still declines geometric MG on single-field solvers — flipping that default needs its own np2/np4 + adaptivity campaign per the issue — it keeps GAMG and records the decline as the migration probe. Verified on the exact #276 err62 geometries (annulus refinement=2 qdegree=3; flat high-degree box), serial and np=2. Found en route: a gmsh-imported base hierarchy level carries section-only coordinates (PetscContainer, no PetscFE) andDMCreateInterpolationfrom such a source silently returns a zero matrix —_DMLevelViewworks on a clone and installs a P1 Lagrange coordinate FE when missing.#486 —
multiplier_schur_pcverified, kept, and instrumented. The issue measured four identical convergence rows and asked verify-or-remove. Tracing PETSc's fieldsplit.c settles it: the flag swaps only the Pmat (h,h) block, and under Stokes_Constrained's defaults (selfp+diag_use_amat) selfp assembles the Schur preconditioner from Amat sub-blocks — the swapped block is provably never read (exactly what was measured). It IS read underschur_precondition="a11"and under a monolithic direct factorisation of the Pmat, so the property stays. An explicit opt-in that cannot reach the PC now records + warns (the one new warning in this PR — an opt-in silently doing nothing is the #477 class). The regression test is matrix-level (assembled Schur pre differs flag-on/off under a11; identical under selfp, pinning the PETSc semantics), immune to "both converge in 4 iterations". Docstring drift corrected: Constrained'ssaddle_preconditionerclaimed selfp used the automatic 1/viscosity mass — it does not; that holds only under a11 — with a TODO(BUG) on whether selfp should see Pmat blocks at all (design question, not changed). What did NOT change: no default, no behaviour of any solve.Tests:
test_1022(fallback observability, every probe fired + silent-on-healthy),test_1023(tolerance ownership, live-object reads, both classes),test_1020rewritten from lockout test to unlock test +ptest_1020(np=2 route parity),test_1024(multiplier Schur regimes). Stakeholders green: 1013, 1014 (x2), 1015, 1016, 1017, 1018 (rotated), 1021, 1061, 1062, 1065, 0820. Fulllevel_1 and tier_agate: 580 passed, 0 failed.Underworld development team with AI support from Claude Code