Skip to content

Dx 5057 further expand cre reconciller - #23440

Open
Tofel wants to merge 16 commits into
developfrom
dx-5057-further-expand-cre-reconciller
Open

Dx 5057 further expand cre reconciller#23440
Tofel wants to merge 16 commits into
developfrom
dx-5057-further-expand-cre-reconciller

Conversation

@Tofel

@Tofel Tofel commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Reworks the CRE reconciler from a single-shot apply into a phase-aware, resumable, mostly-safe-to-rerun tool, and brings the UI up to parity with the CLI.

  • Reconciliation core (A1–A3): per-phase input-hash memoization so a no-change apply does zero on-chain/JD writes; contract deployment is guarded by actual-state (both CapReg + WorkflowRegistry present); a Docker-layer-cache-style cascade means any upstream phase actually running forces every phase after it to run too. No --force flag — the repair path is deleting a phase_hashes entry by hand.
  • Discovery (B1–B4): reads Aptos/Solana keys (capability-gated), validates JD node labels (p2p_id, environment, type) on every run, validates every chain-capability has a matching chain config per member node, and stops reading OCR2 bundles from bootstrap/gateway nodes.
  • Node config (C1–C2): never emits [[EVM]] — chains must pre-exist in desired state; [[chains]] unified with a required family field; adds real aptos/solana end-to-end support (UI catalog, defaults, and a matching CRE blockchain provider at apply time, fixing an aptos error / solana nil-pointer panic).
  • Parallelization (D1): bounded-errgroup for node-spec building and per-node job enrichment; runPreEnvStartup/runPostEnvStartup deliberately left sequential (shared per-DON state).
  • Gateway/UI (E1–E8): replaces the old [[gateway_nodes]] per-gateway assignment list with an explicit family field on every DON (mirrors real CRE's nodesets.don_family) — makes invalid gateway/DON overlap structurally impossible; blocks save when a gateway's family has no served DON; adds a JD settings tab, a per-DON allowlist editor, a smarter default DON selection, and removes infra.chart_values/infra.namespace in favor of a --chart-dir flag.
  • Deployer keys (H1): removes --deployer-key; resolves signing keys from PRIVATE_KEY_<CHAIN_ID>PRIVATE_KEY → Anvil default only.
  • UI apply + diff/status (E3–E4): new Apply tab — POST /api/apply streams NDJSON log events from the same Reconciler.Run the CLI uses, pauses at the TOML breakpoint via an in-process wait/confirm handoff (no more exit-42), localhost-only. New GET /api/diff surfaces contract/DON/workflow-registry/node-config state plus the stored per-phase hash cache.
  • Two fixes: node-label preflight was wrongly requiring an EVM chain config for every node (broke gateway-only node sets) — switched to a raw JD ListNodes call; a coarse onChainComplete() gate in Run() was skipping Apply entirely once contracts/DON IDs looked "done," bypassing the per-phase hash checks and silently no-op'ing capability changes — removed, Apply is always called now. UI also always saves before apply.

Known gaps (tracked in TODO.md, not fixed in this branch)

  • Capability add/remove still doesn't reliably reconfigure CapReg in some repro cases (P0, needs fresh investigation — the onChainComplete() removal wasn't fully sufficient).
  • vault capability panics on apply (missing DKG key discovery step — root-caused, plan in TODO.md, not implemented).
  • Docs (README phase model, breaking schema changes), bootbootstrap rename, and CLD audit artifacts are deferred.
  • Several new UI requests (multi-chain display, dark mode, primary Apply button, richer progress view) are backlog-only.

@github-actions

Copy link
Copy Markdown
Contributor

✅ No conflicts with other open PRs targeting develop

@github-actions

Copy link
Copy Markdown
Contributor

I see you updated files related to core. Please run make gocs in the root directory to add a changeset as well as in the text include at least one of the following tags:

  • #added For any new functionality added.
  • #breaking_change For any functionality that requires manual action for the node to boot.
  • #bugfix For bug fixes.
  • #changed For any change to the existing functionality.
  • #db_update For any feature that introduces updates to database schema.
  • #deprecation_notice For any upcoming deprecation functionality.
  • #internal For changesets that need to be excluded from the final changelog.
  • #nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
  • #removed For any functionality/config that is removed.
  • #updated For any functionality that is updated.
  • #wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

@trunk-io

trunk-io Bot commented Aug 18, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@cl-sonarqube-production

Copy link
Copy Markdown

@Tofel
Tofel marked this pull request as ready for review August 18, 2026 14:43
@Tofel
Tofel requested review from a team as code owners August 18, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant