Skip to content

docs: design for absorbing offchain and solana into this repo - #4239

Open
bgm-malbeclabs wants to merge 11 commits into
mainfrom
docs/monorepo-migration-design
Open

docs: design for absorbing offchain and solana into this repo#4239
bgm-malbeclabs wants to merge 11 commits into
mainfrom
docs/monorepo-migration-design

Conversation

@bgm-malbeclabs

Copy link
Copy Markdown
Contributor

Design for absorbing malbeclabs/doublezero-offchain and malbeclabs/doublezero-solana into this repo. Docs only — no code changes, nothing here is implemented yet.

Prerequisite is already done: both repos moved from doublezerofoundation to malbeclabs on 2026-08-26 (doublezero-offchain#412, doublezero-solana#125, doublezero-shreds#686, doublezero#4236, docs#202).

Summary

  • Two new top-level trees, offchain/ and solana/. Nothing existing moves. Rejects flattening into crates/ and smartcontract/programs/, mainly because the two program sets target different chains — the CLAUDE.md already treats the DoubleZero Ledger / Solana L1 split as load-bearing, and warns it fails silently when confused.
  • One workspace, which deletes ten git-dependency pins. Offchain's 7 malbeclabs/doublezero deps and its 3 doublezero-solana deps all become path deps. network-shapley is the only external pin left.
  • passport and revenue-distribution are held back until measured. They build reproducibly against their own lockfile and are checked against programs/sha256sums_*.txt. Step 0 is a throwaway merge that answers whether a shared lockfile changes the bytes. Decided by measurement, not argument.
  • Sequenced as four PRs plus a throwaway measurement, ordered so the dangerous step fails alone. Step 1 (import + history) touches no build file, which is exactly what makes step 2 (workspace merge) cleanly revertable.
  • The sentinel binary name collides — both repos build a doublezero-sentinel. This repo's is renamed to dz-e2e-sentinel; it has no external contract, and the explicit name makes the later cleanup easy to find.
  • Tags import unprefixed. Across 33 + 14 + 2 tag prefixes there is exactly one collision, sentinel, and it resolves by deleting an orphan tag we want gone anyway.

Why now

A change spanning the three repos currently needs three PRs in a fixed order plus a pin bump in each consumer. On 2026-08-26 that failed badly: shreds pointed one crate at the new org while the offchain revision it pinned still pointed at the old one. Cargo makes a git dep's URL part of the crate identity, so it built two copies and the oracle reported 194 errors of the form "no method named X" on types that plainly had X. Nothing in the error text mentioned a URL. The spec's "Why" section leads with that incident because it is the clearest statement of the cost.

What it means for shreds

Shreds stays a separate repo — it is private and this one is public — so it keeps git deps and coordinated bumps. It still improves a lot: three upstream pins collapse to one, and the duplicate doublezero_sdk it builds today becomes impossible, because no second source will exist. That holds only if shreds repoints every malbeclabs/doublezero dep to a single ref in one commit, so the spec gives that step a lockfile grep as its acceptance test and recommends keeping it as a CI guard.

Testing Verification

Every quantitative claim was measured against the repos rather than estimated:

  • Tag collisions: compared all tag prefixes across the three repos via the GitHub API. Exactly one overlap (sentinel). The spec includes the command to re-run immediately before importing, in case new tags land.
  • Dependency alignment: solana-sdk, solana-program and solana-client resolve identically in all three lockfiles. Only borsh (1.7.0 / 1.7.0 / 1.6.0) and the toolchain (1.97.1 / 1.92.0 / 1.91) differ.
  • Sentinel collision: confirmed both Cargo.toml files declare [[bin]] name = "doublezero-sentinel", and that this repo's is unreleased, feature-gated behind server, and consumed only by the dz-e2e/sentinel image in e2e.yml. The rename touches two files.
  • Duplicate doublezero_sdk: confirmed present in shreds' lockfile today at two sources (?tag=client/v0.31.0 and ?rev=8bb7900e).

Review asks

The five decisions (D1–D5) each carry their rationale and the alternatives rejected. Worth arguing with:

  • D1 — top-level offchain/ and solana/ versus folding into the existing trees.
  • D2 — holding the two Solana programs out of the workspace pending measurement.
  • D3 — offchain's crates moving from 1.92.0 to 1.97.1, and the clippy churn that implies.

@bgm-malbeclabs
bgm-malbeclabs requested review from a team and a lite review from Copilot August 27, 2026 15:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds a design spec that describes how to migrate malbeclabs/doublezero-offchain and malbeclabs/doublezero-solana into this repository as two new top-level trees, and how to sequence the work to reduce risk.

Changes:

  • Add a new migration design spec that defines scope, decisions (D1–D5), sequencing (steps 0–5), rollback, and risk analysis.
  • Document workspace strategy, toolchain pin strategy, tag import strategy, and the sentinel binary rename plan.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +200 to +201
make build-artifacts NETWORK=mainnet-beta
shasum -a 256 -c programs/sha256sums_mainnet_beta.txt

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. The commands were also wrong in a deeper way — see the Bugbot thread on the same step: running them against solana's own lockfile reports no change however the merged workspace resolves.

Step 0 has since run with a resolution-comparison method instead of a rebuild, and the checksum gate in step 4 now specifies running from inside solana/.

Comment on lines +237 to +240
Gate: `cargo check --workspace`, `cargo clippy --all-targets`, the full test suite, e2e,
and the program checksum check. Also confirm the lockfile holds no crate twice:
`grep '^name = ' Cargo.lock | sort | uniq -d` should print nothing that was not already
duplicated before the merge.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, and the real number is worse than the concern suggests: cargo tree -d --workspace reports 140 duplicate groups on main today. All normal. A duplicate-name check would have been pure noise.

Replaced with two gates that actually mean something:

grep -c 'source = "git+https://github.com/malbeclabs' Cargo.lock    # expect 0
cargo tree -d --workspace --locked | grep -cE '^[a-z0-9_-]+ v'      # baseline 140, must not rise

The first is the one that matters. After the dependency flip there should be no internal git sources left at all, because they are all path deps.


Gate: every existing job green, with no existing job definition touched. `git diff`
between each imported tree and its filtered source is empty. The diff is large and needs
no judgement to review.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken. Worth noting the repo had one of each, so there was no convention to break — but judgment matches the US spelling used elsewhere (onchain, etc.), so it is the better default.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7bc5b84. Configure here.

Comment thread docs/superpowers/specs/2026-08-27-monorepo-migration-design.md Outdated
Comment thread docs/superpowers/specs/2026-08-27-monorepo-migration-design.md
Comment thread docs/superpowers/specs/2026-08-27-monorepo-migration-design.md Outdated
Only the monorepo migration design belongs on this branch. Nine other spec and
plan documents under docs/superpowers were untracked working files and were
committed by accident. They stay on disk, they are just no longer tracked.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants