Skip to content

Upgrade seed types from any listed baseline, chosen by recorded origin - #59

Merged
callumalpass merged 3 commits into
mainfrom
feat/seed-upgrade-baselines
Oct 2, 2026
Merged

callumalpass merged 3 commits into
mainfrom
feat/seed-upgrade-baselines

Conversation

@callumalpass

Copy link
Copy Markdown
Collaborator

Problem

upgrade_from holds one baseline: the most recent previous starter. An edited seed installed from an older starter is three-way merged against that baseline, so every difference between the two starters is treated as the user's edit and silently reapplied. Seen in practice: Reader's beta.1 → beta.4 upgrade brought back a match: path_glob the publisher had removed. The lock can't help today: both engines record the pack's resource digest for every seed, even one the pack preserved or a type that existed before installation.

Change (05A, schemas, conformance)

  • upgrade_from: one baseline or a non-empty list, each { digest, document, version? }. A single object stays valid (a list of one).
  • Validation (invalid_type_pack): seed type resources only; digest = SHA-256 of document; distinct digests; not the desired document; same type kind and name; version, if given, matches the document.
  • Seed origin in the lock: seed entries gain optional origin_digest, the publisher document the target descends from. Set on create, on upgrade (exact or merged) and when live already equals the desired document; otherwise carried forward; absent when unknown (pre-existing types, intentionally preserved targets, locks written before this change). A seed's installed digest describes the pack and MUST NOT be used as its origin.
  • Planning, in order (target exists, not preserved, upgrade_from declared):
    1. live = desired → preserve
    2. live = any baseline → update with the exact desired bytes (bytes prove the origin)
    3. origin = desired → preserve (upgraded, then edited)
    4. origin = a baseline → update by three-way merge against that baseline
    5. otherwise → preserve with a reason — never a guessed baseline, and never a pack-wide conflict
  • The assessment reports upgrade_baseline: { digest, version? } for every seed update.

Conformance

  • examples/v0.3/seed-upgrades/: one pack (example.seed-notes) whose note starter evolves v1 (title) → v2 (+created) → v3 (+tags), so merging against the wrong baseline is observable (it drops created). Includes a version that preserves without upgrade_from, a v3 that supports only v2, and one invalid manifest per validation rule. v2 uses the single-object form, v3 a list.
  • tests/v0.3/type-packs/type-packs.yaml: 13 fixtures (requirements seed_type_upgrade, seed_type_upgrade_validation), using new input.history steps (apply, write, replace) and target/lock expectations documented in tests/v0.3/README.md. Merged output is compared structurally since reformatting changed nodes is permitted.
  • scripts/type_pack_model.py: an executable model of these rules that check_v03_tests.py runs for seed fixtures. Mutation check: merging against the newest baseline (today's behaviour) fails 4 fixtures; using the pack digest as origin fails 2; conflicting instead of preserving fails 2.

Compatibility

upgrade_from is still unreleased (see CHANGELOG), so its shape changes before any release. Older engines and validators reject a list; packs should adopt lists only after engines and Connect validation support them. Lock entries without origin_digest are treated as unknown origin: unedited seeds still upgrade (rule 2); edited ones are preserved with a reason instead of being merged against a guess.

Verification

check_test_levels.py, check_v03_tests.py (78 executable, 155 adapter-target), migration prototype, runtime-pack drift check: all pass.

A seed type's `upgrade_from` held one baseline, the most recent previous
starter. An edited seed installed from an older starter was merged against it,
so the differences between the two starters were applied as if they were the
user's edits (a removed rule came back, a new property was dropped). The lock
could not help: it recorded the pack's resource digest for every seed, even a
seed the pack preserved or a type that predated the pack.

`upgrade_from` is now one baseline or a list, each `{ digest, document,
version? }`, validated (digest, seed types only, distinct, not the desired
document, same kind and name, version agrees). The lock records a seed's
`origin_digest`. The engine plans, in order: preserve when live equals the
desired document; replace with the exact desired bytes when live equals any
baseline; preserve when the origin is already the desired document; merge
against the origin baseline; otherwise preserve with a reason, never guessing a
baseline and never making the whole pack conflict. The assessment reports the
baseline used.

Conformance gains seed-upgrade example packs, `input.history` steps, and an
executable model (scripts/type_pack_model.py) that the checker runs; mutated
models that merge against the newest baseline, use the pack digest as origin,
or conflict instead of preserving each fail the fixtures.
…urrent

A preserved target whose bytes equal the desired document descends from it,
whether or not it predated the pack or is an intentionally preserved target.
Recording only an origin is not a reconfiguration.
…; name the frontmatter keys

Edited seeds under locks written before origin_digest are preserved with a
reason until upgraded or recreated; unedited seeds still upgrade. The lock
schema rejects origin_digest on managed resources, and baseline validation
names the frontmatter kind, name and version keys it compares.
@callumalpass
callumalpass merged commit b2ace4c into main Oct 2, 2026
20 checks passed
callumalpass added a commit to callumalpass/mdbase that referenced this pull request Oct 2, 2026
#22)

* Upgrade seed types from any listed baseline, chosen by recorded origin

A seed type's upgrade_from held one baseline, and an edited seed was always
merged against it. A seed installed from an older starter was therefore merged
against the wrong document, so the differences between the two starters were
applied as if they were the user's edits. The lock could not help: it only
recorded the pack's resource digest, even for a seed the pack had preserved or
a type that predated the pack.

Implement mdbase spec f5f5743 (mdbase-dev/mdbase-spec#59):

- upgrade_from is one baseline or a non-empty list of { digest, document,
  version? }, regenerated from the spec schemas. invalid_type_pack when it is
  not on a seed type, a digest does not match its document, digests repeat, a
  baseline is the resource's own document, kind or name differ from the
  desired type, or version differs from the document's declared version.
- Seed lock entries record origin_digest: the desired digest on create, on
  update (exact or merged), and when the live bytes equal the desired
  document; otherwise carried forward by target, or omitted (unknown).
- An existing, non-preserved seed with upgrade_from plans in order: live is
  desired, preserve; live is any baseline, exact update; origin is desired,
  preserve; origin is a baseline, merge against that baseline (conflicts fail
  closed); otherwise preserve with a reason, never a conflict.
- Seed updates report upgrade_baseline { digest, version? } in the assessed
  diff, so it is bound into assessment_digest. An origin-only lock change does
  not make an installed pack reconfigure.

The conformance runner gains input.history and the seed-upgrade expectations,
and now also asserts type-pack runs, status, actions, and target existence,
which it previously ignored. CI pins the spec PR head for now.

* Pin conformance to the clarified seed-origin spec

* Pin conformance to the merged seed-upgrade spec and regenerate its schemas

The lock schema now rejects origin_digest on managed resources.
callumalpass added a commit to mdbase-dev/mdbase-contracts that referenced this pull request Oct 3, 2026
)

* Upgrade every earlier seed starter with upgrade_from baseline lists

mdbase spec 05A (mdbase-dev/mdbase-spec#59) lets a seed type's upgrade_from
list every starter a publisher shipped, and has engines merge an edited seed
only against the baseline its lock records as origin. A starter that is not
listed is never upgraded, and the offered TaskNotes and People packs each
listed only one of their earlier starters.

- Pack definitions accept upgrade_from as one path or a list. The build emits
  a list as [{ digest, version, document }], newest first, with version from
  each baseline's frontmatter, and rejects non-seed baselines, duplicates, the
  resource's own document, and a different type kind or name. The single form
  emits as before, so published provisions are byte-identical.
- tasknotes.task 0.3.0-rc.18 lists task starters 4, 3, 2 and 1; mdbase.contact
  1.4.0 lists Person starters 2 and 1. rc.17 and 1.3.0 are hidden.
- seed-baselines.test.mjs checks that every offered pack's seed upgrade_from
  covers every starter any earlier version shipped at that target (it fails on
  rc.17 and 1.3.0), and that every published baseline follows 05A.
- seed-upgrade.test.mjs installs each earlier version through the selected
  engine and upgrades it, unedited (exact bytes, origin_digest = desired) and
  customised (merged against its own baseline, edits kept).
- Existing tests cover 1.4.0 and rc.18; an edited task type under a lock with
  no origin is now preserved, which blocks the contract upgrade for review.
- Pin mdbase-ts v0.3.0-rc.9, and the CLI to mdbase-connect 4797ba08 with
  mdbase-rs 056db73.

Co-Authored-By: Claude <noreply@anthropic.com>

* Build the verification CLI from merged Connect main ab936db5

mdbase-connect#567 merged; ab936db5 pins mdbase-rs 056db73, matching engine_ref.

---------

Co-authored-by: Claude <noreply@anthropic.com>
callumalpass added a commit to mdbase-dev/mdbase-reader that referenced this pull request Oct 3, 2026
* List every earlier starter as a seed upgrade baseline

Pack beta.4 named only the beta.3 starters as `upgrade_from`, so a
collection installed from an earlier pack was merged against the wrong
document: upgrading an edited beta.1 seed treated beta.1's `match`
rules, which beta.3 removed, as the collection's own edit and restored
them.

mdbase-spec 05A (mdbase-dev/mdbase-spec#59) lets a seed list every
starter it supports and has engines choose the baseline from the lock's
seed origin. Pack beta.5 ships the same version-2 starters and lists,
newest first, each distinct starter Reader has shipped, with its
`version`:

- reader-source: beta.3 (82869c1), beta.2 (54f46e7), beta.1 (e8a3fca)
- reader-annotation: beta.3 (1fb7e75), beta.1 (9b6b0fc)

beta.2 changed only reader-source, and the beta.3 starters were also
served as beta.1 between #20 and #26. The new baselines are the exact
released bytes from 5c55c7d and fde2582; the released beta.2 pack is
added as a fixture.

An unedited starter from any of them is replaced with the exact
version-2 bytes. An edited seed is merged against the starter it was
installed from; for beta.1 and beta.2 that removes the top-level
`match` rule, which 05A leaves to manual review, so the upgrade fails
closed instead of restoring it. A seed whose lock predates recorded
origins is kept with a reason.

Verification installs packs with @callumalpass/mdbase 0.3.0-rc.9, which
supports baseline lists. Until a Connect release includes
mdbase-dev/mdbase-connect#567, @mdbase-dev/connect-dev rejects them, so
connect-protocol 0.1.0-beta.124 is patched with that PR's schema change;
drop the patch when moving to that release.

* Move to Connect SDK 0.1.0-beta.125 and drop the manifest-schema patch

* Restore the production extension manifest
callumalpass added a commit to mdbase-dev/mdbase-writer that referenced this pull request Oct 3, 2026
* List every earlier reader-source starter as an upgrade baseline

Pack beta.4 named only the beta.3 reader-source starter as
`upgrade_from`, so a collection holding an earlier starter was merged
against the wrong document. mdbase-spec 05A (mdbase-dev/mdbase-spec#59)
lets a seed list every starter it supports, and engines choose the
baseline from the lock's seed origin.

Pack beta.5 lists the same baselines as Reader's pack beta.5, byte for
byte and under the same paths, newest first with their `version`:
Reader's beta.3 starter (82869c1, Writer's beta.3), beta.2 (54f46e7)
and beta.1 (e8a3fca, Writer's beta.1 and beta.2). The existing
byte-identity test now covers every listed baseline against Reader's
copies.

Until a Connect release includes mdbase-dev/mdbase-connect#567,
@mdbase-dev/connect-dev rejects baseline lists, so connect-protocol
0.1.0-beta.124 is patched with that PR's schema change. Drop the patch
when moving to that release.

* Move to Connect SDK 0.1.0-beta.125 and drop the manifest-schema patch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant