Reorganise the migration by consuming series, and write down two repoint rules

[mmcky]

Brings PLAN.md up to date after a full verification pass over the 2026-08-04 migration plan, and records two ordering rules that were not written down anywhere. Documentation only — no data, manifests or scripts touched, and the strict audit is unaffected.

Why reorganise by series

The plan groups the remaining 31 datasets by hosting pattern (LFS case, dynamic case, per-repo batches). Grouping by consuming series fits the actual work better, because apart from a single pairing each series now owns its own data.

I checked this rather than assumed it: 16 of the 31 remaining datasets have more than one consuming repo, and every single one is lecture-python-intro + lecture-wasm. There is no other cross-series coupling left anywhere — the last one was the P2 pandas_panel trio, which is already migrated. lecture-dp, lecture-jax and continuous_time_mcs turn out not to be data consumers at all; dp's 10 committed files are inherited orphans, jax embeds graph.txt via %%file, and continuous_time_mcs has one orphan scratch file.

So the tracks table lands beside the phases rather than replacing them: tracks are the execution view, phases stay as the machinery each track passes through. Tracks A–D are mutually independent, and the four real dependencies in the whole programme are named explicitly instead of implied by wave numbering.

The two repoint rules

Both came out of reading what the repos actually do, not out of a failure — which is the good time to write them down.

Never delete a file a sibling repo reads. lecture-wasm/lectures/long_run_growth.md fetches raw.githubusercontent.com/QuantEcon/lecture-python-intro/main/lectures/datasets/mpd2020.xlsx — intro's committed blob. That URL returns 200 today. The standing rule "delete the lecture repo's own copy in the same repoint PR" would therefore 404 the wasm build the moment intro's copy goes, and the 2026-08-04 plan schedules the wasm repoint after those deletions. Every one of the eight landed intro statics is affected.

Repoint every consumer of a dataset together. scripts/build_audit.py fails a migration.yml record marked pending/landed once any consumer reads data-lectures, and fails one marked repointed/final while any consumer still does not. There is no green status for a partially-repointed dataset. That is deliberate and it is what makes the tracker worth trusting — but it means a two-repo dataset cannot move one repo at a time without the drift alarm from #29 firing in the gap.

Also in here

Licensing no longer gates migration (#35). Inherited data — anything the lectures already serve publicly — migrates with its licence recorded as found, including restricted and a null name where that is honest, and anything needing further thought is tracked with alternatives. The gate is the promotion of data.quantecon.org as a public open-data host, not each file's move. This generalises the exception AGENTS.md already carried for countries.csv, and explicitly does not extend to genuinely new datasets.

Phase 4's CORS box is ticked. Verified today: quantecon.github.io/data-lectures/lectures/lingcod_msy_recovery.csv returns access-control-allow-origin: *. The pyodide/JupyterLite requirement (QuantEcon/meta#143) is met on the default Pages domain and does not wait on the custom domain — worth knowing, since the plan defers it to the DNS cutover.

Two stale statements in AGENTS.md corrected. It still described the repo as mid-transition to a flat tree (that landed 2026-07-16) and its repo map said 9 datasets where lectures/ now holds 19. Both would actively mislead someone reading it as the conventions doc, and AGENTS.md's own rule asks for it to be updated in the same PR as PLAN.md.

Verification behind these numbers

The strict audit was re-run against data-lectures@54826bd and freshly-fetched origin/main of all 8 lecture repos: it passes green and reproduces 41 static files, 35 orphans, 22 live-API lectures, 10 migrated, 0 legacy refs, 5 URL forms. The eight landed intro statics were confirmed byte-identical to intro's copies by git blob hash, so the Phase 7 gate genuinely passes for that batch.

Two corrections worth flagging for whoever works the plan next: the ames_house_prices.csv "empty licence field" it lists as an open gap is actually a documented redistribution: permitted decision with reasoning, and high_dim_data's blanket LFS rule is *.csv and *.dta, not *.csv alone.

Part of #8. Related: #13, #14, #26, #35, #15.

[Copilot]

🟡 Changes recommended

There are a couple of documentation accuracy/clarity issues in PLAN.md (wording vs PR description, and an ambiguous/outdated “9 files” bullet) that should be corrected before merge.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

Updates the repo’s migration roadmap and contributor guidance to reflect the verified 2026-08-04→2026-08-06 plan refresh, including a re-organisation of remaining migration work by consuming series and explicit ordering rules for safe cross-repo repoints.

Changes:

• Added a “Repoint rules” section to PLAN.md documenting two ordering constraints for cross-repo repoints and deletions.
• Reframed the remaining migration work into “Migration tracks” by consuming series, and refreshed current audit counts/status notes.
• Updated AGENTS.md to reflect the flattened published tree, the new repoint ordering traps, updated licensing policy framing, and an updated repo map.

File summaries

File	Description
PLAN.md	Refreshes status/audit counts, adds repoint ordering rules, and reorganizes remaining migration work into per-series tracks.
AGENTS.md	Updates repo description and rules to match the now-flat published layout, and records repoint ordering traps + updated licensing policy text.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 2
• Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[mmcky]

Phase 3 storage settled — pushed as a second commit

Resolving the open storage question rather than leaving it for Wave 1, because checking it changed the answer.

The LFS requirement rested on a false premise. Both SCF minis fit plain git — SCF_plus_mini.csv is 31.3 MiB and SCF_plus_mini_no_weights.csv is 72.4 MiB, against GitHub's 100 MiB limit. So the published tree can stay entirely free of LFS objects, which removes the raw-vs-media URL trap for consumers instead of managing it with a sequencing rule. That is a strictly better outcome than what #1 has been planning around.

One file genuinely needs LFS, and it is not a dataset. SCF_plus.dta is 99.1 MiB, has no lecture consumer, and exists only as input to the SCF builder. It goes to a new unpublished sources/ tree under per-path LFS, with sources/README.md as the audit trail — origin, retrieval date, licence, upstream identifier, sha256, and the consuming builder. Files there get no sidecar manifest, because they are not published datasets.

A trap found while checking this. generating_mini.md reads its input over the network from the repo we plan to retire: pd.read_stata('https://github.com/QuantEcon/high_dim_data/blob/main/SCF_plus/SCF_plus.dta?raw=true'). Archiving high_dim_data while that line stands would re-introduce exactly the legacy-repo dependency this project drove to zero — and it would do it quietly, since an archived repo still serves raw URLs. The builder is repointed at sources/ before archiving.

Consequence worth noting for #20's workflow: with nothing published under LFS, the Pages job can check out lfs: false and stop pulling 99 MiB on every push to main plus weekly.

Related: #1 (the LFS issue this largely dissolves), #2 (the fold), #35 (SCF+ licence row).