markfluence.yaml pages: page metadata outside the markdown file

[willkg]

Depends on #100, which owns the markfluence.yaml loader, malformed-file handling, and project-wide settings. This issue adds the pages: key on top of it: per-file page metadata living in the project file instead of in the markdown, so a .md can be published to Confluence while staying pristine.

Motivation

A GitHub Actions workflow (#29) should be able to publish docs/**/*.md to Confluence on push. Today that requires every file to carry markfluence frontmatter, which is exactly what a README or a docs tree that has other readers does not want, and #38 explored sidecar files as the escape hatch without anyone being thrilled about the discovery, XOR, migration and orphan rules it needs.

The flag route does not scale, and it is worth being precise about why. update README.md --page-id 12345 --title X --page-width max already works today. But flags scale with fields while the problem scales with files: --title and --page-id are single-FILE-only, so docs/**/*.md cannot be expressed at all, and every new frontmatter field (labels, #138; layout, #21) would have to grow its own flag to stay usable from CI. A file that maps paths to metadata solves both at once, and the project already has a file for project-scoped truth.

The shape

myrepo/
├── markfluence.yaml
└── docs/
    ├── index.md
    ├── deploy-runbook.md
    └── oncall.md

# project-wide defaults (#100)
space: ENG
page_width: max

# per-file page metadata (this issue)
pages:
  docs/index.md:
    page_id: 12345
    title: Engineering Docs
    labels: [howto]

  docs/deploy-runbook.md:
    page_id: 12346
    title: Deploy Runbook
    parent: docs/index.md
    labels: [runbook, ci/cd]

  docs/oncall.md:
    page_id: 12347
    title: On-call Handbook
    parent: docs/index.md
    page_width: wide

Every .md there has no frontmatter at all. An entry is a whole frontmatter block that lives elsewhere — the same keys, the same values, the same validation, the same canonical order — so internal/frontmatter is reused rather than paralleled and there is one answer to "what fields exist". Deliberately no path: <id> shorthand: one form, not two.

The workflow is then one step with no matrix and no per-file inputs:

- uses: mozilla/markfluence@v1
  with: { command: update, files: "docs/**/*.md", url: …, username: … }
  env:  { CONFLUENCE_TOKEN: ${{ secrets.CONFLUENCE_TOKEN }} }

CI updates; humans create

Page creation stays a human act. Creating in CI would mean a workflow committing a new page_id back to the repo — contents: write, a bot commit per new doc, and a race if two runs create at once — which is a lot of machinery for something that happens rarely and benefits from a person seeing where the page landed.

So the local bootstrap has to be straightforward, and it is:

$ vim docs/new-thing.md      # content only, no frontmatter
$ markfluence create docs/new-thing.md --space ENG --title "New Thing" --parent docs/index.md
$ git add markfluence.yaml docs/new-thing.md && git commit

create writes a complete pages: entry — title, space, parent, page_id, page_width — because the project has a pages: block. Nothing is hand-edited, and CI's update docs/**/*.md picks it up on the next run.

Resolution rules

Both locations are legal, and agreement is silent. A file may carry frontmatter, or have a pages: entry, or both with the same values. That last case is what makes migration incremental: copy values into the manifest, verify, delete them from the files later, with everything working throughout. There is no project "mode" and no all-or-nothing switch.

Disagreement is what gets reported, graded by what it can destroy:

key	on disagreement
page_id, space, parent	error, abort that file	the clobber case: a page_id copy/pasted from an old file publishes over a live page. space/parent decide where a create lands, and #10 wants update to enforce and move, so they are coordinates now rather than after a rule change
title, page_width, labels	warning, frontmatter wins	visible and recoverable; frontmatter wins per #100's flag > frontmatter > project file chain — the answer closer to the content wins

For create, which is transactional and preflights every file, a coordinate disagreement in any file aborts the whole batch. For update, only that file fails and the rest proceed.

Diagnostics are scoped to the files being processed. Running on bar.md must not report anything about foo.md's entry. This constrains the loader: parse errors and unknown top-level keys abort immediately per #100, but a semantically bad entry — a non-numeric page_id, an invalid page_width, an invalid label — is diagnosed only when that entry's file is one of the arguments. One bad entry must not block every invocation in the repo.

A file with no metadata anywhere is skipped, not failed. status: skipped, ok: true, a notice naming it, exit 0. Repositories legitimately contain markdown that is not published to Confluence, and drafts are a normal state; a glob-driven CI run should not go red because someone added a file. This is a behavior change — update currently errors on a file with no page_id — and it is deliberate. A file that is registered but whose entry lacks page_id still errors: someone claimed it and create has not run.

New metadata is written wherever that file's metadata already is, and when there is none, to the manifest if the project has a pages: block, otherwise into the file's frontmatter. A project that has chosen the manifest never accidentally grows frontmatter; a project without one behaves exactly as today.

Path keys are lexically-cleaned, root-relative, slash-form, compared byte-for-byte, and arguments are normalized the same way before lookup — a mismatch now means a silent skip, so the rule has to be exact.

• Lexical only, no EvalSymlinks, matching withinRoot/attachfile.Resolve/destPath. A symlinked docs/ is legitimate, and resolving it would make a key depend on the checkout's layout, which L2 (invocation-independent) forbids.
• A key escaping the root (../elsewhere.md) is a load-time error: the project file declares the project's boundary.
• Two keys normalizing to one path (docs/a.md and ./docs/a.md) is a load-time error naming both; YAML only catches literal duplicates.
• No case folding. Known limit, to document beside pageslug's NFD/NFC note: on a case-insensitive filesystem Docs/a.md opens the file but matches no docs/a.md key, so it reads as unmanaged and is skipped.

Both load-time errors are deliberately not scoped per-file: an escaping or duplicate key means the manifest's structure is wrong, not that one entry is bad.

update loses its page-metadata flags

The rule to extract: flags describe the run; files describe the page.

flag
--title	removed — single-FILE-only, now expressible in frontmatter or an entry
--page-id	removed — same, and it is the flag that made the CI workaround tempting
--page-width	removed — the only batch-ok one, so the real loss; a page_width: project default (#100) covers it better and permanently
--message, --force, --dry-run	kept — invocation metadata and behavior, not page metadata

create keeps its flags, for a principled reason rather than squeamishness: it is the verb that establishes metadata and then persists it, so --space/--parent/--title are how an entry that does not exist yet gets bootstrapped — the local flow above is exactly that. update only ever consumes metadata, so it should have no way to invent any.

This also settles, permanently, the question #138 kept bumping into: there is no --labels, and there will be no flag for whatever field comes after it.

Touchpoints

internal/linkindex is the sharp one. Build walks the tree reading each sibling's frontmatter for page_id/title (linkindex.go:68-76). It must resolve metadata per path through a lookup that consults both locations, or every cross-document link in a manifest project silently degrades to the "exists on disk, not published yet" warning and republishes as plain text. This is #38's "sibling ambiguity" edge, and per-file resolution handles a mixed tree correctly throughout a migration.

check gains a lint — a file with inline markfluence keys in a project that also uses pages: is reported, as a warning, not an error. "No half-and-half" is a real preference and this is how it gets enforced without becoming a wall someone hits mid-migration. Scoped to the files being checked, like everything else.

fix can move a file's inline keys into its manifest entry, reported as an ordinary change. This is a convenience rather than a prerequisite — since agreement is legal, migration needs no special command — but it is what gives the check warning an obvious remedy, and it is read-only against Confluence, so migrating touches no live page.

--json gains metadata_source on update/create results: "frontmatter", "manifest", or null when the file is unmanaged. Debugging "why did it publish to that page" in CI otherwise means reproducing the resolution by hand. Q9's skipped status and the disagreement warnings surface through the existing status/warnings fields.

Docs: README (a project-file section, the CI flow, the removed flags), docs/guarantees.md (L2 is why keys are lexical and root-relative), and a _plans/ file before the branch.

Out of scope

• Creating pages in CI. See above; if it is ever wanted, it is its own issue with its own answer for committing ids back.
• Sidecar files (Explore: sidecar frontmatter files (in-file XOR sidecar) so .md stays clean #38). This supersedes the use case — a pristine .md whose coordinates live elsewhere — without discovery, XOR, migration or orphan rules, because the location is one known file rather than N inferred ones. Explore: sidecar frontmatter files (in-file XOR sidecar) so .md stays clean #38 can be closed as superseded once this lands, or kept for the "metadata next to each file" preference if that turns out to be wanted separately.
• export writing manifest entries. export creates a fresh tree and writes frontmatter; whether an exported tree can be manifest-shaped is a later question.
• A --persist-to file|manifest flag. It is an invocation flag, so it would not violate the rule above, but the inference rule covers every case anyone has named. Add it if someone asks.
• Project-wide defaults (space:, page_width:, message:) — that is markfluence.yaml: support project-wide settings #100.

Related

• markfluence.yaml: support project-wide settings #100 — the loader, malformed-file handling, and project-wide settings. Prerequisite. Its "there is no YAML library in this module" objection is stale: Frontmatter we write is not valid YAML when a value contains a colon #130 made goccy/go-yaml a direct dependency.
• Ship markfluence as a reusable GitHub Action #29 — the GitHub Action this exists to make usable
• Explore: sidecar frontmatter files (in-file XOR sidecar) so .md stays clean #38 — sidecar frontmatter; superseded for the pristine-markdown use case
• Support Confluence page labels via frontmatter #138 — labels; an entry carries labels, which is how labels reach CI without a flag
• Support a layout: article frontmatter directive #21 — layout:; another field that would otherwise want a flag
• update: enforce space/parent and support page moves #10 — update enforcing space/parent and supporting moves; why space/parent are coordinates here