merge queue: checking main (5331a01) and #12142 together#12155
Closed
mergify[bot] wants to merge 2 commits into
Closed
merge queue: checking main (5331a01) and #12142 together#12155mergify[bot] wants to merge 2 commits into
mergify[bot] wants to merge 2 commits into
Conversation
Docs config snippets are the most-copied content on the site, but nothing
stopped them from drifting out of sync with the real Mergify config schema.
A wrong example erodes trust instantly. This adds CI that extracts every
Mergify config example from the MDX and validates it, so a broken snippet
fails CI instead of silently misleading users.
This started as a spike ("not sure it's possible"). It is possible, cleanly.
The three feasibility risks resolved as follows:
1. Schema source. The repo already vendors the exact schema at
`public/mergify-configuration-schema.json` (draft 2020-12), and it is kept
fresh by the automated "chore: sync Mergify JSON Schema files" commits. We
validate against that in-repo copy — offline, no auth, no network. The
`mergify config validate` CLI was evaluated and rejected for CI: it fetches
the schema over the network from docs.mergify.com (the served copy even lags
the just-synced repo file), which is both flaky and circular — validating the
docs against a schema served by the docs. It does the same structural
JSON-Schema check we do, so we lose nothing. It remains the right tool for
deep, single-config semantic validation locally.
2. Extraction / partial-vs-complete. A block is validated only when it has an
unindented top-level Mergify key (`queue_rules`, `pull_request_rules`,
`scopes`, `merge_protections_settings`, ...). Fragments, `...` placeholders,
GitHub Actions YAML, and `# partial`-marked blocks are skipped automatically.
For a complete-looking block that must not be validated (deprecated syntax in
a migration guide), a reader-invisible MDX comment
`{/* validate-config-examples: skip — why */}` before the fence excludes it.
3. CI shape. A new `config-examples` job in the existing GitHub Actions
workflow runs `pnpm check:config-examples`, mirroring the other jobs.
Running the validator over the current docs surfaced 111 complete config
examples; 2 were genuinely broken and are fixed here:
- `comment.mdx`: `message: @{{author}} ...` is invalid YAML (leading `@`), now
quoted.
- `backport.mdx`: `bot_account: {{ author }}` is invalid YAML (flow mapping),
now quoted.
The 2 deprecated `partition_rules` examples in the partitions→scopes migration
guide are marked skipped.
- Validation is structural only. Mergify's custom string formats
(`duration`, `template`, ...) are intentionally not enforced — a generic
JSON-Schema `duration` validator reads it as ISO-8601 and would wrongly reject
valid values like `checks_timeout: 5 min`.
- Consolidated on a single Node implementation (`scripts/validate-config-examples.mjs`,
repo-native, with unit + integration tests) and retired the earlier standalone
Python extractor; the `validate-config-examples` skill now drives the same script.
- `pnpm check:config-examples` — green (111 checked, 2 skipped, 0 invalid)
- `pnpm test` — 73 passing, incl. a negative test proving the validator rejects
an unknown key (not a no-op)
- `pnpm check` — clean (astro check + eslint + biome)
- actionlint — clean on the updated workflow
Change-Id: Iec4826a97e941255bcfcf0133a654edaf1f07585
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎉 This pull request has been checked successfully and will be merged soon. 🎉
Branch main (5331a01) and #12142 are queued together for merge.
This pull request has been created by Mergify to check the mergeability of #12142.
You don't need to do anything. Mergify will close this pull request automatically when it is complete.
Required conditions of queue rule
defaultfor merge:github-review-approved[🛡 GitHub repository ruleset ruleRequire pull request for default branch] (documentation)schedule=Mon-Fri 09:00-17:30[Europe/Paris]Enforce conventional commit]:title ~= ^(fix|feat|internal|docs|style|refactor|perf|test|build|ci|chore|revert|ui)(?:\(.+\))?!?:👀 Review Requirements]:#approved-reviews-by >= 2author = dependabot[bot]author = mergify-ci-bot📕 PR description]:body ~= (?ms:.{48,})🔎 Reviews]:#changes-requested-reviews-by = 0#review-requested = 0#review-threads-unresolved = 0🤖 Continuous Integration]:check-success = buildcheck-success = lintcheck-success = testcheck-success = test-broken-linkslabel = ignore-broken-linkscheck-success=Cloudflare Pages-head-repo-full-name~=^Mergifyio/Required conditions to stay in the queue:
author != dependabot[bot]author != mergify-ci-botgithub-review-approved[🛡 GitHub repository ruleset ruleRequire pull request for default branch] (documentation)Enforce conventional commit]:title ~= ^(fix|feat|internal|docs|style|refactor|perf|test|build|ci|chore|revert|ui)(?:\(.+\))?!?:👀 Review Requirements]:#approved-reviews-by >= 2author = dependabot[bot]author = mergify-ci-bot📕 PR description]:body ~= (?ms:.{48,})🔎 Reviews]:#changes-requested-reviews-by = 0#review-requested = 0#review-threads-unresolved = 0🤖 Continuous Integration]:check-success = buildcheck-success = lintcheck-success = testcheck-success = test-broken-linkslabel = ignore-broken-linkscheck-success=Cloudflare Pages-head-repo-full-name~=^Mergifyio/