From 6675615882df0b68271e58883aba156cf1e4df21 Mon Sep 17 00:00:00 2001 From: iancooper Date: Wed, 12 Aug 2026 21:28:35 +0100 Subject: [PATCH] =?UTF-8?q?spec:=20010=20Task=2010.4=20=E2=80=94=20rule=20?= =?UTF-8?q?7,=20and=20the=20eight=20sentences=20it=20found?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit D6 did not survive Task 10.3, so the rule lands as pagelint rule 7, with CLAUDE.md's ledger and its two new convention sections in the same commit — AC5 of spec 011 failed on exactly that parity gap. Six labels, not the four the task named: the two extra check the description front matter against the sentence it is meant to be. The ledger's first draft had that parity as "review only", which is the one thing this programme has learned not to write down. The ruling the task asked for: the 200 characters are rendered, not source. Eleven pages were over the limit as typed and four are over it as read — seven were over only by the length of URLs nobody sees, and shortening real prose to accommodate an href would have made those pages worse. First run: 8 findings across 8 pages. Four too long, four not unique in two pairs, each an async page that had never been given its own opening line. Design §9.2 predicted double figures; it is eight, recorded as a wrong figure rather than rounded up to meet it. The instrument was wrong twice in one task and in opposite directions. A scratch script written before the rule reported fifteen pages; seven were the source/rendered difference and two were its own bug — it skipped any line starting with * as a list item, so on two scheduler pages it skipped a bold opening sentence and reported a heading-follower two paragraphs down. Eight branches proved red before trusted green, and the red-proof refined a lesson this programme already had. "Assert that your mutation landed" is not strong enough: the first three probes reported SILENT and all three were the probe's fault, each asserting only that the file had changed while producing something the branch was never meant to reject. The assertion has to be that the mutation produced the input the branch rejects. One defect in the rule itself, caught only because the red-proof printed a baseline first and the baseline was red: front_matter_description returned a line number into the slot its caller reads as a reason, so a correct page reported DESCRIPTION UNREADABLE: 2 — the check reporting its own success as a failure. Ledger parity by enumeration found the mirror of spec 011's original defect: NO H1 was in the ledger and missing from the linter's own docstring. The same rule, the other way round, four sessions later. Two defects found in passing and left for Phase 11: 29 internal .html links across 16 pages that linkcheck cannot see, one of them pointing at a page that does not exist; and a second H1 in BuildingAnAsyncPipeline.md that no rule can see. Four of the 29 were fixed here because they sat inside sentences this task was rewriting, with their anchors repointed at headings that exist. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_012tcdwxVb8NmKaX2S6fvyFg --- CLAUDE.md | 78 ++++++++ contents/AsyncDispatchARequest.md | 2 +- contents/BuildingAPipeline.md | 5 +- contents/BuildingAnAsyncPipeline.md | 5 +- contents/CQRSWithBrighterAndDarker.md | 2 +- contents/DispatchingARequest.md | 2 +- contents/OutboxPattern.md | 2 + contents/PolicyFallback.md | 2 +- contents/WhyBrighter.md | 2 +- spec/010-information_architecture/tasks.md | 101 ++++++++++ tools/pagelint.py | 222 ++++++++++++++++++++- 11 files changed, 412 insertions(+), 11 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 5cc9401..f6d66d3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -277,8 +277,80 @@ never silenced. It still counts towards the debt and still says so in its own wo Without that, relocating a block verbatim is indistinguishable from writing a new one, and a page split cannot honour "move the text, do not improve it". +### The opening sentence + +Every page's first sentence after the banner has to survive being read on its own. + +That sentence is not decoration and it is not only for a reader who has already +arrived. It is what GitBook's `.md` variant leads with after the banner, what the +canonical `/llms.txt` prints after the page's title, and what a search engine shows +as the page's snippet — all from the same string. A reader meets it before they meet +the page. + +So it must exist, be **at most 200 characters as rendered**, not end in a colon, and +not be identical to another page's. + +**Rendered, not as typed.** `[Configuration](/contents/BrighterBasicConfiguration.md)` +is fourteen characters to a reader and fifty-four in the source. Measuring the source +fails pages on the length of a URL nobody reads — seven of the eleven pages over the +limit on the first run were over it only in that sense. + +**Uniqueness is the clause that earns its keep.** Two pages introducing themselves +with the same sentence are indistinguishable in an index, and the duplicate is almost +always a copied intro nobody rewrote. On this rule's first run it found +`AzureBlobConfiguration.md` — the *Azure Blob* archive provider page — opening with a +paragraph about *Azure Service Bus*, wrong since 2023 and green under every other +check here, because nothing about it was malformed. + +**What it cannot do**, and this is the point of the three clauses it does check: it +cannot tell whether a sentence is *true*. It checks the properties that correlate +with a sentence nobody reread. + +### Page descriptions + +A page carries its summary to retrieval clients in YAML front matter, above the H1: + +```markdown +--- +description: "The Azure Archive Provider writes messages swept from your Outbox into an Azure Blob Storage container." +layout: + description: + visible: false +--- +``` + +**`description:` is the opening sentence with its markdown stripped**, so the two +cannot drift — a parallel table of hand-written summaries would be stale within a +release, and a sentence that lives in the page cannot be. + +**Always quote the value.** GitBook documents that an unquoted `description:` +containing `:`, `#`, `[`, `]`, `{`, `}`, `&`, `*`, `!`, `|`, `>`, `'`, `"`, `%`, `@` +or `` ` `` causes a **silent** Git Sync failure — the page imports with no title and +no description, and no error anywhere. Sentences carry colons. Quote every value. + +**`layout.description.visible: false` is not optional.** It defaults to *visible*, +and GitBook then renders the description as a subtitle under the H1 — so a page whose +description was derived from its own opening sentence would print that sentence +twice. Measured on a live preview revision: the switch takes the subtitle to zero +while all three meta tags stay. + +**Front matter does not carry the banner, and that has not changed.** The banner stays +a visible blockquote, because a retrieval chunker strips front matter and keeps body +text. What did change is the reason that used to sit beside it: *"GitBook renders +front matter literally into the page body"* is false — front matter is consumed and +the H1 survives. One reason of two was wrong for a year, and the conclusion it +supported was right anyway. + ### llms.txt +> **Superseded 2026-08-12, and not yet rewritten.** The format below describes a file +> this repository does not generate. **GitBook owns `/llms.txt`**, builds it from the +> published tree, and cannot be overridden; our space contributes 143 of its entries +> automatically. The platform's line format is `- [Title](url): description`, with **no +> type field** — the type reaches a reader through the banner, which the `.md` variant +> prints one line below the H1. Spec 010 Phase 11 rewrites this section; it is left +> standing meanwhile so the change is reviewed rather than assumed. + `llms.txt` at the repository root indexes the documentation for retrieval systems. The format is defined here; Spec 010 owns the generator, which builds it from `SUMMARY.md` and reads each page's banner for the type — which is why the banner has @@ -334,6 +406,12 @@ failure the claim in this paragraph is meant to prevent: | Language tag on every fence | 4 | error | error | | "Dispatcher", not "ServiceActivator" or "Service Activator", in prose | 5 | error | error | | `using` directives in C# blocks | 6 | warning, counted | error, unless the block marks its omission `// ...` | +| An opening sentence exists | 7 (`SUMMARY MISSING`) | error | error | +| It is ≤ 200 characters **rendered** | 7 (`SUMMARY TOO LONG`) | error | error | +| It does not end in a colon | 7 (`SUMMARY ENDS IN COLON`) | error | error | +| It is unique across pages | 7 (`SUMMARY NOT UNIQUE`) | error | error | +| `description:` front matter equals it | 7 (`DESCRIPTION MISMATCH`) | error | error | +| That front matter is a quoted single line | 7 (`DESCRIPTION UNREADABLE`) | error | error | | Version markers on code (❌/✅) | — | **review only** | **review only** | Version markers are the one convention with no rule, and deliberately so: whether two diff --git a/contents/AsyncDispatchARequest.md b/contents/AsyncDispatchARequest.md index a6f7c59..8ae8621 100644 --- a/contents/AsyncDispatchARequest.md +++ b/contents/AsyncDispatchARequest.md @@ -2,7 +2,7 @@ > **How-to** · Applies to **Brighter V10** -Once you have [implemented your Request Handler](ImplementingAHandler.html), you will want to dispatch **Commands** or **Events** to that Handler. +Once you have [implemented your async Request Handler](/contents/ImplementingAsyncHandler.md), you will want to dispatch **Commands** and **Events** to it through the Command Processor's asynchronous methods. ## Async Dispatch Usage diff --git a/contents/BuildingAPipeline.md b/contents/BuildingAPipeline.md index c1e7a4e..4b22620 100644 --- a/contents/BuildingAPipeline.md +++ b/contents/BuildingAPipeline.md @@ -2,8 +2,9 @@ > **How-to** · Applies to **Brighter V10** -Once you are using the features of Brighter to act as a [command dispatcher](CommandsCommandDispatcherAndProcessor.html#command-dispatcher) and send or publish messages to a target handler, you may want to use -its [command processor](CommandsCommandDispatcherAndProcessor.html#command-processor) features to handle orthogonal operations. +Once Brighter is dispatching requests to your handlers, you can insert orthogonal operations into the pipeline that runs around each handler. + +This uses Brighter's [command processor](/contents/CommandsCommandDispatcherandProcessor.md#the-command-processor-pattern) features rather than its [command dispatcher](/contents/CommandsCommandDispatcherandProcessor.md#command-dispatcher) features. Common examples of orthogonal operations include: diff --git a/contents/BuildingAnAsyncPipeline.md b/contents/BuildingAnAsyncPipeline.md index 60eade2..6b97114 100644 --- a/contents/BuildingAnAsyncPipeline.md +++ b/contents/BuildingAnAsyncPipeline.md @@ -2,8 +2,9 @@ > **How-to** · Applies to **Brighter V10** -Once you are using the features of Brighter to act as a [command dispatcher](CommandsCommandDispatcherAndProcessor.html#command-dispatcher) and send or publish messages to a target handler, you may want to use -its [command processor](CommandsCommandDispatcherAndProcessor.html#command-processor) features to handle orthogonal operations. +An async pipeline wraps an async request handler in the same orthogonal operations a synchronous pipeline uses, with attributes whose handlers await. + +This uses Brighter's [command processor](/contents/CommandsCommandDispatcherandProcessor.md#the-command-processor-pattern) features rather than its [command dispatcher](/contents/CommandsCommandDispatcherandProcessor.md#command-dispatcher) features. # Implementing a Pipeline diff --git a/contents/CQRSWithBrighterAndDarker.md b/contents/CQRSWithBrighterAndDarker.md index c8dadf6..d5acdf8 100644 --- a/contents/CQRSWithBrighterAndDarker.md +++ b/contents/CQRSWithBrighterAndDarker.md @@ -4,7 +4,7 @@ ## CQRS Introduction -Command Query Responsibility Segregation (CQRS) is an architectural pattern that separates the responsibility for handling commands (operations that change state) from queries (operations that read state). **Brighter** and **Darker** together provide a complete, production-ready CQRS solution for .NET applications, where Brighter handles the command side and Darker handles the query side. +Command Query Responsibility Segregation (CQRS) separates the handling of commands, which change state, from queries, which read it. **Brighter** and **Darker** together provide a complete, production-ready CQRS solution for .NET applications, where Brighter handles the command side and Darker handles the query side. This separation allows you to optimize each side independently: write operations can focus on business logic validation and consistency, while read operations can be optimized for performance and specific client needs. Both frameworks use the same architectural patterns (request handlers, middleware pipelines, policy-based resilience), making it easier for teams to work with both sides of a CQRS architecture using consistent patterns. diff --git a/contents/DispatchingARequest.md b/contents/DispatchingARequest.md index 7efcd4f..cf73173 100644 --- a/contents/DispatchingARequest.md +++ b/contents/DispatchingARequest.md @@ -2,7 +2,7 @@ > **How-to** · Applies to **Brighter V10** -Once you have [implemented your Request Handler](ImplementingAHandler.html), you will want to dispatch **Commands** or **Events** to that Handler. +Once you have [implemented your Request Handler](/contents/ImplementingAHandler.md), you will want to dispatch **Commands** and **Events** to it synchronously, through the Command Processor. ## Request Dispatch Usage diff --git a/contents/OutboxPattern.md b/contents/OutboxPattern.md index c4d8dc8..1bbc9d9 100644 --- a/contents/OutboxPattern.md +++ b/contents/OutboxPattern.md @@ -4,6 +4,8 @@ ## Producer Correctness +The Outbox Pattern is what stops a state change and the event announcing it from diverging. + When a microservice changes the state for which it is the system of record, and then signals to subscribers via an event that it has changed its state, how do we ensure that subscribers receive the event and are therefore consistent with the producer? diff --git a/contents/PolicyFallback.md b/contents/PolicyFallback.md index 05c387d..0d9f4a4 100644 --- a/contents/PolicyFallback.md +++ b/contents/PolicyFallback.md @@ -2,7 +2,7 @@ > **How-to** · Applies to **Brighter V10** -You may want some sort of backstop exception handler, that allows you to take compensating action, such as undoing any partially committed work, issuing a compensating transaction, or queuing work for later delivery (perhaps using the [External Bus](/contents/ImplementingExternalBus.md)). +A Fallback is a backstop exception handler, which runs when a request has failed and lets you take compensating action. You might undo partially committed work, issue a compensating transaction, or queue the work for later delivery (perhaps using the [External Bus](/contents/ImplementingExternalBus.md)). To support this we provide a **IHandleRequests\Fallback** method. In the Fallback method you write your code to run in the event of failure. diff --git a/contents/WhyBrighter.md b/contents/WhyBrighter.md index 54a6e40..717dfb9 100644 --- a/contents/WhyBrighter.md +++ b/contents/WhyBrighter.md @@ -2,7 +2,7 @@ > **Explanation** · Applies to **Brighter V10 and Darker V4** -There are many options for .NET developers looking for either a package to use as a command processor/dispatcher pattern implementation (sometimes confused with the mediator pattern) or package to use as a messaging framework. So why would you choose Brighter & Darker? +Brighter and Darker are two of many options open to a .NET developer who needs a command processor, a command dispatcher, or a messaging framework. The command dispatcher pattern is sometimes confused with the mediator pattern. So why would you choose Brighter & Darker? ## Reactor Pattern diff --git a/spec/010-information_architecture/tasks.md b/spec/010-information_architecture/tasks.md index 5cc5fc1..dd4c713 100644 --- a/spec/010-information_architecture/tasks.md +++ b/spec/010-information_architecture/tasks.md @@ -2484,6 +2484,107 @@ extraction discipline moves into Task 10.4's check, which now has something to f - [x] **Task 10.1** — the mechanism established and measured end to end - [x] **Task 10.3** — ruled *fix it at source*; AC9 narrowed in all three documents +### Task 10.4 as executed — 2026-08-12 + +D6 did not survive Task 10.3, so the rule landed **as `pagelint.py` rule 7**, exactly as the +task's fallback prescribed — and in the same commit as `CLAUDE.md`'s ledger and its two new +convention sections, because AC5 of Spec 011 failed on precisely that parity gap. + +**Six labels, not four.** The four the task named — `SUMMARY MISSING`, `SUMMARY TOO LONG`, +`SUMMARY ENDS IN COLON`, `SUMMARY NOT UNIQUE` — plus **`DESCRIPTION MISMATCH`** and +**`DESCRIPTION UNREADABLE`**, which check the front matter against the sentence it is supposed +to be. The ledger's first draft had that parity as *review only*, which is the one thing this +programme has learned not to write down: **a rule with a tool for half of it will pass the +half nobody checks.** It is mechanical, so it is mechanised. + +**The ruling the task asked for: the 200 characters are *rendered*.** `rendered()` strips link +targets, brackets, code spans and emphasis before measuring. It matters more than it sounds: + +| | source | rendered | +|---|---:|---:| +| pages over the limit | 11 | **4** | + +Seven pages were over the limit **only by the length of URLs nobody reads**. Failing a page on +its hrefs would have been measuring the wrong thing, and the fix — shortening real prose to +accommodate a link target — would have made those pages worse. + +**The first run: 8 findings across 8 pages.** Four `SUMMARY TOO LONG` +(`PolicyFallback.md` 249, `WhyBrighter.md` 226, `OutboxPattern.md` 211, +`CQRSWithBrighterAndDarker.md` 205) and four `SUMMARY NOT UNIQUE` in two pairs — +`DispatchingARequest.md`/`AsyncDispatchARequest.md` and +`BuildingAPipeline.md`/`BuildingAnAsyncPipeline.md`, each an async page that had never been +given its own opening line. **Zero `SUMMARY MISSING` and zero `SUMMARY ENDS IN COLON`.** + +**Design §9.2 predicted "a double-figure number of failures" and it is eight.** The prediction +was made against source length, and against a corpus this spec has since improved. Recorded as +a wrong figure rather than rounded up to meet it. + +> **The instrument was wrong twice in one task, in opposite directions.** A throwaway +> measurement script written before the rule reported **15 pages** — and both of its extra +> findings were its own. Seven were the source/rendered difference above. The other two, +> `AwsScheduler.md` and `AzureScheduler.md` *"ending in a colon"*, were an artefact of the +> script skipping any line starting with `*` as a list item: both pages open +> `**AWS EventBridge Scheduler** is a fully managed…`, bold, not a bullet. The script skipped +> the real introduction and reported the heading-follower two paragraphs down. **The corpus +> was fine and the ruler was bent** — which is *a measurement is only as good as its +> instrument*, met again, and the reason the scratch figure is not quoted anywhere as a result. + +**Proved red before trusted green, eight branches.** `SUMMARY ENDS IN COLON`, +`SUMMARY TOO LONG`, `SUMMARY MISSING`, `DESCRIPTION MISMATCH`, and `DESCRIPTION UNREADABLE` +three ways (unquoted, block scalar, unterminated block). `SUMMARY NOT UNIQUE` needed no probe: +it had four live findings. + +> **And the red-proof refined a lesson this programme already had.** *"When you prove a check +> red, assert that your mutation landed"* is **not strong enough.** The first three probes all +> reported SILENT, and **all three were the probe's fault, not the rule's** — each asserted +> only `text != orig`, which was true, while the mutation produced something the branch was +> never meant to reject: replacing one sentence left the *next* sentence on the line, so the +> summary no longer ended in a colon; deleting the intro paragraph simply promoted the +> following one; the "too long" string was still under 200 rendered. **The assertion has to be +> that the mutation produced the *input the branch rejects*** — here, by calling +> `opening_sentence()` directly and asserting the property before running the linter. Three +> real rules would have been recorded as broken on the strength of the weaker guard. + +**One defect in the rule itself, caught by its own baseline.** `front_matter_description()` +returned `(value, lineno)` on success into a caller reading the second slot as a *reason*, so +a page whose front matter was perfectly correct reported +`DESCRIPTION UNREADABLE: 2` — **the check reporting its own success as a failure, with a line +number for a message.** It was visible only because the red-proof printed a baseline first and +the baseline was red. A run that started at the first mutation would have seen three FIREDs +and called it proven. + +**Ledger parity checked by enumeration, and it found the mirror of the original defect.** Spec +011's acceptance pass found `NO H1` in the linter and missing from `CLAUDE.md`'s ledger. Here +it is in the ledger and **was missing from the linter's own docstring** — the same gap, the +same rule, the other way round, four sessions later. Added. *To check parity, enumerate; do +not read* — and enumerate **both** directions. + +**Two defects found in passing, recorded and deliberately not fixed. Both are Phase 11's.** + +1. **29 internal `.html` links across 16 pages, and `linkcheck.py` cannot see any of them.** + Legacy links from the pre-GitBook site — `ImplementingAHandler.html`, + `CommandsCommandDispatcherAndProcessor.html#command-processor` — which the link checker + skips because they are not `.md`. Twelve distinct targets. **Eleven have an obvious `.md` + counterpart and one, `QualityOfServicePatterns.html`, has no page at all**, so it is a dead + link that has been green in every CI run this repo has ever done. Worse, the conversion is + not purely mechanical: the file is `CommandsCommandDispatcherandProcessor.md`, lower-case + `a`, so the obvious rewrite lands on **WRONG CASE** — the fault class `linkcheck.py` exists + for, hidden behind an extension it does not check. **Four of the 29 were fixed here**, + because they sat inside sentences this task was rewriting anyway, and their anchors were + repointed at headings that exist (`#the-command-processor-pattern`, not + `#command-processor`, which never existed). The other 25 want their own PR. +2. **`BuildingAnAsyncPipeline.md` has two H1s** — line 1 and line 8. No rule sees it: `NO H1` + fires only on *no* H1, and rule 3b starts at `##`. It is the H1 collision Phase 6 recorded + as invisible, in a page nobody split. Nothing links to the second one's anchor, so demoting + it is safe — but it becomes a `##` that must then be unique and qualified, which is a + heading pass, not a sentence fix. + +**Gates after:** linkcheck 144 files, pagelint **0 errors** / 791 warnings / 142 pages, +`--check-shape` 0, `--check-redirects` 0, `--changed origin/master` 0 with everything staged. +**The using-directive debt did not move**, as in Phases 6 through 9. + +- [x] **Task 10.4** — rule 7 in `pagelint.py`, `CLAUDE.md` in the same commit, 8 pages fixed + --- ## Phase 11 — Close (PR 11) diff --git a/tools/pagelint.py b/tools/pagelint.py index ea413bc..112053a 100644 --- a/tools/pagelint.py +++ b/tools/pagelint.py @@ -7,6 +7,7 @@ Rules, and what each is for: + NO H1 a page with no title for a banner to sit under BANNER MISSING no page banner on the first non-blank line after the H1 BANNER MALFORMED a banner is there, but not in the fixed grammar HEADING NOT UNIQUE a `##` text that also appears on another page @@ -15,11 +16,35 @@ SERVICEACTIVATOR "ServiceActivator" in prose where "Dispatcher" is meant USING DIRECTIVES a C# block with no `using` lines (warning, counted; stays a warning under --changed if marked `// ...`) + SUMMARY MISSING no prose after the banner to summarise the page with + SUMMARY TOO LONG an opening sentence over 200 characters, rendered + SUMMARY ENDS IN COLON an opening sentence promising a list an index drops + SUMMARY NOT UNIQUE two pages that introduce themselves identically + DESCRIPTION MISMATCH `description:` front matter that is not that sentence + DESCRIPTION UNREADABLE front matter this tool will not guess the meaning of The banner states the page type, the Brighter/Darker version it applies to and what to read first. It is a visible blockquote rather than front matter because -GitBook renders front matter literally into the page body, and because a -retrieval chunker strips front matter but keeps body text. +a retrieval chunker strips front matter but keeps body text. + + One of that sentence's two reasons used to be "because GitBook renders front + matter literally into the page body", and it was wrong -- or has stopped + being true, which from here is the same thing. Measured on a live preview + revision 2026-08-12 (spec 010 Phase 10): front matter is consumed, the H1 + survives, and `description:` becomes the page's meta, og and twitter + description. There is even a switch, layout.description.visible, for whether + it also renders as a subtitle. The conclusion is untouched -- the banner + stays a blockquote, on the surviving reason, and it demonstrably reaches the + .md variant -- but a premise nobody rechecked for a year was false, so it is + corrected here rather than left to be inherited again. + +Rule 7 is the opening sentence, and it is the one rule here that is not about +form. The other six ask whether a page is well-formed; this asks whether its +first sentence is true and specific enough to stand alone, because that +sentence is what a reader meets as a search snippet and what /llms.txt prints +after the title. It cannot check truth, so it checks the three things that +correlate with a sentence nobody reread -- too long, ends in a colon, or +identical to another page's -- and the third is the one that earns its keep. Heading uniqueness is two rules with deliberately different scopes. Across pages it applies to `##` only: an H3 is read under its H2, so @@ -153,6 +178,12 @@ INLINE_CODE_RE = re.compile(r'`[^`]*`') LINK_TARGET_RE = re.compile(r'\]\([^)]*\)') + +# Rule 7's ceiling, measured on what a reader sees rather than what the author +# typed -- see rendered(). Two hundred is where design 010 §9.2 put it and it +# has held: a sentence longer than this stops being a summary and starts being +# the first half of a paragraph, which is no use as a search snippet. +SUMMARY_LIMIT = 200 OPT_OUT = '' # Both spellings. The API surface uses the closed form, but prose in this repo @@ -643,6 +674,192 @@ def check_terminology(page): return findings +# -------------------------------------------------------------------------- +# Rule 7 — the opening sentence, which is also the page's summary +# -------------------------------------------------------------------------- + +def rendered(text): + """What a reader sees: link text without its target, no emphasis marks. + + The 200-character limit measures this, not the markdown. A sentence can + carry a 90-character documentation URL that nobody reads, and failing a + page on the length of a href would be measuring the wrong thing. Measured + when this rule was written: one sentence came to 199 characters of source + and about 140 rendered, one character under a limit it was never really + near. + """ + text = LINK_TARGET_RE.sub(']', text) # [text](url) -> [text] + text = re.sub(r'\[([^\]]*)\]', r'\1', text) # [text] -> text + text = re.sub(r'`([^`]*)`', r'\1', text) + text = re.sub(r'\*\*([^*]*)\*\*', r'\1', text) + text = re.sub(r'\*([^*]*)\*', r'\1', text) + return text.strip() + + +def first_sentence(text): + """Up to the first sentence-ending period, else the whole line. + + The lookbehinds keep "e.g." and "i.e." from ending a sentence mid-clause; + without them a page opening "Brighter supports several brokers, e.g. Kafka." + would be summarised as four words. + """ + match = re.search(r'(?'): + continue + return (first_sentence(text), lineno), None + return None, 'the page has no prose after its banner' + + +def front_matter_description(page): + """(value, None) from `description:` front matter, or (None, why|None). + + Both slots are returned every time and the second is always a *reason*, + never a line number. An earlier draft returned the line number on success, + which every caller reads as "something went wrong" -- the check reported + its own success as a failure, with the line number as the message. + + Deliberately not a YAML parser. Only the single-line, quoted form this + repository writes is understood, and anything else is reported rather than + guessed at -- a wrong guess here is worse than no check, because GitBook's + own failure mode for malformed front matter is silent. + """ + if not page.lines or page.lines[0].strip() != '---': + return None, None + close = None + for lineno in range(2, len(page.lines) + 1): + if page.lines[lineno - 1].strip() == '---': + close = lineno + break + if close is None: + return None, 'the front matter block is never closed' + for lineno in range(2, close): + line = page.lines[lineno - 1] + if not line.startswith('description:'): + continue + value = line[len('description:'):].strip() + if value[:1] in ('>', '|'): + return None, ('the description uses a YAML block scalar; write it as one ' + 'quoted line so it can be compared with the page') + if not (len(value) >= 2 and value[0] == value[-1] and value[0] in '"\''): + return None, ('the description is unquoted; GitBook fails Git Sync ' + 'silently on an unquoted value containing a colon') + return value[1:-1], None + return None, None + + +def check_summaries(pages, reported): + """Rule 7. Every page opens with a sentence fit to stand alone as its summary. + + That sentence is not decoration. It is what GitBook's `.md` variant leads + with after the banner, and — once it is carried into `description:` front + matter — what the canonical /llms.txt prints after the page's title. A + reader meets it as a search snippet before they ever meet the page. + + Uniqueness is checked across the corpus for the same reason `##` headings + are: two pages with the same opening sentence are indistinguishable in an + index, and the duplicate is usually a copied intro somebody forgot to + rewrite. On this rule's first run that is exactly what it found — a page + about the Azure Blob archive provider opening with a paragraph about Azure + Service Bus, wrong since 2023 and invisible to every other check here, + because nothing about it is malformed. + """ + findings = [] + by_sentence = defaultdict(list) + + for rel in sorted(pages): + got, why = opening_sentence(pages[rel]) + if got is None: + if rel in reported: + findings.append(error( + rel, 1, 'SUMMARY MISSING', + f'{why}; add an introductory sentence below the banner, so the ' + 'page has something to be summarised by')) + continue + sentence, lineno = got + by_sentence[rendered(sentence)].append((rel, lineno)) + if rel not in reported: + continue + shown = rendered(sentence) + if len(shown) > SUMMARY_LIMIT: + findings.append(error( + rel, lineno, 'SUMMARY TOO LONG', + f'the opening sentence is {len(shown)} characters rendered; ' + f'the limit is {SUMMARY_LIMIT}. Split it — the first sentence ' + 'has to survive being read on its own, in a search result')) + if shown.endswith(':'): + findings.append(error( + rel, lineno, 'SUMMARY ENDS IN COLON', + 'the opening sentence ends in a colon, so it promises a list ' + 'that an index will not print. Say what the page is about, then ' + 'introduce the list')) + + # Parity with the front matter, on pages that carry one. The whole point + # of deriving the description from the page is that the two cannot + # drift; nothing enforces that unless something compares them. + described, why = front_matter_description(pages[rel]) + if why: + findings.append(error(rel, 1, 'DESCRIPTION UNREADABLE', why)) + elif described is not None and described != shown: + findings.append(error( + rel, lineno, 'DESCRIPTION MISMATCH', + 'the `description:` front matter is not this page\'s opening ' + f'sentence.\n front matter: {described}\n opening line: {shown}')) + + for shown, where in sorted(by_sentence.items()): + if len(where) < 2: + continue + others = [rel for rel, _ in where] + for rel, lineno in where: + if rel not in reported: + continue + elsewhere = [o for o in others if o != rel] + findings.append(error( + rel, lineno, 'SUMMARY NOT UNIQUE', + 'this opening sentence also opens ' + f'{_also_in(elsewhere, limit=3)}. Two pages that introduce ' + 'themselves identically are indistinguishable in an index — and ' + 'one of them is usually about something else')) + return findings + + # -------------------------------------------------------------------------- # --changed # -------------------------------------------------------------------------- @@ -781,6 +998,7 @@ def main(argv): findings += check_code_blocks(page, strict.get(rel, [])) findings += check_terminology(page) findings += check_headings(pages, reported) + findings += check_summaries(pages, reported) findings.sort(key=lambda f: (f.path, f.line, f.rule))