diff --git a/docs/generation/README.md b/docs/generation/README.md index 03831595..4be316bb 100644 --- a/docs/generation/README.md +++ b/docs/generation/README.md @@ -15,6 +15,7 @@ Both can subscribe to shared values prepared by the [data pipeline](../data/). | Archives, tag indexes, or HTML redirects with page variables and layouts | `*.pages.ts` | One or more DOMStack pages | | Feeds, sitemaps, JSON, text, or fully controlled output | `*.template.ts` | One or more files, without layout wrapping | | An ordinary page with its own source directory and browser assets | [Page files](../pages/#page-files) | A source-backed page | +| Markdown downloads, JSON metadata, or other extra files owned by a source-backed page | [`pageOutputs`](../pages/#page-outputs) | Extra files alongside the page's HTML | ## Table of Contents @@ -48,6 +49,8 @@ A generated-pages module can default-export: | An async iterable, usually returned by `async function*` | Pages are discovered incrementally or the total is not known in advance | Static objects and arrays do not receive factory parameters. +A `null` or `undefined` default export or factory result produces no pages, as does an empty array or async iterable. +Each array entry or yielded value must still be a page-definition object; `null` entries are not skipped. #### One page definition @@ -138,6 +141,29 @@ export default async function * archivePages ({ data }) { } ``` +#### Streaming and failures + +DOMStack consumes generated pages one at a time rather than collecting all definitions before building them. +For each definition, it validates the definition and output path, initializes the page's variables, layouts, and declared data, then renders and writes the HTML before requesting the next definition. +An async generator resumes after `yield` only once that page has been written, so it can release resources associated with the completed page before preparing the next one. +Arrays and single-object results use the same per-page pipeline, although an array-producing factory necessarily creates its array before returning it. + +Definitions marked `draft: true` are skipped unless drafts are enabled. +Skipped drafts retain their position in source identifiers: after a skipped first definition, the next page is identified as `archive.pages.ts#1`, not `archive.pages.ts#0`. +Output paths must be unique across generated pages and must not collide with source-backed pages. +Do not rely on the processing order of sibling factory modules. + +A factory, validation, initialization, or render failure stops the active iterator without requesting later definitions. +Async generators can use `try`/`finally` to release resources when iteration stops. +Earlier completed pages remain written; generated-page builds are not transactional, and a late failure does not roll back earlier HTML. +Conflict errors still identify both sources even when one page has already been written. +Build results retain output metadata and the owning pages-file path for completed pages, including when the build fails. + +In watch mode, failed builds retain both previously owned outputs and any newly written partial outputs without stale-output cleanup. +A later successful rebuild removes obsolete outputs, including partial pages from repeated failures or an initially failed watch build. +Removing the factory or changing it to return no pages also cleans up its tracked outputs after a successful rebuild. +This ownership tracking does not require a public DOMStack manifest and does not remove outputs still owned by sibling factories. + ### Generated-pages factory parameters Functions receive one object with: @@ -150,6 +176,8 @@ Functions receive one object with: Factories do not receive raw source or generated `PageData` collections. Put page-collection logic in [`global.data.ts`](../data/#global-data), return a focused serializable value, and subscribe to its key from the factory. +Source-backed pages remain fully initialized before `global.data.ts` runs, so it can inspect their resolved variables and use their rendering methods subject to the normal data-dependency rules. +Generated pages are not included in that collection, even after earlier yielded pages have been written. This keeps factories downstream of source discovery without exposing generation order or creating page-generation cycles. ### Generated page definitions @@ -163,6 +191,8 @@ This keeps factories downstream of source discovery without exposing generation Generated pages use [global bundles](../global-bundles/) and [layout assets](../layouts/#layout-styles). They do not have page-local `style.css`, `client.js`, or worker entries because they do not have their own source-page directory. +Support for [page outputs](../pages/#page-outputs) on generated pages is deferred. +Generated pages skip all `pageOutputs` hooks, including hooks inherited from layouts. ### Generated-pages types diff --git a/docs/layouts/README.md b/docs/layouts/README.md index 7f887c52..1d0c7752 100644 --- a/docs/layouts/README.md +++ b/docs/layouts/README.md @@ -63,6 +63,7 @@ DOMStack recognizes these exports from a layout module: | `default` | Yes | A synchronous or asynchronous [layout render function](#layout-render-function). | | `vars` | No | An object, or a sync/async function returning an object, providing [layout defaults](#layout-variables). | | `parentLayout` | No | A non-empty string naming the immediate outer layout; see [Declaring nested layouts](#declaring-nested-layouts). | +| `pageOutputs` | No | A build-only function returning [extra files for each source page](../pages/#page-outputs), such as Markdown downloads or JSON metadata. | ## Declaring nested layouts @@ -104,6 +105,45 @@ See [Data subscriptions in nested layouts](../cookbook/nested-layouts/#data-subs See [Compose nested layouts](../cookbook/nested-layouts/) for a complete example and asset guidance. +## Page outputs + +Export `pageOutputs` from a layout to produce extra files for each source-backed page that uses it. +This is the same export used by page modules and vars companions, and it receives `{ page, vars, data }`. +The function runs for each page, and that page owns the returned files. +For example, a documentation layout can publish a Markdown download alongside each rendered page: + +```js +// src/docs.layout.js +export default ({ children }) => children + +export async function pageOutputs ({ page, vars }) { + if (page.type !== 'md' || vars.rawExport === false) return [] + return { + outputName: page.outputName.replace(/\.html$/, '.source.md'), + content: await page.readMarkdownContent(), + } +} +``` + +The filename is relative to the current page's output directory, not the layout directory. +Using the page's HTML filename helps keep destinations unique when several loose Markdown pages share a directory. +Exact duplicate destinations produce best-effort build warnings, not an override contract; avoid sharing output paths between pages or hooks. +Set `rawExport: false` in a page's frontmatter or vars to opt out of this layout's Markdown download. +Returning `[]` from a page hook does not suppress layout files; the layout itself must check the opt-out variable. + +Nested hooks run outermost layout → innermost layout → selected page-level hook, and their files are additive. +If a JS/TS page module and its vars companion both export `pageOutputs`, the page module's hook wins with a warning; layout hooks still run. +Each layout hook receives the fully resolved page `vars` and only that layout renderer's `vars.dataDeps` subscriptions in `data`. +Declare data needed by the hook in the same subscriptions used by the layout render function. + +Hooks may return a `{ outputName, content }` record, an array of records, or an async iterable of records, directly or through a promise. +DOMStack validates and processes each file before requesting the next record, writing it or retaining an unchanged file during watch rebuilds. +A later hook runs only after the preceding hook's files have been processed. +Files are written directly, without transactions or rollback; if a later record or hook fails, earlier writes remain in the destination. +Watch mode tracks these files for cleanup after a successful rebuild or source removal. +Support for generated `*.pages.*` pages is deferred; they skip these hooks, including inherited layout hooks. +See [Page outputs](../pages/#page-outputs) for the complete arguments, public types, path rules, and watch behavior. + ## Layout variables Layouts may also export an optional [`vars` variable provider](../pages/#variable-providers) containing defaults for pages that use the layout: diff --git a/docs/pages/README.md b/docs/pages/README.md index b3868307..a36ccf14 100644 --- a/docs/pages/README.md +++ b/docs/pages/README.md @@ -319,7 +319,172 @@ export default async () => { Page variable files have higher precedence than `global.vars.ts` variables, but lower precedence than frontmatter or `vars` exports from `ts` pages. See [Variables](../../docs/pages/#variables) for the full variable cascade. -### Draft pages +## Page outputs + +Use `pageOutputs` to publish extra files for a source-backed page, such as a Markdown download or a JSON metadata file alongside its HTML. +Export this function from a JS/TS page module, the page's directly associated vars companion, or a [layout](../layouts/#page-outputs). +DOMStack calls the function during the page build and writes the files it returns. +Return output records rather than writing to the destination yourself. + +For example, this page subscribes to `siteMetadata` returned by [`global.data.ts`](../data/#global-data) and writes `article/metadata.json` beneath the destination as well as its normal HTML: + +```js +// src/article/page.js +export const vars = { + title: 'An article', + dataDeps: ['siteMetadata'], +} + +export default ({ vars }) => `

${vars.title}

` + +export const pageOutputs = ({ vars, data }) => ({ + outputName: './metadata.json', + content: JSON.stringify({ title: vars.title, site: data.siteMetadata }), +}) +``` + +### Companion hooks + +For Markdown and HTML pages, put `pageOutputs` in the adjacent `page.vars.js` or `page.vars.ts` companion. +Keep page variables in the companion's default export and export the hook separately: + +```js +// src/article/page.vars.js (alongside page.md) +export default { title: 'An article' } + +export async function* pageOutputs ({ page, vars }) { + yield { + outputName: './source.md', + content: await page.readMarkdownContent(), + } + yield { + outputName: './metadata.json', + content: JSON.stringify({ title: vars.title }), + } +} +``` + +For an HTML page, use the same companion export to return text or JSON rather than calling the Markdown-only `readMarkdownContent()` method. +JS/TS pages can also use a vars companion. +Only the directly associated companion provides a page-level hook; global vars and inherited directory vars do not provide hooks. +If both a JS/TS page module and its companion export `pageOutputs`, DOMStack uses the page module's hook and warns with both provider names. +The companion's hook does not run, but applicable layout hooks still run. + +### Hook arguments and results + +Every hook receives `{ page, vars, data }`: + +- `page` is a read-only handle to the current source page, including `type`, `path`, `url`, `outputName`, `outputRelname`, `draft`, and read-only `pageFile` metadata. + Its `readMarkdownContent()` method reads the Markdown body with YAML frontmatter removed, without rendering Markdown, substituting Handlebars, or rewriting links. + The method throws for non-Markdown pages. +- `vars` contains the fully resolved page variables and is read-only. +- `data` contains only the declaring renderer's subscribed global data. + Page-module and companion hooks share the page renderer's subscriptions; each layout hook shares that specific layout renderer's subscriptions. + Declare required keys through the renderer's existing `vars.dataDeps`, `dataDeps` in a companion's default vars object, or page frontmatter. + Each layout declares its own keys through its `vars.dataDeps`. + Undeclared keys are unavailable, even if another hook subscribes to them. + See [Data subscriptions](../data/#data-subscriptions). + +A hook returns one `{ outputName: string, content: string }` record, an array of records, or an async iterable of records, directly or through a promise. +Use a single record for one file, an array for a fixed set, or an async generator to produce files incrementally. +`outputName` must be a non-empty file path, and `content` must be a string; serialize JSON with `JSON.stringify()`. +Bare strings, `null`, and `undefined` are not valid results. +Return `[]` or yield no records when the hook has no files to produce. + +### Composition and opt-out + +Applicable hooks run in outermost layout → innermost layout → selected page-level hook order. +Layout outputs and page-level outputs are additive; the page hook does not replace layout outputs. +Returning `[]` from the page hook only skips that hook's files. +To let a page opt out of a layout's files, have the layout inspect a resolved variable such as `rawExport: false` and return `[]` itself, as in the [layout example](../layouts/#page-outputs). +Each layout that supports the opt-out must check that variable. + +Hooks run only when building the owning page's output, not when collection or global-data code calls `renderInnerPage()` or `renderFullPage()`. +Support for generated `*.pages.*` pages is deferred; those pages skip all `pageOutputs` hooks, including inherited layout hooks. + +### Streaming results + +DOMStack consumes records sequentially rather than buffering all hook results. +It validates each record and its destination, then writes the file or retains an unchanged file before requesting the next record. +An async generator resumes after `yield` only once that file has been processed, so it can release resources before preparing the next one. +Arrays use the same per-record processing, but the hook must create the array before returning it. +A later layout or page hook starts only after the preceding hook's files have been processed. + +### Page-output types + +Import `PageOutputsFunction` from `@domstack/static/types.js` to type a hook, where `T` is the resolved variables shape and `D` is the declaring renderer's subscribed data shape: + +```ts +// src/article/page.vars.ts +import type { PageOutputsFunction } from '@domstack/static/types.js' + +type ArticleVars = { title: string } +type ArticleData = { siteMetadata: { name: string } } + +export default { + title: 'An article', + dataDeps: ['siteMetadata'], +} + +export const pageOutputs: PageOutputsFunction = ({ vars, data }) => ({ + outputName: './metadata.json', + content: JSON.stringify({ title: vars.title, site: data.siteMetadata.name }), +}) +``` + +`PageOutput` types an individual output record, and `PageOutputsResult` describes the record, array, or async iterable returned by a hook. +`PageOutputsFunctionParams` types the argument object, and `PageOutputsPage` types its read-only `page` handle. +Types describe the values but do not subscribe to global data; keep the runtime `dataDeps` declaration. + +### Output paths and failures + +Output names resolve beneath the configured destination, including custom destinations: + +| Output name | Resolution for a page at `docs/article/index.html` | +| --- | --- | +| `metadata.json` or `./metadata.json` | `docs/article/metadata.json` | +| `../source/article.md` | `docs/source/article.md` | +| `/raw/article.md` | `raw/article.md` at the destination root | + +A leading `/` means destination-root-relative, never filesystem-absolute. +Relative paths use the current page's output directory even when a layout declares them. +Parent traversal is allowed only while the resolved target remains inside the destination. +Use portable file names; drive-letter paths, UNC paths, reserved names, and paths ending in a separator are invalid. +Targets that escape the destination, traverse symlinks, or name a directory instead of a file fail the build. + +Duplicate destinations produce best-effort warnings based on build output reports, including exact duplicates between hooks and conflicts with normal HTML, other pages, templates, copied assets, or bundles. +They do not reject the build, even when content differs. +Watch warnings cover outputs observed in the current page/template phase and may miss conflicts with files from earlier builds or names that differ only in case. +Choose unique destinations; do not rely on write order or cleanup behavior for conflicting outputs. + +Page outputs are written directly to the destination as their records arrive. +If a later hook, iterator step, output validation, or write fails, earlier writes remain, including updates to existing files and newly created files. +Processing stops at the failure without requesting subsequent records or invoking later hooks. +Async generators can use `try`/`finally` to release resources when iteration stops. +The page's HTML is rendered before its hooks run and written only after they succeed, so a hook failure leaves any previous HTML in place. +Writes are not transactional and are not rolled back: other pages and build phases may already have written their outputs, and filesystem write failures can leave partial updates. + +### Watch behavior and ownership + +Page outputs belong to the source page and appear in page build reports and, when enabled, the build manifest with kind `page-output`. +Ownership tracking and cleanup also work when public build-manifest generation is disabled. +A failed build retains previously tracked files and tracks any files successfully written or retained unchanged before the failure; it does not clean up the page's old outputs. +After a successful rebuild, DOMStack removes previously owned files no longer returned, including renamed outputs, files from removed hooks, and partial outputs retained from failed attempts. +Files written before a failure are tracked even when the initial watch build fails, so recovery, hook removal, or source deletion can clean them up. +Source deletion, source rename, or draft exclusion also removes the page's old outputs. +Adding, editing, removing, or renaming a companion updates the owning page's hook and output set. + +Hooks rerun when the owning page rebuilds, including changes to applicable page or layout data subscriptions. +A dependency used only by a hook still triggers an HTML rebuild because both outputs rebuild together. +During watch rebuilds, DOMStack skips rewriting a previously written page output when its content matches and the destination's filesystem metadata has not changed. +These unchanged files remain owned by the page and keep their modification times. +Missing files or files with changed metadata are written again. +On a fresh build, page outputs are written even if identical files already exist. + +An article body edit can update that article's HTML and Markdown download without rebuilding sibling pages, while a shared data change can rerun affected hooks without rewriting their unchanged files. +Use [templates](../generation/#templates) for collection-wide search indexes and feeds, and page outputs for per-page files. + +## Draft pages A complete draft page can use the same colocated files as a published page: diff --git a/index.js b/index.js index 35074735..492ae6c1 100644 --- a/index.js +++ b/index.js @@ -11,6 +11,7 @@ * @import { Logger as PinoLogger } from 'pino' * @import { DomstackManifestRecord } from './lib/domstack-manifest/index.js' * @import { WatchDependencyState } from './lib/build-pages/watch-dependencies.js' + * @import { PageOutputCache } from './lib/build-pages/page-builders/page-output-writer.js' * @import { WatchSnapshot, WatchEvent, WatchPlan } from './lib/watch-plan.js' * @typedef {{ dispose: () => Promise }} DisposableBuildContext * @typedef {{ pageFilePath: string, sourcePageFilePath?: string | undefined, pagesFilePath?: string | undefined, layoutNames: string[], outputs?: DomstackManifestRecord[] | undefined }} WatchedPageReport @@ -22,10 +23,10 @@ */ import { once } from 'events' import assert from 'node:assert' -import { mkdtemp, readFile, rm } from 'node:fs/promises' +import { lstat, mkdtemp, readFile, rm } from 'node:fs/promises' import { tmpdir } from 'node:os' import chokidar from 'chokidar' -import { basename, join, relative, resolve } from 'node:path' +import { basename, dirname, join, relative, resolve } from 'node:path' // @ts-expect-error import makeArray from 'make-array' import ignore from 'ignore' @@ -101,10 +102,12 @@ export class DomStack { #globalDataDepPaths = new Set() /** @type {Set} absolute filepaths of esbuild entry points */ #esbuildEntryPoints = new Set() - /** @type {Set} destination-relative outputs from the last successful page builds */ - #pageOutputRelnames = new Set() - /** @type {Map>} *.pages.* filepath → owned destination-relative outputs */ - #pagesFileOutputMap = new Map() + /** @type {Map>} source page or *.pages.* filepath → owned absolute output paths */ + #pageOutputMap = new Map() + /** @type {PageOutputCache} Successful writes, including those before an iterator failure. */ + #pageOutputCache = new Map() + /** @type {Map>} template filepath → currently claimed absolute output paths */ + #templateOutputMap = new Map() /** @type {Map>} *.pages.* filepath → layouts used by its generated pages */ #pagesFileLayoutMap = new Map() /** @type {WatchDependencyState | null} subscriptions and fingerprints from the last successful page build */ @@ -280,7 +283,10 @@ export class DomStack { ...this.opts, trackWatchDependencies: true, }) + this.#pageOutputCache = pageBuildResults.report.pageOutputCache ?? this.#pageOutputCache + delete pageBuildResults.report.pageOutputCache if (pageBuildResults.errors.length > 0) { + this.#rememberPartialPageOutputs(pageBuildResults) throw new DomStackAggregateError(pageBuildResults.errors, 'Page build finished but there were errors.', { siteData, pageBuildResults, @@ -291,8 +297,7 @@ export class DomStack { siteData, pageBuildResults, } - this.#pageOutputRelnames = getPageOutputRelnames(pageBuildResults.outputs) - this.#pagesFileOutputMap = getPagesFileOutputMap(pageBuildResults.report.pages) + await this.#removeObsoletePageOutputs(pageBuildResults, false) this.#pagesFileLayoutMap = getPagesFileLayoutMap(pageBuildResults.report.pages) this.#updatePageLayoutNames(pageBuildResults.report.pages, true) this.#pageBuildFailed = false @@ -534,22 +539,24 @@ ${siteData.errors.map(err => ` ${err.message}`).join('\n')}`) ...(templateFilterPaths ? { templateFilterPaths } : {}), ...(pagesFileFilterPaths ? { pagesFileFilterPaths } : {}), previousWatchDependencies: this.#watchDependencies, + previousPageOutputCache: this.#pageOutputCache, trackWatchDependencies: true, }) + this.#pageOutputCache = pageBuildResults.report.pageOutputCache ?? this.#pageOutputCache + delete pageBuildResults.report.pageOutputCache if (pageBuildResults.errors.length > 0) { + this.#rememberPartialPageOutputs(pageBuildResults) throw new DomStackAggregateError(pageBuildResults.errors, 'Page build finished but there were errors.', { siteData, pageBuildResults, }) } const isFiltered = pageFilterPaths !== null || templateFilterPaths !== null || pagesFileFilterPaths !== null + await this.#removeObsoletePageOutputs(pageBuildResults, isFiltered) this.#updatePageLayoutNames(pageBuildResults.report.pages, !isFiltered) if (!isFiltered) { - await this.#removeObsoletePageOutputs(pageBuildResults.outputs) - this.#pagesFileOutputMap = getPagesFileOutputMap(pageBuildResults.report.pages) this.#pagesFileLayoutMap = getPagesFileLayoutMap(pageBuildResults.report.pages) - } else if ((pageBuildResults.report.rebuiltPagesFilePaths?.length ?? 0) > 0) { - await this.#removeObsoleteGeneratedPageOutputs(pageBuildResults.report.rebuiltPagesFilePaths ?? [], pageBuildResults.report.pages) + } else { updatePagesFileLayoutMap(this.#pagesFileLayoutMap, pageBuildResults.report.rebuiltPagesFilePaths ?? [], pageBuildResults.report.pages) } this.#watchDependencies = pageBuildResults.report.watchDependencies ?? this.#watchDependencies @@ -570,61 +577,59 @@ ${siteData.errors.map(err => ` ${err.message}`).join('\n')}`) } /** - * Remove page files that were emitted by the previous successful full build - * but are no longer claimed by the current page or template build. - * - * @param {DomstackManifestRecord[]} outputs + * Failed direct builds can leave new files. Keep their paths alongside prior + * ownership without cleaning anything up until a successful rebuild. + * @param {Pick} results */ - async #removeObsoletePageOutputs (outputs) { - const currentOutputRelnames = new Set(outputs.map(output => output.outputRelname)) - const currentPageOutputRelnames = getPageOutputRelnames(outputs) - const dest = resolve(this.#dest) - - await Promise.all(Array.from(this.#pageOutputRelnames, async outputRelname => { - if (currentOutputRelnames.has(outputRelname)) return - const filepath = resolve(dest, outputRelname) - assertInsideDest(dest, filepath) - if (filepath === dest) throw new Error('Refusing to remove the build destination') - await rm(filepath, { force: true }) - })) - - this.#pageOutputRelnames = currentPageOutputRelnames + #rememberPartialPageOutputs (results) { + for (const [owner, outputs] of getPageOutputMap(resolve(this.#dest), results.report.pages)) { + const previous = this.#pageOutputMap.get(owner) ?? new Set() + for (const path of outputs) previous.add(path) + this.#pageOutputMap.set(owner, previous) + } } /** - * Remove outputs no longer emitted by the selected generated-pages owners. - * Ownership state changes only after a successful targeted build. + * Reconcile page ownership only after a successful page phase. Untouched page + * and template owners still protect their outputs during targeted builds. * - * @param {string[]} pagesFileFilterPaths - * @param {WatchedPageReport[]} pageReports + * @param {Pick} results + * @param {boolean} isFiltered */ - async #removeObsoleteGeneratedPageOutputs (pagesFileFilterPaths, pageReports) { - const currentByOwner = getPagesFileOutputMap(pageReports) + async #removeObsoletePageOutputs (results, isFiltered) { const dest = resolve(this.#dest) + const rebuiltPages = getPageOutputMap(dest, results.report.pages) + const pages = isFiltered ? new Map(this.#pageOutputMap) : new Map() + const templates = isFiltered ? new Map(this.#templateOutputMap) : new Map() + + // Factories can successfully rebuild to zero pages; regular pages always + // report their HTML output, even when their page-output hook is gone. + for (const owner of results.report.rebuiltPagesFilePaths ?? []) pages.delete(owner) + for (const [owner, outputs] of rebuiltPages) pages.set(owner, outputs) + for (const report of results.report.templates) { + templates.set(report.templateInfo.templateFile.filepath, new Set( + report.outputs.map(output => resolve(dest, report.templateInfo.path, output)) + )) + } - for (const pagesFilePath of pagesFileFilterPaths) { - const previousOutputs = this.#pagesFileOutputMap.get(pagesFilePath) ?? new Set() - const currentOutputs = currentByOwner.get(pagesFilePath) ?? new Set() - - await Promise.all(Array.from(previousOutputs, async outputRelname => { - if (currentOutputs.has(outputRelname)) return - const filepath = resolve(dest, outputRelname) - assertInsideDest(dest, filepath) - if (filepath === dest) throw new Error('Refusing to remove the build destination') - await rm(filepath, { force: true }) - })) - - for (const outputRelname of previousOutputs) this.#pageOutputRelnames.delete(outputRelname) - for (const report of pageReports) { - if (report.pagesFilePath !== pagesFilePath) continue - for (const output of report.outputs ?? []) { - if (output.kind === 'page') this.#pageOutputRelnames.add(output.outputRelname) - } + const claimed = new Set(results.outputs.map(output => resolve(dest, output.outputRelname))) + for (const outputs of [...pages.values(), ...templates.values()]) { + for (const filepath of outputs) claimed.add(filepath) + } + const stale = new Set() + for (const outputs of this.#pageOutputMap.values()) { + for (const filepath of outputs) { + if (!claimed.has(filepath)) stale.add(filepath) } + } + for (const filepath of stale) await removeStalePageOutput(dest, filepath) - if (currentOutputs.size > 0) this.#pagesFileOutputMap.set(pagesFilePath, currentOutputs) - else this.#pagesFileOutputMap.delete(pagesFilePath) + const pageOwnedPaths = new Set([...pages.values()].flatMap(outputs => [...outputs])) + for (const filepath of this.#pageOutputCache.keys()) { + if (!pageOwnedPaths.has(filepath)) this.#pageOutputCache.delete(filepath) } + this.#pageOutputMap = pages + this.#templateOutputMap = templates } /** @@ -846,35 +851,45 @@ ${siteData.errors.map(err => ` ${err.message}`).join('\n')}`) } /** - * @param {DomstackManifestRecord[]} outputs - * @returns {Set} - */ -function getPageOutputRelnames (outputs) { - return new Set(outputs - .filter(output => output.kind === 'page') - .map(output => output.outputRelname)) -} - -/** - * Group generated-page outputs by their owning *.pages.* filepath. - * + * @param {string} dest * @param {WatchedPageReport[]} pageReports * @returns {Map>} */ -function getPagesFileOutputMap (pageReports) { +function getPageOutputMap (dest, pageReports) { /** @type {Map>} */ const outputsByOwner = new Map() - for (const report of pageReports) { - if (!report.pagesFilePath) continue - const outputs = outputsByOwner.get(report.pagesFilePath) ?? new Set() - for (const output of report.outputs ?? []) outputs.add(output.outputRelname) - outputsByOwner.set(report.pagesFilePath, outputs) + const owner = report.pagesFilePath ?? report.sourcePageFilePath + if (!owner) continue + const outputs = outputsByOwner.get(owner) ?? new Set() + for (const output of report.outputs ?? []) outputs.add(resolve(dest, output.outputRelname)) + outputsByOwner.set(owner, outputs) } - return outputsByOwner } +/** + * Never follow a replaced output directory outside the destination. A symlink + * at the output itself is safe to unlink; directories are never removed. + * @param {string} dest + * @param {string} filepath + */ +async function removeStalePageOutput (dest, filepath) { + assertInsideDest(dest, filepath) + if (filepath === dest) throw new Error('Refusing to remove the build destination') + try { + for (let ancestor = dirname(filepath); ; ancestor = dirname(ancestor)) { + const stats = await lstat(ancestor) + if (stats.isSymbolicLink() || !stats.isDirectory()) return + if (ancestor === dest) break + } + const stats = await lstat(filepath) + if (!stats.isDirectory()) await rm(filepath, { force: true }) + } catch (err) { + if (/** @type {NodeJS.ErrnoException} */ (err).code !== 'ENOENT') throw err + } +} + /** * Group layouts used by generated pages by their owning *.pages.* filepath. * diff --git a/lib/build-pages/index.js b/lib/build-pages/index.js index e89392c3..d8ba54fb 100644 --- a/lib/build-pages/index.js +++ b/lib/build-pages/index.js @@ -6,6 +6,7 @@ * @import { ResolvedLayout } from './page-data.js' * @import { DomstackManifestRecord } from '../domstack-manifest/index.js' * @import { WatchDependencyState, WatchConsumer, WatchDependencyTracker } from './watch-dependencies.js' + * @import { PageOutputCache } from './page-builders/page-output-writer.js' */ import { Worker } from 'worker_threads' @@ -23,6 +24,7 @@ import { DomStackDataError, DomStackOutputConflictError } from '../helpers/domst import { isAsyncIterable, isPlainObject } from '../helpers/type-guards.js' import { createSubscribedData, resolveDataDeps } from './data-deps.js' import { WatchDependencyTracker as WatchDependencyTrackerClass } from './watch-dependencies.js' +import { outputWarnings } from '../helpers/output-warnings.js' const MAX_CONCURRENCY = Math.min(cpus().length, 24) @@ -43,6 +45,7 @@ const __dirname = import.meta.dirname * @property {PageReport[]} pages * @property {TemplateReport[]} templates * @property {WatchDependencyState | undefined} [watchDependencies] + * @property {PageOutputCache | undefined} [pageOutputCache] * @property {string[] | undefined} [rebuiltPagesFilePaths] */ @@ -89,6 +92,7 @@ const __dirname = import.meta.dirname * @property {boolean | undefined} [buildDrafts] - Include generated page definitions marked as drafts. * @property {WatchDependencyState | null | undefined} [previousWatchDependencies] - Dependency state from the previous successful watch build. * @property {boolean | undefined} [trackWatchDependencies] - Collect subscriptions for incremental watch builds. + * @property {PageOutputCache | undefined} [previousPageOutputCache] - Successful output hashes and metadata retained across watch workers. */ /** @@ -282,25 +286,16 @@ function validateGeneratedPageDefinition (value) { /** * @param {unknown} value - * @returns {Promise} + * @returns {AsyncGenerator} */ -async function collectGeneratedPageDefinitions (value) { - if (value == null) return [] +async function * iterateGeneratedPageDefinitions (value) { + if (value == null) return - if (Array.isArray(value)) { - return value.map(validateGeneratedPageDefinition) + if (Array.isArray(value) || isAsyncIterable(value)) { + for await (const definition of value) yield validateGeneratedPageDefinition(definition) + } else { + yield validateGeneratedPageDefinition(value) } - - if (isAsyncIterable(value)) { - /** @type {GeneratedPageDefinition[]} */ - const definitions = [] - for await (const definition of value) { - definitions.push(validateGeneratedPageDefinition(definition)) - } - return definitions - } - - return [validateGeneratedPageDefinition(value)] } /** @@ -364,11 +359,9 @@ function generatedDefinitionToPageInfo ({ definition, pagesFile, index }) { * @param {Set | null} params.pagesFileFilterSet * @param {boolean | undefined} params.buildDrafts * @param {WatchDependencyTracker} params.watchDependencyTracker - * @returns {Promise} + * @returns {AsyncGenerator} */ -async function resolveGeneratedPageInfos ({ siteData, factoryVars, globalData, pagesFileFilterSet, buildDrafts, watchDependencyTracker }) { - /** @type {PageInfo[]} */ - const generatedPageInfos = [] +async function * resolveGeneratedPageInfos ({ siteData, factoryVars, globalData, pagesFileFilterSet, buildDrafts, watchDependencyTracker }) { /** @type {Map} */ const pageOutputClaims = new Map() @@ -419,10 +412,9 @@ async function resolveGeneratedPageInfos ({ siteData, factoryVars, globalData, p }) : pagesExport - const definitions = await collectGeneratedPageDefinitions(pagesResults) - - for (const [index, definition] of definitions.entries()) { - const generatedPageInfo = generatedDefinitionToPageInfo({ definition, pagesFile, index }) + let index = 0 + for await (const definition of iterateGeneratedPageDefinitions(pagesResults)) { + const generatedPageInfo = generatedDefinitionToPageInfo({ definition, pagesFile, index: index++ }) if (generatedPageInfo.draft && !buildDrafts) continue const outputKey = resolve(generatedPageInfo.outputRelname) @@ -443,7 +435,7 @@ async function resolveGeneratedPageInfos ({ siteData, factoryVars, globalData, p } pageOutputClaims.set(outputKey, generatedClaim) - generatedPageInfos.push(generatedPageInfo) + yield generatedPageInfo } } catch (err) { const error = err instanceof Error @@ -453,8 +445,6 @@ async function resolveGeneratedPageInfos ({ siteData, factoryVars, globalData, p throw error } } - - return generatedPageInfos } /** @@ -474,6 +464,7 @@ export function buildPages (src, dest, siteData, opts) { buildDrafts: opts?.buildDrafts, previousWatchDependencies: opts?.previousWatchDependencies, trackWatchDependencies: opts?.trackWatchDependencies, + previousPageOutputCache: opts?.previousPageOutputCache, } return new Promise((resolve, reject) => { @@ -532,6 +523,9 @@ export async function buildPagesDirect (_src, dest, siteData, opts) { warnings: [], } + const outputCache = opts?.trackWatchDependencies ? new Map(opts.previousPageOutputCache) : undefined + result.report.pageOutputCache = outputCache + const pageFilterSet = opts?.pageFilterPaths ? new Set(opts.pageFilterPaths) : null const templateFilterSet = opts?.templateFilterPaths ? new Set(opts.templateFilterPaths) : null const pagesFileFilterSet = opts?.pagesFileFilterPaths ? new Set(opts.pagesFileFilterPaths) : null @@ -610,6 +604,7 @@ export async function buildPagesDirect (_src, dest, siteData, opts) { } catch (err) { result.errors.push(serializeBuildError(err, { page: pageInfoForWorker(pageInfo) }, 'Error resolving page vars')) } + result.warnings.push(...pageData.warnings) return pageData } @@ -650,39 +645,6 @@ export async function buildPagesDirect (_src, dest, siteData, opts) { }) } - let generatedPageInfos = /** @type {PageInfo[]} */ ([]) - try { - generatedPageInfos = await resolveGeneratedPageInfos({ - siteData, - factoryVars: globalVars, - globalData, - pagesFileFilterSet, - buildDrafts: opts?.buildDrafts, - watchDependencyTracker, - }) - } catch (err) { - const pagesFile = /** @type {PagesFileInfo | undefined} */ (err instanceof Error && 'pagesFile' in err ? err.pagesFile : undefined) - result.errors.push(serializeBuildError(err, { pagesFile }, `Error resolving generated pages: ${err instanceof Error ? err.message : String(err)}`)) - } - - if (result.errors.length > 0) return result - - const generatedPages = await pMap(generatedPageInfos, initPageData, { concurrency: MAX_CONCURRENCY }) - if (result.errors.length > 0) return result - - for (const page of generatedPages) { - page.setGlobalData(globalData) - const generatedOwnerPath = page.pageInfo.generated?.pagesFile.pagesFile.filepath - watchDependencyTracker.registerConsumer( - 'page', - page.pageInfo.outputRelname, - page.dataDeps, - generatedOwnerPath ? { ownerPath: generatedOwnerPath } : {} - ) - } - - if (result.errors.length > 0) return result - /** @type {PageData[]} */ const pagesToWrite = [] @@ -692,10 +654,6 @@ export async function buildPagesDirect (_src, dest, siteData, opts) { } } - for (const page of generatedPages) { - pagesToWrite.push(page) - } - /** @type {[number, number]} Divided concurrency values */ const dividedConcurrency = MAX_CONCURRENCY % 2 ? [((MAX_CONCURRENCY - 1) / 2) + 1, (MAX_CONCURRENCY - 1) / 2] // odd @@ -710,27 +668,78 @@ export async function buildPagesDirect (_src, dest, siteData, opts) { : (siteData.pagesFiles ?? []).map(pagesFile => pagesFile.pagesFile.filepath) } - await Promise.all([ - pMap(pagesToWrite, async (page) => { - try { - const buildResult = await pageWriter({ - dest, - page, - }) - + /** @param {PageData} page */ + const writePage = async (page) => { + try { + const buildResult = await pageWriter({ + dest, + page, + outputCache, + }) + + result.report.pages.push({ + pageFilePath: buildResult.pageFilePath, + sourcePageFilePath: page.pageInfo.generated ? undefined : page.pageInfo.pageFile.filepath, + pagesFilePath: page.pageInfo.generated?.pagesFile.pagesFile.filepath, + layoutName: page.layout?.name, + layoutNames: page.layoutChain.map(layout => layout.name), + outputs: buildResult.outputs, + }) + result.outputs.push(...buildResult.outputs) + return true + } catch (err) { + // Direct writes already emitted by a failed iterator still need ownership + // so a later successful watch rebuild can remove them. + if (page.outputRecords.length > 0) { result.report.pages.push({ - pageFilePath: buildResult.pageFilePath, + pageFilePath: join(dest, page.pageInfo.outputRelname), sourcePageFilePath: page.pageInfo.generated ? undefined : page.pageInfo.pageFile.filepath, pagesFilePath: page.pageInfo.generated?.pagesFile.pagesFile.filepath, layoutName: page.layout?.name, layoutNames: page.layoutChain.map(layout => layout.name), - outputs: buildResult.outputs, + outputs: page.outputRecords, }) - result.outputs.push(...buildResult.outputs) - } catch (err) { - result.errors.push(serializeBuildError(err, { page: pageInfoForWorker(page.pageInfo) }, 'Error building page')) + result.outputs.push(...page.outputRecords) + } + result.errors.push(serializeBuildError(err, { page: pageInfoForWorker(page.pageInfo) }, `Error building page "${page.pageInfo.pageFile.relname}"`)) + return false + } + } + + // Keep output names for dependency pruning, not generated definitions or PageData instances. + const generatedOutputRelnames = new Set() + const writeGeneratedPages = async () => { + try { + for await (const pageInfo of resolveGeneratedPageInfos({ + siteData, + factoryVars: globalVars, + globalData, + pagesFileFilterSet, + buildDrafts: opts?.buildDrafts, + watchDependencyTracker, + })) { + const errorCount = result.errors.length + const page = await initPageData(pageInfo) + if (result.errors.length > errorCount) break + page.setGlobalData(globalData) + watchDependencyTracker.registerConsumer( + 'page', + pageInfo.outputRelname, + page.dataDeps, + { ownerPath: pageInfo.pageFile.filepath } + ) + if (!await writePage(page)) break + generatedOutputRelnames.add(pageInfo.outputRelname) } - }, { concurrency: dividedConcurrency[0] }), + } catch (err) { + const pagesFile = /** @type {PagesFileInfo | undefined} */ (err instanceof Error && 'pagesFile' in err ? err.pagesFile : undefined) + result.errors.push(serializeBuildError(err, { pagesFile }, `Error building generated pages: ${err instanceof Error ? err.message : String(err)}`)) + } + } + + await Promise.all([ + pMap(pagesToWrite, writePage, { concurrency: dividedConcurrency[0] }), + writeGeneratedPages(), pMap(templatesToRender, async (template) => { try { const buildResult = await templateBuilder({ @@ -750,8 +759,9 @@ export async function buildPagesDirect (_src, dest, siteData, opts) { ]) if (opts?.trackWatchDependencies) { + result.warnings.push(...outputWarnings(result.outputs)) watchDependencyTracker.pruneGeneratedPages( - new Set(generatedPages.map(page => page.pageInfo.outputRelname)), + generatedOutputRelnames, pagesFileFilterSet ) result.report.watchDependencies = watchDependencyTracker.state diff --git a/lib/build-pages/page-builders/js/index.js b/lib/build-pages/page-builders/js/index.js index 53c7f62a..2eb2a02f 100644 --- a/lib/build-pages/page-builders/js/index.js +++ b/lib/build-pages/page-builders/js/index.js @@ -4,6 +4,7 @@ */ import assert from 'node:assert' +import { validatePageOutputsHook } from '../../page-outputs.js' /** * Resolve a JavaScript page module. @@ -26,10 +27,11 @@ export async function jsBuilder ({ pageInfo }) { } } - const { default: pageLayout, vars } = await import(pageInfo.pageFile.filepath) + const { default: pageLayout, vars, pageOutputs } = await import(pageInfo.pageFile.filepath) assert(pageLayout, 'js pages must export a page layout default export') assert(typeof pageLayout === 'function', 'js pages pageLayout must be a function') - return { vars, pageLayout } + const hook = validatePageOutputsHook(pageOutputs, pageInfo.pageFile.filepath) + return { vars, pageLayout, pageOutputs: hook } } diff --git a/lib/build-pages/page-builders/page-output-writer.js b/lib/build-pages/page-builders/page-output-writer.js new file mode 100644 index 00000000..d650dfaa --- /dev/null +++ b/lib/build-pages/page-builders/page-output-writer.js @@ -0,0 +1,131 @@ +/** + * @import { Stats } from 'node:fs' + * @import { PageData } from '../page-data.js' + * + * @typedef {Map} PageOutputCache + */ +import { createHash } from 'node:crypto' +import { lstat, mkdir, writeFile } from 'node:fs/promises' +import { dirname, relative, resolve, sep } from 'node:path' +import { assertInsideDest, toPosix } from '../../helpers/path.js' +import { createDomstackManifestRecord } from '../../domstack-manifest/index.js' + +/** + * Resolve sidecar names independently of legacy page/template path semantics. + * Backslashes are separators on every platform; only a single leading forward + * slash means destination-root relative (drive paths and UNC paths are invalid). + * + * @param {string} dest + * @param {string} pageFilePath + * @param {string} outputName + */ +export function resolvePageOutputPath (dest, pageFilePath, outputName) { + if (typeof outputName !== 'string' || !outputName.trim()) { + throw new TypeError('Page outputName must be a non-empty file path') + } + const name = outputName.replaceAll('\\', '/') + if (outputName.startsWith('\\') || name.startsWith('//') || /^[a-z]:/i.test(name)) { + throw new Error(`Page outputName must not be a drive or UNC path: ${outputName}`) + } + const parts = name.split('/') + if (!parts.at(-1) || ['.', '..'].includes(parts.at(-1) ?? '')) { + throw new Error(`Page outputName must name a file: ${outputName}`) + } + for (const part of parts) { + if (!part || part === '.' || part === '..') continue + // Reject Windows aliases and special files even when building on POSIX. + if (/[<>:"|?*]/u.test(part) || Array.from(part).some(character => character.charCodeAt(0) < 32) || /[. ]$/.test(part) || /^(con|prn|aux|nul|com[1-9¹²³]|lpt[1-9¹²³])(?:\.|$)/i.test(part)) { + throw new Error(`Page outputName contains an invalid file path component: ${outputName}`) + } + } + const filepath = name.startsWith('/') + ? resolve(dest, name.slice(1)) + : resolve(dirname(pageFilePath), name) + const relname = relative(resolve(dest), filepath) + const message = `Page outputName escapes dest or names its directory: ${outputName}` + assertInsideDest(dest, filepath, message) + if (!relname) throw new Error(message) + return { filepath, outputRelname: toPosix(relname) } +} + +/** + * Check existing components, including the leaf, without following symlinks. + * This is not a defense against concurrent external filesystem mutations. + * + * @param {string} dest + * @param {string} filepath + */ +async function assertWritablePath (dest, filepath) { + let current = resolve(dest) + const components = relative(current, filepath).split(sep) + for (let index = -1; index < components.length; index++) { + const component = components[index] + if (component !== undefined) current = resolve(current, component) + let info + try { + info = await lstat(current) + } catch (error) { + if (/** @type {NodeJS.ErrnoException} */ (error).code === 'ENOENT') continue + throw error + } + if (info.isSymbolicLink()) throw new Error(`Page output path contains a symlink: ${current}`) + const leaf = index === components.length - 1 + if (leaf ? !info.isFile() : !info.isDirectory()) { + throw new Error(`Page output path is not a ${leaf ? 'file' : 'directory'}: ${current}`) + } + if (leaf) return info + } +} + +/** + * Write sidecars directly, retaining ownership records even for unchanged bytes. + * + * @param {object} params + * @param {string} params.dest + * @param {string} params.pageFilePath + * @param {Pick, 'pageInfo' | 'outputRecords'>} params.page + * @param {Iterable<{outputName: string, content: string}> | AsyncIterable<{outputName: string, content: string}>} params.pageOutputs + * @param {PageOutputCache | undefined} [params.outputCache] + */ +export async function writePageOutputs ({ dest, pageFilePath, page, pageOutputs, outputCache }) { + const { pageInfo, outputRecords } = page + for await (const output of pageOutputs) { + if (typeof output.content !== 'string') throw new TypeError('Page output content must be a string') + const { filepath, outputRelname } = resolvePageOutputPath(dest, pageFilePath, output.outputName) + const info = await assertWritablePath(dest, filepath) + const hash = outputCache ? createHash('sha256').update(output.content, 'utf8').digest('hex') : undefined + const cached = outputCache?.get(filepath) + const unchanged = cached && cached.hash === hash && info && cached.metadata === fileMetadata(info) + if (!unchanged) { + outputCache?.delete(filepath) + await mkdir(dirname(filepath), { recursive: true }) + await writeFile(filepath, output.content) + } + const record = { + ...createDomstackManifestRecord({ + dest, + filepath, + outputRelname, + kind: 'page-output', + sourceRelname: pageInfo.pageFile.relname, + pagePath: pageInfo.path, + pageUrl: pageInfo.url, + }), + pagePath: pageInfo.path, + } + outputRecords.push(record) + if (!unchanged && outputCache && hash) { + outputCache.set(filepath, { hash, metadata: fileMetadata(await lstat(filepath)) }) + } + } + return outputRecords +} + +/** + * Best-effort detection of replacement or external edits, even if mtime is restored. + * Reuses the path validation stat; never rereads destination content. + * @param {Stats} info + */ +function fileMetadata (info) { + return [info.dev, info.ino, info.size, info.mtimeMs, info.ctimeMs].join(':') +} diff --git a/lib/build-pages/page-builders/page-output-writer.test.js b/lib/build-pages/page-builders/page-output-writer.test.js new file mode 100644 index 00000000..5a0951b3 --- /dev/null +++ b/lib/build-pages/page-builders/page-output-writer.test.js @@ -0,0 +1,308 @@ +/** + * @import { PageInfo } from '../../identify-pages.js' + * @import { DomstackManifestRecord } from '../../domstack-manifest/index.js' + * @import { PageOutputCache } from './page-output-writer.js' + */ +import { test } from 'node:test' +import assert from 'node:assert/strict' +import { createHash } from 'node:crypto' +import fs, { lstat, mkdtemp, mkdir, readFile, rm, symlink, writeFile } from 'node:fs/promises' +import { syncBuiltinESMExports } from 'node:module' +import { tmpdir } from 'node:os' +import { join, resolve } from 'node:path' +import { resolvePageOutputPath, writePageOutputs } from './page-output-writer.js' + +import { createEntry } from '../../domstack-manifest/records.js' + +const pageInfo = /** @type {PageInfo} */ ({ + path: 'posts', + outputName: 'index.html', + outputRelname: 'posts/index.html', + url: '/posts/', + pageFile: { relname: 'posts/page.js' }, +}) + +test('page output paths use the actual page output directory and destination root', () => { + const dest = resolve('public') + const page = join(dest, 'posts', 'nested', 'index.html') + /** @type {[string, string][]} */ + const cases = [ + ['data.json', 'posts/nested/data.json'], + ['../feed.json', 'posts/feed.json'], + ['../../feed.json', 'feed.json'], + ['/feed.json', 'feed.json'], + ['..\\feed.json', 'posts/feed.json'], + ['./data.json', 'posts/nested/data.json'], + ['..hidden/data.json', 'posts/nested/..hidden/data.json'], + ] + for (const [name, expected] of cases) { + const result = resolvePageOutputPath(dest, page, name) + assert.equal(result.outputRelname, expected) + assert.equal(result.filepath, join(dest, expected)) + } +}) + +test('page output paths reject escapes, directory names and nonportable file names', () => { + const dest = resolve('public') + const page = join(dest, 'posts/index.html') + for (const name of ['', ' ', '/', '.', '..', 'foo/', 'foo\\', 'foo/.', 'foo/..', '../../escape.json', '/../escape.json', '//server/file', '\\file', '\\\\server\\file', 'C:/file', 'C:file', 'file:stream', 'file\u0000.json', 'file?', 'NUL.json', 'aux', 'COM1.txt', 'dir./file', 'dir /file']) { + assert.throws(() => resolvePageOutputPath(dest, page, name), Error, name) + } + for (const name of ['../../public', '../../escape.json']) { + assert.throws(() => resolvePageOutputPath(dest, page, name), { + message: `Page outputName escapes dest or names its directory: ${name}`, + }) + } +}) + +test('writer writes sidecars with page ownership and non-navigation JSON records', async t => { + const dest = await mkdtemp(join(tmpdir(), 'domstack-page-output-')) + t.after(() => rm(dest, { recursive: true, force: true })) + const page = { pageInfo, outputRecords: /** @type {DomstackManifestRecord[]} */ ([]) } + const outputs = await writePageOutputs({ + dest, + pageFilePath: join(dest, 'posts/index.html'), + page, + pageOutputs: [{ outputName: '/feed.json', content: '{"ok":true}' }, { outputName: 'nested/data.txt', content: 'hello' }], + }) + assert.equal(outputs, page.outputRecords) + assert.equal(await readFile(join(dest, 'feed.json'), 'utf8'), '{"ok":true}') + assert.equal(await readFile(join(dest, 'posts/nested/data.txt'), 'utf8'), 'hello') + assert.ok(outputs[0]) + assert.equal(outputs[0].kind, 'page-output') + assert.equal(outputs[0].sourceRelname, 'posts/page.js') + assert.equal(outputs[0].pagePath, 'posts') + assert.equal(outputs[0].pageUrl, '/posts/') + assert.equal(outputs[0].url, '/feed.json') + assert.equal(outputs[0].page, undefined) + const entry = await createEntry({ dest, record: outputs[0] }) + assert.equal(entry?.role, 'subresource') +}) + +test('writer rejects symlink components and existing directories without touching their targets', async t => { + const root = await mkdtemp(join(tmpdir(), 'domstack-page-output-links-')) + t.after(() => rm(root, { recursive: true, force: true })) + const dest = join(root, 'dest') + const outside = join(root, 'outside') + await mkdir(dest) + await mkdir(outside) + await writeFile(join(outside, 'data.json'), 'unchanged') + await symlink(outside, join(dest, 'linked'), 'dir') + await symlink(join(outside, 'data.json'), join(dest, 'leaf.json'), 'file') + await mkdir(join(dest, 'directory')) + for (const outputName of ['linked/data.json', 'leaf.json', 'directory']) { + const page = { pageInfo, outputRecords: [] } + /** @type {PageOutputCache} */ + const outputCache = new Map() + await assert.rejects(writePageOutputs({ + dest, + pageFilePath: join(dest, 'index.html'), + page, + pageOutputs: [{ outputName, content: 'changed' }], + outputCache, + }), /symlink|not a file/) + assert.deepEqual(page.outputRecords, []) + assert.equal(outputCache.size, 0) + } + assert.equal(await readFile(join(outside, 'data.json'), 'utf8'), 'unchanged') +}) + +test('writer validates each path before writing it, retaining earlier writes', async t => { + const dest = await mkdtemp(join(tmpdir(), 'domstack-page-output-invalid-')) + t.after(() => rm(dest, { recursive: true, force: true })) + const page = { pageInfo, outputRecords: /** @type {DomstackManifestRecord[]} */ ([]) } + /** @type {PageOutputCache} */ + const outputCache = new Map() + await assert.rejects(writePageOutputs({ + dest, + pageFilePath: join(dest, 'index.html'), + page, + pageOutputs: [{ outputName: 'valid.json', content: '{}' }, { outputName: '../escape.json', content: '{}' }], + outputCache, + }), /escapes dest/) + assert.equal(await readFile(join(dest, 'valid.json'), 'utf8'), '{}') + assert.deepEqual(page.outputRecords.map(output => output.outputRelname), ['valid.json']) + assert.deepEqual([...outputCache.keys()], [join(dest, 'valid.json')]) +}) + +test('writer preserves duplicate records for duplicate-output warnings', async t => { + const dest = await mkdtemp(join(tmpdir(), 'domstack-page-output-claims-')) + t.after(() => rm(dest, { recursive: true, force: true })) + const outputs = await writePageOutputs({ + dest, + pageFilePath: join(dest, 'index.html'), + page: { pageInfo, outputRecords: [] }, + pageOutputs: [{ outputName: 'data.json', content: 'first' }, { outputName: './data.json', content: 'second' }], + }) + assert.equal(outputs.length, 2) + assert.ok(outputs[0]) + assert.ok(outputs[1]) + assert.equal(outputs[0].outputRelname, outputs[1].outputRelname) + assert.equal(outputs[0].sourceRelname, outputs[1].sourceRelname) +}) + +test('writer caches SHA256 of UTF8 bytes and stat metadata, not output content', async t => { + const dest = await mkdtemp(join(tmpdir(), 'domstack-page-output-hash-')) + t.after(() => rm(dest, { recursive: true, force: true })) + const page = { pageInfo, outputRecords: [] } + /** @type {PageOutputCache} */ + const outputCache = new Map() + const content = 'café 🌊\n' + const filepath = join(dest, 'data.txt') + const outputs = await writePageOutputs({ + dest, + pageFilePath: join(dest, 'index.html'), + page, + pageOutputs: [{ outputName: './data.txt', content }], + outputCache, + }) + const info = await lstat(filepath) + assert.equal(info.size, Buffer.byteLength(content, 'utf8')) + assert.deepEqual([...outputCache], [[filepath, { + hash: createHash('sha256').update(Buffer.from(content, 'utf8')).digest('hex'), + metadata: [info.dev, info.ino, info.size, info.mtimeMs, info.ctimeMs].join(':'), + }]]) + assert.equal(outputs, page.outputRecords) + assert.equal(outputs.length, 1) + assert.ok(outputs[0]) + assert.equal('content' in outputs[0], false) + assert.equal(await readFile(filepath, 'utf8'), content) +}) + +for (const cached of [false, true]) { + test(`writer overwrites existing equal bytes without reading with ${cached ? 'a cold' : 'no'} cache`, async t => { + const dest = await mkdtemp(join(tmpdir(), 'domstack-page-output-cold-')) + t.after(() => rm(dest, { recursive: true, force: true })) + const filepath = join(dest, 'data.txt') + await writeFile(filepath, 'equal bytes') + const writes = t.mock.method(fs, 'writeFile') + const reads = t.mock.method(fs, 'readFile', async () => { throw new Error('must not read destination content') }) + syncBuiltinESMExports() + t.after(() => { t.mock.restoreAll(); syncBuiltinESMExports() }) + const page = { pageInfo, outputRecords: [] } + /** @type {PageOutputCache | undefined} */ + const outputCache = cached ? new Map() : undefined + const outputs = await writePageOutputs({ + dest, + pageFilePath: join(dest, 'index.html'), + page, + pageOutputs: [{ outputName: 'data.txt', content: 'equal bytes' }], + outputCache, + }) + assert.equal(writes.mock.callCount(), 1) + assert.deepEqual(writes.mock.calls[0]?.arguments, [filepath, 'equal bytes']) + assert.equal(reads.mock.callCount(), 0) + assert.equal(outputs, page.outputRecords) + assert.equal(outputs.length, 1) + assert.equal(outputCache?.has(filepath), cached ? true : undefined) + }) +} + +test('writer skips matching cached bytes and metadata but still records every claim', async t => { + const dest = await mkdtemp(join(tmpdir(), 'domstack-page-output-warm-')) + t.after(() => rm(dest, { recursive: true, force: true })) + /** @type {PageOutputCache} */ + const outputCache = new Map() + const page = { pageInfo, outputRecords: [] } + const params = { dest, pageFilePath: join(dest, 'index.html'), page, outputCache } + await writePageOutputs({ ...params, pageOutputs: [{ outputName: 'data.txt', content: 'same' }] }) + const previousCache = outputCache.get(join(dest, 'data.txt')) + const writes = t.mock.method(fs, 'writeFile') + const reads = t.mock.method(fs, 'readFile', async () => { throw new Error('must not read destination content') }) + syncBuiltinESMExports() + t.after(() => { t.mock.restoreAll(); syncBuiltinESMExports() }) + const outputs = await writePageOutputs({ + ...params, + pageOutputs: [{ outputName: './data.txt', content: 'same' }, { outputName: '/data.txt', content: 'same' }], + }) + assert.equal(writes.mock.callCount(), 0) + assert.equal(reads.mock.callCount(), 0) + assert.equal(outputs, page.outputRecords) + assert.equal(outputs.length, 3) + assert.deepEqual(outputs[1], outputs[0]) + assert.deepEqual(outputs[2], outputs[0]) + assert.equal(outputCache.get(join(dest, 'data.txt')), previousCache) +}) + +test('writer rewrites on a hash mismatch or any stat metadata mismatch', async t => { + const dest = await mkdtemp(join(tmpdir(), 'domstack-page-output-mismatch-')) + t.after(() => rm(dest, { recursive: true, force: true })) + /** @type {PageOutputCache} */ + const outputCache = new Map() + const filepath = join(dest, 'data.txt') + const params = { dest, pageFilePath: join(dest, 'index.html'), page: { pageInfo, outputRecords: [] }, outputCache } + const pageOutputs = [{ outputName: 'data.txt', content: 'same' }] + await writePageOutputs({ ...params, pageOutputs }) + const writes = t.mock.method(fs, 'writeFile') + syncBuiltinESMExports() + t.after(() => { t.mock.restoreAll(); syncBuiltinESMExports() }) + for (const field of ['hash', 'dev', 'ino', 'size', 'mtimeMs', 'ctimeMs']) { + const cached = outputCache.get(filepath) + assert.ok(cached) + const metadata = cached.metadata.split(':') + if (field === 'hash') cached.hash = 'stale' + else metadata[['dev', 'ino', 'size', 'mtimeMs', 'ctimeMs'].indexOf(field)] = 'stale' + cached.metadata = metadata.join(':') + writes.mock.resetCalls() + await writePageOutputs({ ...params, pageOutputs }) + assert.equal(writes.mock.callCount(), 1, field) + assert.notEqual(outputCache.get(filepath), cached, field) + } +}) + +test('failed writes leave no successful records or cache entries, including stale cache entries', async t => { + const dest = await mkdtemp(join(tmpdir(), 'domstack-page-output-failure-')) + t.after(() => rm(dest, { recursive: true, force: true })) + const filepath = join(dest, 'data.txt') + const cause = new Error('disk full') + const writes = t.mock.method(fs, 'writeFile', async () => { throw cause }) + syncBuiltinESMExports() + t.after(() => { t.mock.restoreAll(); syncBuiltinESMExports() }) + for (const stale of [false, true]) { + const page = { pageInfo, outputRecords: [] } + /** @type {PageOutputCache} */ + const outputCache = new Map(stale ? [[filepath, { hash: 'stale', metadata: 'stale' }]] : []) + await assert.rejects(writePageOutputs({ + dest, + pageFilePath: join(dest, 'index.html'), + page, + pageOutputs: [{ outputName: 'data.txt', content: 'not written' }], + outputCache, + }), error => error === cause) + assert.deepEqual(page.outputRecords, []) + assert.equal(outputCache.size, 0) + } + assert.equal(writes.mock.callCount(), 2) +}) + +test('writer streams each output to disk and retains successful records and cache after iterator failure', async t => { + const dest = await mkdtemp(join(tmpdir(), 'domstack-page-output-iterator-')) + t.after(() => rm(dest, { recursive: true, force: true })) + /** @type {PageOutputCache} */ + const outputCache = new Map() + const page = { pageInfo, outputRecords: /** @type {DomstackManifestRecord[]} */ ([]) } + const filepath = join(dest, 'first.txt') + const cause = new Error('iterator failed') + let closed = false + async function * pageOutputs () { + try { + yield { outputName: 'first.txt', content: 'first' } + assert.equal(await readFile(filepath, 'utf8'), 'first') + assert.deepEqual(page.outputRecords.map(output => output.outputRelname), ['first.txt']) + assert.equal(outputCache.has(filepath), true) + throw cause + } finally { + closed = true + } + } + await assert.rejects(writePageOutputs({ + dest, + pageFilePath: join(dest, 'index.html'), + page, + pageOutputs: pageOutputs(), + outputCache, + }), error => error === cause) + assert.equal(closed, true) + assert.deepEqual(page.outputRecords.map(output => output.outputRelname), ['first.txt']) + assert.deepEqual([...outputCache.keys()], [filepath]) +}) diff --git a/lib/build-pages/page-builders/page-writer.js b/lib/build-pages/page-builders/page-writer.js index 4461e2e0..99c8f1b7 100644 --- a/lib/build-pages/page-builders/page-writer.js +++ b/lib/build-pages/page-builders/page-writer.js @@ -2,11 +2,14 @@ * @import { PageInfo } from '../../identify-pages.js' * @import { PageData as PageDataClass } from '../page-data.js' * @import { DomstackManifestRecord } from '../../domstack-manifest/index.js' + * @import { PageOutputsFunction } from '../page-outputs.js' + * @import { PageOutputCache } from './page-output-writer.js' */ import { join } from 'path' import { writeFile, mkdir } from 'fs/promises' import { createDomstackManifestRecord } from '../../domstack-manifest/index.js' +import { writePageOutputs } from './page-output-writer.js' /** * @typedef {Object} BuilderOptions @@ -74,6 +77,7 @@ import { createDomstackManifestRecord } from '../../domstack-manifest/index.js' * @typedef PageBuilderResult * @property {Partial} vars - Any variables resolved by the builder * @property {InternalPageFunction} pageLayout - The function that returns the rendered page + * @property {PageOutputsFunction | undefined} [pageOutputs] - Optional build-only page-output hook. */ /** @@ -96,41 +100,48 @@ import { createDomstackManifestRecord } from '../../domstack-manifest/index.js' * @param {object} params * @param {string} params.dest - The dest folder. * @param {PageData} params.page - The PageInfo object of the current page + * @param {PageOutputCache | undefined} [params.outputCache] * @returns {Promise<{ pageFilePath: string, outputs: DomstackManifestRecord[] }>} */ export async function pageWriter ({ dest, page, + outputCache, }) { if (!page.pageInfo) throw new Error('Uninitialzied page detected') const pageDir = join(dest, page.pageInfo.path) const pageFilePath = join(pageDir, page.pageInfo.outputName) + page.outputRecords.length = 0 const formattedPageOutput = await page.renderFullPage() + await writePageOutputs({ + dest, + pageFilePath, + page, + pageOutputs: page.collectPageOutputs(), + outputCache, + }) const vars = page.vars const manifestRole = extractManifestRole(vars) await mkdir(pageDir, { recursive: true }) await writeFile(pageFilePath, formattedPageOutput) - /** @type {DomstackManifestRecord[]} */ - const outputs = [ - createDomstackManifestRecord({ - dest, - filepath: pageFilePath, - outputRelname: page.pageInfo.outputRelname, - kind: 'page', + page.outputRecords.unshift(createDomstackManifestRecord({ + dest, + filepath: pageFilePath, + outputRelname: page.pageInfo.outputRelname, + kind: 'page', + url: page.pageInfo.url, + sourceRelname: page.pageInfo.pageFile.relname, + pagePath: page.pageInfo.path, + pageUrl: page.pageInfo.url, + pageVars: copyPageVars(vars), + manifestRole, + page: { + path: page.pageInfo.path, url: page.pageInfo.url, - sourceRelname: page.pageInfo.pageFile.relname, - pagePath: page.pageInfo.path, - pageUrl: page.pageInfo.url, - pageVars: copyPageVars(vars), - manifestRole, - page: { - path: page.pageInfo.path, - url: page.pageInfo.url, - }, - }), - ] + }, + })) // Generate meta.json with worker mappings if page has workers if (page.pageInfo?.workers) { @@ -151,18 +162,19 @@ export async function pageWriter ({ const workersFilePath = join(pageDir, 'workers.json') const workersContent = JSON.stringify(workerMappings, null, 2) await writeFile(workersFilePath, workersContent) - outputs.push(createDomstackManifestRecord({ + page.outputRecords.push(createDomstackManifestRecord({ dest, filepath: workersFilePath, outputRelname: join(page.pageInfo.path, 'workers.json'), kind: 'worker-manifest', + sourceRelname: page.pageInfo.pageFile.relname, pagePath: page.pageInfo.path, pageUrl: page.pageInfo.url, })) } } - return { pageFilePath, outputs } + return { pageFilePath, outputs: page.outputRecords } } /** diff --git a/lib/build-pages/page-data-page-outputs.test.js b/lib/build-pages/page-data-page-outputs.test.js new file mode 100644 index 00000000..2a701a7b --- /dev/null +++ b/lib/build-pages/page-data-page-outputs.test.js @@ -0,0 +1,450 @@ +/** + * @import { PageInfo } from '../identify-pages.js' + * @import { ResolvedLayout } from './page-data.js' + * @import { TestContext } from 'node:test' + * @import { PageOutputCache } from './page-builders/page-output-writer.js' + */ +import { test } from 'node:test' +import assert from 'node:assert/strict' +import { mkdtemp, readFile, writeFile, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { PageData, resolveLayout } from './page-data.js' +import { identifyPages } from '../identify-pages.js' +import { pageWriter } from './page-builders/page-writer.js' + +/** + * @param {TestContext} t + * @param {{ module?: string, companion?: string, extension?: string }} [options] + */ +async function fixture (t, { module, companion, extension = 'mjs' } = {}) { + const dir = await mkdtemp(join(tmpdir(), 'domstack-hooks-')) + t.after(() => rm(dir, { recursive: true, force: true })) + const filepath = join(dir, module ? `page.${extension}` : 'page.md') + await writeFile(filepath, module ?? '---\ntitle: Source\n---\n# Body\n') + const file = (/** @type {string} */ path) => ({ root: dir, filepath: join(dir, path), relname: path, basename: path, parentName: '' }) + /** @type {PageInfo} */ + const pageInfo = { + pageFile: file(module ? `page.${extension}` : 'page.md'), + type: module ? 'js' : 'md', + path: '', + url: '/', + outputName: 'index.html', + outputRelname: 'index.html', + draft: false, + } + if (companion !== undefined) { + await writeFile(join(dir, 'page.vars.mjs'), companion) + pageInfo.pageVars = file('page.vars.mjs') + } + const pd = new PageData({ pageInfo, globalVars: { layout: 'inner', pageOutputs: () => { throw new Error('global vars are not providers') } }, globalStyle: undefined, globalClient: undefined, defaultStyle: null, defaultClient: null, builderOptions: {} }) + /** @type {Record>} */ + const layouts = { + outer: { name: 'outer', render: ({ children }) => children, vars: {}, layoutStylePath: null, layoutClientPath: null }, + inner: { name: 'inner', parentLayout: 'outer', render: ({ children }) => children, vars: {}, layoutStylePath: null, layoutClientPath: null }, + } + return { pd, layouts, dir } +} + +test('explicit collection runs outer -> inner -> page with renderer subscriptions, not on rendering', async t => { + const { pd, layouts, dir } = await fixture(t, { + module: `export const vars = { dataDeps: ['pageKey'] } + export let hookCalls = 0 + export default ({ data }) => data.pageKey + export const pageOutputs = ({ page, vars, data }) => { + hookCalls++ + return { outputName: 'page.txt', content: data.pageKey + data.companionKey + page.url } + }`, + companion: "export default { dataDeps: ['companionKey'] }", + }) + /** @type {string[]} */ + const calls = [] + for (const name of ['outer', 'inner']) { + const path = join(dir, `${name}.layout.mjs`) + await writeFile(path, `export const vars = { dataDeps: ['${name}Key'] }; export default ({ children }) => children; + export const pageOutputs = ({ data }) => ({ outputName: '${name}.txt', content: data.${name}Key });`) + const layout = layouts[name] + assert.ok(layout) + const resolved = await resolveLayout(path) + Object.assign(layout, resolved, { parentLayout: name === 'inner' ? 'outer' : undefined }) + const hook = layout.pageOutputs + assert.ok(hook) + layout.pageOutputs = params => { + calls.push(name) + assert.deepEqual(Object.keys(params.data), [`${name}Key`]) + assert.throws(() => params.data.pageKey, /undeclared/) + return hook(params) + } + } + await assert.rejects(Array.fromAsync(pd.collectPageOutputs()), /initialized/) + await pd.init({ layouts }) + await assert.rejects(Array.fromAsync(pd.collectPageOutputs()), /outer.*not available/) + assert.deepEqual(pd.dataDeps, ['companionKey', 'innerKey', 'outerKey', 'pageKey']) + pd.setGlobalData({ pageKey: 'p', companionKey: 'c', outerKey: 'o', innerKey: 'i', secret: 'hidden' }) + const outputRecords = pd.outputRecords + assert.deepEqual(outputRecords, []) + await pd.renderInnerPage() + assert.deepEqual(pd.outputRecords, []) + await pd.renderFullPage() + assert.equal(pd.outputRecords, outputRecords) + assert.deepEqual(pd.outputRecords, []) + assert.deepEqual(calls, []) + const pageModule = await import(pd.pageInfo.pageFile.filepath) + assert.equal(pageModule.hookCalls, 0) + const outputs = await Array.fromAsync(pd.collectPageOutputs()) + assert.equal(pageModule.hookCalls, 1) + assert.deepEqual(calls, ['outer', 'inner']) + assert.deepEqual(outputs.map(({ outputName, content }) => ({ outputName, content })), [ + { outputName: 'outer.txt', content: 'o' }, { outputName: 'inner.txt', content: 'i' }, { outputName: 'page.txt', content: 'pc/' }, + ]) + assert.deepEqual(outputs[0]?.provenance, { kind: 'layout', source: join(dir, 'outer.layout.mjs'), layoutName: 'outer' }) + assert.deepEqual(outputs[2]?.provenance, { kind: 'page', source: join(dir, 'page.mjs') }) + assert.deepEqual(pd.outputRecords, []) +}) + +test('page writer records HTML and streamed sidecars on the actual PageData handle', async t => { + const { pd, layouts, dir } = await fixture(t, { + module: `export const events = [] + export default () => { events.push('render'); return '

Page

' } + export async function * pageOutputs () { + events.push('collect') + yield { outputName: 'data.json', content: '{"ok":true}' } + }`, + }) + await pd.init({ layouts }) + const pageModule = await import(pd.pageInfo.pageFile.filepath) + const outputRecords = pd.outputRecords + /** @type {PageOutputCache} */ + const outputCache = new Map() + const dest = join(dir, 'public') + const result = await pageWriter({ dest, page: pd, outputCache }) + assert.deepEqual(pageModule.events, ['render', 'collect']) + assert.equal(pd.outputRecords, outputRecords) + assert.equal(result.outputs, outputRecords) + assert.deepEqual(outputRecords.map(output => [output.kind, output.outputRelname]), [ + ['page', 'index.html'], ['page-output', 'data.json'], + ]) + for (const output of outputRecords) { + assert.equal(output.sourceRelname, 'page.mjs') + assert.equal(output.pageUrl, '/') + assert.equal('content' in output, false) + } + assert.equal(await readFile(result.pageFilePath, 'utf8'), '

Page

') + assert.equal(await readFile(join(dest, 'data.json'), 'utf8'), '{"ok":true}') + assert.deepEqual([...outputCache.keys()], [join(dest, 'data.json')]) + const cached = outputCache.get(join(dest, 'data.json')) + const rebuilt = await pageWriter({ dest, page: pd, outputCache }) + assert.equal(rebuilt.outputs, outputRecords) + assert.equal(outputRecords.length, 2) + assert.deepEqual(pageModule.events, ['render', 'collect', 'render', 'collect']) + assert.equal(outputCache.get(join(dest, 'data.json')), cached) +}) + +test('page writer retains emitted sidecars on PageData when a provider fails mid-stream', async t => { + const { pd, layouts, dir } = await fixture(t, { + module: "export default () => '

Page

'; export const pageOutputs = () => { throw new Error('page must not run') }", + }) + const dest = join(dir, 'public') + const filepath = join(dest, 'first.txt') + /** @type {PageOutputCache} */ + const outputCache = new Map() + const { outer } = layouts + assert.ok(outer) + const cause = new Error('iterator failed') + outer.pageOutputs = async function * () { + yield { outputName: 'first.txt', content: 'first' } + assert.equal(await readFile(filepath, 'utf8'), 'first') + assert.deepEqual(pd.outputRecords.map(output => output.outputRelname), ['first.txt']) + assert.equal(outputCache.has(filepath), true) + throw cause + } + await pd.init({ layouts }) + const outputRecords = pd.outputRecords + await assert.rejects(pageWriter({ dest, page: pd, outputCache }), error => { + assert.ok(error instanceof Error) + assert.match(error.message, /pageOutputs for page "page.mjs" from layout "outer" failed: Invalid pageOutputs.*iterator failed/) + assert.ok(error.cause instanceof Error) + assert.equal(error.cause.cause, cause) + return true + }) + assert.equal(pd.outputRecords, outputRecords) + assert.deepEqual(outputRecords.map(output => [output.kind, output.outputRelname]), [['page-output', 'first.txt']]) + assert.deepEqual([...outputCache.keys()], [filepath]) +}) + +test('collection invokes providers lazily in outer -> inner -> page order', async t => { + const { pd, layouts } = await fixture(t, { + module: `export default () => '' + export let hookCalls = 0 + export function pageOutputs () { + hookCalls++ + return [{ outputName: 'page.txt', content: 'page' }] + }`, + }) + /** @type {string[]} */ + const events = [] + for (const name of ['outer', 'inner']) { + const layout = layouts[name] + assert.ok(layout) + layout.pageOutputs = () => { + events.push(`${name}:called`) + return (async function * () { + try { + for (const index of [1, 2]) { + events.push(`${name}:${index}`) + yield { outputName: `${name}-${index}.txt`, content: `${index}` } + } + } finally { + events.push(`${name}:closed`) + } + })() + } + } + await pd.init({ layouts }) + const pageModule = await import(pd.pageInfo.pageFile.filepath) + const outputs = pd.collectPageOutputs() + assert.equal(outputs[Symbol.asyncIterator](), outputs) + assert.deepEqual(events, []) + assert.equal(pageModule.hookCalls, 0) + assert.equal((await outputs.next()).value?.outputName, 'outer-1.txt') + assert.deepEqual(events, ['outer:called', 'outer:1']) + assert.equal((await outputs.next()).value?.outputName, 'outer-2.txt') + assert.deepEqual(events, ['outer:called', 'outer:1', 'outer:2']) + assert.equal(pageModule.hookCalls, 0) + assert.equal((await outputs.next()).value?.outputName, 'inner-1.txt') + assert.deepEqual(events, ['outer:called', 'outer:1', 'outer:2', 'outer:closed', 'inner:called', 'inner:1']) + assert.equal((await outputs.next()).value?.outputName, 'inner-2.txt') + assert.equal(pageModule.hookCalls, 0) + assert.equal((await outputs.next()).value?.outputName, 'page.txt') + assert.deepEqual(events, ['outer:called', 'outer:1', 'outer:2', 'outer:closed', 'inner:called', 'inner:1', 'inner:2', 'inner:closed']) + assert.equal(pageModule.hookCalls, 1) + assert.deepEqual(await outputs.next(), { value: undefined, done: true }) +}) + +test('closing collection runs the active provider finally and never invokes later providers', async t => { + for (const close of ['return', 'break']) { + const { pd, layouts } = await fixture(t, { + module: `export default () => '' + export let hookCalls = 0 + export function pageOutputs () { hookCalls++; return [] }`, + }) + /** @type {string[]} */ + const events = [] + const { outer, inner } = layouts + assert.ok(outer) + assert.ok(inner) + outer.pageOutputs = () => { + events.push('outer:called') + return (async function * () { + try { + yield { outputName: 'first.txt', content: 'first' } + events.push('outer:second') + yield { outputName: 'second.txt', content: 'second' } + } finally { + await Promise.resolve() + events.push('outer:closed') + } + })() + } + inner.vars = { dataDeps: ['unready'] } + inner.pageOutputs = () => { events.push('inner:called'); return [] } + await pd.init({ layouts }) + const pageModule = await import(pd.pageInfo.pageFile.filepath) + const unopened = pd.collectPageOutputs() + await unopened.return() + assert.deepEqual(events, []) + const outputs = pd.collectPageOutputs() + if (close === 'return') { + assert.equal((await outputs.next()).value?.outputName, 'first.txt') + assert.deepEqual(events, ['outer:called']) + assert.deepEqual(await outputs.return(), { value: undefined, done: true }) + } else { + // eslint-disable-next-line no-unreachable-loop -- Exercise iterator cleanup on an early break. + for await (const output of outputs) { + assert.equal(output.outputName, 'first.txt') + assert.deepEqual(events, ['outer:called']) + break + } + } + assert.deepEqual(events, ['outer:called', 'outer:closed']) + assert.equal(pageModule.hookCalls, 0) + assert.deepEqual(await outputs.next(), { value: undefined, done: true }) + } +}) + +test('streamed provider errors retain page context and causes after earlier records', async t => { + const { pd, layouts } = await fixture(t, { + module: "export default () => ''; export const pageOutputs = () => { throw new Error('page must not run') }", + }) + const cause = new Error('iterator failed') + let closed = false + const { outer } = layouts + assert.ok(outer) + outer.pageOutputs = async function * () { + try { + yield { outputName: 'first.txt', content: 'first' } + throw cause + } finally { + closed = true + } + } + await pd.init({ layouts }) + const outputs = pd.collectPageOutputs() + assert.equal((await outputs.next()).value?.outputName, 'first.txt') + assert.equal(closed, false) + await assert.rejects(outputs.next(), error => { + assert.ok(error instanceof Error) + assert.match(error.message, /pageOutputs for page "page.mjs" from layout "outer" failed: Invalid pageOutputs.*iterator failed/) + assert.ok(error.cause instanceof Error) + assert.equal(error.cause.cause, cause) + return true + }) + assert.equal(closed, true) +}) + +test('markdown companion gets a frozen narrow source handle and subscribed data', async t => { + const { pd, layouts, dir } = await fixture(t, { + companion: ` + import assert from 'node:assert/strict' + export default { dataDeps: ['selected'] } + export const dataDeps = ['secret'] // A named export is not a subscription declaration. + export async function pageOutputs ({ page, vars, data }) { + assert.equal(Object.isFrozen(page), true) + assert.equal(Object.isFrozen(page.pageFile), true) + assert.equal(Object.isFrozen(vars), true) + for (const key of ['data', 'pageInfo', 'renderInnerPage', 'renderFullPage', 'setGlobalData', 'collectPageOutputs', 'outputRecords']) assert.equal(key in page, false) + assert.throws(() => { page.pageFile.filepath = '/other.md' }, TypeError) + assert.throws(() => data.secret, /undeclared/) + const read = page.readMarkdownContent + return { outputName: 'body.md', content: data.selected + await read.call({ pageInfo: {} }) } + }` + }) + await pd.init({ layouts }) + await assert.rejects(Array.fromAsync(pd.collectPageOutputs()), /companion.*not available/) + pd.setGlobalData({ selected: 'selected:', secret: 'hidden' }) + assert.deepEqual(await Array.fromAsync(pd.collectPageOutputs()), [{ outputName: 'body.md', content: 'selected:\n# Body\n', provenance: { kind: 'companion', source: join(dir, 'page.vars.mjs') } }]) +}) + +test('JS and TS page providers work and cannot read markdown or undeclared data', async t => { + for (const extension of ['mjs', 'ts']) { + const { pd, layouts } = await fixture(t, { + extension, module: ` + import assert from 'node:assert/strict' + export default () => 'page' + export async function pageOutputs ({ page, data }) { + await assert.rejects(page.readMarkdownContent(), /only.*markdown/) + assert.throws(() => data.secret, /undeclared/) + return [] + }` + }) + await pd.init({ layouts }) + pd.setGlobalData({ secret: 'hidden' }) + assert.deepEqual(await Array.fromAsync(pd.collectPageOutputs()), []) + } +}) + +test('companions also provide outputs for JS and HTML pages', async t => { + for (const type of ['js', 'html']) { + const { pd, layouts } = await fixture(t, { + ...(type === 'js' ? { module: "export const vars = { dataDeps: ['selected'] }; export default () => ''" } : {}), + companion: `export default { dataDeps: ['selected'] }; + export const pageOutputs = async ({ data }) => [{ outputName: 'extra.txt', content: data.selected }]`, + }) + if (type === 'html') { + pd.pageInfo.type = 'html' + await writeFile(pd.pageInfo.pageFile.filepath, '

HTML source

') + } + await pd.init({ layouts }) + pd.setGlobalData({ selected: type }) + const outputs = await Array.fromAsync(pd.collectPageOutputs()) + assert.equal(outputs[0]?.content, type) + assert.equal(outputs[0]?.provenance.kind, 'companion') + } +}) + +test('page module outputs take precedence over companion outputs with a warning and additive layouts', async t => { + const warn = t.mock.method(console, 'warn', () => {}) + for (const extension of ['mjs', 'ts']) { + const { pd, layouts, dir } = await fixture(t, { + extension, + module: "export default () => ''; export const pageOutputs = () => ({ outputName: 'page.txt', content: 'page' })", + companion: "export const pageOutputs = () => { throw new Error('ignored companion must not run') }", + }) + for (const name of ['outer', 'inner']) { + const layout = layouts[name] + assert.ok(layout) + layout.pageOutputs = () => ({ outputName: `${name}.txt`, content: name }) + } + await pd.init({ layouts }) + await pd.init({ layouts }) + assert.equal(warn.mock.callCount(), 0) + assert.equal(pd.warnings.length, 1) + const warning = pd.warnings[0] + assert.ok(warning) + assert.equal(warning.code, 'DOM_STACK_WARNING_DUPLICATE_PAGE_OUTPUTS_PROVIDER') + const { message } = warning + assert.ok(message.includes(join(dir, `page.${extension}`))) + assert.ok(message.includes(join(dir, 'page.vars.mjs'))) + assert.match(message, /both export pageOutputs; using the page module export and ignoring the companion export/) + const outputs = await Array.fromAsync(pd.collectPageOutputs()) + assert.deepEqual(outputs.map(({ outputName, content }) => ({ outputName, content })), [ + { outputName: 'outer.txt', content: 'outer' }, + { outputName: 'inner.txt', content: 'inner' }, + { outputName: 'page.txt', content: 'page' }, + ]) + assert.deepEqual(outputs[2]?.provenance, { kind: 'page', source: join(dir, `page.${extension}`) }) + } +}) + +test('invalid output exports identify their sources', async t => { + for (const options of [{ module: "export default () => ''; export const pageOutputs = 1" }, { companion: 'export const pageOutputs = null' }]) { + const { pd, layouts } = await fixture(t, options) + await assert.rejects(pd.init({ layouts }), /pageOutputs.*page.*must be a function/) + } + const { dir } = await fixture(t) + const path = join(dir, 'bad.layout.mjs') + await writeFile(path, "export default () => ''; export const pageOutputs = {}") + await assert.rejects(resolveLayout(path), /pageOutputs.*bad.layout.mjs.*function/) +}) + +test('generated pages skip page, companion and all layout hooks', async t => { + const { pd, layouts } = await fixture(t, { module: "throw new Error('must not import source module')", companion: 'export const pageOutputs = 123' }) + pd.pageInfo.generated = { pagesFile: { pagesFile: pd.pageInfo.pageFile, path: '', name: 'generated' }, children: 'generated' } + for (const layout of Object.values(layouts)) { + layout.vars = { dataDeps: ['unready'] } + layout.pageOutputs = () => { throw new Error('generated hook ran') } + } + await pd.init({ layouts }) + assert.deepEqual(await pd.collectPageOutputs().next(), { value: undefined, done: true }) +}) + +test('discovery excludes ignored providers and disabled drafts but enabled drafts collect outputs', async t => { + const { pd, layouts, dir } = await fixture(t, { + companion: "export const pageOutputs = () => ({ outputName: 'draft.txt', content: 'draft output' })", + }) + await rm(pd.pageInfo.pageFile.filepath) + await writeFile(join(dir, 'page.draft.md'), '# Draft') + await writeFile(join(dir, 'page.js'), "throw new Error('ignored provider must not load')") + const disabled = await identifyPages(dir, { ignore: ['page.js'] }) + assert.equal(disabled.pages.length, 0) + const enabled = await identifyPages(dir, { ignore: ['page.js'], buildDrafts: true }) + assert.equal(enabled.pages.length, 1) + const [pageInfo] = enabled.pages + assert.ok(pageInfo) + assert.equal(pageInfo.draft, true) + pd.pageInfo = pageInfo + await pd.init({ layouts }) + pd.setGlobalData({}) + assert.equal((await Array.fromAsync(pd.collectPageOutputs()))[0]?.content, 'draft output') +}) + +test('page hook failures include page and provider context, and no providers is valid', async t => { + for (const body of ["throw new Error('hook failed')", "return 'bare string'", "return (async function * () { throw new Error('iterator failed') })()"]) { + const { pd, layouts } = await fixture(t, { companion: `export function pageOutputs () { ${body} }` }) + await pd.init({ layouts }) + await assert.rejects(Array.fromAsync(pd.collectPageOutputs()), /pageOutputs for page "page.md" from companion.*page.vars.mjs.*(failed|Record)/) + } + const { pd, layouts } = await fixture(t) + await pd.init({ layouts }) + assert.deepEqual(await Array.fromAsync(pd.collectPageOutputs()), []) +}) diff --git a/lib/build-pages/page-data.js b/lib/build-pages/page-data.js index 0abe277c..d326cd3d 100644 --- a/lib/build-pages/page-data.js +++ b/lib/build-pages/page-data.js @@ -1,6 +1,9 @@ /** * @import { PageInfo } from '../identify-pages.js' + * @import { DomstackManifestRecord } from '../domstack-manifest/index.js' + * @import { DomStackWarning } from '../helpers/domstack-warning.js' * @import { BuilderOptions, InternalPageFunction } from './page-builders/page-writer.js' + * @import { PageOutputsFunction, PageOutputProvenance, CollectedPageOutput } from './page-outputs.js' */ import { readFile } from 'node:fs/promises' @@ -11,6 +14,7 @@ import { createSubscribedData, extractDataDeps } from './data-deps.js' import { DomStackDataError } from '../helpers/domstack-error.js' import pretty from 'pretty' import { resolveLayoutChain } from './resolve-layout-chain.js' +import { createPageOutputsPage, normalizePageOutputs, validatePageOutputsHook } from './page-outputs.js' /** * @typedef {Object} WorkerFiles @@ -25,10 +29,10 @@ import { resolveLayoutChain } from './resolve-layout-chain.js' * @template [V=string] V - The return type of the layout function (defaults to string) * @template {object} [D=Record] - Declared global data. * @param {string} layoutPath - The string path to the layout ESM module. - * @returns {Promise<{ render: InternalLayoutFunction, vars: Partial, parentLayout: string | undefined }>} The resolved layout module exports. + * @returns {Promise<{ render: InternalLayoutFunction, vars: Partial, parentLayout: string | undefined, pageOutputs: PageOutputsFunction | undefined, source: string }>} The resolved layout module exports. */ export async function resolveLayout (layoutPath) { - const { default: layout, vars, parentLayout } = await import(layoutPath) + const { default: layout, vars, parentLayout, pageOutputs } = await import(layoutPath) if (typeof layout !== 'function') throw new TypeError(`Layout "${layoutPath}" must export a default render function`) if (parentLayout !== undefined && (typeof parentLayout !== 'string' || !parentLayout.trim())) { throw new TypeError(`Layout "${layoutPath}" parentLayout must be a non-empty string`) @@ -37,6 +41,8 @@ export async function resolveLayout (layoutPath) { return { render: layout, parentLayout, + source: layoutPath, + pageOutputs: validatePageOutputsHook(pageOutputs, layoutPath), vars: /** @type {Partial} */ (await resolveVarsExport(vars, 'Layout vars')), } } @@ -128,6 +134,8 @@ export async function resolveLayout (layoutPath) { * @typedef ResolvedLayout * @property {InternalLayoutFunction} render - The layout function * @property {Partial} [vars] - Variables exported by the layout module. + * @property {PageOutputsFunction | undefined} [pageOutputs] - Explicit output-phase hook. + * @property {string} [source] - Layout module path for diagnostics. * @property {string} name - The name of the layout * @property {string | undefined} [parentLayout] - Name of the optional outer layout. * @property {string | null} layoutStylePath - The string path to the layout style @@ -143,6 +151,8 @@ export async function resolveLayout (layoutPath) { */ export class PageData { /** @type {PageInfo} */ pageInfo + /** @type {DomStackWarning[]} */ warnings = [] + /** @type {DomstackManifestRecord[]} Successfully emitted files, including partial builds; never populated by rendering alone. */ outputRecords = [] /** @type {ResolvedLayout | null | undefined} */ layout /** @type {ResolvedLayout[]} Each parent has its own render and data contracts. */ layoutChain = [] /** @type {Partial} */ globalVars @@ -151,6 +161,7 @@ export class PageData { /** @type {Partial | null} */ builderVars = null /** @type {string[]} Union of the page and entire layout chain, for output invalidation. */ dataDeps = [] /** @type {string[]} */ #pageDataDeps = [] + /** @type {{ hook: PageOutputsFunction, provenance: PageOutputProvenance } | undefined} */ #pageOutputs /** @type {Map }>} */ #layoutSubscriptions = new Map() /** @type {string[]} */ styles = [] /** @type {string[]} */ scripts = [] @@ -310,7 +321,30 @@ export class PageData { await resolvePostVars({ varsPath: pageVars?.filepath }) // throws if postVars export is detected const builder = pageBuilders[type] - const { vars: builderVars } = await builder({ pageInfo, options: this.builderOptions }) + const built = await builder({ pageInfo, options: this.builderOptions }) + const { vars: builderVars } = built + if (!pageInfo.generated) { + const pageModuleOutputs = type === 'js' ? built.pageOutputs : undefined + const varsCompanionOutputs = pageVars?.filepath + ? validatePageOutputsHook((await import(pageVars.filepath)).pageOutputs, pageVars.filepath) + : undefined + if (pageModuleOutputs && varsCompanionOutputs) { + this.warnings.push({ + code: 'DOM_STACK_WARNING_DUPLICATE_PAGE_OUTPUTS_PROVIDER', + message: `Page "${pageInfo.pageFile.filepath}" and companion "${pageVars?.filepath}" both export pageOutputs; using the page module export and ignoring the companion export`, + }) + } + const hook = pageModuleOutputs ?? varsCompanionOutputs + if (hook) { + this.#pageOutputs = { + hook, + provenance: { + kind: pageModuleOutputs ? 'page' : 'companion', + source: pageModuleOutputs ? pageInfo.pageFile.filepath : /** @type {string} */ (pageVars?.filepath), + }, + } + } + } const layoutName = resolveLayoutName(globalVars, resolvedPageVars, builderVars) @@ -366,6 +400,50 @@ export class PageData { this.#initialized = true } + /** + * Run hooks lazily as the output phase consumes each record. + * Layouts run outermost first, followed by the single page-level provider. + * @returns {AsyncGenerator} + */ + async * collectPageOutputs () { + if (!this.#initialized) throw new Error('Must be initialized before collecting pageOutputs') + if (this.pageInfo.generated) return + // Capture source metadata so rebinding the reader cannot change its source. + const sourceInfo = { ...this.pageInfo, pageFile: { ...this.pageInfo.pageFile } } + const page = createPageOutputsPage(sourceInfo, async () => { + if (sourceInfo.type !== 'md') throw new Error('Markdown content can only be read from markdown pages') + return parseMdFileContents(await readFile(sourceInfo.pageFile.filepath, 'utf8')).markdownContent + }) + const pageData = this + /** + * @param {PageOutputsFunction} hook + * @param {PageOutputProvenance} provenance + * @param {() => object} getData + * @returns {AsyncGenerator} + */ + const collect = async function * (hook, provenance, getData) { + try { + yield * normalizePageOutputs(hook({ page, vars: pageData.vars, data: getData() }), provenance) + } catch (cause) { + throw new Error(`pageOutputs for page "${pageData.pageInfo.pageFile.relname}" from ${provenance.kind} "${provenance.source}" failed: ${cause instanceof Error ? cause.message : String(cause)}`, { cause }) + } + } + for (const layout of this.layoutChain) { + const source = layout.source ?? layout.name + const hook = validatePageOutputsHook(layout.pageOutputs, source) + if (!hook) continue + yield * collect(hook, { kind: 'layout', source, layoutName: layout.name }, () => { + const subscription = this.#layoutSubscriptions.get(layout.name) + if (!this.#dataReady && subscription?.keys.length) throw this.#dataNotReadyError() + return subscription?.data ?? Object.freeze({}) + }) + } + if (this.#pageOutputs) { + const { hook, provenance } = this.#pageOutputs + yield * collect(hook, provenance, () => this.data) + } + } + /** * Render the inner contents of a page. * @returns {Promise>} The page's render value, before any layout runs. diff --git a/lib/build-pages/page-outputs-types.test.ts b/lib/build-pages/page-outputs-types.test.ts new file mode 100644 index 00000000..4b82d6b3 --- /dev/null +++ b/lib/build-pages/page-outputs-types.test.ts @@ -0,0 +1,68 @@ +import type { + PageOutput, + PageOutputProvenance, + PageOutputsFunction, + PageOutputsFunctionParams, + PageOutputsPage, + PageOutputsResult, + CollectedPageOutput, + PageData, + DomstackManifestRecord, +} from '../../types.ts' +import { normalizePageOutputs } from './page-outputs.js' +import { writePageOutputs } from './page-builders/page-output-writer.js' +import type { PageOutputCache } from './page-builders/page-output-writer.js' + +// Compile-only assertions for the public type entry and the narrow hook contract. +export function checkPageOutputsTypes (pageData: PageData<{ title: string }>, page: PageOutputsPage) { + const output: PageOutput = { outputName: 'feed.json', content: '' } + const provenance: PageOutputProvenance = { kind: 'layout', source: 'base.layout.ts', layoutName: 'base' } + const collected: CollectedPageOutput = { ...output, provenance } + const result: PageOutputsResult = [output] + const hook: PageOutputsFunction<{ title: string }, { posts: string[] }> = async ({ page, vars, data }) => ({ + outputName: 'feed.json', content: vars.title + page.url + data.posts.join(','), + }) + const params: PageOutputsFunctionParams<{ title: string }, { posts: string[] }> = { page, vars: { title: 'title' }, data: { posts: [] } } + const outputs: AsyncGenerator = pageData.collectPageOutputs() + const normalized: AsyncGenerator = normalizePageOutputs(result, provenance) + const iterable: AsyncIterable = outputs + const outputRecords: DomstackManifestRecord[] = pageData.outputRecords + const outputCache: PageOutputCache = new Map() + const written: Promise = writePageOutputs({ + dest: 'public', + pageFilePath: 'public/index.html', + page: { pageInfo: pageData.pageInfo, outputRecords }, + pageOutputs: outputs, + outputCache, + }) + const uncached: Promise = writePageOutputs({ + dest: 'public', pageFilePath: 'public/index.html', page: pageData, pageOutputs: [output], + }) + // @ts-expect-error Emitted records describe files, not buffered output content. + const bufferedContent = outputRecords[0]?.content + // @ts-expect-error Hooks cannot access the internal emitted-file records. + const hookRecords = page.outputRecords + // @ts-expect-error Collection is streamed, not a promise of buffered records. + const buffered: Promise = pageData.collectPageOutputs() + const iterator: PageOutputsFunction = async function * () { yield output } + const promisedIterator: PageOutputsFunction = async () => (async function * () { yield output })() + // @ts-expect-error Bare strings are not hook results. + const badHook: PageOutputsFunction = () => 'html' + // @ts-expect-error Content must be a string. + const badOutput: PageOutput = { outputName: 'feed.json', content: {} } + // @ts-expect-error Page metadata is read-only. + page.url = '/other' + // @ts-expect-error Source metadata is read-only. + page.pageFile.filepath = '/other.md' + // @ts-expect-error Hooks cannot render pages. + page.renderFullPage() + // @ts-expect-error Hooks cannot access a PageData instance. + const bypass = page.pageInfo + // @ts-expect-error Global data is only accessible through the subscribed params.data. + const globalData = page.data + // @ts-expect-error Vars are read-only. + params.vars.title = 'other' + // @ts-expect-error Only declared data is available. + const secret = params.data.secret + return { hook, params, outputs, normalized, iterable, outputRecords, outputCache, written, uncached, bufferedContent, hookRecords, buffered, collected, result, iterator, promisedIterator, badHook, badOutput, bypass, globalData, secret } +} diff --git a/lib/build-pages/page-outputs.js b/lib/build-pages/page-outputs.js new file mode 100644 index 00000000..f8f8668f --- /dev/null +++ b/lib/build-pages/page-outputs.js @@ -0,0 +1,99 @@ +/** + * @import { PageInfo } from '../identify-pages.js' + * + * @typedef {object} PageOutput + * @property {string} outputName + * @property {string} content + * + * @typedef {object} PageOutputProvenance + * @property {'page' | 'companion' | 'layout'} kind + * @property {string} source - Provider module path (layout name when no path is available). + * @property {string} [layoutName] + * + * @typedef {PageOutput & { provenance: PageOutputProvenance }} CollectedPageOutput + * @typedef {PageOutput | PageOutput[] | AsyncIterable} PageOutputsResult + * @typedef {Readonly> & { readonly pageFile: Readonly, readonly readMarkdownContent: () => Promise }} PageOutputsPage + */ + +/** + * @template {Record} [T=Record] + * @template {object} [D=Record] + * @typedef {object} PageOutputsFunctionParams + * @property {PageOutputsPage} page - Read-only source metadata, without rendering or global-data access. + * @property {Readonly} vars + * @property {D} data - The same subscriptions as this provider's renderer. + */ + +/** + * @template {Record} [T=Record] + * @template {object} [D=Record] + * @callback PageOutputsFunction + * @param {PageOutputsFunctionParams} params + * @returns {PageOutputsResult | Promise} + */ + +/** + * @param {unknown} hook + * @param {string} source + * @returns {PageOutputsFunction | undefined} + */ +export function validatePageOutputsHook (hook, source) { + if (hook === undefined) return undefined + if (typeof hook !== 'function') throw new TypeError(`pageOutputs in "${source}" must be a function`) + return /** @type {PageOutputsFunction} */ (hook) +} + +/** + * Normalize one provider's result, preserving order and diagnostic context. + * The writer checks each destination and writes it before requesting another record. + * @param {unknown} result + * @param {PageOutputProvenance} provenance + * @returns {AsyncGenerator} + */ +export async function * normalizePageOutputs (result, provenance) { + let recordNumber = 0 + /** + * @param {unknown} record + * @returns {CollectedPageOutput} + */ + const validate = (record) => { + recordNumber++ + if (!record || typeof record !== 'object' || + !('outputName' in record) || typeof record.outputName !== 'string' || !record.outputName.trim() || + !('content' in record) || typeof record.content !== 'string') { + throw new TypeError(`Record ${recordNumber} must be { outputName: non-empty string, content: string }`) + } + return { outputName: record.outputName, content: record.content, provenance: { ...provenance } } + } + try { + const resolved = await result + if (Array.isArray(resolved)) { + for (const record of resolved) yield validate(record) + } else if (resolved && typeof resolved === 'object' && Symbol.asyncIterator in resolved) { + for await (const record of /** @type {AsyncIterable} */ (resolved)) yield validate(record) + } else { + yield validate(resolved) + } + } catch (cause) { + throw new Error(`Invalid pageOutputs from ${provenance.kind} "${provenance.source}": ${cause instanceof Error ? cause.message : String(cause)}`, { cause }) + } +} + +/** + * @param {PageInfo} info + * @param {() => Promise} readMarkdownContent - Bound to the source, never to caller-supplied metadata. + * @returns {PageOutputsPage} + */ +export function createPageOutputsPage (info, readMarkdownContent) { + const { type, path, url, outputName, outputRelname, draft, pageFile } = info + return Object.freeze({ + type, + path, + url, + outputName, + outputRelname, + draft, + pageFile: Object.freeze({ ...pageFile }), + readMarkdownContent, + }) +} diff --git a/lib/build-pages/page-outputs.test.js b/lib/build-pages/page-outputs.test.js new file mode 100644 index 00000000..78a3dd43 --- /dev/null +++ b/lib/build-pages/page-outputs.test.js @@ -0,0 +1,106 @@ +/** @import { PageOutputProvenance } from './page-outputs.js' */ +import { test } from 'node:test' +import assert from 'node:assert/strict' +import { normalizePageOutputs, validatePageOutputsHook } from './page-outputs.js' + +/** @type {PageOutputProvenance} */ +const provenance = { kind: 'page', source: '/src/page.ts' } +const record = { outputName: 'feed.json', content: '' } + +test('pageOutputs normalizes records, arrays, promises and async iterables', async () => { + const expected = [{ ...record, provenance }] + assert.deepEqual(await Array.fromAsync(normalizePageOutputs(record, provenance)), expected) + assert.deepEqual(await Array.fromAsync(normalizePageOutputs(Promise.resolve([record]), provenance)), expected) + async function * records () { yield record; yield { ...record, outputName: 'second.json' } } + assert.deepEqual(await Array.fromAsync(normalizePageOutputs(Promise.resolve(records()), provenance)), [...expected, { ...record, outputName: 'second.json', provenance }]) + assert.deepEqual(await Array.fromAsync(normalizePageOutputs([], provenance)), []) + async function * empty () {} + assert.deepEqual(await Array.fromAsync(normalizePageOutputs(empty(), provenance)), []) +}) + +test('pageOutputs rejects invalid hooks and records with source context', async () => { + for (const hook of [null, true, {}, 'content']) { + assert.throws(() => validatePageOutputsHook(hook, provenance.source), /pageOutputs.*\/src\/page.ts.*function/) + } + for (const result of ['bare string', undefined, null, {}, { outputName: '', content: '' }, { outputName: 'x', content: 1 }, [record, 'bad'], new Set([record])]) { + await assert.rejects(Array.fromAsync(normalizePageOutputs(result, provenance)), /Invalid pageOutputs.*\/src\/page.ts.*Record/) + } + async function * broken () { yield record; throw new Error('iterator failed') } + await assert.rejects(Array.fromAsync(normalizePageOutputs(broken(), provenance)), /\/src\/page.ts.*iterator failed/) + await assert.rejects(Array.fromAsync(normalizePageOutputs(Promise.reject(new Error('promise failed')), provenance)), /\/src\/page.ts.*promise failed/) +}) + +test('normalization pulls one record at a time and closes the provider on return or break', async () => { + for (const close of ['return', 'break']) { + /** @type {string[]} */ + const events = [] + async function * records () { + try { + events.push('first') + yield record + events.push('second') + yield { ...record, outputName: 'second.json' } + } finally { + events.push('closed') + } + } + const outputs = normalizePageOutputs(records(), provenance) + assert.equal(outputs[Symbol.asyncIterator](), outputs) + assert.deepEqual(events, []) + if (close === 'return') { + const first = await outputs.next() + assert.deepEqual(first, { value: { ...record, provenance }, done: false }) + assert.notEqual(first.value?.provenance, provenance) + assert.deepEqual(events, ['first']) + assert.deepEqual(await outputs.return(), { value: undefined, done: true }) + } else { + // eslint-disable-next-line no-unreachable-loop -- Exercise iterator cleanup on an early break. + for await (const output of outputs) { + assert.deepEqual(output, { ...record, provenance }) + assert.deepEqual(events, ['first']) + break + } + } + assert.deepEqual(events, ['first', 'closed']) + assert.deepEqual(await outputs.next(), { value: undefined, done: true }) + } +}) + +test('normalization validates each record only when requested with its record number', async () => { + let closed = false + async function * records () { + try { + yield record + yield { outputName: 'invalid.json', content: 1 } + } finally { + closed = true + } + } + for (const result of [[record, { outputName: 'invalid.json', content: 1 }], records()]) { + const outputs = normalizePageOutputs(result, provenance) + assert.deepEqual(await outputs.next(), { value: { ...record, provenance }, done: false }) + await assert.rejects(outputs.next(), error => { + assert.ok(error instanceof Error) + assert.match(error.message, /Invalid pageOutputs from page "\/src\/page.ts": Record 2/) + assert.ok(error.cause instanceof TypeError) + return true + }) + } + assert.equal(closed, true) +}) + +test('normalization preserves iterator failure causes after yielding valid records', async () => { + const cause = new Error('iterator failed') + async function * records () { + yield record + throw cause + } + const outputs = normalizePageOutputs(records(), provenance) + assert.deepEqual(await outputs.next(), { value: { ...record, provenance }, done: false }) + await assert.rejects(outputs.next(), error => { + assert.ok(error instanceof Error) + assert.match(error.message, /\/src\/page.ts.*iterator failed/) + assert.equal(error.cause, cause) + return true + }) +}) diff --git a/lib/builder.js b/lib/builder.js index 238ba455..57dea202 100644 --- a/lib/builder.js +++ b/lib/builder.js @@ -16,6 +16,7 @@ import { buildCopy } from './build-copy/index.js' import { buildEsbuild, buildServiceWorkerEsbuild } from './build-esbuild/index.js' import { DomStackAggregateError } from './helpers/domstack-aggregate-error.js' import { ensureDest } from './helpers/ensure-dest.js' +import { outputWarnings } from './helpers/output-warnings.js' import { isDomstackManifestEnabled, reconcileDomstackManifest, @@ -197,6 +198,8 @@ export async function builder (src, dest, opts) { pageBuildResults ) + warnings.push(...outputWarnings(baseOutputRecords)) + const domstackManifestReconciliation = domstackManifestEnabled ? await reconcileDomstackManifest({ dest, diff --git a/lib/domstack-manifest/schema.js b/lib/domstack-manifest/schema.js index d973c148..fb8bd34c 100644 --- a/lib/domstack-manifest/schema.js +++ b/lib/domstack-manifest/schema.js @@ -18,6 +18,7 @@ export const domstackManifestKindSchema = /** @satisfies {JSONSchema} */ (/** @t description: 'Classifies the build pipeline step or artifact type that produced this output.', enum: [ 'page', + 'page-output', 'template', 'script', 'style', diff --git a/lib/domstack-manifest/schema.json b/lib/domstack-manifest/schema.json index df3308f9..2f618bc8 100644 --- a/lib/domstack-manifest/schema.json +++ b/lib/domstack-manifest/schema.json @@ -33,6 +33,7 @@ "description": "Classifies the build pipeline step or artifact type that produced this output.", "enum": [ "page", + "page-output", "template", "script", "style", diff --git a/lib/helpers/domstack-warning.js b/lib/helpers/domstack-warning.js index 15d5d1a1..d741d0db 100644 --- a/lib/helpers/domstack-warning.js +++ b/lib/helpers/domstack-warning.js @@ -12,6 +12,8 @@ * 'DOM_STACK_WARNING_DUPLICATE_MARKDOWN_IT_SETTINGS' | * 'DOM_STACK_WARNING_DUPLICATE_DOMSTACK_MANIFEST_SETTINGS' | * 'DOM_STACK_WARNING_CONFLICTING_MANIFEST_OUTPUT' | + * 'DOM_STACK_WARNING_DUPLICATE_OUTPUT' | + * 'DOM_STACK_WARNING_DUPLICATE_PAGE_OUTPUTS_PROVIDER' | * 'DOM_STACK_WARNING_DUPLICATE_GLOBAL_VARS' | * 'DOM_STACK_WARNING_DUPLICATE_GLOBAL_DATA' | * 'DOM_STACK_WARNING_PAGE_MD_SHADOWS_README' diff --git a/lib/helpers/output-warnings.js b/lib/helpers/output-warnings.js new file mode 100644 index 00000000..17b11a5d --- /dev/null +++ b/lib/helpers/output-warnings.js @@ -0,0 +1,27 @@ +/** + * @import { DomstackManifestRecord } from '../domstack-manifest/index.js' + * @import { DomStackWarning } from './domstack-warning.js' + */ +import { resolve } from 'node:path' + +/** + * Report duplicate destinations observed in build reports. This does not reserve + * paths or control writes; watch phases can only report the outputs they see. + * @param {DomstackManifestRecord[]} outputs + * @returns {DomStackWarning[]} + */ +export function outputWarnings (outputs) { + const destinations = new Map() + const warnings = /** @type {DomStackWarning[]} */ ([]) + for (const output of outputs) { + const path = resolve(output.filepath) + const previous = destinations.get(path) + if (previous) { + warnings.push({ + code: 'DOM_STACK_WARNING_DUPLICATE_OUTPUT', + message: `Duplicate output "${output.outputRelname}": ${previous.sourceRelname ?? previous.kind} (${previous.kind}) and ${output.sourceRelname ?? output.kind} (${output.kind}) write the same destination; output may be overwritten.`, + }) + } else destinations.set(path, output) + } + return warnings +} diff --git a/lib/helpers/path.js b/lib/helpers/path.js index a042cdcd..0cbf45bc 100644 --- a/lib/helpers/path.js +++ b/lib/helpers/path.js @@ -20,7 +20,7 @@ export function assertInsideDest (dest, filepath, message = `Output path escapes const absDest = resolve(dest) const absFilepath = resolve(filepath) const rel = relative(absDest, absFilepath) - if (rel !== '' && (rel.startsWith('..') || isAbsolute(rel))) { + if (rel !== '' && (rel === '..' || rel.startsWith(`..${sep}`) || isAbsolute(rel))) { throw new Error(message) } } diff --git a/test-cases/generated-pages/index.test.js b/test-cases/generated-pages/index.test.js index 009246ce..270b293b 100644 --- a/test-cases/generated-pages/index.test.js +++ b/test-cases/generated-pages/index.test.js @@ -529,7 +529,7 @@ export default function indexesPages ({ data }) { await withTempFixture({ 'root.layout.js': minimalRootLayout, 'global.vars.js': minimalGlobalVars, - 'invalid.pages.js': 'export default [{ outputName: "valid/index.html" }, 42]\n', + 'invalid.pages.js': 'export default [{ outputName: "valid/index.html", children: "Published before validation failure" }, 42]\n', }, async ({ src, dest }) => { const domstack = new DomStack(src, dest) await assert.rejects( @@ -544,6 +544,7 @@ export default function indexesPages ({ data }) { return true } ) + assert.match(await readFile(join(dest, 'valid/index.html'), 'utf8'), /Published before validation failure/) }) }) diff --git a/test-cases/generated-pages/streaming.test.js b/test-cases/generated-pages/streaming.test.js new file mode 100644 index 00000000..69e7c611 --- /dev/null +++ b/test-cases/generated-pages/streaming.test.js @@ -0,0 +1,323 @@ +/** + * @import { TestContext } from 'node:test' + * @import { Results } from '../../lib/builder.js' + */ +import { test } from 'node:test' +import assert from 'node:assert/strict' +import { mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises' +import { join } from 'node:path' +import pino from 'pino' +import { DomStack } from '../../index.js' +import { builder } from '../../lib/builder.js' +import { DomStackAggregateError } from '../../lib/helpers/domstack-aggregate-error.js' +import { errorText, settle, writeFiles } from '../page-outputs/helpers.js' + +/** @param {TestContext} t @param {Record} files @param {boolean} [buildDrafts] */ +async function setup (t, files, buildDrafts = false) { + // index.test.js sweeps .tmp-* directories; these fixtures must survive parallel test files. + const root = await mkdtemp(join(import.meta.dirname, '.streaming-')) + const src = join(root, 'src') + const dest = join(root, 'custom-output') + const logs = /** @type {string[]} */ ([]) + const options = { static: true, domstackManifest: false, buildDrafts, logger: pino({ level: 'debug' }, { write: line => logs.push(line) }) } + const site = new DomStack(src, dest, options) + t.after(async () => { + if (site.watching) await site.stopWatching() + await rm(root, { recursive: true, force: true }) + }) + await writeFiles(src, { + 'global.vars.js': `export default { layout: 'root', title: 'Global', testRoot: ${JSON.stringify(root)}, testDest: ${JSON.stringify(dest)} }`, + 'root.layout.js': "export default ({ children }) => '
' + children + '
'", + ...files, + }) + return { + src, + dest, + root, + site, + logs, + build: () => builder(src, dest, options), + /** @param {string} name */ + read: name => readFile(join(dest, name), 'utf8'), + } +} + +/** + * @param {Results['pageBuildResults']} result + * @param {string} src + * @param {string} dest + * @param {string} outputRelname + * @param {string} owner + * @param {number} index + */ +function assertReported (result, src, dest, outputRelname, owner, index) { + assert.ok(result, 'page build results survive worker transport') + const output = result.outputs.find(output => output.outputRelname === outputRelname) + assert.ok(output, `${outputRelname} retains output metadata`) + assert.equal(output.kind, 'page') + assert.equal(output.filepath, join(dest, outputRelname)) + assert.equal(output.sourceRelname, `${owner}#${index}`) + const report = result.report.pages.find(page => page.outputs.some(output => output.outputRelname === outputRelname)) + assert.ok(report, `${outputRelname} retains an ownership report`) + assert.equal(report.pagesFilePath, join(src, owner)) + assert.equal(report.sourcePageFilePath, undefined) + assert.equal(report.pageFilePath, join(dest, outputRelname)) + assert.deepEqual(report.outputs.find(record => record.outputRelname === outputRelname), output) +} + +for (const [form, factoryExport] of Object.entries({ + 'generator function': 'export default pages', + 'static async iterable': 'export default pages()', + 'async function returning an iterable': 'export default async () => pages()', +})) { + test(`${form} renders and writes each page before requesting the next definition`, async t => { + const { build, src, dest, read } = await setup(t, { + 'concrete/page.js': 'export default ({ vars }) => vars.title', + 'concrete/page.vars.js': "export default async () => ({ title: 'Initialized concrete' })", + 'global.data.js': `import assert from 'node:assert/strict' + export default async ({ pages }) => { + assert.equal(pages.length, 1) + assert.equal(pages[0].pageInfo.generated, undefined) + assert.equal(pages[0].vars.title, 'Initialized concrete') + assert.equal(await pages[0].renderInnerPage(), 'Initialized concrete') + return { collection: pages.map(page => page.vars.title), pageValue: 'Page data', layoutValue: 'Layout data' } + }`, + 'generated.layout.js': `import assert from 'node:assert/strict' + export const vars = { title: 'Layout', layoutOnly: 'Resolved layout', dataDeps: ['layoutValue'] } + export default ({ vars, children, data }) => { + assert.throws(() => data.pageValue, /undeclared/) + return '
' + vars.title + ':' + vars.layoutOnly + ':' + data.layoutValue + ':' + children + '
' + } + export const pageOutputs = () => { throw Error('generated layout output hook must be skipped') }`, + 'stream.pages.js': `import assert from 'node:assert/strict' + import { readFile } from 'node:fs/promises' + import { join } from 'node:path' + import globals from './global.vars.js' + export const dataDeps = ['collection'] + export const pageOutputs = () => { throw Error('pages-file output hook must be skipped') } + async function* pages (context) { + if (context) { + assert.deepEqual(context.data.collection, ['Initialized concrete']) + assert.equal(context.pagesFile.pagesFile.relname, 'stream.pages.js') + assert.throws(() => context.data.pageValue, /undeclared/) + } + for (const name of ['first', 'second']) { + yield { + outputName: name + '/index.html', + vars: { layout: 'generated', title: name, dataDeps: ['pageValue'] }, + children: async ({ vars, data }) => { + assert.equal(vars.layoutOnly, 'Resolved layout') + assert.throws(() => data.layoutValue, /undeclared/) + return data.pageValue + }, + } + assert.equal(await readFile(join(globals.testDest, name, 'index.html'), 'utf8'), + '
' + name + ':Resolved layout:Layout data:Page data
') + } + } + ${factoryExport}`, + }) + await writeFiles(dest, { 'first/index.html': 'stale HTML must be replaced before the next pull' }) + const result = await build().catch(error => { + t.diagnostic(errorText(error)) + throw error + }) + for (const [index, name] of ['first', 'second'].entries()) { + assert.equal(await read(`${name}/index.html`), `
${name}:Resolved layout:Layout data:Page data
`) + assertReported(result.pageBuildResults, src, dest, `${name}/index.html`, 'stream.pages.js', index) + } + assert.equal(await read('concrete/index.html'), '
Initialized concrete
') + assert.equal(result.pageBuildResults?.outputs.length, 3, 'generated hooks produce no extra files') + }) +} + +test('single, array and function exports preserve defaults, empty children and nullish results', async t => { + const { build, read } = await setup(t, { + 'nested/single.pages.js': 'export default {}', + 'array.pages.js': `export default [ + { children: undefined, vars: undefined, outputName: undefined }, + { outputName: 'null-child.html', children: null }, + { outputName: 'inline.html', children: async () => 'Inline' }, + ]`, + 'sync.pages.js': 'export default ({ vars }) => ({ children: vars.title })', + 'async.pages.js': "export default async () => [{ outputName: 'async.html', children: 'Async' }]", + 'null.pages.js': 'export default null', + 'undefined.pages.js': 'export default undefined', + 'null-function.pages.js': 'export default () => null', + 'undefined-function.pages.js': 'export default async () => undefined', + 'empty.pages.js': 'export default []', + 'empty-iterator.pages.js': 'export default async function* () {}', + }) + const result = await build() + const expected = { + 'nested/single/index.html': '', + 'array/index.html': '', + 'null-child.html': '', + 'inline.html': 'Inline', + 'sync/index.html': 'Global', + 'async.html': 'Async', + } + assert.deepEqual(result.pageBuildResults?.outputs.map(output => output.outputRelname).sort(), Object.keys(expected).sort()) + for (const [name, content] of Object.entries(expected)) assert.equal(await read(name), `
${content}
`) +}) + +for (const buildDrafts of [false, true]) { + test(`draft yields retain their source indices with buildDrafts=${buildDrafts}`, async t => { + const { build, src, dest, read } = await setup(t, { + 'drafts.pages.js': `export default async function* () { + yield { outputName: 'draft.html', draft: true, children: 'Draft' } + yield { outputName: 'published.html', children: 'Published' } + yield { outputName: 'another-draft.html', draft: true, children: 'Another draft' } + yield { outputName: 'last.html', children: 'Last' } + }`, + }, buildDrafts) + const result = await build() + assertReported(result.pageBuildResults, src, dest, 'published.html', 'drafts.pages.js', 1) + assertReported(result.pageBuildResults, src, dest, 'last.html', 'drafts.pages.js', 3) + assert.equal(result.pageBuildResults?.outputs.length, buildDrafts ? 4 : 2) + for (const [name, index, content] of /** @type {[string, number, string][]} */ ([['draft.html', 0, 'Draft'], ['another-draft.html', 2, 'Another draft']])) { + if (buildDrafts) { + assertReported(result.pageBuildResults, src, dest, name, 'drafts.pages.js', index) + assert.equal(await read(name), `
${content}
`) + } else { + await assert.rejects(stat(join(dest, name)), { code: 'ENOENT' }) + } + } + }) +} + +for (const scenario of [ + { name: 'invalid definition', operation: 'yield 42', message: /Generated page definition must be an object/ }, + { name: 'null yielded definition', operation: 'yield null', message: /Generated page definition must be an object/ }, + { name: 'invalid path', operation: "yield { outputName: '../escape.html' }", message: /must not contain "\.\." segments/ }, + { name: 'collision', operation: "yield { outputName: 'first.html', children: 'Must not overwrite' }", message: /Output path conflict/ }, + { name: 'page render', operation: "yield { outputName: 'broken.html', children: () => { throw Error('stream render failed') } }", message: /stream render failed/ }, + { name: 'layout render', operation: "yield { outputName: 'broken.html', vars: { layout: 'broken' } }", message: /stream layout failed/ }, + { name: 'vars initialization', operation: "yield { outputName: 'broken.html', vars: { dataDeps: false } }", message: /dataDeps/ }, + { name: 'data binding', operation: "yield { outputName: 'broken.html', vars: { dataDeps: ['missing'] } }", message: /missing/ }, + { name: 'factory', operation: "throw Error('stream factory failed')", message: /stream factory failed/ }, +]) { + test(`${scenario.name} failure closes the generator without pulling later pages and retains earlier reports`, async t => { + const { build, src, dest, root, read } = await setup(t, { + 'broken.layout.js': "export default () => { throw Error('stream layout failed') }", + 'stream.pages.js': String.raw`import { appendFile } from 'node:fs/promises' + import { join } from 'node:path' + export default async function* ({ vars }) { + const trace = join(vars.testRoot, 'trace.txt') + try { + await appendFile(trace, 'first\n') + yield { outputName: 'first.html', vars: { title: 'Published' }, children: 'Published' } + await appendFile(trace, 'bad\n') + ${scenario.operation} + await appendFile(trace, 'later\n') + yield { outputName: 'later.html', children: 'Must not render' } + } finally { + await appendFile(trace, 'closed\n') + } + }`, + }) + await writeFiles(dest, { 'broken.html': 'previous HTML' }) + await assert.rejects(build(), error => { + assert.ok(error instanceof DomStackAggregateError) + assert.match(errorText(error), scenario.message) + assert.match(errorText(error), /stream\.pages\.js/) + if (scenario.name === 'collision') { + assert.deepEqual(error.errors[0].conflict, { + outputPath: 'first.html', + a: { type: 'page', path: 'stream.pages.js#0' }, + b: { type: 'page', path: 'stream.pages.js#1' }, + }) + } + const results = /** @type {Results} */ (error.results) + assertReported(results.pageBuildResults, src, dest, 'first.html', 'stream.pages.js', 0) + assert.equal(results.pageBuildResults?.outputs.length, 1) + assert.equal(results.pageBuildResults?.outputs[0]?.pageVars?.['title'], 'Published') + return true + }) + assert.equal(await readFile(join(root, 'trace.txt'), 'utf8'), 'first\nbad\nclosed\n', 'finally is awaited and no following yield is requested') + assert.equal(await read('first.html'), '
Published
') + assert.equal(await read('broken.html'), 'previous HTML') + await assert.rejects(stat(join(dest, 'later.html')), { code: 'ENOENT' }) + await assert.rejects(stat(join(root, 'escape.html')), { code: 'ENOENT' }) + }) +} + +test('sibling factories publish unique outputs with independent owner metadata', async t => { + const { build, src, dest, read } = await setup(t, Object.fromEntries(['a', 'b'].map(name => [ + `${name}.pages.js`, + `export default async function* () { + yield { outputName: '${name}/one.html', children: '${name} one' } + yield { outputName: '${name}/two.html', children: '${name} two' } + }`, + ]))) + const result = await build() + assert.equal(result.pageBuildResults?.outputs.length, 4) + for (const owner of ['a', 'b']) { + for (const [index, name] of ['one', 'two'].entries()) { + const output = `${owner}/${name}.html` + assert.equal(await read(output), `
${owner} ${name}
`) + assertReported(result.pageBuildResults, src, dest, output, `${owner}.pages.js`, index) + } + } +}) + +/** @param {string[]} names @param {string} [failure] */ +function watchFactory (names, failure) { + return `export default async function* () { + ${names.map(name => `yield { outputName: '${name}.html', children: '${name}' }`).join('\n')} + ${failure ? `throw Error('${failure}')` : ''} + }` +} + +for (const change of ['recovery', 'deletion', 'empty result']) { + test(`watch ${change} cleans successful and repeated partial factory ownership`, { timeout: 30_000 }, async t => { + const { site, src, dest, read, logs } = await setup(t, { + 'stream.pages.js': watchFactory(['old', 'stale']), + 'sibling.pages.js': watchFactory(['sibling']), + }) + await site.watch({ serve: false }) + const sibling = await read('sibling.html') + for (const name of ['partial', 'second-partial']) { + await settle(site, logs, async () => { + await writeFile(join(src, 'stream.pages.js'), watchFactory([name], `${name} failure`)) + }, `${name} failure`) + assert.equal(await read(`${name}.html`), `
${name}
`) + assert.equal(await read('old.html'), '
old
') + assert.equal(await read('stale.html'), '
stale
') + assert.equal(await read('partial.html'), '
partial
', 'repeated failure keeps earlier partial ownership') + } + await settle(site, logs, async () => { + if (change === 'deletion') await rm(join(src, 'stream.pages.js')) + else await writeFile(join(src, 'stream.pages.js'), change === 'empty result' ? 'export default null' : watchFactory(['recovered'])) + }) + for (const name of ['old', 'stale', 'partial', 'second-partial']) { + await assert.rejects(stat(join(dest, `${name}.html`)), { code: 'ENOENT' }) + } + if (change === 'recovery') assert.equal(await read('recovered.html'), '
recovered
') + assert.equal(await read('sibling.html'), sibling, 'cleanup preserves sibling factory output') + await assert.rejects(stat(join(dest, 'domstack-manifest.json')), { code: 'ENOENT' }) + }) +} + +for (const change of ['recovery', 'deletion', 'empty result']) { + test(`initial failed watch retains partial generated page reports for ${change}`, { timeout: 30_000 }, async t => { + const { site, src, dest, read, logs } = await setup(t, { + 'stream.pages.js': watchFactory(['partial', 'nested/partial'], 'initial stream failure'), + }) + const result = await site.watch({ serve: false }) + assert.ok(logs.some(line => JSON.parse(line).msg === 'Build Failed!')) + assert.ok(logs.some(line => line.includes('initial stream failure'))) + for (const [index, name] of ['partial', 'nested/partial'].entries()) { + assert.equal(await read(`${name}.html`), `
${name}
`) + assertReported(result.pageBuildResults, src, dest, `${name}.html`, 'stream.pages.js', index) + } + await settle(site, logs, async () => { + if (change === 'deletion') await rm(join(src, 'stream.pages.js')) + else await writeFile(join(src, 'stream.pages.js'), change === 'empty result' ? 'export default []' : watchFactory(['recovered'])) + }) + for (const name of ['partial', 'nested/partial']) { + await assert.rejects(stat(join(dest, `${name}.html`)), { code: 'ENOENT' }) + } + if (change === 'recovery') assert.equal(await read('recovered.html'), '
recovered
') + }) +} diff --git a/test-cases/page-outputs/cache.test.js b/test-cases/page-outputs/cache.test.js new file mode 100644 index 00000000..bd77de57 --- /dev/null +++ b/test-cases/page-outputs/cache.test.js @@ -0,0 +1,183 @@ +import { test } from 'node:test' +import assert from 'node:assert/strict' +import { rm, stat, utimes, writeFile } from 'node:fs/promises' +import { join } from 'node:path' +import { hook, setup, settle } from './helpers.js' + +// Install the guard inside each build worker; parent-side reads remain available +// for assertions, and syncBuiltinESMExports also guards already-imported bindings. +const noOutputReadsLayout = `import fs from 'node:fs/promises' +import { syncBuiltinESMExports } from 'node:module' +const readFile = fs.readFile +fs.readFile = async (...args) => { + if (String(args[0]).endsWith('/cached.txt')) throw Error('page-output writer reread destination bytes') + return readFile(...args) +} +syncBuiltinESMExports() +export default ({ children }) => children` + +test('watch caches identical hook bytes across workers without rereads and recreates deleted or cleaned outputs', { timeout: 30_000 }, async t => { + const companionSource = 'export default {}; ' + hook('cached.txt', 'cached bytes') + const { site, src, dest, read, mtime, logs } = await setup(t, { + 'root.layout.js': noOutputReadsLayout, + 'page.html': 'initial main', + 'page.vars.js': companionSource, + }) + await site.watch({ serve: false }) + assert.equal(await read('cached.txt'), 'cached bytes') + const originalTime = await mtime('cached.txt') + for (const content of ['first rebuild', 'second rebuild']) { + await settle(site, logs, async () => { + await writeFile(join(src, 'page.html'), content) + }) + assert.equal(await read('index.html'), content, 'the hook owner actually rebuilt') + assert.equal(await read('cached.txt'), 'cached bytes') + assert.equal(await mtime('cached.txt'), originalTime, 'identical bytes retain their mtime across workers') + } + + await rm(join(dest, 'cached.txt')) + await assert.rejects(read('cached.txt'), { code: 'ENOENT' }) + await settle(site, logs, async () => { + await writeFile(join(src, 'page.html'), 'rebuild after destination deletion') + }) + assert.equal(await read('cached.txt'), 'cached bytes', 'a cache hit must still validate destination existence') + assert.notEqual(await mtime('cached.txt'), originalTime) + + await settle(site, logs, async () => { + await writeFile(join(src, 'page.vars.js'), 'export default {}') + }) + await assert.rejects(read('cached.txt'), { code: 'ENOENT' }) + await settle(site, logs, async () => { + await writeFile(join(src, 'page.vars.js'), companionSource) + }) + assert.equal(await read('cached.txt'), 'cached bytes', 're-adding the same hook recreates the same destination and content') + const restoredTime = await mtime('cached.txt') + await settle(site, logs, async () => { + await writeFile(join(src, 'page.html'), 'rebuild after hook re-addition') + }) + assert.equal(await mtime('cached.txt'), restoredTime, 're-added outputs participate in caching again') +}) + +test('watch repairs same-size external edits with exactly restored mtime using changed ctime', { timeout: 15_000 }, async t => { + // Normalize writes before the writer records metadata, so utimes can restore + // mtime exactly even on filesystems whose native timestamps have nanoseconds. + const timestamp = 1_600_000_000 + const { site, src, dest, read, logs } = await setup(t, { + 'root.layout.js': `import fs from 'node:fs/promises' + import { syncBuiltinESMExports } from 'node:module' + const writeFile = fs.writeFile + fs.writeFile = async (...args) => { + await writeFile(...args) + if (String(args[0]).endsWith('/metadata.txt')) await fs.utimes(args[0], ${timestamp}, ${timestamp}) + } + syncBuiltinESMExports() + export default ({ children }) => children`, + 'page.html': 'initial main', + 'page.vars.js': 'export default {}; ' + hook('metadata.txt', 'original'), + }) + await site.watch({ serve: false }) + const output = join(dest, 'metadata.txt') + const original = await stat(output, { bigint: true }) + assert.equal(original.mtimeNs, BigInt(timestamp) * 1_000_000_000n) + await settle(site, logs, async () => { + await writeFile(join(src, 'page.html'), 'unchanged sidecar rebuild') + }) + assert.equal((await stat(output, { bigint: true })).ctimeNs, original.ctimeNs, 'the normalized output was cached, not rewritten') + + await writeFile(output, 'tampered') + await utimes(output, timestamp, timestamp) + const modified = await stat(output, { bigint: true }) + assert.equal(modified.size, original.size) + assert.equal(modified.mtimeNs, original.mtimeNs, 'mtime is restored exactly, not merely within a tolerance') + assert.equal(modified.ino, original.ino) + assert.equal(modified.dev, original.dev) + assert.notEqual(modified.ctimeNs, original.ctimeNs, 'ctime is the changed cache-validation metadata') + assert.equal(await read('metadata.txt'), 'tampered') + await settle(site, logs, async () => { + await writeFile(join(src, 'page.html'), 'rebuild after external edit') + }) + assert.equal(await read('metadata.txt'), 'original', 'matching hash, size, inode and mtime cannot hide an external edit') +}) + +test('watch retains cached writes before iterator failure through another failed worker and recovery', { timeout: 30_000 }, async t => { + const outputs = `yield { outputName: 'existing.txt', content: 'updated before failure' } + yield { outputName: 'partial.txt', content: 'new before failure' }` + const { site, src, dest, read, mtime, logs } = await setup(t, { + 'page.js': `export default () => 'initial main'; export const pageOutputs = () => [ + { outputName: 'existing.txt', content: 'initial sidecar' }, + { outputName: 'stale.txt', content: 'keep until recovery' }, + ]`, + }) + await site.watch({ serve: false }) + const mainTime = await mtime('index.html') + const existingTime = await mtime('existing.txt') + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), `export default () => 'failed main'; export async function* pageOutputs () { + ${outputs} + throw Error('first cache iterator failure') + }`) + }, 'first cache iterator failure') + assert.equal(await read('existing.txt'), 'updated before failure') + assert.equal(await read('partial.txt'), 'new before failure') + assert.notEqual(await mtime('existing.txt'), existingTime) + const cachedTimes = [await mtime('existing.txt'), await mtime('partial.txt')] + + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), `export default () => 'failed again'; export async function* pageOutputs () { + ${outputs} + throw Error('second cache iterator failure') + }`) + }, 'second cache iterator failure') + assert.deepEqual([await mtime('existing.txt'), await mtime('partial.txt')], cachedTimes, 'failed workers retain both updated and newly created cache entries') + assert.equal(await read('index.html'), 'initial main') + assert.equal(await mtime('index.html'), mainTime) + assert.equal(await read('stale.txt'), 'keep until recovery') + + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), `export default () => 'recovered main'; export async function* pageOutputs () { + ${outputs} + }`) + }) + assert.equal(await read('index.html'), 'recovered main') + assert.equal(await read('existing.txt'), 'updated before failure') + assert.equal(await read('partial.txt'), 'new before failure') + assert.deepEqual([await mtime('existing.txt'), await mtime('partial.txt')], cachedTimes, 'recovery also skips unchanged outputs from failed workers') + await assert.rejects(stat(join(dest, 'stale.txt')), { code: 'ENOENT' }) +}) + +for (const writer of ['template', 'page']) { + test(`watch repairs a cached hook destination overwritten by another ${writer}`, { timeout: 20_000 }, async t => { + const otherSource = writer === 'template' ? 'shared.template.js' : 'shared.md' + const otherContent = writer === 'template' + ? "export default () => ({ outputName: 'shared.html', content: 'other writer' })" + : 'other writer' + const { site, src, read, mtime, logs } = await setup(t, { + 'page.js': "export default () => 'initial owner'", + [otherSource]: writer === 'template' + ? "export default () => ({ outputName: 'shared.html', content: 'initial other writer' })" + : 'initial other writer', + }) + await site.watch({ serve: false }) + // Add the hook only after the competing output exists, avoiding concurrent + // writes to the same destination during the initial build. + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), "export default () => 'seeded owner'; " + hook('shared.html', 'hook content')) + }) + assert.equal(await read('shared.html'), 'hook content') + const ownerTime = await mtime('index.html') + await settle(site, logs, async () => { + await writeFile(join(src, otherSource), otherContent) + }) + if (writer === 'template') { + assert.equal(await read('shared.html'), 'other writer') + } else { + assert.match(await read('shared.html'), /

other writer<\/p>/) + } + assert.equal(await mtime('index.html'), ownerTime, 'the competing writer rebuild leaves the hook owner untouched') + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), "export default () => 'repaired owner'; " + hook('shared.html', 'hook content')) + }) + assert.equal(await read('index.html'), 'repaired owner') + assert.equal(await read('shared.html'), 'hook content', 'the next hook owner rebuild detects another writer despite unchanged hook bytes') + }) +} diff --git a/test-cases/page-outputs/helpers.js b/test-cases/page-outputs/helpers.js new file mode 100644 index 00000000..58d9e524 --- /dev/null +++ b/test-cases/page-outputs/helpers.js @@ -0,0 +1,90 @@ +/** + * @import { TestContext } from 'node:test' + */ +import { mkdir, mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises' +import { dirname, join } from 'node:path' +import { inspect } from 'node:util' +import pino from 'pino' +import { DomStack } from '../../index.js' +import { builder } from '../../lib/builder.js' + +/** @param {string} root @param {Record} files */ +export async function writeFiles (root, files) { + for (const [name, content] of Object.entries(files)) { + const path = join(root, name) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, content) + } +} + +/** @param {TestContext} t @param {Record} files */ +export async function setup (t, files) { + const tmp = await mkdtemp(join(import.meta.dirname, '.tmp-')) + const src = join(tmp, 'src') + const dest = join(tmp, 'custom-output') + await writeFiles(src, { + 'global.vars.js': "export default { layout: 'root' }", + 'root.layout.js': 'export default ({ children }) => children', + ...files, + }) + const logs = /** @type {string[]} */ ([]) + const options = { static: true, domstackManifest: false, logger: pino({ level: 'debug' }, { write: line => logs.push(line) }) } + const site = new DomStack(src, dest, options) + t.after(async () => { + if (site.watching) await site.stopWatching() + await rm(tmp, { recursive: true, force: true }) + }) + return { + src, + dest, + site, + logs, + build: () => builder(src, dest, options), + /** @param {string} name */ + read: name => readFile(join(dest, name), 'utf8'), + /** @param {string} name */ + mtime: async name => (await stat(join(dest, name))).mtimeMs, + } +} + +/** + * Capture the cursor before mutating sources so startup/previous builds cannot + * satisfy the wait, even when chokidar has not detected the change yet. + * @param {DomStack} site + * @param {string[]} logs + * @param {() => Promise} mutate + * @param {string} [expectedError] + */ +export async function settle (site, logs, mutate, expectedError) { + const cursor = logs.length + await mutate() + const deadline = performance.now() + 10_000 + while (true) { + const messages = logs.slice(cursor).map(line => JSON.parse(line).msg) + if (messages.includes('Build Failed!')) { + if (!expectedError || !messages.some(message => message.includes(expectedError))) { + throw new Error(`Unexpected watch build failure:\n${messages.join('\n')}`) + } + break + } + if (messages.includes('Build Success!') && !expectedError) break + if (performance.now() >= deadline) { + throw new Error(`Timed out waiting for watch ${expectedError ? `failure: ${expectedError}` : 'build success'}:\n${messages.join('\n')}`) + } + await new Promise(resolve => setTimeout(resolve, 25)) + } + await site.settled() + const errors = logs.slice(cursor).filter(line => JSON.parse(line).level >= 50) + if (!expectedError && errors.length) throw new Error(`Unexpected watch errors:\n${errors.join('\n')}`) +} + +/** @param {unknown} error @returns {string} */ +export function errorText (error) { + if (!(error instanceof Error)) return inspect(error, { depth: null }) + return [error.message, error.cause ? errorText(error.cause) : '', + ...('errors' in error && Array.isArray(error.errors) ? error.errors.map(errorText) : []), + ].join('\n') +} + +/** @param {string} outputName @param {string} [content] */ +export const hook = (outputName, content = 'sidecar') => `export const pageOutputs = () => ({ outputName: ${JSON.stringify(outputName)}, content: ${JSON.stringify(content)} })` diff --git a/test-cases/page-outputs/index.test.js b/test-cases/page-outputs/index.test.js new file mode 100644 index 00000000..00d802bf --- /dev/null +++ b/test-cases/page-outputs/index.test.js @@ -0,0 +1,321 @@ +import { test } from 'node:test' +import assert from 'node:assert/strict' +import { stat, utimes } from 'node:fs/promises' +import { join } from 'node:path' +import { errorText, hook, setup, writeFiles } from './helpers.js' + +const rawLayout = `export default ({ children }) => '

' + children + '
' +export const pageOutputs = async ({ page }) => ({ outputName: './source.txt', content: await page.readMarkdownContent() })` + +test('builder renders Markdown and exports the unrendered body from its layout at a custom destination', async t => { + const body = '# Article\n\nKeep **Markdown**, {{ vars.title }}, and [links](./other.md).\n' + const { build, read, dest } = await setup(t, { + 'root.layout.js': rawLayout, + 'docs/page.md': '---\ntitle: Resolved title\n---\n' + body, + }) + const result = await build() + assert.match(await read('docs/index.html'), /
\s*

Markdown<\/strong>/) + assert.equal(await read('docs/source.txt'), '\n' + body) + const record = result.pageBuildResults?.outputs.find(output => output.outputRelname === 'docs/source.txt') + assert.ok(record, 'page output is included in the page build report') + assert.equal(record.filepath, join(dest, 'docs/source.txt')) + assert.equal(record.sourceRelname, 'docs/page.md') +}) + +test('nested hooks run outer -> inner -> companion with isolated renderer data and resolved vars', async t => { + const { build, read } = await setup(t, { + 'global.vars.js': "export default { layout: 'inner', title: 'global' }; export const pageOutputs = () => { throw Error('global provider ran') }", + 'global.data.js': "export default { outer: 'O', inner: 'I', selected: 'P', secret: 'hidden' }", + 'root.layout.js': `import assert from 'node:assert/strict' + export const vars = { dataDeps: ['outer'] } + export default ({ children, data }) => data.outer + children + export const pageOutputs = ({ page, vars, data }) => { + assert.equal(vars.title, 'page title') + assert.throws(() => data.selected, /undeclared/) + assert.equal('renderFullPage' in page, false) + assert.equal('data' in page, false) + globalThis[page.pageFile.filepath] = ['outer'] + return { outputName: 'outer.txt', content: data.outer } + }`, + 'inner.layout.js': `import assert from 'node:assert/strict' + import { readFile } from 'node:fs/promises' + import { dirname, join } from 'node:path' + export const parentLayout = 'root' + export const vars = { dataDeps: ['inner'] } + export default ({ children, data }) => data.inner + children + export async function* pageOutputs ({ page, data }) { + assert.throws(() => data.outer, /undeclared/) + assert.equal(await readFile(join(dirname(page.pageFile.filepath), '../../custom-output/docs/outer.txt'), 'utf8'), 'O') + globalThis[page.pageFile.filepath].push('inner') + yield { outputName: './inner.txt', content: data.inner } + }`, + 'docs/page.md': '---\ntitle: page title\n---\n# Body\n', + 'docs/page.vars.js': `import assert from 'node:assert/strict' + import { readFile } from 'node:fs/promises' + import { dirname, join } from 'node:path' + export default { dataDeps: ['selected'] } + export const pageOutputs = async ({ page, vars, data }) => { + assert.throws(() => data.secret, /undeclared/) + assert.throws(() => data.inner, /undeclared/) + assert.equal(Object.isFrozen(page), true) + assert.equal(Object.isFrozen(vars), true) + const outputDir = join(dirname(page.pageFile.filepath), '../../custom-output/docs') + assert.equal(await readFile(join(outputDir, 'outer.txt'), 'utf8'), 'O') + assert.equal(await readFile(join(outputDir, 'inner.txt'), 'utf8'), 'I') + const order = globalThis[page.pageFile.filepath] + delete globalThis[page.pageFile.filepath] + return [ + { outputName: '/metadata.json', content: JSON.stringify({ title: vars.title, selected: data.selected, order: [...order, 'page'] }) }, + { outputName: '../source/article.txt', content: await page.readMarkdownContent() }, + ] + }`, + }) + await build() + assert.deepEqual(JSON.parse(await read('metadata.json')), { title: 'page title', selected: 'P', order: ['outer', 'inner', 'page'] }) + assert.equal(await read('docs/outer.txt'), 'O') + assert.equal(await read('docs/inner.txt'), 'I') + assert.equal(await read('source/article.txt'), '\n# Body\n') + assert.match(await read('docs/index.html'), /^OI\s*

{ + const { build, read } = await setup(t, { + 'global.data.js': "export default { selected: 'subscribed', secret: 'private' }", + [`article/page.${extension}`]: extension === 'html' ? '

{{ vars.title }}

' : 'export default ({ vars, data }) => vars.title + data.selected', + 'article/page.vars.js': `import assert from 'node:assert/strict' + export default { title: 'Companion', dataDeps: ['selected'] } + export async function pageOutputs ({ page, vars, data }) { + await assert.rejects(page.readMarkdownContent()) + assert.throws(() => data.secret, /undeclared/) + return { outputName: 'metadata.json', content: JSON.stringify({ title: vars.title, value: data.selected }) } + }`, + }) + await build() + assert.match(await read('article/index.html'), /Companion/) + assert.deepEqual(JSON.parse(await read('article/metadata.json')), { title: 'Companion', value: 'subscribed' }) + }) +} + +test('JS page modules support promised async iterables, arrays, and empty results', async t => { + const { build, read } = await setup(t, { + 'page.js': `export default () => 'main'; export const pageOutputs = async () => (async function* () { + yield { outputName: 'one.txt', content: 'one' }; yield { outputName: './two.txt', content: 'two' } + })()`, + 'array/page.js': "export default () => 'array'; export const pageOutputs = () => [{ outputName: 'array.txt', content: 'array' }]", + 'empty/page.js': "export default () => 'empty'; export const pageOutputs = () => []", + 'iterator/page.js': "export default () => 'empty iterator'; export async function* pageOutputs () {}", + }) + await build() + for (const name of ['one', 'two']) assert.equal(await read(`${name}.txt`), name) + assert.equal(await read('array/array.txt'), 'array') + assert.equal(await read('empty/index.html'), 'empty') + assert.equal(await read('iterator/index.html'), 'empty iterator') +}) + +test('async generators publish each record before requesting the next at a custom destination', async t => { + const { build, dest, read, mtime } = await setup(t, { + 'page.js': `import assert from 'node:assert/strict' + import { readFile, stat } from 'node:fs/promises' + import { dirname, join } from 'node:path' + export default () => 'main' + export async function* pageOutputs ({ page }) { + const dest = join(dirname(page.pageFile.filepath), '../custom-output') + yield { outputName: 'replaced.txt', content: 'replacement' } + assert.equal(await readFile(join(dest, 'replaced.txt'), 'utf8'), 'replacement') + yield { outputName: 'nested/new.txt', content: 'new sidecar' } + assert.equal(await readFile(join(dest, 'nested/new.txt'), 'utf8'), 'new sidecar') + const unchangedTime = (await stat(join(dest, 'unchanged.txt'))).mtimeMs + yield { outputName: 'unchanged.txt', content: 'same bytes' } + assert.equal(await readFile(join(dest, 'unchanged.txt'), 'utf8'), 'same bytes') + assert.notEqual((await stat(join(dest, 'unchanged.txt'))).mtimeMs, unchangedTime, 'a cold build writes even identical bytes') + }`, + }) + await writeFiles(dest, { 'replaced.txt': 'old sidecar', 'unchanged.txt': 'same bytes' }) + await utimes(join(dest, 'unchanged.txt'), 1, 1) + const unchangedTime = await mtime('unchanged.txt') + const result = await build() + assert.equal(await read('replaced.txt'), 'replacement') + assert.equal(await read('nested/new.txt'), 'new sidecar') + assert.notEqual(await mtime('unchanged.txt'), unchangedTime) + assert.equal(await read('index.html'), 'main') + for (const outputRelname of ['replaced.txt', 'nested/new.txt', 'unchanged.txt']) { + assert.ok(result.pageBuildResults?.outputs.some(output => output.outputRelname === outputRelname), `${outputRelname} is reported, including unchanged content`) + } +}) + +test('generated pages skip inherited layout hooks', async t => { + const { build, read } = await setup(t, { + 'root.layout.js': "export default ({ children }) => children; export const pageOutputs = () => { throw Error('generated hook ran') }", + 'items.pages.js': "export default { outputName: 'generated.html', children: 'Generated' }", + }) + await build() + assert.equal(await read('generated.html'), 'Generated') +}) + +test('JS page outputs take precedence over companion outputs while layouts remain additive', async t => { + const { build, read, src } = await setup(t, { + 'root.layout.js': 'export default ({ children }) => children; ' + hook('layout.txt', 'layout'), + 'page.js': "export default () => 'main'; " + hook('page.txt', 'page'), + 'page.vars.js': "export default {}; export const pageOutputs = () => { throw Error('ignored companion must not run') }", + }) + const result = await build() + const warnings = result.pageBuildResults?.warnings.filter(warning => 'code' in warning && warning.code === 'DOM_STACK_WARNING_DUPLICATE_PAGE_OUTPUTS_PROVIDER') + assert.equal(warnings?.length, 1) + const warning = warnings?.[0] + assert.ok(warning && 'message' in warning) + assert.ok(warning.message.includes(join(src, 'page.js'))) + assert.ok(warning.message.includes(join(src, 'page.vars.js'))) + assert.ok(result.warnings.includes(warning), 'worker warnings propagate to the aggregate build result') + assert.equal(await read('index.html'), 'main') + assert.equal(await read('layout.txt'), 'layout') + assert.equal(await read('page.txt'), 'page') + await assert.rejects(read('companion.txt'), { code: 'ENOENT' }) +}) + +for (const scenario of [ + { name: 'own HTML', output: 'index.html', files: {} }, + { name: 'other page HTML', output: 'other/index.html', files: { 'other/page.html': 'Other' } }, + { name: 'template', output: 'shared.txt', files: { 'shared.txt.template.js': "export default () => 'template'" } }, + { name: 'asset', output: 'shared.txt', files: { 'shared.txt': 'asset' } }, + { name: 'bundle', output: 'client.js', files: { 'client.js': 'console.log(1)', 'esbuild.settings.js': "export default opts => ({ ...opts, entryNames: '[dir]/[name]' })" } }, + { name: 'layout hook', output: 'shared.txt', files: { 'root.layout.js': 'export default ({ children }) => children; ' + hook('shared.txt') } }, + { name: 'other page hook', output: 'shared.txt', files: { 'other/page.js': "export default () => 'other'; " + hook('/shared.txt') } }, +]) { + test(`builder warns about a duplicate sidecar destination with ${scenario.name}`, async t => { + const { build } = await setup(t, { + 'page.js': "export default () => 'new main'; " + hook(scenario.output), + ...scenario.files, + }) + const result = await build() + assert.ok(result.warnings.some(warning => { + const message = errorText(warning) + return /duplicate|conflict/i.test(message) && message.includes(scenario.output) + }), `expected a duplicate destination warning for ${scenario.output}: ${errorText(result.warnings)}`) + }) +} + +for (const result of [ + "'bare string'", + "{ outputName: 'bad.txt', content: 42 }", + + "{ outputName: '../escape.txt', content: 'bad' }", + "{ outputName: '/', content: 'bad' }", +]) { + test(`builder rejects invalid page output: ${result}`, async t => { + const { build, dest, read } = await setup(t, { + 'page.js': `export default () => 'new'; export const pageOutputs = () => (${result})`, + }) + await writeFiles(dest, { 'index.html': 'old' }) + await assert.rejects(build()) + assert.equal(await read('index.html'), 'old') + await assert.rejects(stat(join(dest, '../escape.txt')), { code: 'ENOENT' }) + }) +} + +test('iterator failure retains earlier sidecar writes and the previous HTML', async t => { + const { build, dest, read } = await setup(t, { + 'a/page.js': "export default () => 'new sibling'; " + hook('sibling.txt', 'new sibling sidecar'), + 'z/page.js': `export default () => 'new main'; export async function* pageOutputs () { + yield { outputName: 'old.txt', content: 'replacement' } + yield { outputName: 'partial.txt', content: 'published before failure' } + throw Error('iterator exploded') + }`, + }) + const previous = { 'z/index.html': 'old main', 'z/old.txt': 'old sidecar', 'z/stale.txt': 'retain on failure' } + await writeFiles(dest, previous) + await assert.rejects(build(), error => { + assert.match(errorText(error), /iterator exploded/) + return true + }) + assert.equal(await read('z/index.html'), 'old main') + assert.equal(await read('z/old.txt'), 'replacement') + assert.equal(await read('z/partial.txt'), 'published before failure') + assert.equal(await read('z/stale.txt'), 'retain on failure') +}) + +for (const provider of ['layout', 'page']) { + test(`a later ${provider} provider failure retains earlier layout files`, async t => { + const failingHook = 'export const pageOutputs = () => { throw Error(\'later provider exploded\') }' + const { build, dest, read } = await setup(t, { + 'global.vars.js': "export default { layout: 'inner' }", + 'root.layout.js': `export default ({ children }) => children + export async function* pageOutputs () { + yield { outputName: 'old.txt', content: 'replacement' } + yield { outputName: 'partial.txt', content: 'partial' } + }`, + 'inner.layout.js': `export const parentLayout = 'root'; export default ({ children }) => children; + ${provider === 'layout' ? failingHook : 'export const pageOutputs = () => []'}`, + 'page.js': `export default () => 'new main'; + ${provider === 'page' ? failingHook : "export const pageOutputs = () => { throw Error('page provider must not run') }"}`, + }) + await writeFiles(dest, { 'index.html': 'old main', 'old.txt': 'old sidecar' }) + await assert.rejects(build(), error => { + assert.match(errorText(error), /later provider exploded/) + assert.doesNotMatch(errorText(error), /page provider must not run/) + return true + }) + assert.equal(await read('index.html'), 'old main') + assert.equal(await read('old.txt'), 'replacement') + assert.equal(await read('partial.txt'), 'partial') + }) +} + +for (const invalid of [ + { record: "{ outputName: '../escape.txt', content: 'invalid' }", message: /escapes dest/ }, + { record: "{ outputName: 'invalid.txt', content: 42 }", message: /content.*string/i }, +]) { + test(`a later invalid record stops the stream without requesting following yields: ${invalid.record}`, async t => { + const { build, dest, read } = await setup(t, { + 'page.js': `import { writeFile } from 'node:fs/promises' + import { dirname, join } from 'node:path' + export default () => 'new main' + export async function* pageOutputs ({ page }) { + yield { outputName: 'first.txt', content: 'published' } + yield ${invalid.record} + await writeFile(join(dirname(page.pageFile.filepath), '../following-yield-requested'), 'requested') + yield { outputName: 'following.txt', content: 'must not publish' } + }`, + }) + await writeFiles(dest, { 'index.html': 'old main' }) + await assert.rejects(build(), error => { + assert.match(errorText(error), invalid.message) + return true + }) + assert.equal(await read('first.txt'), 'published') + assert.equal(await read('index.html'), 'old main') + for (const name of ['../escape.txt', 'invalid.txt', '../following-yield-requested', 'following.txt']) { + await assert.rejects(stat(join(dest, name)), { code: 'ENOENT' }) + } + }) +} + +test('identical duplicate records from one hook warn rather than reject the build', async t => { + const { build, read } = await setup(t, { + 'page.js': `export default () => 'main'; export const pageOutputs = () => [ + { outputName: 'same.txt', content: 'same' }, + { outputName: './same.txt', content: 'same' }, + ]`, + }) + const result = await build() + assert.equal(await read('index.html'), 'main') + assert.equal(await read('same.txt'), 'same') + assert.ok(result.warnings.some(warning => { + const message = errorText(warning) + return /duplicate|conflict/i.test(message) && message.includes('same.txt') + }), `expected a duplicate destination warning: ${errorText(result.warnings)}`) +}) + +test('render failure leaves the owning page HTML and sidecars unchanged', async t => { + const { build, dest, read } = await setup(t, { + 'page.js': "export default () => { throw Error('render exploded') }; " + hook('old.txt', 'replacement'), + }) + await writeFiles(dest, { 'index.html': 'old main', 'old.txt': 'old sidecar' }) + await assert.rejects(build(), error => { + assert.match(errorText(error), /render exploded/) + return true + }) + assert.equal(await read('index.html'), 'old main') + assert.equal(await read('old.txt'), 'old sidecar') +}) diff --git a/test-cases/page-outputs/ownership.test.js b/test-cases/page-outputs/ownership.test.js new file mode 100644 index 00000000..02dcd7a3 --- /dev/null +++ b/test-cases/page-outputs/ownership.test.js @@ -0,0 +1,122 @@ +import { test } from 'node:test' +import assert from 'node:assert/strict' +import { mkdir, readFile, rm, stat, symlink, writeFile } from 'node:fs/promises' +import { join } from 'node:path' +import { hook, setup, settle } from './helpers.js' + +test('data-invalidated pages replace ownership using actual reports', { timeout: 15_000 }, async t => { + const { site, src, dest, read, logs } = await setup(t, { + 'global.data.js': "export default { name: 'old.txt' }", + 'page.js': "export const vars = { dataDeps: ['name'] }; export default () => 'main'; export const pageOutputs = ({ data }) => ({ outputName: data.name, content: 'sidecar' })", + }) + await site.watch({ serve: false }) + await settle(site, logs, async () => { + await writeFile(join(src, 'global.data.js'), "export default { name: 'new.txt' }") + }) + assert.equal(await read('new.txt'), 'sidecar') + await assert.rejects(stat(join(dest, 'old.txt')), { code: 'ENOENT' }) +}) + +for (const owner of ['page', 'template']) { + test(`targeted cleanup protects an untouched ${owner} claim`, { timeout: 15_000 }, async t => { + const { site, src, read, logs } = await setup(t, { + 'page.js': "export default () => 'main'; " + hook('shared.html'), + ...(owner === 'page' + ? { 'shared.md': 'other page' } + : { 'shared.template.js': "export default () => ({ outputName: 'shared.html', content: 'template' })" }), + }) + await site.watch({ serve: false }) + const shared = await read('shared.html') + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), "export default () => 'updated main'") + }) + assert.equal(await read('shared.html'), shared) + }) +} + +test('repeated failed watch builds union partial paths with successful ownership for recovery', { timeout: 15_000 }, async t => { + const { site, src, dest, read, logs } = await setup(t, { + 'page.js': "export default () => 'old main'; " + hook('old.txt'), + }) + await site.watch({ serve: false }) + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), `export default () => 'failed main'; export async function* pageOutputs () { + yield { outputName: 'partial.txt', content: 'partial' } + throw Error('ownership failure') + }`) + }, 'ownership failure') + assert.equal(await read('old.txt'), 'sidecar', 'failure must not clean up the previous successful output') + assert.equal(await read('partial.txt'), 'partial') + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), `export default () => 'failed again'; export async function* pageOutputs () { + yield { outputName: 'second-partial.txt', content: 'second partial' } + throw Error('second ownership failure') + }`) + }, 'second ownership failure') + assert.equal(await read('index.html'), 'old main') + assert.equal(await read('old.txt'), 'sidecar') + assert.equal(await read('partial.txt'), 'partial') + assert.equal(await read('second-partial.txt'), 'second partial') + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), "export default () => 'recovered'; " + hook('new.txt')) + }) + assert.equal(await read('new.txt'), 'sidecar') + for (const name of ['old.txt', 'partial.txt', 'second-partial.txt']) { + await assert.rejects(stat(join(dest, name)), { code: 'ENOENT' }) + } +}) + +for (const change of ['recovery', 'source deletion', 'hook removal']) { + test(`initial failed watch tracks partial ownership for ${change}`, { timeout: 15_000 }, async t => { + const { site, src, dest, read, logs } = await setup(t, { + 'article/page.html': 'Article', + 'article/page.vars.js': `export default {}; export async function* pageOutputs () { + yield { outputName: 'partial.txt', content: 'partial' } + yield { outputName: '/root-partial.txt', content: 'root partial' } + throw Error('initial ownership failure') + }`, + }) + const result = await site.watch({ serve: false }) + assert.ok(logs.some(line => JSON.parse(line).msg === 'Build Failed!')) + assert.ok(logs.some(line => line.includes('initial ownership failure'))) + assert.equal(await read('article/partial.txt'), 'partial') + assert.equal(await read('root-partial.txt'), 'root partial') + for (const outputRelname of ['article/partial.txt', 'root-partial.txt']) { + const record = result.pageBuildResults?.outputs.find(output => output.outputRelname === outputRelname) + assert.ok(record, `${outputRelname} is included in the failed page build report`) + assert.equal(record.sourceRelname, 'article/page.html') + assert.equal(record.filepath, join(dest, outputRelname)) + } + await assert.rejects(stat(join(dest, 'article/index.html')), { code: 'ENOENT' }) + await settle(site, logs, async () => { + if (change === 'recovery') await writeFile(join(src, 'article/page.vars.js'), 'export default {}; ' + hook('recovered.txt', 'recovered')) + if (change === 'source deletion') await rm(join(src, 'article/page.html')) + if (change === 'hook removal') await writeFile(join(src, 'article/page.vars.js'), 'export default {}') + }) + for (const name of ['article/partial.txt', 'root-partial.txt']) { + await assert.rejects(stat(join(dest, name)), { code: 'ENOENT' }) + } + if (change === 'source deletion') { + await assert.rejects(stat(join(dest, 'article/index.html')), { code: 'ENOENT' }) + } else { + assert.equal(await read('article/index.html'), 'Article') + if (change === 'recovery') assert.equal(await read('article/recovered.txt'), 'recovered') + } + }) +} + +test('stale cleanup does not follow symlink ancestors outside dest', { timeout: 15_000 }, async t => { + const { site, src, dest, logs } = await setup(t, { + 'page.js': "export default () => 'main'; " + hook('nested/owned.txt'), + }) + await site.watch({ serve: false }) + const outside = join(dest, '..', 'outside') + await mkdir(outside) + await writeFile(join(outside, 'owned.txt'), 'keep') + await rm(join(dest, 'nested'), { recursive: true }) + await symlink(outside, join(dest, 'nested'), 'dir') + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), "export default () => 'main without hook'") + }) + assert.equal(await readFile(join(outside, 'owned.txt'), 'utf8'), 'keep') +}) diff --git a/test-cases/page-outputs/watch.test.js b/test-cases/page-outputs/watch.test.js new file mode 100644 index 00000000..aa68ca96 --- /dev/null +++ b/test-cases/page-outputs/watch.test.js @@ -0,0 +1,202 @@ +import { test } from 'node:test' +import assert from 'node:assert/strict' +import { rename, rm, stat, writeFile } from 'node:fs/promises' +import { join } from 'node:path' +import { hook, setup, settle, writeFiles } from './helpers.js' + +const rawLayout = `export const vars = { dataDeps: ['navigation'] } +export default ({ children, data }) => data.navigation + children +export const pageOutputs = async ({ page }) => ({ outputName: 'source.txt', content: await page.readMarkdownContent() })` + +test('watch updates only changed raw content and retains unchanged ownership without a public manifest', { timeout: 30_000 }, async t => { + const { site, src, dest, read, mtime, logs } = await setup(t, { + 'global.data.js': "export default { navigation: 'Navigation one' }", + 'root.layout.js': rawLayout, + 'a/page.md': '# Article A\n', + 'b/page.md': '# Article B\n', + }) + await site.watch({ serve: false }) + const siblingRaw = await mtime('b/source.txt') + const siblingHtml = await mtime('b/index.html') + const originalRaw = await mtime('a/source.txt') + await settle(site, logs, async () => { + await writeFile(join(src, 'a/page.md'), '# Edited A\n') + }) + assert.equal(await read('a/source.txt'), '# Edited A\n') + assert.match(await read('a/index.html'), /Edited A/) + assert.notEqual(await mtime('a/source.txt'), originalRaw) + assert.equal(await mtime('b/source.txt'), siblingRaw) + assert.equal(await mtime('b/index.html'), siblingHtml, 'body edit must not invalidate an unrelated sibling') + const editedRaw = await mtime('a/source.txt') + await settle(site, logs, async () => { + await writeFile(join(src, 'global.data.js'), "export default { navigation: 'Navigation two' }") + }) + for (const name of ['a', 'b']) assert.match(await read(`${name}/index.html`), /Navigation two/) + assert.equal(await mtime('a/source.txt'), editedRaw, 'navigation rebuild does not rewrite identical raw content') + assert.equal(await mtime('b/source.txt'), siblingRaw) + await settle(site, logs, async () => { + await rm(join(src, 'b/page.md')) + }) + await assert.rejects(stat(join(dest, 'b/source.txt')), { code: 'ENOENT' }) + await assert.rejects(stat(join(dest, 'b/index.html')), { code: 'ENOENT' }) + assert.equal(await read('a/source.txt'), '# Edited A\n') + await assert.rejects(stat(join(dest, 'domstack-manifest.json')), { code: 'ENOENT' }) +}) + +test('watch reconciles companion addition, output rename, hook removal, companion removal and re-addition', { timeout: 30_000 }, async t => { + const { site, src, dest, read, logs } = await setup(t, { 'article/page.html': '

Article

' }) + await site.watch({ serve: false }) + const companion = join(src, 'article/page.vars.js') + await settle(site, logs, async () => { + await writeFile(companion, 'export default {}; ' + hook('first.txt', 'first')) + }) + assert.equal(await read('article/first.txt'), 'first') + await settle(site, logs, async () => { + await writeFile(companion, 'export default {}; ' + hook('renamed.txt', 'renamed')) + }) + assert.equal(await read('article/renamed.txt'), 'renamed') + await assert.rejects(stat(join(dest, 'article/first.txt')), { code: 'ENOENT' }) + await settle(site, logs, async () => { + await writeFile(companion, 'export default { title: "no hook" }') + }) + await assert.rejects(stat(join(dest, 'article/renamed.txt')), { code: 'ENOENT' }) + await settle(site, logs, async () => { + await writeFile(companion, 'export default {}; ' + hook('again.txt')) + }) + assert.equal(await read('article/again.txt'), 'sidecar') + await settle(site, logs, async () => { + await rm(companion) + }) + await assert.rejects(stat(join(dest, 'article/again.txt')), { code: 'ENOENT' }) + await settle(site, logs, async () => { + await writeFile(companion, 'export default {}; ' + hook('restored.txt')) + }) + assert.equal(await read('article/restored.txt'), 'sidecar') + await settle(site, logs, async () => { + await rename(companion, join(src, 'article/unassociated.vars.js')) + }) + await assert.rejects(stat(join(dest, 'article/restored.txt')), { code: 'ENOENT' }) + assert.match(await read('article/index.html'), /Article/) +}) + +test('watch removes sidecars on source rename and draft exclusion', { timeout: 30_000 }, async t => { + const { site, src, dest, read, logs } = await setup(t, { + 'root.layout.js': `export default ({ children }) => children + export const pageOutputs = async ({ page }) => ({ outputName: page.outputName + '.txt', content: await page.readMarkdownContent() })`, + 'article.md': '# Article\n', + }) + await site.watch({ serve: false }) + assert.equal(await read('article.html.txt'), '# Article\n') + await settle(site, logs, async () => { + await rename(join(src, 'article.md'), join(src, 'renamed.md')) + }) + assert.equal(await read('renamed.html.txt'), '# Article\n') + for (const name of ['article.html', 'article.html.txt']) await assert.rejects(stat(join(dest, name)), { code: 'ENOENT' }) + await settle(site, logs, async () => { + await rename(join(src, 'renamed.md'), join(src, 'renamed.draft.md')) + }) + for (const name of ['renamed.html', 'renamed.html.txt']) await assert.rejects(stat(join(dest, name)), { code: 'ENOENT' }) + await settle(site, logs, async () => { + await rename(join(src, 'renamed.draft.md'), join(src, 'renamed.md')) + }) + assert.equal(await read('renamed.html.txt'), '# Article\n') +}) + +test('watch hook failure retains partial writes and recovery removes old and partial outputs', { timeout: 30_000 }, async t => { + const { site, src, dest, read, mtime, logs } = await setup(t, { + 'page.js': `export default () => 'old main'; export const pageOutputs = () => [ + { outputName: 'old.txt', content: 'old sidecar' }, + { outputName: 'stale.txt', content: 'retain until recovery' }, + ]`, + }) + await site.watch({ serve: false }) + const oldHtmlTime = await mtime('index.html') + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), `export default () => 'failed main'; export async function* pageOutputs () { + yield { outputName: 'old.txt', content: 'failed replacement' } + yield { outputName: 'partial.txt', content: 'partial' } + throw Error('watch iterator exploded') + }`) + }, 'watch iterator exploded') + assert.ok(logs.some(line => line.includes('watch iterator exploded')), 'watch reports the hook failure') + assert.equal(await read('index.html'), 'old main') + assert.equal(await mtime('index.html'), oldHtmlTime) + assert.equal(await read('old.txt'), 'failed replacement') + assert.equal(await read('partial.txt'), 'partial') + assert.equal(await read('stale.txt'), 'retain until recovery') + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), "export default () => 'recovered main'; " + hook('new.txt', 'recovered')) + }) + assert.equal(await read('index.html'), 'recovered main') + assert.equal(await read('new.txt'), 'recovered') + for (const name of ['old.txt', 'stale.txt', 'partial.txt']) { + await assert.rejects(stat(join(dest, name)), { code: 'ENOENT' }) + } +}) + +for (const change of ['source deletion', 'draft exclusion', 'hook removal', 'companion deletion', 'companion rename']) { + test(`watch independently cleans up after ${change}`, { timeout: 15_000 }, async t => { + const { site, src, dest, read, logs } = await setup(t, { + 'article/page.html': 'Article', + 'article/page.vars.js': 'export default {}; ' + hook('owned.txt'), + }) + await site.watch({ serve: false }) + assert.equal(await read('article/owned.txt'), 'sidecar') + const page = join(src, 'article/page.html') + const companion = join(src, 'article/page.vars.js') + await settle(site, logs, async () => { + if (change === 'source deletion') await rm(page) + if (change === 'draft exclusion') await rename(page, join(src, 'article/page.draft.html')) + if (change === 'hook removal') await writeFile(companion, 'export default {}') + if (change === 'companion deletion') await rm(companion) + if (change === 'companion rename') await rename(companion, join(src, 'article/unassociated.vars.js')) + }) + await assert.rejects(stat(join(dest, 'article/owned.txt')), { code: 'ENOENT' }) + if (change === 'source deletion' || change === 'draft exclusion') { + await assert.rejects(stat(join(dest, 'article/index.html')), { code: 'ENOENT' }) + } else { + assert.equal(await read('article/index.html'), 'Article') + } + }) +} + +test('provider precedence warnings reach the configured logger on initial and incremental watch builds', { timeout: 15_000 }, async t => { + const { site, src, read, logs } = await setup(t, { + 'page.js': "export default () => 'initial'; " + hook('selected.txt', 'page module'), + 'page.vars.js': "export default {}; export const pageOutputs = () => { throw Error('ignored companion ran') }", + }) + const result = await site.watch({ serve: false }) + assert.equal(result.pageBuildResults?.warnings.filter(warning => 'code' in warning && warning.code === 'DOM_STACK_WARNING_DUPLICATE_PAGE_OUTPUTS_PROVIDER').length, 1) + const cursor = logs.length + await settle(site, logs, async () => { + await writeFile(join(src, 'page.js'), "export default () => 'rebuilt'; " + hook('selected.txt', 'page module')) + }) + for (const messages of [logs.slice(0, cursor), logs.slice(cursor)]) { + const warnings = messages.map(line => JSON.parse(line)).filter(entry => entry.level === 40 && entry.msg.includes('both export pageOutputs')) + assert.ok(warnings.length > 0, 'the configured logger receives provider warnings for this watch phase') + for (const warning of warnings) { + assert.ok(warning.msg.includes(join(src, 'page.js'))) + assert.ok(warning.msg.includes(join(src, 'page.vars.js'))) + } + } + assert.equal(await read('index.html'), 'rebuilt') + assert.equal(await read('selected.txt'), 'page module') +}) + +test('hook-only data subscriptions invalidate their owner but not an unrelated sibling', { timeout: 30_000 }, async t => { + const { site, src, read, mtime, logs } = await setup(t, { + 'global.data.js': "export default { selected: 'first', unrelated: 'unchanged' }", + 'a/page.html': 'A', + 'a/page.vars.js': "export default { dataDeps: ['selected'] }; export const pageOutputs = ({ data }) => ({ outputName: 'data.txt', content: data.selected })", + 'b/page.html': 'B', + }) + await site.watch({ serve: false }) + const mainTime = await mtime('a/index.html') + const siblingTime = await mtime('b/index.html') + await settle(site, logs, async () => { + await writeFiles(src, { 'global.data.js': "export default { selected: 'second', unrelated: 'unchanged' }" }) + }) + assert.equal(await read('a/data.txt'), 'second') + assert.notEqual(await mtime('a/index.html'), mainTime) + assert.equal(await mtime('b/index.html'), siblingTime) +}) diff --git a/test-cases/watch/index.test.js b/test-cases/watch/index.test.js index 3c4438ec..8bb9a897 100644 --- a/test-cases/watch/index.test.js +++ b/test-cases/watch/index.test.js @@ -114,13 +114,13 @@ test('targeted factories reserve untouched owners outputs and recover after a co 'b.pages.js': "export default { outputName: 'b.html', children: 'Owner B' }", }, }) - const retainedTime = (await stat(path.join(dest, 'b.html'))).mtimeMs await writeFile(path.join(src, 'a.pages.js'), "export default { outputName: 'b.html', children: 'Collision' }") await settle(domStack) assert.ok(logs.some(line => line.includes('Output path conflict: b.html is produced by both b.pages.js and a.pages.js#0.')), 'both conflicting producers use source-relative names') assert.match(await readFile(path.join(dest, 'b.html'), 'utf8'), /Owner B/) assert.match(await readFile(path.join(dest, 'a.html'), 'utf8'), /Owner A/) - assert.equal((await stat(path.join(dest, 'b.html'))).mtimeMs, retainedTime) + // A repeated watcher event can trigger a full retry after failure. Streaming + // may rewrite B with its own content before encountering A's collision again. await writeFile(path.join(src, 'a.pages.js'), "export default { outputName: 'c.html', children: 'Recovered' }") await settle(domStack) assert.match(await readFile(path.join(dest, 'c.html'), 'utf8'), /Recovered/) diff --git a/types.ts b/types.ts index da8a48e1..32f40804 100644 --- a/types.ts +++ b/types.ts @@ -6,6 +6,15 @@ import type { Results } from './lib/builder.js' export type { DataDeps } from './lib/build-pages/data-deps.js' +export type { + PageOutput, + PageOutputProvenance, + PageOutputsFunction, + PageOutputsFunctionParams, + PageOutputsPage, + PageOutputsResult, + CollectedPageOutput, +} from './lib/build-pages/page-outputs.js' export type { BuildOptions } from 'esbuild' export type { DomStackOpts, Results, SiteData } from './lib/builder.js'