Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,41 @@ jobs:
# the flag onto the repo's own test script, so the strip-types flag stays single-sourced.
- name: Run tests (with coverage)
run: node --run test -- --experimental-test-coverage

# The `ci` job installs the newest versions. This one runs the same checks on the oldest
# supported Node major (the newest release of it) with every peer at the lowest version its
# range allows, so a declared minimum that stopped working is caught here and not in a host.
min-versions:
runs-on: ubuntu-latest

permissions:
contents: read

env:
MONGOMS_DISABLE_POSTINSTALL: '1'

steps:
- uses: actions/checkout@v6

- uses: actions/setup-node@v6
with:
node-version: '24'
cache: 'npm'

- name: Cache mongod binary (mongodb-memory-server)
uses: actions/cache@v4
with:
path: ~/.cache/mongodb-binaries
key: mongod-${{ runner.os }}-mms11

- name: npm clean install
run: npm ci

# Read the minimums from package.json, so this job follows the peer ranges.
- name: Install the minimum peer versions
run: |
npm install --no-save $(node -p "Object.entries(require('./package.json').peerDependencies).map(([name, range]) => name + '@' + range.replace(/^[^0-9]*/, '')).join(' ')")

- run: node --run types:check
- run: node --run build
- run: node --run test
2 changes: 1 addition & 1 deletion .github/workflows/packaging.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
run: npm ci

# Builds dist, packs the tarball, installs it into a throwaway consumer and verifies
# the PUBLISHED surface (17 core exports, the exports map, the optional subpaths'
# the PUBLISHED surface (the expected core exports, the exports map, the optional subpaths'
# loud-fail without their AWS SDKs, the always-safe subpaths, the resize-scaffold bin)
# — dist-only breakage the TS-source test suite can't see.
- name: Packaging smoke test
Expand Down
94 changes: 68 additions & 26 deletions AGENTS.md

Large diffs are not rendered by default.

193 changes: 162 additions & 31 deletions CHANGELOG.md

Large diffs are not rendered by default.

124 changes: 88 additions & 36 deletions README.md

Large diffs are not rendered by default.

47 changes: 31 additions & 16 deletions RELEASE.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
# Releasing `@adaptivestone/framework-module-resize`

Releases are **manual** (same as the sibling repos — `@adaptivestone/framework` and
`@adaptivestone/framework-module-email` ship without a publish workflow). CI (`.github/workflows/ci.yml`)
and the packaging smoke (`.github/workflows/packaging.yml`) gate every push/PR; publishing is a
human step run from a clean `main`.
`@adaptivestone/framework-module-email` ship without a publish workflow). CI
(`.github/workflows/ci.yml`: the latest Node with the locked dependencies, plus a `min-versions`
job on the oldest supported Node major, at its newest 24.x release, with every peer at its
declared minimum) and the packaging smoke
(`.github/workflows/packaging.yml`) gate every push/PR; publishing is a human step run from a
clean `main`.

## 1. Pre-flight gates (all must be green)

Expand Down Expand Up @@ -35,9 +38,14 @@ Semver. Pre-1.0 (`0.x`), the minor is the breaking channel:
Bump with `npm version <patch|minor|major>` (updates `package.json` + creates the git tag — see
§4), or edit `version` by hand and tag manually.

> **Check `version` first.** For 0.3.0 no bump is needed: `package.json` already holds `0.3.0`
> and `CHANGELOG.md` has the `# 0.3.0` heading, so tag the release commit by hand. In general,
> compare `version` with `npm view @adaptivestone/framework-module-resize version`; if it is
> already ahead, do not run `npm version` on top of it (that would skip a version).

**Update `CHANGELOG.md` in the same commit as the bump.** Newest version first, `# X.Y.Z`
heading, with `**Breaking changes**` / `**Features**` / `**Internal**` groups — the format the
sibling `framework-module-email` uses. It is NOT in `files`, so it stays on GitHub and never
heading, with `**Breaking changes**` / `**Features**` / `**Fixes**` / `**Internal**` groups —
the format the sibling `framework-module-email` uses. It is NOT in `files`, so it stays on GitHub and never
ships in the tarball (same as the sibling). Write it for a host developer deciding whether to
upgrade: what breaks, what is new, and what they must change.

Expand Down Expand Up @@ -70,10 +78,11 @@ all that ships.

Tag the release commit `vX.Y.Z`.

> **Note:** the sibling repos actually tag **without** the `v` prefix (`framework` uses `5.3.1`,
> `framework-module-email` uses `2.0.0`) — an earlier version of this file claimed otherwise. This
> repo already published `v0.1.0` and `v0.2.0` with the prefix, so it keeps `v` for internal
> consistency rather than switching mid-stream. Use `v` here; don't "fix" it to match the siblings.
> **Note:** the sibling repos tag **without** the `v` prefix (`framework` uses `5.3.1`,
> `framework-module-email` uses `2.0.0`). This repo's existing tags are mixed: `v0.1.0` has the
> prefix, `0.2.0` does not, and 0.2.1 has no tag at all (`git tag` lists them). This file keeps
> the `v` prefix, npm's default; pick one convention before the next release and use it from
> then on.

```bash
git tag v0.1.0
Expand Down Expand Up @@ -106,13 +115,19 @@ _Placeholder — the one check the automated smoke can't do: wire the published
actual framework host and run it against real infra._

- [ ] In a real `@adaptivestone/framework` host: `npm i @adaptivestone/framework-module-resize`,
run `npx resize-scaffold`, fill the `storage` TODO in `src/resizer.ts` + `mediaModelName`
in `src/config/resize.ts`, `import ./resizer.ts` from `src/server.ts`.
- [ ] Confirm `npm run gen` (the framework's AST codegen) types the scaffolded
`class ResizeTask extends ResizeTaskModel {}`.
- [ ] Boot the server, upload an image, confirm a `resolve()` read enqueues and the
`ResizeWorker` command generates + persists previews (mongo transport + S3 storage).
- [ ] Confirm eager mode (`generate()`, no transport) on a host wired without a queue.
run `npx resize-scaffold`, and in `src/config/resize.ts` set `mediaModelName`, `storage`
(S3), `queue: { driver: 'database' }` and `worker.enabled: true`. `src/resizer.ts` stays
`new FrameworkResizer({ pipelines })`. Create the `ResizeTask` indexes through the host's
migration, and call `await resizer.verify()` after `server.init()`.
- [ ] `npx resize-scaffold --check` passes, and `npm run gen` (the framework's AST codegen) types
`getModel('ResizeTask')` through the scaffolded shim.
- [ ] Boot the server, upload an image, confirm a `resolve()` read queues the missing variants and
`npm run cli ResizeWorker` generates and persists the previews (database queue + S3
storage). If the host uses SQS, repeat with `queue: { driver: 'sqs', … }`.
- [ ] Stop the worker during a task (SIGTERM): the task goes back to `pending` and the next
worker finishes it.
- [ ] Confirm eager mode (`generate()`, no `queue` in the config) on a host scaffolded with
`--eager`; `npx resize-scaffold --check --eager` passes.

## Notes

Expand Down
5 changes: 3 additions & 2 deletions docs/design/2026-09-30-multi-resizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,11 +272,12 @@ published; the page still describes 0.2.

**P5 — Later, each optional and separate.**
- `worker.parallelTasks`.
- Merge `prewarm` into `enqueueRequired`.
- Merge `prewarm` into `enqueueRequired`. **Done:** `enqueueRequired` was merged into `prewarm`.
- Strip archived-spec section references from comments.
- Check the AVIF/WebP quality defaults on real photos.
- Export a framework-free `ResizeTask` schema definition (fields + indexes). Today a host
without the framework copies them from `src/models/ResizeTask.ts`.
without the framework copies them from `src/models/ResizeTask.ts`. **Done:**
`resizeTaskFields` and `resizeTaskIndexes` (and `createResizeModels`) from `…/drivers/mongo.js`.

## 7. Non-goals

Expand Down
2 changes: 1 addition & 1 deletion docs/design/2026-10-05-database-queue-framework-wrapper.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Database, queue and framework wrapper

Status: **proposal**, 2026-10-05. Follows
Status: **implemented** in pull requests #46 and #47 (proposed 2026-10-05). Follows
[drivers and framework independence](./2026-10-02-drivers-and-framework-independence.md) (Q1–Q4,
merged). Pre-release: breaking changes are fine; hosts migrate their own data.

Expand Down
55 changes: 55 additions & 0 deletions docs/host-adoption.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,58 @@ Retain `minimize: false` on the media schema so empty objects inside opaque refs
survive persistence. Framework `BaseModel` already supplies this default; direct
Mongoose schemas must set it explicitly. The field fragment alone cannot configure
the enclosing schema's options.

## Other changes a host must make for this version

`CHANGELOG.md` (`# 0.3.0`) has the details.

- Construct the Resizer with `new FrameworkResizer({ pipelines, hooks })` in
`src/resizer.ts`, and move `storage` and `queue` into `src/config/resize.ts`
(`queue: { driver: 'database' }` for background generation). Move
`worker.concurrency` to the top-level `concurrency`.
- Create the `ResizeTask` indexes through the host's migration: the claim index
`{ queue: 1, status: 1, availableAt: 1 }` before or together with the new
workers, then drop the old lease index (`{ status: 1, createdAt: 1 }`, or
`{ queue: 1, status: 1, createdAt: 1 }` from a pre-release build). The partial
unique index on `{ fileId, pipeline, requestKey }` is required for
de-duplication. Backfilling `availableAt` is optional: rows without it still wait
for the retry time or lease end stored in `leaseExpiresAt`.
- Regenerate the model shim, which now imports `…/framework/ResizeTaskModel.js`:
delete `src/models/ResizeTask.ts` and re-run `npx resize-scaffold` (`--force`
would also overwrite `src/resizer.ts` and `src/config/resize.ts`). An ejected or
hand-written model must add the `resizer`, `queue`, `requestKey` and
`availableAt` fields and the claim index; `resize-scaffold --check` and
`resizer.verify()` report a model without the fields.
- A hand-written media schema that declares preview rows as sub-documents must add
`identity: { type: String }` to them (the fragment already has it);
`resizer.verify()` throws `RESIZE_MONGO_MEDIA_MODEL_OUTDATED` without it. Rows
written earlier have no identity and are not migrated. A custom model-shaped
`MongoMediaModel` needs `findById` and `findOneAndUpdate`. A `findOneAndUpdate`
middleware on the media model now runs once per preview write (and once for the
dimension backfill) and receives a document with only `_id`, or `null` when the
preview was already stored or the media is gone.
- The original is never served. Code that read `isOriginal`, or relied on
`ctx.isOwner` / `ctx.isAdmin` to get the original from `resolve()`, must change:
to give an owner the private original, call
`storage.signedUrl(original.storageRef, ttlSeconds)`. For a pipeline without
`variantSteps`, a small raster original now gets a normal preview at its own
size, made by the worker.
- SVG `beforeSteps` now receive the rendered PNG, not SVG markup. The render runs
in a child process: a host under Node's permission model needs
`--allow-child-process`, and a bundle must keep `svgRasterChild.js` next to
`svgRaster.js` (otherwise `RESIZE_SVG_RENDER_UNAVAILABLE`).
- Resizers on the database queue (and Resizers with identical SQS settings) share
one task queue with one timing: config files that share a queue must set the same
timing keys (and effective SQS `waitTimeSeconds`), or `verify()` and worker start
fail with
`RESIZE_CONFIG_QUEUE_TIMING_CONFLICT`. Without the framework, call
`mongoDatabase()` once and pass its `tasks` to every Resizer.
- A host whose Resizers read only named config files (no `resize.ts`) starts the
worker with `npm run cli ResizeWorker -- --config=<name>`.
- Previews stored before preview identity included the Resizer and the pipeline
have no `resizer` or `pipeline` field and count as the `default` Resizer's
`default` pipeline. A named Resizer or pipeline generates its previews again, and a
`default` read may serve an old preview that was rendered by another pipeline;
remove such rows if that matters.
- Register every pipeline in `src/resizer.ts`, and deploy the worker before the API
requests a new or renamed pipeline: an unregistered pipeline is never rendered.
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
"./package.json": "./package.json",
"./config/resize.js": "./dist/config/resize.js",
"./framework.js": "./dist/framework/index.js",
"./framework/ResizeTaskModel.js": "./dist/framework/ResizeTaskModel.js",
"./drivers/fs.js": "./dist/drivers/fs.js",
"./drivers/s3.js": "./dist/drivers/s3.js",
"./drivers/mongo.js": "./dist/drivers/mongo/index.js",
Expand All @@ -23,7 +24,7 @@
},
"repository": {
"type": "git",
"url": "git+https://github.com/adaptivestone/framework-module-resize.git"
"url": "git+https://github.com/adaptivestone/framework-module-resizer.git"
},
"homepage": "https://framework.adaptivestone.com/docs/resize",
"keywords": [
Expand Down
29 changes: 21 additions & 8 deletions smokeTest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,7 @@ const PKG = '@adaptivestone/framework-module-resize';
// (a) main entry: exactly the expected runtime exports, and no driver class leaks into it.
const mod = await import(PKG);
const expected = [
'calculateResizedDimensions',
'formatPictureUrls',
'getFilterSig',
'getImageContentType',
'getPreviewIdentity',
'getSizeKey',
'isCatalogCovered',
'parseSizeKey',
Expand All @@ -58,15 +54,11 @@ const expected = [
'ResizeNoOriginalError',
'ResizeOriginalError',
'getResizer',
'listResizers',
'consumeQueue',
'timingOf',
'Resizer',
'ResizeDatabase',
'ResizeStorage',
'TaskQueue',
'resetResizerForTests',
'processTask',
'runWorker',
];
for (const name of expected) {
Expand Down Expand Up @@ -133,6 +125,7 @@ const safe = [
['/drivers/fs.js', 'LocalFsStorage'],
['/framework.js', 'FrameworkDatabase'],
['/framework.js', 'ResizeTaskModel'],
['/framework/ResizeTaskModel.js', 'default'],
['/framework.js', 'ResizeWorker'],
['/framework.js', 'FrameworkResizer'],
['/framework.js', 'runResizeWorker'],
Expand All @@ -142,6 +135,21 @@ for (const [sub, exp] of safe) {
assert.ok(exp in m, sub + ' should export ' + exp);
console.log(' ok ' + sub + ' imports (exports ' + exp + ')');
}
const taskModel = await import(PKG + '/framework/ResizeTaskModel.js');
const framework = await import(PKG + '/framework.js');
const expectedFramework = [
'FrameworkDatabase',
'FrameworkResizer',
'ResizeTaskModel',
'ResizeWorker',
'runResizeWorker',
];
assert.deepEqual(
Object.keys(framework).sort(),
[...expectedFramework].sort(),
'framework entry export surface drift',
);
assert.strictEqual(taskModel.default, framework.ResizeTaskModel);
const { MongoTaskQueue, MongoDatabase } = await import(PKG + '/drivers/mongo.js');
for (const Driver of [MongoTaskQueue, MongoDatabase]) {
assert.equal(
Expand Down Expand Up @@ -224,8 +232,13 @@ import {
FrameworkResizer,
type FrameworkResizeConfig,
} from '@adaptivestone/framework-module-resize/framework.js';
import ResizeTaskModel from '@adaptivestone/framework-module-resize/framework/ResizeTaskModel.js';
import { defaultFrameworkResizeConfig } from '@adaptivestone/framework-module-resize/config/resize.js';

// The scaffold imports the defining model file so framework codegen can detect its ancestor.
class ResizeTask extends ResizeTaskModel {}
void ResizeTask;

// The scaffolded config with its commented-out worker line enabled must stay a complete config.
const hostConfig = {
...defaultFrameworkResizeConfig,
Expand Down
50 changes: 50 additions & 0 deletions src/config/resize.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,24 @@ describe('getResizeConfig', () => {
}
});

test('rejects worker.concurrency and names the config file and replacement', () => {
for (const value of [8, undefined]) {
install({
...makeResizeConfig(),
worker: { ...defaultWorkerOptions, concurrency: value },
});
assert.throws(
() => getResizeConfig('resizeListings'),
(error: unknown) =>
error instanceof ResizeConfigError &&
error.code === 'RESIZE_CONFIG_REMOVED_KEY' &&
error.message.includes('`worker.concurrency`') &&
error.message.includes('src/config/resizeListings.ts') &&
error.message.includes('top-level `concurrency`'),
);
}
});

test('requires an encode.formats entry for every generated format', () => {
// 'jpg' is a Sharp alias: it would encode JPEG without the 'jpeg' options or flatten.
install(makeResizeConfig({ formats: ['jpg', 'webp'] }));
Expand Down Expand Up @@ -244,6 +262,38 @@ describe('config split: core validation vs framework loading', () => {
assert.strictEqual(getResizeConfig().worker, defaultWorkerOptions);
});

test('the dead-letter cooldown lockTtlMs.failed defaults to 10 minutes and is validated', () => {
assert.equal(defaultQueueOptions.lockTtlMs.failed, 600_000);
install(
makeResizeConfig({
queue: {
driver: 'database',
lockTtlMs: { dispatch: 30_000, worker: 30_000 },
},
}),
);
assert.deepEqual(getResizeConfig().timing.lockTtlMs, {
dispatch: 30_000,
worker: 30_000,
failed: 600_000,
});
resetAppInstance();
install(
makeResizeConfig({
queue: {
driver: 'database',
lockTtlMs: { dispatch: 30_000, worker: 30_000, failed: -5 },
},
}),
);
assert.throws(
() => getResizeConfig(),
(err: unknown) =>
err instanceof ResizeConfigError &&
err.code === 'RESIZE_CONFIG_QUEUE_LOCK_TTL_INVALID',
);
});

test('queue: false is eager only, and invalid queue timing is a config error', () => {
install(makeResizeConfig({ queue: false }));
assert.equal(getResizeConfig().queue, false);
Expand Down
8 changes: 4 additions & 4 deletions src/config/resize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ const defaultResizeConfig: ResizeConfig = {
animated: false,
encode: {
formats: {
jpeg: { quality: 80, mozjpeg: true, chromaSubsampling: '4:2:0' },
jpeg: { quality: 88, mozjpeg: true, chromaSubsampling: '4:2:0' },
webp: { quality: 82, effort: 4 },
avif: { quality: 64, effort: 4 },
},
Expand All @@ -37,14 +37,14 @@ const defaultResizeConfig: ResizeConfig = {
};

/** Queue timing and lock TTL defaults (a task queue's missing timing values). */
export const defaultQueueOptions: QueueTimingOptions = {
lockTtlMs: { dispatch: 60_000, worker: 60_000 },
export const defaultQueueOptions = {
lockTtlMs: { dispatch: 60_000, worker: 60_000, failed: 600_000 },
leaseMs: 60_000,
retryBackoffMs: { base: 5_000, max: 300_000 },
maxAttempts: 5,
idlePollMs: 1_000,
taskTimeoutMs: 600_000,
};
} satisfies QueueTimingOptions;

/** Framework worker command defaults. */
export const defaultWorkerOptions: FrameworkWorkerConfig = {
Expand Down
Loading
Loading