diff --git a/AGENTS.md b/AGENTS.md index a3e15317..a71edcda 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -113,7 +113,11 @@ Azure, `git push`, `npm publish`. `package-lock.json` in sync. TypeScript strict, ESM only, `src/` uses no Node APIs (`types: []`). 8. **Telemetry** stays disclosed (console notice, `TELEMETRY.md`, both READMEs, CHANGELOG), sends no page text unless the app opts in, keeps every opt-out (`telemetry: false`, `?genclass=no-telemetry`, localStorage, GPC) and - never feeds a decision. Widening what is sent needs the owner's OK. + never feeds a decision. Widening what is sent needs the owner's OK. Exception (owner, 2026-11-12; CONTRACT.md + §17): the collector's reply to a batch may carry `x-genclass-observe` when the app's plan has used its live fixes + for the month (Free: 100); the runtime then caps its mode at observe until the UTC month ends + (`src/telemetry/client.ts` planReply, `runtime.ts` capToObserve). It only ever lowers the mode, the batch's + content does not change, and with telemetry off there is no reply and no cap. 9. **Publish only through the release workflows**: `.github/workflows/release.yml` on a `v*` tag whose version equals `packages/runtime/package.json`; `.github/workflows/release-mcp.yml` on a `mcp-v*` tag for `@genclass/mcp` (its first version is published once by the owner from the workflow's tarball, because npm sets up diff --git a/README.md b/README.md index 339f308e..31cef0c3 100644 --- a/README.md +++ b/README.md @@ -377,6 +377,10 @@ rows and 837,623 real-browser rows (§6). Recall is the open problem. ## Privacy and telemetry - **Decisions are local.** The model runs in the browser; nothing is sent to make a decision. +- **Free's live-fix limit.** On genclass.dev's Free plan an app gets 100 problems fixed live a month. Past it, the + collector's reply to a diagnostics batch tells the page to observe only until the month ends (it keeps finding and + reporting problems, and says so once in the console). Nothing more is sent for it; with diagnostics off it never + happens. - **Anonymous diagnostics are on by default** with `GenClass.init()`, `/auto` and the script tag on public hosts, sent to the GenClass maintainers' collector ([`telemetry-worker/`](telemetry-worker), a Cloudflare Worker writing to a private R2 bucket) to improve the model. One console line per page says it is on. Off on local and private diff --git a/docs/runtime/API.md b/docs/runtime/API.md index 01b5ac03..6ebce116 100644 --- a/docs/runtime/API.md +++ b/docs/runtime/API.md @@ -211,7 +211,9 @@ prints one console line either way. Gate order (each request/decision): protected → cross-origin → op scope (created under off/observe) → mode/allow/deny/requireDiagnosis (at the effective mode) → thresholds → `policy.actionLimits` (`limit:perMinute|perSubject|perSession`; defaults 60/min global, 10/min per subject, 200 per session) → `onBeforeAction` (`vetoed`, `would-veto`, or -`limit:hold` past the budget) → execute. effectiveMode = min(mode, sample cap, breaker cap, route rule); URL overrides +`limit:hold` past the budget) → execute. effectiveMode = min(mode, sample cap, breaker cap, plan cap, route rule) +(the plan cap: the collector's reply said this app's plan used its live fixes for the month, so the page observes only +until the UTC month ends; CONTRACT.md §17, TELEMETRY.md "Where it goes"); URL overrides (`?genclass`, `?genclass-mode`, `?genclass-aggr`, `?genclass-sample`, `localStorage.genclass`) only demote, from the app's mode or from `observe` when it sets none; `?genclass=` / `localStorage.genclass` may also raise on a development host (http(s) on a loopback name with an explicit port, or `*.test`; `src/devhost.ts`), and since 0.2.3 and 0.3.0-beta.2 diff --git a/docs/runtime/CONTRACT.md b/docs/runtime/CONTRACT.md index badcf5b2..8b4d2118 100644 --- a/docs/runtime/CONTRACT.md +++ b/docs/runtime/CONTRACT.md @@ -606,3 +606,22 @@ anything" (section 13), on these terms: maps no fact to a diagnosis or an action. Model-visible text is unchanged (`situation-v2.3`). - **Nothing leaves the machine.** It talks only to a loopback URL; telemetry is unchanged. - **Determinism (section 0 rule 3).** Timing uses the injected `Clock`; schedules are hashes of a seed and a request key. + +## 17. The plan's observe cap (added 2026-11-12; owner request) + +Plans are per project on genclass.dev (billing/plans.json version 4): Free fixes at most 100 problems a month on +visitors' screens for one app (`limits.live_fixes_per_month`), then GenClass observes only until the month ends. + +- **Where the word comes from.** Only the collector's reply to a telemetry batch: the header `x-genclass-observe` + (CORS-exposed; `telemetry-worker/src/usage.ts`). The transport hands it to the client (`{ observe }`); a reply the + page cannot read (a beacon on page exit, a custom transport) changes nothing. +- **What it does.** `rt.capToObserve(true)` adds one more cap to the session mode: effectiveMode = min(mode, sample + cap, breaker cap, plan cap, route rule). It only ever lowers the mode: observe finds and reports everything and + changes nothing (section 0.5), so a correct app is never made worse by it. One console warning, once. +- **How long.** The client remembers it in `localStorage["genclass:observe"]` as `" "`, so the next page starts capped before its first reply; a reply without the header lifts the cap (the plan + was paid for) and clears the key. Another app's key is never read. +- **What does not change.** What is collected and sent, every telemetry opt-out (with telemetry off there is no reply + and no cap), situations, facts, model-visible text (`situation-v2.3`), the gate's order. Determinism: the month's + end is computed from the injected `Clock`. + diff --git a/docs/runtime/RESULTS.md b/docs/runtime/RESULTS.md index bde0925f..14af5293 100644 --- a/docs/runtime/RESULTS.md +++ b/docs/runtime/RESULTS.md @@ -514,6 +514,7 @@ walk, which missed the two shared chunks (2.74 + 0.19 KB). | + review of the mode-raise fix: `normalizeRoute`, the `mode` validation and its warning, the sample bucket's `act@` form, `readWindowConfig`'s mode check (branch `security/advisory-mode-raise`, 0.3.0-beta.5; local, 2026-10-27, `test/first-load-size.mjs` and gzip -9 against main `49c8c03`'s dist: 108.36 / 100.28 / 128.55 KB) | **108.63 KB** (+0.27) | **100.46 KB** (+0.17) | **128.83 KB** (387.4 KB minified, +0.28) | | limit in `bundle.test.ts` for 0.3.0-beta.5, owner OK 2026-10-28 (`/auto`: CI's Node 22 gzip measures 108.83 KB where Node 25 measures 108.63 KB, so the limit follows CI plus 0.07 KB; the main entry: CI measures 100.55 KB where Node 25 measures 100.46 KB, limit 100.60 KB) | 108.90 KB | 100.60 KB | none | | limit in `bundle.test.ts` for 0.3.0-beta.5 with the measured visual checks' counts in telemetry (VISUAL.md section 10), owner OK 2026-11-02: the `session` event's `visual` and `fix` fields grow both entries by 0.06 KB on Node 22 (`/auto` 108.89 KB, the main entry 100.61 KB); the summary counts live in the checks' own chunk | 109.00 KB | 100.70 KB | none | +| limit in `bundle.test.ts` for the plan's observe cap (CONTRACT.md §17), owner OK 2026-11-12 ("raise it", 0.3 KB): the telemetry client reads the collector's `x-genclass-observe` reply and the runtime caps its mode at observe; growth over main at e4f2eaa7 on Node 22.23 (`test/first-load-size.mjs`): `/auto` 108.939 → 109.239 KB (0.300 KB), the main entry 100.672 → 100.964 KB (0.292 KB) | 109.30 KB | 101.00 KB | none | Removing the vendor adapters changes no first load (they were lazy chunks of `/auto` and files the script tag loaded from the CDN). The one warning for the names `sentry`, `posthog` and `datadog` (`src/cdn/sinks.ts` `applySinks`: the diff --git a/packages/runtime/CHANGELOG.md b/packages/runtime/CHANGELOG.md index f5135e55..3dba3ad4 100644 --- a/packages/runtime/CHANGELOG.md +++ b/packages/runtime/CHANGELOG.md @@ -2,6 +2,14 @@ ## Unreleased +- **Free's live-fix limit: observe only past it.** genclass.dev's plans are per project (Free: 3 apps, 100 problems + fixed live a month per app; Team, US$15 a project a month, no limit). When an app's plan has used its live fixes for + the month, the collector's reply to a telemetry batch says so (`x-genclass-observe`), and the runtime observes only + until the UTC month ends: everything is still found and reported, nothing is changed, one console warning. It is + remembered per app in `localStorage["genclass:observe"]`; a reply without the header (the plan was paid for) lifts + it. It only ever lowers the mode; nothing more is sent, and with telemetry off nothing is capped (TELEMETRY.md "Where + it goes", CONTRACT.md §17). First load: +0.30 KB for `/auto` (109.24 KB), +0.29 KB for the main entry (100.96 KB). + - **`init`/`connect` recognise more existing setups:** `@genclass/nuxt` and `@genclass/astro` in a config, and the key of a hand-written setup (`bin/lib/init.mjs` `inspectProject`). `connect --yes` refuses to replace a key that is none of the account's apps unless `--app` or `--name` says which (`bin/lib/login.mjs`). Run by a host CLI diff --git a/packages/runtime/README.md b/packages/runtime/README.md index 16b7acd3..7bc57cca 100644 --- a/packages/runtime/README.md +++ b/packages/runtime/README.md @@ -355,6 +355,10 @@ observing with no download and no worker. ## Privacy and telemetry - **Decisions are local.** Situations are built and decided in the browser; nothing is sent to make a decision. +- **Free's live-fix limit.** On genclass.dev's Free plan an app gets 100 problems fixed live a month. Past it, the + collector's reply to a diagnostics batch tells the page to observe only until the month ends (it keeps finding and + reporting problems, and says so once in the console). Nothing more is sent for it; with diagnostics off it never + happens. - **Anonymous diagnostics are on by default** with `GenClass.init()` in a browser (and so with `/auto`, the script tag and `init` setups), except on local and private hosts (`localhost`, `*.localhost`, `*.local`, `*.test`, `127.0.0.1`, `::1`, private IP ranges), in Node/SSR, and in `createRuntime()`. `0.1.0-beta.3` to `0.2.1` send from @@ -1109,7 +1113,7 @@ retraining, which has not been measured ([RESULTS.md §10](https://github.com/genclass-dev/GenClass/blob/main/docs/runtime/RESULTS.md#10-release-030-beta0-first-load-and-telemetry-volume-ci)). The interaction-analytics stub measured about 108.5 / 100.4 KB on its branch (§15); `0.3.0-beta.1` to `beta.4` are not re-measured here (`summary().requests` added 34 B in `0.3.0-beta.2`, §23), and `test/bundle.test.ts` keeps them - under 109.0 KB and 100.7 KB (the limits follow CI's gzip on Node 22; RESULTS.md §14 lists each raise). The model worker, ONNX + under 109.3 KB and 101.0 KB (the limits follow CI's gzip on Node 22; RESULTS.md §14 lists each raise). The model worker, ONNX Runtime, the devtools, the sinks and the visual checks load on demand; ONNX Runtime's `.wasm` (3.1 MB brotli; 5.5 MB with WebGPU, per the install tests) is never part of your build. The script-tag file is about 129 KB gzip. - **Model:** with onnxruntime-web 1.30 on single-thread WASM, a forward pass takes an estimated 182 / 330 / 599 ms at diff --git a/packages/runtime/TELEMETRY.md b/packages/runtime/TELEMETRY.md index 56c770d7..c716eb0f 100644 --- a/packages/runtime/TELEMETRY.md +++ b/packages/runtime/TELEMETRY.md @@ -538,6 +538,12 @@ section 4.5. With a Content-Security-Policy, allow `https://genclass.dev` in `connect-src` or turn telemetry off; after 3 failed requests in a row a page stops sending (a 429 is a back-off, not a failure, since 0.3.0). +- **The reply:** the collector answers a batch with its status, a `Retry-After` on a 429 and, when the app's plan + has used its live fixes for the month (Free: 100 problems fixed on visitors' screens a month per app), the header + `x-genclass-observe`. Then GenClass observes only until the UTC month ends: it keeps finding and reporting every + problem and changes nothing, and it says so once in the console. It remembers this for the app in + `localStorage["genclass:observe"]` (the app's token and the month's end). A reply without the header lifts it. + Nothing more is sent for this; with telemetry off there is no reply, so nothing is capped. - **Storage:** a private Cloudflare R2 bucket (`genclass-telemetry`), gzip JSON Lines, one object per batch, partitioned by UTC date, runtime version and model version. The collector adds only the time it received the batch and the visitor's coarse **country** (two letters, from Cloudflare). It does not store IP addresses, user diff --git a/packages/runtime/src/runtime.ts b/packages/runtime/src/runtime.ts index df1fe6b6..88d7b019 100644 --- a/packages/runtime/src/runtime.ts +++ b/packages/runtime/src/runtime.ts @@ -794,11 +794,28 @@ export class RuntimeImpl implements Runtime { } private routeWarned = new Set(); - /** min(global mode, sample cap, breaker cap): the session-wide part of the effective mode. */ + /** The app's plan has used its live fixes for the month (the collector's reply, src/telemetry/client.ts planReply). */ + private planObserve = false; + + /** + * Caps this page at observe (true) or lifts that cap (false). Only the collector's reply about the app's plan calls + * it; it never raises the mode above what the app asked for. Observe finds and reports everything and changes + * nothing, which is what an app without GenClass's actions gets: a correct app is never made worse by it. + */ + capToObserve(on: boolean): void { + if (on === this.planObserve) return; + this.planObserve = on; + if (on && this._mode !== "observe") { + this.warnOnce("plan-observe", "GenClass: live fixes used for the month, observing only. genclass.dev/account/billing"); + } + } + + /** min(global mode, sample cap, breaker cap, plan cap): the session-wide part of the effective mode. */ private sessionMode(): ModeOrOff { if (this.enabledState !== "on") return "off"; let m: ModeOrOff = this._mode; if (!this.sampled) m = minMode(m, "observe"); + if (this.planObserve) m = minMode(m, "observe"); if (this.breakerImpl?.tripped) m = minMode(m, this.breakerImpl.cfg.downgradeTo); return m; } diff --git a/packages/runtime/src/telemetry/client.ts b/packages/runtime/src/telemetry/client.ts index 9a337b94..35904a9d 100644 --- a/packages/runtime/src/telemetry/client.ts +++ b/packages/runtime/src/telemetry/client.ts @@ -122,6 +122,9 @@ function compact(o: Ev): Ev { return o; } +/** localStorage: " " while this app's plan has the runtime observe only (planReply). */ +export const OBSERVE_KEY = "genclass:observe"; + export class TelemetryClient implements TelemetryStatus { readonly enabled = true; private queue: Ev[] = []; @@ -183,6 +186,8 @@ export class TelemetryClient implements TelemetryStatus { info: TelemetrySessionInfo, ) { this.t0 = rt.clock.now(); + // this app's plan said observe only on an earlier page, this month: from the first request on, not after a reply + if (this.rememberedObserve()) rt.capToObserve(true); this.lastSummaryAt = this.t0; this.session(info); // skipped before the runtime could report a change (the device floor): say why right after the session @@ -194,6 +199,38 @@ export class TelemetryClient implements TelemetryStatus { get endpoint(): string { return this.cfg.endpoint; } + + /** + * The collector's reply to a batch (the transport's answer): its OBSERVE_HEADER caps the runtime at observe and is + * remembered for this app's key until the UTC month ends; a reply without it lifts a cap that came from it. A reply + * the page could not read (a beacon on page exit, a custom transport) changes nothing. It only ever lowers the mode. + */ + private planReply(res: unknown): void { + // undefined: a reply the page could not read (a beacon, a custom transport); null: no cap + const o = (res as { observe?: unknown } | undefined)?.observe; + if (o === undefined || !this.cfg.token) return; + const on = !!o; + try { + const ls = this.g.localStorage as Storage; + const d = new Date(this.rt.clock.now()); + if (on) ls.setItem(OBSERVE_KEY, `${this.cfg.token} ${Date.UTC(d.getUTCFullYear(), d.getUTCMonth() + 1, 1)}`); + else ls.removeItem(OBSERVE_KEY); + } catch { + /* the cap holds for this page */ + } + this.rt.capToObserve(on); + } + + /** This app's key was told to observe only this month, on an earlier page (localStorage, OBSERVE_KEY: " "). */ + private rememberedObserve(): boolean { + try { + const [t, until] = String((this.g.localStorage as Storage).getItem(OBSERVE_KEY)).split(" "); + return !!this.cfg.token && t === this.cfg.token && Number(until) > this.rt.clock.now(); + } catch { + return false; + } + } + get sessionId(): string { return this.cfg.sessionId; } @@ -703,10 +740,11 @@ export class TelemetryClient implements TelemetryStatus { let p: Promise; try { p = Promise.resolve(this.cfg.transport.send(this.cfg.endpoint, body, { beacon })).then( - () => { + (res: unknown) => { this.sentEvents += batch.length; this.failStreak = 0; this.backoffs = 0; + this.planReply(res); }, (e: unknown) => this.refused(e, batch), ); diff --git a/packages/runtime/src/telemetry/transport.ts b/packages/runtime/src/telemetry/transport.ts index f7e57ffb..749dad49 100644 --- a/packages/runtime/src/telemetry/transport.ts +++ b/packages/runtime/src/telemetry/transport.ts @@ -18,6 +18,12 @@ const NATIVE_BEACON: ((url: string, data?: BodyInit | null) => boolean) | undefi } })(); +/** + * The header of the collector's reply that says this app's plan has used its live fixes for the month: the runtime + * then observes only (it finds and reports, and changes nothing) until the month ends or the reply stops saying so. + */ +export const OBSERVE_HEADER = "x-genclass-observe"; + /** Whether this environment can send at all (fetch or sendBeacon captured at module load). */ export function nativeTransportAvailable(): boolean { return !!NATIVE_FETCH || !!NATIVE_BEACON; @@ -56,6 +62,8 @@ export const nativeTransport: TelemetryTransport = { }).then((r) => { // the status, and a Retry-After the collector exposes, decide back-off (429) vs failure (client.ts backoffOf) if (!r.ok) throw Object.assign(new Error(`HTTP ${r.status}`), { status: r.status, retryAfterMs: retryAfterMs(r.headers?.get?.("retry-after")) }); + // the collector's word on this app's plan: observe only (client.ts planReply) + return { observe: r.headers?.get?.(OBSERVE_HEADER) ?? null }; }); } catch (e) { return Promise.reject(e); diff --git a/packages/runtime/test/bundle.test.ts b/packages/runtime/test/bundle.test.ts index 5dc6e2a7..6b0a7cbf 100644 --- a/packages/runtime/test/bundle.test.ts +++ b/packages/runtime/test/bundle.test.ts @@ -99,7 +99,12 @@ describe.skipIf(!built)("dist as an app bundles it", () => { // The main entry likewise: CI measures 100.55 KB (Node 25: 100.46 KB); its growth from the same fixes is 0.15 KB. // 0.3.0-beta.5 (owner OK 2026-11-02): the session event's two visual-checks fields (`visual`, `fix`; VISUAL.md // section 10) grow both by 0.06 KB (Node 22: /auto 108.89 KB, the main entry 100.61 KB); the limits follow. - const LIMIT_KB: Record = { "auto.js": 109.0, "index.js": 100.7 }; + // The plan's observe cap (owner OK 2026-11-12, "raise it" by 0.3 KB): the telemetry client reads the collector's + // x-genclass-observe reply header and remembers it for the month, and the runtime caps its mode at observe (src/ + // telemetry/client.ts planReply, runtime.ts capToObserve). Growth over main at e4f2eaa7 (Node 22.23, + // test/first-load-size.mjs): /auto 108.939 -> 109.239 KB (0.300 KB), the main entry 100.672 -> 100.964 KB + // (0.292 KB). Both limits are raised by 0.3 KB. + const LIMIT_KB: Record = { "auto.js": 109.3, "index.js": 101.0 }; it("first-load cost of /auto and the main entry, minified + gzip: under the limits", async () => { const { build } = await import("esbuild"); const uses: Record = { "auto.js": "", "index.js": "GenClass.init({});" }; diff --git a/packages/runtime/test/telemetry.test.ts b/packages/runtime/test/telemetry.test.ts index e2985b4a..80b6cd2c 100644 --- a/packages/runtime/test/telemetry.test.ts +++ b/packages/runtime/test/telemetry.test.ts @@ -862,3 +862,54 @@ describe("telemetry: a full queue drops routine decision records first (TELEMETR }); void createRuntime; + +describe("telemetry: the collector's word on the app's plan (observe only past its live fixes, owner 2026-11-12)", () => { + const TOKEN = "gc_AbCdEfGhIjKlMnOpQrStUv"; + /** A localStorage stand-in. */ + const storage = () => { + const m = new Map(); + return { getItem: (k: string) => m.get(k) ?? null, setItem: (k: string, v: string) => void m.set(k, String(v)), removeItem: (k: string) => void m.delete(k), m }; + }; + /** A transport whose replies say observe (or not), as the collector's header does through the native transport. */ + function replying(observe: () => string | null) { + const sent: unknown[] = []; + return { sent, send(_u: string, body: string) { sent.push(JSON.parse(body)); return Promise.resolve({ observe: observe() }); } }; + } + + it("a reply that says observe caps the page at observe; it is remembered for this key until the month ends; a reply without it lifts it", async () => { + let say: string | null = "live-fix-limit"; + const t = replying(() => say); + const ls = storage(); + const s = setup({ token: TOKEN, telemetry: { transport: t }, extraGlobal: { crypto: globalThis.crypto, localStorage: ls } }); + const rt = s.rt as unknown as RuntimeImpl; + expect(rt.effectiveMode()).toBe("guard"); + await rt.telemetry!.flush(); + expect(rt.effectiveMode()).toBe("observe"); + const now = new Date(s.clock.now()); + expect(ls.m.get("genclass:observe")).toBe(`${TOKEN} ${Date.UTC(now.getUTCFullYear(), now.getUTCMonth() + 1, 1)}`); + // the next page of the same app starts in observe, before any reply + const t2 = replying(() => null); + const next = setup({ token: TOKEN, telemetry: { transport: t2 }, extraGlobal: { crypto: globalThis.crypto, localStorage: ls } }); + expect((next.rt as unknown as RuntimeImpl).effectiveMode()).toBe("observe"); + // another app on the same origin is not capped by it + const other = setup({ token: "gc_ZzZzZzZzZzZzZzZzZzZzZz", telemetry: { transport: replying(() => null) }, extraGlobal: { crypto: globalThis.crypto, localStorage: ls } }); + expect((other.rt as unknown as RuntimeImpl).effectiveMode()).toBe("guard"); + // the plan changed (paid): a reply without the header lifts the cap and forgets it + say = null; + await (next.rt as unknown as RuntimeImpl).telemetry!.flush(); + expect((next.rt as unknown as RuntimeImpl).effectiveMode()).toBe("guard"); + expect(ls.m.has("genclass:observe")).toBe(false); + }); + + it("it never raises a mode: an app in observe stays in observe, and a reply the page cannot read changes nothing", async () => { + const ls = storage(); + const s = setup({ mode: "observe", token: TOKEN, telemetry: { transport: replying(() => null) }, extraGlobal: { crypto: globalThis.crypto, localStorage: ls } }); + await (s.rt as unknown as RuntimeImpl).telemetry!.flush(); + expect((s.rt as unknown as RuntimeImpl).effectiveMode()).toBe("observe"); + // a beacon or a custom transport answers nothing readable + const quiet = setup({ token: TOKEN, telemetry: { transport: { send: () => Promise.resolve() } }, extraGlobal: { crypto: globalThis.crypto, localStorage: ls } }); + (quiet.rt as unknown as RuntimeImpl).capToObserve(true); + await (quiet.rt as unknown as RuntimeImpl).telemetry!.flush(); + expect((quiet.rt as unknown as RuntimeImpl).effectiveMode()).toBe("observe"); + }); +});