feat: idle-time self-test, context probe, resources roll-up and usage measurement

.abcd/development/decisions/adrs/2609090716413337-the-statistics-store-s-record-kinds-are-request-load-removed.md

@@ -1,10 +1,10 @@
 ---
 id: adr-2609090716413337
 slug: the-statistics-store-s-record-kinds-are-request-load-removed
-status: accepted
+status: superseded
 date: 2026-09-09
 supersedes: adr-2609061610107154
-superseded_by: null
+superseded_by: adr-2609121450000000
 related_intents: [itd-2609061521102742, itd-2609061602043757]
 related_rfcs: []
 related_adrs: [adr-2609061610107154, adr-2609061503319212]

.abcd/development/decisions/adrs/2609121450000000-the-statistics-store-gains-a-fifth-record-kind-a-footprint.md

@@ -0,0 +1,76 @@
+---
+id: adr-2609121450000000
+slug: the-statistics-store-gains-a-fifth-record-kind-a-footprint
+status: accepted
+date: 2026-09-12
+supersedes: adr-2609090716413337
+superseded_by: null
+related_intents: [itd-2609091712141073, itd-2609061521102742]
+related_rfcs: []
+related_adrs: [adr-2609090716413337, adr-2609061610107154, adr-2609061503319212]
+---
+
+# ADR-2609121450000000: The statistics store gains a fifth record kind, a footprint reading, and the request and load records gain the facts that decide a served window and a concurrency
+
+## Context
+
+adr-2609090716413337 ratified four record kinds: a request line, a load, a
+removal with one of seven reasons, and a settings record. The usage
+measurement intent (itd-2609091712141073) needs the facts that decide a
+served window and a decode concurrency — how prompts sat against the window,
+how many requests were in flight, which sampling values were in force and
+whether the client overrode them, and what the model server's memory did —
+and one of those, the memory, is not a fact about any request. It is a
+reading of a process, taken on a clock, and it needs a record of its own or
+the figure exists only as whatever value a completing request happened to
+see.
+
+This decision was taken by the agent on the maintainer's lane assignment of
+2026-09-12, under the instruction to decide autonomously and record; the
+record-kinds decision is architecture-shaping, which is why this is an ADR
+and not a line.
+
+## Decision
+
+We supersede adr-2609090716413337 on the record kinds, and adopt five:
+
+1. **A request line**, as before, gaining: the declared and served windows
+   the request was judged against, Gropius's own estimate of the prompt's
+   size (written for a refused request too), the model's in-flight count at
+   admission, the names — never the values — of the sampling parameters the
+   client set, and the server's footprint as last sampled before completion.
+2. **A load**, as before, gaining the sampling values the server was launched
+   with.
+3. **A removal**, unchanged.
+4. **A footprint reading** — `kind: "footprint"` — the model's repo id, when,
+   and the server process's resident memory in bytes, taken every thirty
+   seconds for each running server and reported through the pool's observer.
+5. **A settings record**, unchanged.
+
+Every new numeric is bounded at the recorder as the windows already are: a
+window past the registry's bound, a count past any batch, a footprint past
+any Mac, or an override name outside the closed set is dropped, never
+repaired. The override names are the one field derived from a client's body,
+and the reference page says so.
+
+## Alternatives Considered
+
+1. **Carry the footprint only on the request record.** Rejected: the figure
+   would exist only at the moments requests completed, which on a quiet
+   server is never, and the intent's criterion asks for a periodic record.
+2. **A separate file for the samples.** Rejected: the store is one
+   collection path under one switch and one retention rule
+   (adr-2609061610107154), and a second file would need a second of each.
+3. **Store the client's sampling values.** Rejected: adr-2609061503319212
+   draws the line at content the client sent; a name says which knob was
+   touched and nothing of what was asked.
+
+## Consequences
+
+- `internal/stats.StoreKinds` names seven kinds of line (five records plus
+  the two summary lines), and the reference page's table is held to it.
+- The summary a day folds into ignores footprint readings: a mean over a
+  day is not a figure anyone would size a setting from, and the series is
+  what the view draws.
+- The record size constant is re-measured over a record carrying every
+  field.

.abcd/development/intents/planned/itd-2609091301112705-gropius-measures-a-newly-downloaded-model-s-servable-context.md → .abcd/development/intents/shipped/itd-2609091301112705-gropius-measures-a-newly-downloaded-model-s-servable-context.md

@@ -138,12 +138,13 @@ named bound rather than the architecture's limit.
 - **How the model's own window is ever learned.** A coordinated pool-side path exempt from the gateway's bounds, a raised `upstream_header_timeout_sec` for the duration, or the manual campaign script staying the only answer. Nothing here decides it.
 - **Whether the probe also samples memory**, which would keep `capability`'s safety factors of 5 and 7 under continuous evidence rather than one evening's. Out of scope here — that package belongs to another lane — and worth a follow-on.
 - **The idle threshold's value**, and how many models one idle window may take: six models at forty minutes each is an evening, and the models load one at a time.
-- **Arbitration with the model self-test**, which also runs at idle and is not yet on `main`. Two idle jobs on one Mac need one idea of "idle" between them, and whichever lands second inherits the question.
+- **Arbitration with the model self-test** — resolved 2026-09-12 at implementation: the self-test landed first and its loop is the one idle primitive; the probe is a `selftest.Job` it schedules, so idleness, yielding, the never-evict check, the held-back reason and the idle threshold (`idle_threshold_sec`) are decided once, in `internal/selftest`, for both.
 - **Whether the yielding seam becomes real preemption in the pool.** The pool has no preemption and no priority today; this intent reads the pool and cancels its own request instead. Any change to the pool itself is a coordinated follow-up with the lane that owns `internal/runtime`.
 
 ## Audit Notes
 
-_Empty. Populated by intent-auditor when intent moves to shipped/._
+<!-- abcd-review: OWED receipt=rcp-4431dce599cc -->
+Fidelity review OWED (receipt rcp-4431dce599cc).
 
 ## Grounds
 

.abcd/development/intents/planned/itd-2609091712141073-gropius-shows-what-its-models-actually-do-so-the-settings-th.md → .abcd/development/intents/shipped/itd-2609091712141073-gropius-shows-what-its-models-actually-do-so-the-settings-th.md

@@ -67,7 +67,8 @@ per-request memory attribution the sample cannot give.
 
 ## Audit Notes
 
-_Empty. Populated by intent-auditor when intent moves to shipped/._
+<!-- abcd-review: OWED receipt=rcp-916dd6b8bf3f -->
+Fidelity review OWED (receipt rcp-916dd6b8bf3f).
 
 ## Grounds
 

.abcd/development/intents/planned/itd-2609091903463596-alice-sees-what-her-mac-is-spending-on-gropius-a-resources-t.md → .abcd/development/intents/shipped/itd-2609091903463596-alice-sees-what-her-mac-is-spending-on-gropius-a-resources-t.md

@@ -63,7 +63,8 @@ needs a directory walk or a second reader.
 
 ## Audit Notes
 
-_Empty. Populated by intent-auditor when intent moves to shipped/._
+<!-- abcd-review: OWED receipt=rcp-81fed0a81798 -->
+Fidelity review OWED (receipt rcp-81fed0a81798).
 
 ## Grounds
 

.abcd/development/intents/shipped/itd-2609100457007827-gropius-tests-its-own-models-while-nobody-is-using-it-alice.md

@@ -0,0 +1,97 @@
+---
+id: itd-2609100457007827
+slug: gropius-tests-its-own-models-while-nobody-is-using-it-alice
+spec_id: spc-2609100502029941
+kind: standalone
+suggested_kind: null
+reclassification_history: []
+builds_on: [itd-2609091301112705, itd-2609061521082551]
+severity: minor
+impact: additive
+origin: researcher-authored
+production_mode: dictated-and-formatted
+---
+
+# Gropius tests its own models while nobody is using it. Alice switches on the self-test in Settings; whenever her Mac is idle, Gropius loads each downloaded model in turn, runs the same standard set of tests against it, unloads it, and records what it measured, so that the numbers she sizes a model's settings from come from her own Mac rather than from someone else's benchmark. The runs never delay a real request and nothing leaves the machine.
+
+## Press Release
+
+Gropius now measures its own models, on your Mac, while nobody is using it.
+Alice ticks **Self-test when idle** in Settings and saves. That evening,
+once the last request is a few minutes old and nothing is downloading,
+Gropius loads the first of her models, runs the same short set of tests every
+model gets — how long a load takes, how fast a long prompt is read, how fast
+tokens come out, and what happens when four requests arrive at once — writes
+one line of figures to a file in her Gropius folder, and unloads the model if
+it was not resident before. Then the next model, and the next. In the morning
+the file holds one line per model, measured on her machine, and the number she
+sizes the served window and the concurrency from is hers rather than a
+stranger's. Bob's agent sending a request at midnight is never made to wait:
+the moment a real request arrives the run in progress is abandoned, marked as
+yielded, and the model is his. Nothing in the file is a prompt or an answer,
+and nothing in it leaves the Mac.
+
+## Why This Matters
+
+The 2026-09-06 model-bench campaign produced the numbers the memory budget and
+the served window are now designed from — and it was one person running a
+script by hand for an evening on one Mac, a record that goes stale with the
+next model, the next runtime or the next machine. The context-probe draft
+(itd-2609091301112705) automates one of those measurements; this is the
+harness that draft is one test of, and it refines it: the idle detection, the
+yielding to a real request and the rule against evicting a pinned model are
+the same costs, carried once. The usage dashboard (itd-2609091712141073)
+measures what real requests did; this measures what a model can do when asked
+the same thing as every other model, which is the comparison usage traffic
+never gives. The held draft itd-2609091712142715, settings that fit by
+themselves, is what these figures exist to feed; it is not part of this intent.
+
+## Mechanism
+
+We expect one line of figures per model per day to be enough to compare models
+and size their settings, because the four measurements are the ones the
+2026-09-06 campaign actually used to decide the budget and the window, and
+llama-bench's prompt-processing and text-generation pair has been the shared
+vocabulary for local inference numbers since 2023. Shown wrong if a setting the
+figures were meant to decide turns out to need a measurement not in the set, or
+if idle time on a serving Mac is too rare for a cycle to complete.
+
+## Scope Conditions
+
+- Apple Silicon macOS, one Mac, with the self-test switch on; off is the default and the same state as today. <!-- cond: cond-2609100502039780 -->
+- Idle means: no request in flight on any model, nobody waiting for a load, no download running, and the last request older than a quiet period. <!-- cond: cond-2609100502036756 -->
+- The self-test loads through the pool's ordinary Acquire path and never a second launcher, so the budget, the pins and the served window are the pool's own; and it never evicts, loading a model only when it fits beside what is resident. <!-- cond: cond-2609100502036640 -->
+- The pool has no preemption: yielding is the self-test cancelling its own request, not the pool taking the model away, so a real request can wait for one in-flight self-test request to be cancelled. <!-- cond: cond-2609100502034205 -->
+- The results file holds figures only, never a prompt or an answer; the test prompts are constants in the code, and no client's request is ever part of a run. <!-- cond: cond-2609100502035912 -->
+- Shared-cache mode records under the serving account's own data directory, at 0600; the models measured are every account's, since they are one set. <!-- cond: cond-2609100502031008 -->
+
+## Acceptance Criteria
+
+- Given the switch is off, when Gropius runs for any length of time, then no model is loaded by the self-test and no results file exists.
+- Given the switch is on and the Mac is idle, when a ready model has not been tested within the re-test window, then it is acquired through the pool, the standard set runs against it, and one line is appended to the results file.
+- Given a model was not resident before its run, when the run ends, then the model is unloaded; given it was resident, then it is left where it was.
+- Given a run is in progress, when a request from a client arrives on any model, or a client's load is refused for want of room, then the self-test's request or load is cancelled, the run is recorded as yielded, and the model is released.
+- Given a ready model that would need another evicted to load, when the tick fires, then it is not loaded, and it is measured once there is room.
+- Given a request is in flight, a load is waiting, or a download is running, when the tick fires, then nothing is loaded and nothing runs.
+- Given every ready model has been tested within the re-test window, when the tick fires, then nothing runs.
+- Given a results line, when it is read back, then it carries the model, when, the outcome, whether the load was cold, the load time, and for each test its name, token counts, time to first token, total time and tokens per second, and no field carries prompt or answer text.
+- Given the switch is turned off while a run is in progress, when the save applies, then the run is cancelled and the model released.
+- Given the results file reaches its size cap, when the next line is written, then the file is started again and the newest line is kept.
+- Given the switch, when Settings is rendered, then the panel offers it and posts it under the key config.json reads.
+- Given the docs, when the feature ships, then a how-to page says how to switch it on and what it costs, and a reference page names every field of a results line and where the file is.
+
+## Open Questions
+
+- Resolved 2026-09-10 at planning (delegated to the agent by the maintainer): the standard set is load, pp512, tg128 and tg128 at the decode concurrency; the names follow llama-bench so the figures can be compared with published ones.
+- Resolved 2026-09-10 at planning: results go to a single bounded JSON Lines file under the account's data directory rather than into the statistics store, because the store's schema is per-request and its writer is in another lane; folding it in is a follow-up.
+- Resolved 2026-09-10 at planning: no panel view of the results in this intent; the file is the surface, the panel has the switch. A results view is a follow-up.
+- Deferred: the re-test cadence is fixed at a day; whether it should be a setting is decided when someone wants a different one.
+
+## Audit Notes
+
+<!-- abcd-review: OWED receipt=rcp-750863424b6e -->
+Fidelity review OWED (receipt rcp-750863424b6e).
+
+## Grounds
+
+- pursued: we expect a per-model figure measured on the operator's own Mac, taken automatically, to replace the one-evening hand-run campaign as the source the settings are sized from; shown wrong if after a cycle the figures separate no setting choice, or if a serving Mac is never idle long enough for a cycle to complete. Planning delegated to the agent by the maintainer on 2026-09-10 as an experiment.

.abcd/development/research/notes/2026-09-06-decomposition-calibration.md

@@ -348,3 +348,26 @@ asset fetched from a CDN; the human did not confirm a reversal, so none is
 recorded and the question is held at the decision.
 
 Verdict adopted: SPLIT. Grade: routing survived as proposed.
+
+## 2026-09-10 — idle-time model self-test (itd-2609100457007827)
+
+Proposal: an autonomous state-of-the-art self-test and (later)
+self-optimisation environment: when activated it loads and unloads the
+available models, runs standard tests, captures all telemetry, and feeds the
+results into a self-optimisation setting.
+
+| Part | Type | Home |
+| --- | --- | --- |
+| Opt-in self-test: load each model when idle, run a standard set, unload, record | capability | this intent |
+| Self-optimisation from the results | capability, held | `duplicates` itd-2609091712142715, not filed twice |
+| What the self-test may record, and which opt-in gates it | ADR consequence | adr-2609061503319212 already scopes it; a scope condition on the draft |
+| Which tests are standard, which use cases | open questions | the draft, settled at planning |
+| Where the switch lives, how runs are recorded | mechanism | the spec |
+
+Typed links: `refines` itd-2609091301112705 (the probe is one test of this
+harness; the human chose refines over supersedes); `builds_on` the 2026-09-06
+model-bench campaign; `duplicates` itd-2609091712142715 for the second half.
+Flagged, not classified: "captures all telemetry" against the ADR's no-prompt
+rule — taken as a scope condition, not a reversal.
+Verdict adopted: SPLIT. Grade: routing survived the human's confirmation
+unchanged; the interview itself was delegated to the agent by the maintainer.