From 18f85e059ecc98159fa5b494d42dd23a4398e16e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Mon, 6 Jul 2026 04:39:18 +0000 Subject: [PATCH 01/31] feat(data-contracts): add SimulatableRegistrarExt and ObservableRegistrarExt for enhanced data integration --- docs/design/041-data-contracts-integration.md | 275 ++++++++++++++++++ 1 file changed, 275 insertions(+) create mode 100644 docs/design/041-data-contracts-integration.md diff --git a/docs/design/041-data-contracts-integration.md b/docs/design/041-data-contracts-integration.md new file mode 100644 index 0000000..021a97f --- /dev/null +++ b/docs/design/041-data-contracts-integration.md @@ -0,0 +1,275 @@ +# 041 — Data-Contracts Integration: make the capability story real (D1/D12) + +**Status:** Proposed 2026-07-06. Follow-up design doc for the design-038 **D1/D12** decision (tracking issue [#161](https://github.com/aimdb-dev/aimdb/issues/161)). Companion to the claims-assessment track (039, 040). +**Scope:** `aimdb-data-contracts` (add two registrar extension traits — one of which also gains a build-time sim-to-real source selector), the README capability table, and the weather-mesh example (adopt the new methods). **No changes to the contract trait definitions themselves** — `Simulatable`, `Observable`, `Streamable`, `Linkable`, `Migratable` keep their current shape. +**Goal:** Turn `aimdb-data-contracts` from a *side-channel* (traits nothing consumes) into a *consumed layer*, so the README's "capability traits" story is backed by code the engine actually runs — using the exact mechanism a third party would use. + +--- + +## 1. Problem + +Design 038 findings **D1** ("building abstractions for consumers that never arrived") and **D12** ("marketing vocabulary in API design") both landed on the same subject: the data-contract capability traits are advertised as a headline feature, but core consumes none of them, and two of them — `Simulatable` and `Observable` — have **no non-example consumer at all**. + +Concretely, today: + +- `Simulatable::simulate()` is only ever called **by hand**, inside example producer loops. Each weather station repeats ~30 lines of "make an RNG → loop → `T::simulate(...)` → `produce` → sleep" boilerplate ([`weather-station-beta/src/main.rs:132-189`](../../examples/weather-mesh-demo/weather-station-beta/src/main.rs), [`weather-station-gamma/src/main.rs:80-125`](../../examples/weather-mesh-demo/weather-station-gamma/src/main.rs)). +- `Observable`'s only non-example consumer is the opt-in `log_tap()` helper, which the user must wire manually as a tap ([`weather-hub/src/main.rs:36`](../../examples/weather-mesh-demo/weather-hub/src/main.rs)). +- `RecordRegistrar::source()` / `.tap()` are fully generic with **no trait bounds** — and they must stay that way (they are general-purpose stage installers, not contract-specific). + +The 2026-07-01 owner decision (recorded in 038 §4 D1/D12) resolved the tension **by completion, not deletion**: data contracts are core to AimDB's identity, so the fix is to *add the missing consumer*, not remove the API. This doc specifies that consumer. + +## 2. The model to copy: `.persist()` + +`aimdb-persistence` already demonstrates the pattern this doc generalizes. `RecordRegistrarPersistExt::persist()` ([`aimdb-persistence/src/ext.rs`](../../aimdb-persistence/src/ext.rs)) is an **extension trait on `RecordRegistrar`** that installs a `.tap()` — the engine drives it exactly like any other tap, and `aimdb-core` never learns persistence exists: + +```rust +// aimdb-persistence/src/ext.rs (abbreviated) +impl<'a, T> RecordRegistrarPersistExt<'a, T> for RecordRegistrar<'a, T> { + fn persist(&mut self, record_name: impl Into) -> &mut RecordRegistrar<'a, T> { + // ... pull backend from Extensions TypeMap ... + self.tap(move |_ctx, consumer| async move { /* subscribe → serialize → store */ }) + } +} +``` + +The two properties we copy: + +1. **Extension trait on `RecordRegistrar`**, not a bound on `.source()`/`.tap()`. The generic stage installers stay generic; the contract-specific ergonomics live in the contract crate. +2. **The engine consumes the trait through the same mechanism third parties would use** — `.source()`/`.tap()` are public API; a `simulate()` that calls `.source()` is doing nothing a downstream crate couldn't do itself. + +**One difference from `.persist()`:** persistence needed builder-side state (a backend registered via `.with_persistence()`, retrieved from the `Extensions` TypeMap). The contract extensions need **no builder-side state** — `simulate(cfg)` receives its config inline and `observe(node_id)` receives its label inline. So there is **no `.with_*()` builder extension** in this design; both methods are pure registrar-time sugar. + +## 3. Design + +Two extension traits added to `aimdb-data-contracts`, each gated behind the feature that already carries its contract trait. + +### 3.1 `SimulatableRegistrarExt::simulate(cfg, rng)` (feature `simulatable`) + +Installs the generate → produce → sleep source loop that examples write by hand. **The caller supplies the RNG** (decision §5.1): `Simulatable::simulate` is generic over the RNG, and no_std/Embassy has no default entropy source, so `.simulate()` takes an owned `R: rand::Rng + Send + 'static` that moves into the source future. + +```rust +// aimdb-data-contracts/src/simulatable.rs (new ext, feature = "simulatable") +pub trait SimulatableRegistrarExt<'a, T> +where + T: Simulatable + Send + Sync + Clone + 'static, +{ + /// Install a source that emits `T::simulate(...)` every `cfg.interval_ms`, + /// driving the supplied RNG. Works on any runtime — the caller chooses the + /// RNG (OS entropy on std, a seeded PRNG on no_std). + fn simulate(&mut self, cfg: SimulationConfig, rng: R) -> &mut RecordRegistrar<'a, T> + where + R: rand::Rng + Send + 'static; +} + +impl<'a, T> SimulatableRegistrarExt<'a, T> for RecordRegistrar<'a, T> +where + T: Simulatable + Send + Sync + Clone + 'static, +{ + fn simulate(&mut self, cfg: SimulationConfig, mut rng: R) -> &mut RecordRegistrar<'a, T> + where + R: rand::Rng + Send + 'static, + { + if !cfg.enabled { + // Simulation gated off by config → install no source. The record + // can still receive a producer elsewhere. (Mirrors `.persist()`'s + // no-op-when-unconfigured precedent.) + return self; + } + self.source(move |ctx, producer| async move { + let mut prev: Option = None; + loop { + let now_ms = ctx + .time() + .unix_time() + .map(|(s, ns)| s.saturating_mul(1000) + (ns / 1_000_000) as u64) + .unwrap_or(0); + let next = T::simulate(&cfg, prev.as_ref(), &mut rng, now_ms); + producer.produce(next.clone()); + prev = Some(next); + ctx.time().sleep_millis(cfg.interval_ms.max(1)).await; + } + }) + } +} +``` + +Notes: +- **Caller-supplied RNG** keeps `.simulate()` runtime-agnostic and drops the hard dependency on `rand`'s `std_rng` feature: std callers pass `StdRng::from_rng(&mut rand::rng())` (what the demo already constructs), an Embassy target passes a seeded PRNG. Deterministic tests get a fixed seed for free. +- Uses `RuntimeContext::time()` for both the timestamp (`unix_time()`) and the delay (`sleep_millis()`) — so the loop is runtime-neutral (works on any adapter), exactly as `.persist()`'s cleanup loop is. +- `.simulate()` installs a **source**, so it participates in the existing writer-exclusivity rule (a simulated record cannot also be a `.transform()`/`.link_from()` source — the conflict is reported from `build()`, unchanged). +- Removes ~30 lines of boilerplate per station and deletes the manual `db.producer::(...)` + `tokio::spawn` dance. + +### 3.2 `ObservableRegistrarExt::observe(node_id)` (feature `observable`) + +Installs `log_tap` as a tap — the one-liner that replaces the manual `.tap(|ctx, c| log_tap(ctx, c, id))`. + +```rust +// aimdb-data-contracts/src/observable.rs (new ext, feature = "observable") +pub trait ObservableRegistrarExt<'a, T> +where + T: Observable + Send + Sync + Clone + core::fmt::Debug + 'static, +{ + /// Install a logging tap that prints `T::format_log(node_id)` per value. + fn observe(&mut self, node_id: &'static str) -> &mut RecordRegistrar<'a, T>; +} + +impl<'a, T> ObservableRegistrarExt<'a, T> for RecordRegistrar<'a, T> +where + T: Observable + Send + Sync + Clone + core::fmt::Debug + 'static, +{ + fn observe(&mut self, node_id: &'static str) -> &mut RecordRegistrar<'a, T> { + self.tap(move |ctx, consumer| log_tap(ctx, consumer, node_id)) + .with_name("observe") // surfaces the tap in stage profiling (no-op w/o `observability`) + } +} +``` + +Notes: +- `log_tap` is unchanged and stays exported for callers who want the raw form. +- `.with_name("observe")` gives the tap a stable identity in the stage-profiling snapshot when the `observability` feature is on — a small, honest link to the observability subsystem (see §4). + +### 3.3 `SimulatableRegistrarExt::source_or_simulate(mode, cfg, rng, real)` (feature `simulatable`) + +A record has **exactly one producer**: `set_producer` rejects a second `.source()`, and a source is mutually exclusive with `.transform()`/`.link_from()` — validated once in `build()`, not panicked ([`typed_record.rs:570`](../../aimdb-core/src/typed_record.rs)). Both `.simulate()` (§3.1) and a real `.source()` install *that same slot*. So the sim-to-real workflow — "run against a real sensor in production, against `Simulatable` in dev / CI / demo / when the hardware is absent" — is **not** a coexistence problem. It is a **build-time source selection**: choose which producer fills the one slot. This keeps single-writer-per-key true *by construction* (only one branch ever installs a producer, so `build()` never sees a conflict) and costs nothing at runtime. + +The combinator adds a second method to the §3.1 trait, wrapping the branch a caller would otherwise write by hand: + +```rust +// aimdb-data-contracts/src/simulatable.rs (feature = "simulatable") + +/// Which producer fills a record's single source slot. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum SourceMode { + /// Install the real `.source()` closure (hardware / network feed). + Real, + /// Install the `Simulatable` generate → produce → sleep loop (§3.1). + Simulated, +} + +pub trait SimulatableRegistrarExt<'a, T> +where + T: Simulatable + Send + Sync + Clone + 'static, +{ + // fn simulate(&mut self, cfg: SimulationConfig, rng: R) -> ... // §3.1 + + /// Install `real` when `mode == Real`, else the §3.1 simulation loop. + /// Exactly one producer is installed either way, so writer-exclusivity + /// holds and `build()` never reports a conflict. + fn source_or_simulate( + &mut self, + mode: SourceMode, + cfg: SimulationConfig, + rng: R, + real: F, + ) -> &mut RecordRegistrar<'a, T> + where + F: FnOnce(RuntimeContext, Producer) -> Fut + Send + 'static, + Fut: core::future::Future + Send + 'static, + R: rand::Rng + Send + 'static; +} + +impl<'a, T> SimulatableRegistrarExt<'a, T> for RecordRegistrar<'a, T> +where + T: Simulatable + Send + Sync + Clone + 'static, +{ + fn source_or_simulate( + &mut self, + mode: SourceMode, + cfg: SimulationConfig, + rng: R, + real: F, + ) -> &mut RecordRegistrar<'a, T> + where + F: FnOnce(RuntimeContext, Producer) -> Fut + Send + 'static, + Fut: core::future::Future + Send + 'static, + R: rand::Rng + Send + 'static, + { + match mode { + SourceMode::Real => self.source(real), // the one general-purpose stage installer + SourceMode::Simulated => self.simulate(cfg, rng), // §3.1 + } + } +} +``` + +Notes: +- **`mode` is caller-derived, exactly like `rng` (§5.1).** The ext stays runtime-agnostic — it does not read env or entropy itself. A std station derives it from an env var / config field (`AIMDB_SOURCE=sim|real`); a no_std target selects with a **Cargo feature** so the real-hardware source only *compiles* on the board (`#[cfg(feature = "hw")]`) and simulation is the fall-through. The record wiring is identical both ways — this is the embedded "develop against sim, flash against silicon" story made first-class, not a demo detail. +- **`cfg.enabled` stays sim's internal gate, not the selector.** `mode` picks the branch; within `Simulated`, `enabled: false` still installs no source (§3.1). `Simulated` + `enabled: false` is therefore the explicit "leave this record writer-less" case — identical to calling `.simulate()` with a disabled config today. +- **Both `real` and `rng` are owned and moved into the chosen branch;** the unused one is dropped at registration. No boxing of the unused arm, no runtime flag — the selection is fully resolved before `build()`. +- **`SourceMode` is re-exported next to `SimulationConfig`/`SimulationParams`** from `aimdb-data-contracts` (lib.rs `pub use`). +- **Runtime failover is out of scope.** Falling over to simulated values *live* when a sensor drops, then back, cannot be two sources (that breaks writer-exclusivity); it needs a single source future that internally multiplexes real vs. `T::simulate(...)`. Recorded as a follow-up in §4 / §7. + +### 3.4 What stays out + +- **No trait bounds added to `.source()`/`.tap()`/`.transform()`.** They remain general-purpose. +- **No new dependency in `aimdb-core`.** The direction of the dependency is `aimdb-data-contracts → aimdb-core`, never the reverse — same as persistence. +- **No change to the five contract trait definitions.** Per the issue and the user's steer: keep the traits as they are; only add the integration layer. +- **No runtime sim↔real hot-swap.** `source_or_simulate` selects once, at registration; a live failover multiplexer is a separate design (§7). + +## 4. Every capability trait: current state → target + +The issue asks for the `Observable` claim to be made honest. Rather than fix that row in isolation, we set a **current state → target** entry for *every* capability trait — this is the map the README table and future integration work should track against. The through-line: a trait's "target" is *the engine consuming it*, and the README should describe the **current** state, not the target. + +| Trait | Current state (verified) | Target | +|---|---|---| +| `Streamable` | **Consumed.** Bound by `aimdb-websocket-connector`'s `StreamableRegistry::register()` ([`server/registry.rs:33`](../../aimdb-websocket-connector/src/server/registry.rs), `server/builder.rs:250`) as the schema-name gate for browser streaming. Marker trait (no methods). | Keep as the "may cross a serialization boundary" marker + registry key. Already real — README row is accurate. | +| `Migratable` | **Consumed.** `migration_chain!` drives runtime migration; used by benches (`b0_alloc_migration`) and the weather example. Works `no_std` (design 039). | Keep. Stretch: auto-apply the chain on the deserialize/`link_from` path so connectors migrate old wire data without a manual call. | +| `Observable` | **Barely consumed.** Signal extraction + `format_log()` only; sole consumer is the opt-in `log_tap()`, wired by hand. **Not** connected to metrics. | `.observe(node_id)` installs the log tap (§3.2). README reworded (below). Statistics come from the separate `observability` feature, not this trait. | +| `Linkable` | **Convention only.** No engine bound anywhere — examples call `T::to_bytes()`/`from_bytes()` by hand inside `with_serializer`/`with_deserializer` closures. | `.link_from`/`.link_to` default their (de)serializer to `Linkable::{from_bytes,to_bytes}` when `T: Linkable`, so the connector consumes the trait directly. *(Own follow-up — not in this doc's PR.)* | +| `Simulatable` | **Unconsumed.** Called by hand in example producer loops only. | `.simulate(cfg, rng)` installs the source loop (§3.1); `.source_or_simulate(mode, …)` picks real-vs-sim for the single producer slot (§3.3). *Stretch:* runtime sim↔real failover (live switch on sensor staleness) — its own doc. | +| `Settable` | **Unconsumed.** `set(value, ts)`; examples `impl Settable`, nothing calls it outside codegen templates. | Back a typed `.set(value)` producer/one-shot helper. *(Own follow-up — noted for completeness; not in scope here.)* | + +This doc delivers the `Observable` and `Simulatable` rows. `Linkable`/`Settable` targets are recorded so the map is complete, but are separate follow-ups (each needs its own caller-in-the-same-PR). + +### 4.1 The `Observable` README claim — reword (decided) + +The capability table ([`README.md:137`](../../README.md)) currently reads `Observable | Automatic per-record metrics`. This is wrong on two counts: + +1. **`Observable` is about log formatting, not metrics.** Its surface is `signal()`, `ICON`, `UNIT`, `format_log()` ([`aimdb-data-contracts/src/lib.rs:192-227`](../../aimdb-data-contracts/src/lib.rs)) — domain-signal extraction + human-readable output. +2. **The per-record metrics that *do* exist are independent of `Observable`.** `BufferMetricsSnapshot` (produced/consumed/dropped/occupancy) is collected by the buffer for **every** record under the `observability` feature ([`aimdb-core/src/buffer/traits.rs:278`](../../aimdb-core/src/buffer/traits.rs)), regardless of whether the type is `Observable`. + +**Decision (2026-07-06): reword to match reality.** `Observable` lets the user *tap and log* the record; the `observability` **feature flag** is what provides statistics — two separate things the README wrongly fused. New row: + +| Trait | What it unlocks | Runtimes | +|---|---|---| +| `Observable` | Built-in logging tap (`.observe()`) over an extracted signal | std, no_std | + +The README's separate feature-flag section should carry the "per-record buffer statistics" story where it belongs (`observability`). `.observe()`'s `.with_name("observe")` (§3.2) still gives the tap a real identity in the stage-profiling snapshot — an honest, modest link, no overclaim. A signal-metrics sink (min/max/mean of `signal()` into the snapshot) remains possible later but is out of scope and would get its own design doc. + +## 5. Decisions (resolved 2026-07-06) + +### 5.1 RNG source for `.simulate()` — **caller-supplied** + +`Simulatable::simulate` is generic over the RNG; `.simulate()` must *pick* one, and no_std/Embassy has no default entropy source. **Decision: the caller supplies the RNG** — `.simulate(cfg, rng)` takes an owned `R: rand::Rng + Send + 'static` (§3.1). std callers pass `StdRng::from_rng(&mut rand::rng())` (already in the demo); embedded callers pass a seeded PRNG; tests pass a fixed seed. This keeps the method a single signature that works on every runtime and drops the need for `rand`'s `std_rng` feature in the ext path. + +### 5.2 `Observable` README claim — **reword** (see §4.1) + +Reworded to "built-in logging tap over an extracted signal"; statistics stay attributed to the `observability` feature. Signal-metrics sink deferred to a possible future doc. + +### 5.3 New coupling: `simulatable` now pulls `aimdb-core` + +`observable` already depends on `aimdb-core` (for `log_tap`). Adding `SimulatableRegistrarExt` means **`simulatable` gains the same `aimdb-core` dependency** (for `RecordRegistrar`/`Producer`/`RuntimeContext`). This is the intended shape of the D1/D12 fix — "turns `aimdb-data-contracts` into a consumed layer" — but it means selecting `simulatable` on a flash-constrained target now compiles core's `alloc` surface. `aimdb-core` is pulled with `default-features = false, features = ["alloc"]` (matching the existing `observable` wiring), so `serde_json`/AimX stay out. If someone ever needs the bare trait + `SimulationConfig` without core, split via a `simulatable-integration` sub-feature — not proposed now (no such caller). + +### 5.4 Sim-to-real is build-time selection, not runtime coexistence — **decided** + +"Real feed in production, `Simulatable` in dev / CI / demo / absent hardware" is delivered as `source_or_simulate(mode, cfg, rng, real)` (§3.3): a registrar-time branch over the single producer slot. **Rationale:** a record has exactly one producer ([`typed_record.rs:570`](../../aimdb-core/src/typed_record.rs)), so sim and real can never both write — attempting to install both is a writer-exclusivity violation `build()` already rejects, and would contradict single-writer-per-key. Selecting one at registration is zero-cost and invariant-safe, and it reuses the general-purpose `.source()` installer unchanged (no new stage type). **Live** failover (sim↔real at runtime, e.g. on sensor staleness) would instead require a *single* source future that multiplexes real vs. `T::simulate(...)` internally; it is deferred to its own doc (§7), not folded into this selector. + +## 6. Migration & examples + +- **weather-station-beta / gamma:** replace the manual producer loops with `reg.simulate(cfg, rng)` inside `configure::()`, passing the `StdRng` they already build. Deletes the `tokio::spawn`, the `db.producer::()` handles, and the prev-value bookkeeping. +- **weather-hub:** replace `.tap(move |ctx, c| log_tap(ctx, c, key.as_str()))` with `.observe(node_id)`. +- **`source_or_simulate` caller:** the demo stations are simulation-only (they publish synthetic data; there is no real sensor to select), so the combinator's in-PR caller is an `aimdb-data-contracts` integration test — a fixed-seed RNG plus a stub real source, asserting each `SourceMode` installs the expected single producer and that `build()` reports **no** writer conflict. This satisfies the D1 caller-in-PR rule without fabricating hardware; a station that toggles a *genuine* feed to sim is future work (§7). +- These examples/tests become the **caller** that proves the API — satisfying the D1 rule ("no public API without a caller in the same PR"). +- `CHANGELOG.md`: entry under `aimdb-data-contracts` ("added `SimulatableRegistrarExt` (`simulate` + `source_or_simulate`) / `ObservableRegistrarExt`") and README. + +## 7. Sequencing + +1. Land `ObservableRegistrarExt::observe()` + the README reword (self-contained). +2. Land `SimulatableRegistrarExt::simulate(cfg, rng)`. +3. Land `SimulatableRegistrarExt::source_or_simulate(mode, cfg, rng, real)` on the same trait, with its integration-test caller (§3.3, §6). +4. Convert the weather-mesh examples to the new methods in the same commit as the API they exercise (the caller requirement). +5. Update the README capability table with the current→target-accurate rows (§4). + +Per [[feedback_no_stacked_prs_for_design_docs]]: land as separate commits on one branch, one per work item — not a stacked-PR chain. + +**Out of scope / recorded for later** (each needs its own caller-in-PR): the `Linkable` and `Settable` targets from §4; the **runtime sim↔real failover** multiplexer (§3.3 note) — a single source future that switches live between the real feed and `T::simulate(...)` on sensor staleness, kept single-writer-safe. From 33e96924a0fd0f9619594e1db034cc2a0b1400c2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Mon, 6 Jul 2026 19:04:34 +0000 Subject: [PATCH 02/31] Implement code changes to enhance functionality and improve performance --- docs/design/041-data-contracts-integration.md | 434 +++++++++++------- 1 file changed, 269 insertions(+), 165 deletions(-) diff --git a/docs/design/041-data-contracts-integration.md b/docs/design/041-data-contracts-integration.md index 021a97f..4659b0f 100644 --- a/docs/design/041-data-contracts-integration.md +++ b/docs/design/041-data-contracts-integration.md @@ -1,62 +1,111 @@ -# 041 — Data-Contracts Integration: make the capability story real (D1/D12) +# 041 — Data contracts as first-class capabilities (v2) -**Status:** Proposed 2026-07-06. Follow-up design doc for the design-038 **D1/D12** decision (tracking issue [#161](https://github.com/aimdb-dev/aimdb/issues/161)). Companion to the claims-assessment track (039, 040). -**Scope:** `aimdb-data-contracts` (add two registrar extension traits — one of which also gains a build-time sim-to-real source selector), the README capability table, and the weather-mesh example (adopt the new methods). **No changes to the contract trait definitions themselves** — `Simulatable`, `Observable`, `Streamable`, `Linkable`, `Migratable` keep their current shape. -**Goal:** Turn `aimdb-data-contracts` from a *side-channel* (traits nothing consumes) into a *consumed layer*, so the README's "capability traits" story is backed by code the engine actually runs — using the exact mechanism a third party would use. +**Status:** Proposed 2026-07-06, **v2**. Supersedes v1 (same date, same number). v1's two anchors — "no changes to the contract trait definitions" and the `SourceMode` build-time sim selector (§3.3 of v1) — were overturned by owner steer on 2026-07-06: simulation must be a **compile-time** decision (no sim code in production binaries, at all), `Observable` must earn the name (metrics, not logging), and `Streamable`/`Linkable`/`Settable` need to become intuitive. Follow-up to design-038 **D1/D12** (tracking issue [#161](https://github.com/aimdb-dev/aimdb/issues/161)). + +**Scope:** `aimdb-data-contracts` (trait reshapes + three registrar ext traits; `Simulatable` stays here behind its feature), `aimdb-sync` (consumes `Settable`), one small `aimdb-core` observability surface (signal gauges), the README capability table, the weather-mesh example, and a CI guard. + +**Goal:** every capability trait is a promise the engine keeps — implemented ⇒ one specific verb becomes available, and that verb is consumed by real code in the same PR (D1 rule). --- ## 1. Problem -Design 038 findings **D1** ("building abstractions for consumers that never arrived") and **D12** ("marketing vocabulary in API design") both landed on the same subject: the data-contract capability traits are advertised as a headline feature, but core consumes none of them, and two of them — `Simulatable` and `Observable` — have **no non-example consumer at all**. +Design 038 D1/D12: the capability traits are advertised as a headline feature, but core consumes almost none of them. v1 of this doc fixed that by *adding consumers* without touching the traits. Three problems survived v1: -Concretely, today: +1. **Simulation was still a build-time value, not a compile-time fact.** v1 §3.3's `source_or_simulate(mode, cfg, rng, real)` selected the producer with a runtime-valued `SourceMode` — which means **both arms compile into the binary**. Even env-var selection ships the sim loop, every `T::simulate` impl, `SimulationConfig`, and `rand` in the production image. The owner requirement is stronger: production binaries must contain **zero** simulation code, verifiable from the dependency graph. +2. **`Observable` remained logging.** v1 resolved the false README claim ("automatic per-record metrics") by *rewording downward* to "built-in logging tap". The right fix is upward: make the trait actually feed metrics, then the strong claim is true. +3. **`Streamable`/`Linkable`/`Settable` are unintuitive** because a user cannot answer *"what does implementing this let me write?"* `Linkable` exists but every example still hand-writes `with_deserializer` closures; `Settable` was built for `aimdb-sync` but sync never references it (`SyncProducer::set` takes a full `T` — [`producer.rs:136`](../../aimdb-sync/src/producer.rs)); `Streamable` works but nothing says its verb is the ws-connector's `.register::()`. -- `Simulatable::simulate()` is only ever called **by hand**, inside example producer loops. Each weather station repeats ~30 lines of "make an RNG → loop → `T::simulate(...)` → `produce` → sleep" boilerplate ([`weather-station-beta/src/main.rs:132-189`](../../examples/weather-mesh-demo/weather-station-beta/src/main.rs), [`weather-station-gamma/src/main.rs:80-125`](../../examples/weather-mesh-demo/weather-station-gamma/src/main.rs)). -- `Observable`'s only non-example consumer is the opt-in `log_tap()` helper, which the user must wire manually as a tap ([`weather-hub/src/main.rs:36`](../../examples/weather-mesh-demo/weather-hub/src/main.rs)). -- `RecordRegistrar::source()` / `.tap()` are fully generic with **no trait bounds** — and they must stay that way (they are general-purpose stage installers, not contract-specific). +## 2. Organizing principle: one verb per contract, tiered by deployment role -The 2026-07-01 owner decision (recorded in 038 §4 D1/D12) resolved the tension **by completion, not deletion**: data contracts are core to AimDB's identity, so the fix is to *add the missing consumer*, not remove the API. This doc specifies that consumer. +A capability trait is a compile-time promise about a schema type. Each promise unlocks **exactly one verb**, and each contract belongs to a **tier** that states whether it may exist in a production binary: -## 2. The model to copy: `.persist()` +| Contract | Implement when… | Verb it unlocks | Consumer | Tier | +|---|---|---|---|---| +| `SchemaType` | always (identity) | `configure::(key, …)` | core | identity | +| `Linkable` | the type crosses a per-URL byte boundary (MQTT/KNX/serial/UDS) | `.linked_from(url)` / `.linked_to(url)` (§3.3) | connectors | wire (prod) | +| `Streamable` | the type streams as schema-named JSON (browser/WASM) | ws-connector `.register::()` | [`server/registry.rs:33`](../../aimdb-websocket-connector/src/server/registry.rs) | wire (prod) | +| `Migratable` | the schema evolved across versions | `migration_chain!` | core runtime migration (design 039) | wire (prod) | +| `Settable` | callers outside the AimDB thread set the record from a primitive | `SyncProducer::set_value(v)` (§3.4) | `aimdb-sync` | wire (prod) | +| `Observable` | the type carries a domain signal worth watching | `.observe()` → live signal metrics (§3.2) | introspection surface | introspection (prod, optional) | +| `Simulatable` | the type can generate realistic synthetic data | `.simulate(profile, rng)` (§3.1) | `simulatable` feature ext | **dev-only — never in prod** | -`aimdb-persistence` already demonstrates the pattern this doc generalizes. `RecordRegistrarPersistExt::persist()` ([`aimdb-persistence/src/ext.rs`](../../aimdb-persistence/src/ext.rs)) is an **extension trait on `RecordRegistrar`** that installs a `.tap()` — the engine drives it exactly like any other tap, and `aimdb-core` never learns persistence exists: +Mechanism (unchanged from v1, copied from `.persist()` — [`aimdb-persistence/src/ext.rs`](../../aimdb-persistence/src/ext.rs)): extension traits over `RecordRegistrar`/`SyncProducer` in the crate that owns the contract, installing plain `.source()`/`.tap()`/`.link_*()` stages. `.source()`/`.tap()` stay unbounded; `aimdb-core` never learns the contracts exist; dependency direction is always *contracts → core*, never the reverse. -```rust -// aimdb-persistence/src/ext.rs (abbreviated) -impl<'a, T> RecordRegistrarPersistExt<'a, T> for RecordRegistrar<'a, T> { - fn persist(&mut self, record_name: impl Into) -> &mut RecordRegistrar<'a, T> { - // ... pull backend from Extensions TypeMap ... - self.tap(move |_ctx, consumer| async move { /* subscribe → serialize → store */ }) - } -} -``` +The tier column is the second half of the fix: *wire* contracts ship in production, *introspection* is prod-optional, and the *dev* tier must be **structurally excludable** — which drives §3.1's crate split. -The two properties we copy: +## 3. Design -1. **Extension trait on `RecordRegistrar`**, not a bound on `.source()`/`.tap()`. The generic stage installers stay generic; the contract-specific ergonomics live in the contract crate. -2. **The engine consumes the trait through the same mechanism third parties would use** — `.source()`/`.tap()` are public API; a `simulate()` that calls `.source()` is doing nothing a downstream crate couldn't do itself. +### 3.1 `Simulatable` → compile-time only, behind the `simulatable` feature -**One difference from `.persist()`:** persistence needed builder-side state (a backend registered via `.with_persistence()`, retrieved from the `Extensions` TypeMap). The contract extensions need **no builder-side state** — `simulate(cfg)` receives its config inline and `observe(node_id)` receives its label inline. So there is **no `.with_*()` builder extension** in this design; both methods are pure registrar-time sugar. +#### 3.1.1 Stays in `aimdb-data-contracts`, as the dev-tier feature -## 3. Design +`Simulatable` remains in the contracts crate behind the existing `simulatable` feature. (A separate `aimdb-simulation` crate was considered for the structural guarantee and **rejected 2026-07-06** — owner call: not worth another crate in the monorepo.) The compile-time guarantee never depended on a crate boundary; it rests on three enforceable properties: -Two extension traits added to `aimdb-data-contracts`, each gated behind the feature that already carries its contract trait. +1. **`simulatable` is not (and never becomes) a default feature.** Already true today ([`Cargo.toml:14`](../../aimdb-data-contracts/Cargo.toml)); CI asserts it stays that way (§3.1.5). +2. **Production binaries resolve features per-package.** The workspace uses `resolver = "2"` ([`Cargo.toml:45`](../../Cargo.toml)), so `cargo build -p --release` unifies features only over that binary's own graph — a sim-enabled example elsewhere in the workspace cannot leak the feature into the release artifact. Corollary, worth one sentence in CONTRIBUTING: release artifacts are built per-package, never lifted out of a `--workspace` build (where unification does cross targets). +3. **`rand` is the tracer.** `Simulatable::simulate` makes `rand` reachable iff the feature is on, so an inverse-dependency check on `rand` proves sim absence from a production graph (§3.1.5). -### 3.1 `SimulatableRegistrarExt::simulate(cfg, rng)` (feature `simulatable`) +Feature wiring: `simulatable = ["rand", "aimdb-core"]` — the ext trait needs `RecordRegistrar`/`Producer`/`RuntimeContext`, the same core coupling `observable` already has and `linkable` gains in §3.3. `rand` stays `default-features = false` and **loses `std_rng`** (the RNG is caller-supplied, carried over from v1 §5.1). The feature is `no_std`-compatible: the "develop against sim, flash against silicon" story is embedded-first. -Installs the generate → produce → sleep source loop that examples write by hand. **The caller supplies the RNG** (decision §5.1): `Simulatable::simulate` is generic over the RNG, and no_std/Embassy has no default entropy source, so `.simulate()` takes an owned `R: rand::Rng + Send + 'static` that moves into the source future. +#### 3.1.2 Trait reshape: domain params, no runtime gate + +The current `SimulationParams` (base/variation/trend/step — [`simulatable.rs:53`](../../aimdb-data-contracts/src/simulatable.rs)) is a random-walk config pretending to be universal, and `SimulationConfig.enabled` is a runtime gate that contradicts the compile-time stance. Both go: + +```rust +// aimdb-data-contracts/src/simulatable.rs (feature = "simulatable") + +/// Dev-only capability: generate realistic synthetic data. +pub trait Simulatable: SchemaType { + /// Type-specific generation parameters (a temperature defines walk bounds, + /// a GPS track defines waypoints, …). + type Params: Clone + Send + Sync + Default + 'static; + + /// Generate the next sample. `previous` enables walks/trends; + /// `timestamp_ms` is Unix millis supplied by the driving loop. + fn simulate( + params: &Self::Params, + previous: Option<&Self>, + rng: &mut R, + timestamp_ms: u64, + ) -> Self; +} + +/// Loop policy + generation params for one record. +#[derive(Clone, Debug, Default)] +pub struct SimProfile

{ + /// Interval between samples (milliseconds). + pub interval_ms: u64, + pub params: P, +} + +/// Off-the-shelf `Params` for scalar random walks — what today's +/// `SimulationParams` actually was. Existing impls migrate by setting +/// `type Params = RandomWalkParams`. +#[derive(Clone, Debug, serde::Serialize, serde::Deserialize)] +pub struct RandomWalkParams { + pub base: f64, + pub variation: f64, + pub trend: f64, + pub step: f64, +} +``` + +There is **no `enabled` field**. "Sim but off" is not a state: production excludes the crate (§3.1.4), and a std dev binary that wants sim conditionally simply branches before calling `.simulate()`. + +#### 3.1.3 `SimulatableRegistrarExt::simulate(profile, rng)` + +The v1 §3.1 loop, minus the `enabled` branch, over the reshaped types: ```rust -// aimdb-data-contracts/src/simulatable.rs (new ext, feature = "simulatable") pub trait SimulatableRegistrarExt<'a, T> where T: Simulatable + Send + Sync + Clone + 'static, { - /// Install a source that emits `T::simulate(...)` every `cfg.interval_ms`, - /// driving the supplied RNG. Works on any runtime — the caller chooses the - /// RNG (OS entropy on std, a seeded PRNG on no_std). - fn simulate(&mut self, cfg: SimulationConfig, rng: R) -> &mut RecordRegistrar<'a, T> + /// Install a source that emits `T::simulate(...)` every `interval_ms`, + /// driving the caller-supplied RNG (OS entropy on std, seeded PRNG on + /// no_std, fixed seed in tests). + fn simulate(&mut self, profile: SimProfile, rng: R) -> &mut RecordRegistrar<'a, T> where R: rand::Rng + Send + 'static; } @@ -65,16 +114,10 @@ impl<'a, T> SimulatableRegistrarExt<'a, T> for RecordRegistrar<'a, T> where T: Simulatable + Send + Sync + Clone + 'static, { - fn simulate(&mut self, cfg: SimulationConfig, mut rng: R) -> &mut RecordRegistrar<'a, T> + fn simulate(&mut self, profile: SimProfile, mut rng: R) -> &mut RecordRegistrar<'a, T> where R: rand::Rng + Send + 'static, { - if !cfg.enabled { - // Simulation gated off by config → install no source. The record - // can still receive a producer elsewhere. (Mirrors `.persist()`'s - // no-op-when-unconfigured precedent.) - return self; - } self.source(move |ctx, producer| async move { let mut prev: Option = None; loop { @@ -83,193 +126,254 @@ where .unix_time() .map(|(s, ns)| s.saturating_mul(1000) + (ns / 1_000_000) as u64) .unwrap_or(0); - let next = T::simulate(&cfg, prev.as_ref(), &mut rng, now_ms); + let next = T::simulate(&profile.params, prev.as_ref(), &mut rng, now_ms); producer.produce(next.clone()); prev = Some(next); - ctx.time().sleep_millis(cfg.interval_ms.max(1)).await; + ctx.time().sleep_millis(profile.interval_ms.max(1)).await; } }) + .with_name("simulate") } } ``` -Notes: -- **Caller-supplied RNG** keeps `.simulate()` runtime-agnostic and drops the hard dependency on `rand`'s `std_rng` feature: std callers pass `StdRng::from_rng(&mut rand::rng())` (what the demo already constructs), an Embassy target passes a seeded PRNG. Deterministic tests get a fixed seed for free. -- Uses `RuntimeContext::time()` for both the timestamp (`unix_time()`) and the delay (`sleep_millis()`) — so the loop is runtime-neutral (works on any adapter), exactly as `.persist()`'s cleanup loop is. -- `.simulate()` installs a **source**, so it participates in the existing writer-exclusivity rule (a simulated record cannot also be a `.transform()`/`.link_from()` source — the conflict is reported from `build()`, unchanged). -- Removes ~30 lines of boilerplate per station and deletes the manual `db.producer::(...)` + `tokio::spawn` dance. +Runtime-neutral (`ctx.time()` for clock and delay, like `.persist()`'s cleanup loop), installs a **source** so writer-exclusivity is enforced by `build()` unchanged ([`typed_record.rs:570`](../../aimdb-core/src/typed_record.rs)). -### 3.2 `ObservableRegistrarExt::observe(node_id)` (feature `observable`) +#### 3.1.4 Sim-to-real is a `#[cfg]`, not an API — `SourceMode` is deleted -Installs `log_tap` as a tap — the one-liner that replaces the manual `.tap(|ctx, c| log_tap(ctx, c, id))`. +v1 §3.3 (`SourceMode` + `source_or_simulate`) is **removed without replacement API**. The sim-to-real selection is the app's Cargo feature, and this doc makes that pattern canonical instead of wrapping it: + +```toml +# app Cargo.toml +[features] +sim = ["aimdb-data-contracts/simulatable", "weather-mesh-common/sim"] +``` ```rust -// aimdb-data-contracts/src/observable.rs (new ext, feature = "observable") +// app wiring — the record configuration is identical either way +builder.configure::(KEY, |reg| { + #[cfg(feature = "sim")] + reg.simulate(profile, rng); + #[cfg(not(feature = "sim"))] + reg.source(read_hardware); + + reg.buffer_cfg(BufferCfg::SingleLatest); +}); +``` + +The app's contracts crate gates its impls the same way (`#[cfg(feature = "sim")] impl Simulatable for Temperature { … }`). With `sim` off there is nothing to audit: no impls, no `SimProfile`, no `rand` anywhere in the binary's graph. Exactly one producer is installed on either path, so single-writer-per-key holds by construction — same argument as v1, now with the selection resolved by the compiler instead of a value. + +#### 3.1.5 CI guard: prove production is sim-free + +A CI step (Makefile target `check-no-sim`) builds the production configuration of each example that has a `sim` feature and asserts the exclusion from the resolved graph — `cargo tree -p -e normal -i rand` must report no matching package when `sim` is off (`rand` is the tracer, §3.1.1) — and additionally asserts that `simulatable` never appears in `aimdb-data-contracts`' default features. This turns "at all cost" from a convention into a gate. + +### 3.2 `Observable` → the signal, not the log line + +#### 3.2.1 Trait reshape: keep the kernel, drop the cosmetics + +The trait's valuable kernel is the numeric projection (`Signal`, `signal()`, `UNIT`); `ICON` and `format_log()` are presentation and made the whole trait read as a log formatter ([`lib.rs:192-227`](../../aimdb-data-contracts/src/lib.rs)). Reshape: + +```rust +pub trait Observable: SchemaType { + /// Numeric type of the domain signal. + type Signal: PartialOrd + Copy; + + /// What the signal means, for metrics/UI labels (e.g. "celsius"). + /// Defaults to the schema name. + const SIGNAL: &'static str = Self::NAME; + + /// Unit label (e.g. "°C", "%", "hPa"). + const UNIT: &'static str = ""; + + /// Extract the signal from this instance. + fn signal(&self) -> Self::Signal; +} +``` + +`ICON` and `format_log()` are deleted from the trait. The logging tap (below) formats from `Debug` + `UNIT`; demo emoji live in the demo. + +#### 3.2.2 `.observe()` installs a signal-metrics tap + +`.observe()` folds `signal()` into per-record **signal stats** (last/min/max/count/mean as `f64`) that surface on the existing introspection paths. Mechanism — a small, feature-honest core addition mirroring `with_name` ([`typed_api.rs:433`](../../aimdb-core/src/typed_api.rs)): + +- `aimdb-core` gains `RecordRegistrar::signal_gauge(name: &'static str, unit: &'static str) -> SignalGaugeHandle`. Under the `observability` feature it registers atomic `SignalStats` on the record's profiling state and returns a live handle; without the feature it is always-available but returns an inert handle (the `with_name` precedent — callers never `cfg` on core's features). +- Exposure: `RecordMetadata` ([`builder.rs:191`](../../aimdb-core/src/builder.rs) `list_records`) gains an optional signal-stats field, and the AimX `record.get` response carries it — so the signal shows up in `record.list`/`record.get`, the MCP tools built on them, and stage-profiling output. *Implement `Observable`, call `.observe()`, and your domain signal is live on every introspection surface.* That is the README claim, and after this change it is true. + +```rust +// aimdb-data-contracts/src/observable.rs (feature = "observable") pub trait ObservableRegistrarExt<'a, T> where - T: Observable + Send + Sync + Clone + core::fmt::Debug + 'static, + T: Observable + Send + Sync + Clone + 'static, + T::Signal: Into, { - /// Install a logging tap that prints `T::format_log(node_id)` per value. - fn observe(&mut self, node_id: &'static str) -> &mut RecordRegistrar<'a, T>; + /// Feed `T::signal()` into the record's signal gauge (last/min/max/mean), + /// visible via record.list / record.get / stage profiling. + fn observe(&mut self) -> &mut RecordRegistrar<'a, T>; } impl<'a, T> ObservableRegistrarExt<'a, T> for RecordRegistrar<'a, T> where - T: Observable + Send + Sync + Clone + core::fmt::Debug + 'static, + T: Observable + Send + Sync + Clone + 'static, + T::Signal: Into, { - fn observe(&mut self, node_id: &'static str) -> &mut RecordRegistrar<'a, T> { - self.tap(move |ctx, consumer| log_tap(ctx, consumer, node_id)) - .with_name("observe") // surfaces the tap in stage profiling (no-op w/o `observability`) + fn observe(&mut self) -> &mut RecordRegistrar<'a, T> { + let gauge = self.signal_gauge(T::SIGNAL, T::UNIT); + self.tap(move |_ctx, consumer| async move { + let mut reader = consumer.subscribe(); + while let Ok(value) = reader.recv().await { + gauge.update(value.signal().into()); + } + }) + .with_name("observe") } } ``` Notes: -- `log_tap` is unchanged and stays exported for callers who want the raw form. -- `.with_name("observe")` gives the tap a stable identity in the stage-profiling snapshot when the `observability` feature is on — a small, honest link to the observability subsystem (see §4). +- `T::Signal: Into` is bounded **at the consumption site**, not in the trait — `f32`/`i32`/`u32` signals qualify; a type with an exotic signal can still implement `Observable` and write its own tap. +- `.observe()` takes **no `node_id`** — the record key already identifies the gauge; `node_id` was a logging concern. +- Landing order inside the work item: the core `signal_gauge` surface and the ext land together; if the core PR needs to decouple, an interim `Extensions`-TypeMap registry owned by the contracts crate is acceptable, but the gauge API is the target and the README claim waits for it. -### 3.3 `SimulatableRegistrarExt::source_or_simulate(mode, cfg, rng, real)` (feature `simulatable`) +#### 3.2.3 Logging stays, honestly named -A record has **exactly one producer**: `set_producer` rejects a second `.source()`, and a source is mutually exclusive with `.transform()`/`.link_from()` — validated once in `build()`, not panicked ([`typed_record.rs:570`](../../aimdb-core/src/typed_record.rs)). Both `.simulate()` (§3.1) and a real `.source()` install *that same slot*. So the sim-to-real workflow — "run against a real sensor in production, against `Simulatable` in dev / CI / demo / when the hardware is absent" — is **not** a coexistence problem. It is a **build-time source selection**: choose which producer fills the one slot. This keeps single-writer-per-key true *by construction* (only one branch ever installs a producer, so `build()` never sees a conflict) and costs nothing at runtime. +`log_tap` remains for humans watching a console, exposed as `.log(node_id)` on the same ext trait (formats `[node_id] key: {signal}{UNIT} {value:?}` from `Debug` — no trait items needed). The README row for `Observable` says metrics; logging is a bullet under it, not the definition. -The combinator adds a second method to the §3.1 trait, wrapping the branch a caller would otherwise write by hand: +### 3.3 `Linkable` → the default codec that `.link_*` actually consumes -```rust -// aimdb-data-contracts/src/simulatable.rs (feature = "simulatable") +Implementing `Linkable` today changes nothing — every example still hand-wires closures ([`embassy-mqtt-connector-demo/src/main.rs:350-393`](../../examples/embassy-mqtt-connector-demo/src/main.rs)). Fix: one-line link verbs, ext trait in the contracts crate (dependency direction preserved): -/// Which producer fills a record's single source slot. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -pub enum SourceMode { - /// Install the real `.source()` closure (hardware / network feed). - Real, - /// Install the `Simulatable` generate → produce → sleep loop (§3.1). - Simulated, -} - -pub trait SimulatableRegistrarExt<'a, T> +```rust +// aimdb-data-contracts/src/linkable.rs (feature = "linkable") +pub trait LinkableRegistrarExt<'a, T> where - T: Simulatable + Send + Sync + Clone + 'static, + T: Linkable + Send + Sync + Clone + core::fmt::Debug + 'static, { - // fn simulate(&mut self, cfg: SimulationConfig, rng: R) -> ... // §3.1 - - /// Install `real` when `mode == Real`, else the §3.1 simulation loop. - /// Exactly one producer is installed either way, so writer-exclusivity - /// holds and `build()` never reports a conflict. - fn source_or_simulate( - &mut self, - mode: SourceMode, - cfg: SimulationConfig, - rng: R, - real: F, - ) -> &mut RecordRegistrar<'a, T> - where - F: FnOnce(RuntimeContext, Producer) -> Fut + Send + 'static, - Fut: core::future::Future + Send + 'static, - R: rand::Rng + Send + 'static; + /// `.link_from(url)` with the codec defaulted to `T::from_bytes`. + fn linked_from(&mut self, url: &str) -> &mut RecordRegistrar<'a, T>; + /// `.link_to(url)` with the codec defaulted to `T::to_bytes`. + fn linked_to(&mut self, url: &str) -> &mut RecordRegistrar<'a, T>; } -impl<'a, T> SimulatableRegistrarExt<'a, T> for RecordRegistrar<'a, T> +impl<'a, T> LinkableRegistrarExt<'a, T> for RecordRegistrar<'a, T> where - T: Simulatable + Send + Sync + Clone + 'static, + T: Linkable + Send + Sync + Clone + core::fmt::Debug + 'static, { - fn source_or_simulate( - &mut self, - mode: SourceMode, - cfg: SimulationConfig, - rng: R, - real: F, - ) -> &mut RecordRegistrar<'a, T> - where - F: FnOnce(RuntimeContext, Producer) -> Fut + Send + 'static, - Fut: core::future::Future + Send + 'static, - R: rand::Rng + Send + 'static, - { - match mode { - SourceMode::Real => self.source(real), // the one general-purpose stage installer - SourceMode::Simulated => self.simulate(cfg, rng), // §3.1 - } + fn linked_from(&mut self, url: &str) -> &mut RecordRegistrar<'a, T> { + self.link_from(url) + .with_deserializer(|_ctx, bytes| T::from_bytes(bytes)) + .finish() + } + + fn linked_to(&mut self, url: &str) -> &mut RecordRegistrar<'a, T> { + self.link_to(url) + .with_serializer(|_ctx, value| { + value.to_bytes().map_err(|_| SerializeError::InvalidData) + }) + .finish() } } ``` Notes: -- **`mode` is caller-derived, exactly like `rng` (§5.1).** The ext stays runtime-agnostic — it does not read env or entropy itself. A std station derives it from an env var / config field (`AIMDB_SOURCE=sim|real`); a no_std target selects with a **Cargo feature** so the real-hardware source only *compiles* on the board (`#[cfg(feature = "hw")]`) and simulation is the fall-through. The record wiring is identical both ways — this is the embedded "develop against sim, flash against silicon" story made first-class, not a demo detail. -- **`cfg.enabled` stays sim's internal gate, not the selector.** `mode` picks the branch; within `Simulated`, `enabled: false` still installs no source (§3.1). `Simulated` + `enabled: false` is therefore the explicit "leave this record writer-less" case — identical to calling `.simulate()` with a disabled config today. -- **Both `real` and `rng` are owned and moved into the chosen branch;** the unused one is dropped at registration. No boxing of the unused arm, no runtime flag — the selection is fully resolved before `build()`. -- **`SourceMode` is re-exported next to `SimulationConfig`/`SimulationParams`** from `aimdb-data-contracts` (lib.rs `pub use`). -- **Runtime failover is out of scope.** Falling over to simulated values *live* when a sensor drops, then back, cannot be two sources (that breaks writer-exclusivity); it needs a single source future that internally multiplexes real vs. `T::simulate(...)`. Recorded as a follow-up in §4 / §7. +- **Inbound matches exactly** (`with_deserializer` takes `Result` — [`typed_api.rs:872`](../../aimdb-core/src/typed_api.rs)). **Outbound needs one lossy mapping**: `with_serializer` returns `Result, SerializeError>` ([`typed_api.rs:661`](../../aimdb-core/src/typed_api.rs)), so the `String` detail is dropped to `SerializeError::InvalidData`. Aligning `Linkable`'s error type with the connector layer (a shared `CodecError` instead of `String` — which also removes an alloc on embedded) is a recorded follow-up (§7); it touches core connector signatures and doesn't block this verb. +- **The raw builders remain the escape hatch** for per-link options (`with_config`, QoS ext traits, topic providers/resolvers). `.linked_from`/`.linked_to` are the 80% path. +- **JSON boilerplate gets a derive:** `#[derive(Linkable)]` in `aimdb-derive` emitting `serde_json::to_vec`/`from_slice` (JSON is the default format; binary formats implement by hand, as the KNX DPT codecs rightly do). D1 caller: the derive replaces at least one hand-written JSON impl in the same commit. +- **Coupling note (the v1 §5.3 move, now for `linkable`):** the `linkable` feature currently has no `aimdb-core` dependency; the ext adds one (`default-features = false, features = ["alloc"]`, same wiring as `observable`). +- **Three serialization stories, stated once** so users stop guessing: -### 3.4 What stays out + | Boundary | Mechanism | Contract | + |---|---|---| + | Schema-named JSON streaming (browser/WASM) | ws registry keyed by `SchemaType::NAME` | `Streamable` | + | Per-URL opaque payloads (MQTT/KNX/serial/UDS) | fused per-link codec | `Linkable` | + | AimX `record.get`/`record.set`/`subscribe` JSON | blanket `RemoteSerialize` ([`codec.rs:33`](../../aimdb-core/src/codec.rs)), automatic for serde types | none needed | -- **No trait bounds added to `.source()`/`.tap()`/`.transform()`.** They remain general-purpose. -- **No new dependency in `aimdb-core`.** The direction of the dependency is `aimdb-data-contracts → aimdb-core`, never the reverse — same as persistence. -- **No change to the five contract trait definitions.** Per the issue and the user's steer: keep the traits as they are; only add the integration layer. -- **No runtime sim↔real hot-swap.** `source_or_simulate` selects once, at registration; a live failover multiplexer is a separate design (§7). +### 3.4 `Settable` → consumed by `aimdb-sync` -## 4. Every capability trait: current state → target +`Settable` was built for the sync bridge, but `aimdb-sync` never references it: `SyncProducer::set(value: T)` takes a fully constructed record, so every outside-the-thread caller hand-assembles the struct. The verb it was named for is the missing ten lines — and note this is distinct from AimX's existing `record.set {name, value}` (full JSON value through `JsonCodec`); `Settable` is **set-by-primitive**: -The issue asks for the `Observable` claim to be made honest. Rather than fix that row in isolation, we set a **current state → target** entry for *every* capability trait — this is the map the README table and future integration work should track against. The through-line: a trait's "target" is *the engine consuming it*, and the README should describe the **current** state, not the target. - -| Trait | Current state (verified) | Target | -|---|---|---| -| `Streamable` | **Consumed.** Bound by `aimdb-websocket-connector`'s `StreamableRegistry::register()` ([`server/registry.rs:33`](../../aimdb-websocket-connector/src/server/registry.rs), `server/builder.rs:250`) as the schema-name gate for browser streaming. Marker trait (no methods). | Keep as the "may cross a serialization boundary" marker + registry key. Already real — README row is accurate. | -| `Migratable` | **Consumed.** `migration_chain!` drives runtime migration; used by benches (`b0_alloc_migration`) and the weather example. Works `no_std` (design 039). | Keep. Stretch: auto-apply the chain on the deserialize/`link_from` path so connectors migrate old wire data without a manual call. | -| `Observable` | **Barely consumed.** Signal extraction + `format_log()` only; sole consumer is the opt-in `log_tap()`, wired by hand. **Not** connected to metrics. | `.observe(node_id)` installs the log tap (§3.2). README reworded (below). Statistics come from the separate `observability` feature, not this trait. | -| `Linkable` | **Convention only.** No engine bound anywhere — examples call `T::to_bytes()`/`from_bytes()` by hand inside `with_serializer`/`with_deserializer` closures. | `.link_from`/`.link_to` default their (de)serializer to `Linkable::{from_bytes,to_bytes}` when `T: Linkable`, so the connector consumes the trait directly. *(Own follow-up — not in this doc's PR.)* | -| `Simulatable` | **Unconsumed.** Called by hand in example producer loops only. | `.simulate(cfg, rng)` installs the source loop (§3.1); `.source_or_simulate(mode, …)` picks real-vs-sim for the single producer slot (§3.3). *Stretch:* runtime sim↔real failover (live switch on sensor staleness) — its own doc. | -| `Settable` | **Unconsumed.** `set(value, ts)`; examples `impl Settable`, nothing calls it outside codegen templates. | Back a typed `.set(value)` producer/one-shot helper. *(Own follow-up — noted for completeness; not in scope here.)* | - -This doc delivers the `Observable` and `Simulatable` rows. `Linkable`/`Settable` targets are recorded so the map is complete, but are separate follow-ups (each needs its own caller-in-the-same-PR). - -### 4.1 The `Observable` README claim — reword (decided) +```rust +// aimdb-sync/src/producer.rs (inherent impl, feature = "data-contracts") +impl SyncProducer +where + T: aimdb_data_contracts::Settable + Send + 'static + Debug + Clone, +{ + /// Construct via `T::set(value, now)` and send. Blocking, like `set()`. + pub fn set_value(&self, value: T::Value) -> SyncResult<()> { + self.set(T::set(value, unix_now_ms())) + } -The capability table ([`README.md:137`](../../README.md)) currently reads `Observable | Automatic per-record metrics`. This is wrong on two counts: + /// Non-blocking variant, like `try_set()`. + pub fn try_set_value(&self, value: T::Value) -> SyncResult<()> { + self.try_set(T::set(value, unix_now_ms())) + } -1. **`Observable` is about log formatting, not metrics.** Its surface is `signal()`, `ICON`, `UNIT`, `format_log()` ([`aimdb-data-contracts/src/lib.rs:192-227`](../../aimdb-data-contracts/src/lib.rs)) — domain-signal extraction + human-readable output. -2. **The per-record metrics that *do* exist are independent of `Observable`.** `BufferMetricsSnapshot` (produced/consumed/dropped/occupancy) is collected by the buffer for **every** record under the `observability` feature ([`aimdb-core/src/buffer/traits.rs:278`](../../aimdb-core/src/buffer/traits.rs)), regardless of whether the type is `Observable`. + /// Explicit-timestamp variant (replay, testing). + pub fn set_value_at(&self, value: T::Value, timestamp_ms: u64) -> SyncResult<()> { + self.set(T::set(value, timestamp_ms)) + } +} +``` -**Decision (2026-07-06): reword to match reality.** `Observable` lets the user *tap and log* the record; the `observability` **feature flag** is what provides statistics — two separate things the README wrongly fused. New row: +Notes: +- **Dependency direction:** `aimdb-sync` gains `aimdb-data-contracts = { optional = true, default-features = false, features = ["settable"] }` behind a `data-contracts` feature. The contracts crate does **not** grow a `sync` feature. +- **Clock semantics (documented, one sentence):** `set_value` stamps with the *caller's* `SystemTime` (sample time at the edge), not the engine's `ctx.time()`; `set_value_at` exists for callers that need control. `aimdb-sync` is std-only, so `SystemTime` is always available. +- **Single-writer untouched:** `set_value` flows through the same channel as `set()`; the sync bridge remains the record's one producer, and concurrent `SyncProducer` clones already arbitrate through that one request stream. +- **Feature gate:** `Settable` is currently compiled unconditionally in the contracts crate; it moves behind a `settable` feature for tier symmetry (codegen templates that emit `impl Settable` enable it). +- A future AimX **set-by-primitive** verb (`aimdb set temperature 22.5` from CLI/MCP) becomes a second consumer of the same trait — recorded in §7, not in scope. The read/write symmetry is now explicit: `Observable::signal()` is the generic *read* projection, `Settable::set()` the generic *write* injection. -| Trait | What it unlocks | Runtimes | -|---|---|---| -| `Observable` | Built-in logging tap (`.observe()`) over an extracted signal | std, no_std | +### 3.5 `Streamable` — unchanged -The README's separate feature-flag section should carry the "per-record buffer statistics" story where it belongs (`observability`). `.observe()`'s `.with_name("observe")` (§3.2) still gives the tap a real identity in the stage-profiling snapshot — an honest, modest link, no overclaim. A signal-metrics sink (min/max/mean of `signal()` into the snapshot) remains possible later but is out of scope and would get its own design doc. +Already consumed (ws-connector registry bound). Its fix is the verb table (§2) and the serialization-stories table (§3.3): the README states its verb is `.register::()` on the ws connector builder. No trait change. -## 5. Decisions (resolved 2026-07-06) +### 3.6 `Migratable` — unchanged -### 5.1 RNG source for `.simulate()` — **caller-supplied** +Consumed since design 039, works no_std. Stretch (recorded, not in scope): auto-apply the migration chain on the `linked_from` deserialize path so connectors migrate old wire data without a manual call. -`Simulatable::simulate` is generic over the RNG; `.simulate()` must *pick* one, and no_std/Embassy has no default entropy source. **Decision: the caller supplies the RNG** — `.simulate(cfg, rng)` takes an owned `R: rand::Rng + Send + 'static` (§3.1). std callers pass `StdRng::from_rng(&mut rand::rng())` (already in the demo); embedded callers pass a seeded PRNG; tests pass a fixed seed. This keeps the method a single signature that works on every runtime and drops the need for `rand`'s `std_rng` feature in the ext path. +## 4. Crate & feature layout after this doc -### 5.2 `Observable` README claim — **reword** (see §4.1) +``` +aimdb-data-contracts + features: linkable (+core), observable (+core), migratable, settable, + simulatable (+core, rand nf — no std_rng) # dev tier — never a default feature + (no new crate: aimdb-simulation split considered and rejected, §3.1.1) -Reworded to "built-in logging tap over an extracted signal"; statistics stay attributed to the `observability` feature. Signal-metrics sink deferred to a possible future doc. +aimdb-sync + features: data-contracts → aimdb-data-contracts (nf, settable) -### 5.3 New coupling: `simulatable` now pulls `aimdb-core` +aimdb-core + observability: + SignalStats / RecordRegistrar::signal_gauge (inert handle when off) + RecordMetadata: + optional signal stats; AimX record.get carries them +``` -`observable` already depends on `aimdb-core` (for `log_tap`). Adding `SimulatableRegistrarExt` means **`simulatable` gains the same `aimdb-core` dependency** (for `RecordRegistrar`/`Producer`/`RuntimeContext`). This is the intended shape of the D1/D12 fix — "turns `aimdb-data-contracts` into a consumed layer" — but it means selecting `simulatable` on a flash-constrained target now compiles core's `alloc` surface. `aimdb-core` is pulled with `default-features = false, features = ["alloc"]` (matching the existing `observable` wiring), so `serde_json`/AimX stay out. If someone ever needs the bare trait + `SimulationConfig` without core, split via a `simulatable-integration` sub-feature — not proposed now (no such caller). +`aimdb-data-contracts` bumps 0.2.0 → 0.3.0 (breaking: `Simulatable` reshaped — `type Params`, `SimulationConfig`/`SimulationParams` replaced by `SimProfile`/`RandomWalkParams`; `Observable` loses `ICON`/`format_log`; `Settable` gains a feature gate). -### 5.4 Sim-to-real is build-time selection, not runtime coexistence — **decided** +## 5. Decisions (resolved 2026-07-06, v2) -"Real feed in production, `Simulatable` in dev / CI / demo / absent hardware" is delivered as `source_or_simulate(mode, cfg, rng, real)` (§3.3): a registrar-time branch over the single producer slot. **Rationale:** a record has exactly one producer ([`typed_record.rs:570`](../../aimdb-core/src/typed_record.rs)), so sim and real can never both write — attempting to install both is a writer-exclusivity violation `build()` already rejects, and would contradict single-writer-per-key. Selecting one at registration is zero-cost and invariant-safe, and it reuses the general-purpose `.source()` installer unchanged (no new stage type). **Live** failover (sim↔real at runtime, e.g. on sensor staleness) would instead require a *single* source future that multiplexes real vs. `T::simulate(...)` internally; it is deferred to its own doc (§7), not folded into this selector. +1. **Simulation is compile-time.** `SourceMode`/`source_or_simulate` deleted; `enabled` deleted; `#[cfg]` branch is the canonical sim-to-real pattern; CI guard (`rand` tracer + never-a-default-feature assert). `Simulatable` **stays in `aimdb-data-contracts`** behind `simulatable` — a separate `aimdb-simulation` crate was considered and rejected (2026-07-06, owner: keep the monorepo lean); the guarantee rests on §3.1.1's three properties, not on a crate boundary. (Overturns v1 §3.3/§5.4.) +2. **Caller-supplied RNG stands** (v1 §5.1 carried over); `rand` loses `std_rng` everywhere. +3. **`Simulatable::Params` is an associated type**; `RandomWalkParams` ships as the migration path for today's scalar walks. +4. **`Observable` claims metrics because it delivers metrics**: kernel-only trait, `.observe()` → core signal gauges surfaced via `record.list`/`record.get`/profiling. (Overturns v1 §4.1's reword-downward.) +5. **`Linkable` ext is in scope** (was a v1 follow-up): `.linked_from`/`.linked_to` + `#[derive(Linkable)]`; `String`→`SerializeError::InvalidData` mapping accepted for now. +6. **`Settable` is wired to its original purpose**: `SyncProducer::set_value` family in `aimdb-sync`, caller-side clock, `settable` feature gate. +7. **No trait bounds on `.source()`/`.tap()`/`.transform()`; no core→contracts dependency** — unchanged from v1. -## 6. Migration & examples +## 6. Migration & examples (the D1 callers) -- **weather-station-beta / gamma:** replace the manual producer loops with `reg.simulate(cfg, rng)` inside `configure::()`, passing the `StdRng` they already build. Deletes the `tokio::spawn`, the `db.producer::()` handles, and the prev-value bookkeeping. -- **weather-hub:** replace `.tap(move |ctx, c| log_tap(ctx, c, key.as_str()))` with `.observe(node_id)`. -- **`source_or_simulate` caller:** the demo stations are simulation-only (they publish synthetic data; there is no real sensor to select), so the combinator's in-PR caller is an `aimdb-data-contracts` integration test — a fixed-seed RNG plus a stub real source, asserting each `SourceMode` installs the expected single producer and that `build()` reports **no** writer conflict. This satisfies the D1 caller-in-PR rule without fabricating hardware; a station that toggles a *genuine* feed to sim is future work (§7). -- These examples/tests become the **caller** that proves the API — satisfying the D1 rule ("no public API without a caller in the same PR"). -- `CHANGELOG.md`: entry under `aimdb-data-contracts` ("added `SimulatableRegistrarExt` (`simulate` + `source_or_simulate`) / `ObservableRegistrarExt`") and README. +- **weather-station-beta/gamma:** delete the manual RNG-loop producers; `reg.simulate(SimProfile { interval_ms, params: RandomWalkParams { … } }, StdRng::…)` under the app's `sim` feature; contracts impls move to `type Params = RandomWalkParams` and `#[cfg(feature = "sim")]`. The station is the §3.1.4 pattern's reference implementation (with the CI guard building its prod configuration). +- **weather-hub:** `.observe()` replaces the manual `log_tap` tap; keep one `.log(node_id)` where console output is the point of the demo. Hub verifies the signal appears in `record.list`/`record.get`. +- **one connector demo (tokio-mqtt):** `#[derive(Linkable)]` + `.linked_from`/`.linked_to` replace the hand-written closures. +- **aimdb-sync doctest/integration test:** `producer.set_value(22.5)` end-to-end (construct → produce → consume), plus `set_value_at` determinism in a unit test. +- **README:** capability table becomes the §2 verb table (Contract / Implement when / Verb / Tier); the `observability` feature keeps the buffer-statistics story; the `simulatable` feature gets a "dev tier — never ships" call-out with the `#[cfg]` pattern. +- **CHANGELOG:** entries for all four crates touched. ## 7. Sequencing -1. Land `ObservableRegistrarExt::observe()` + the README reword (self-contained). -2. Land `SimulatableRegistrarExt::simulate(cfg, rng)`. -3. Land `SimulatableRegistrarExt::source_or_simulate(mode, cfg, rng, real)` on the same trait, with its integration-test caller (§3.3, §6). -4. Convert the weather-mesh examples to the new methods in the same commit as the API they exercise (the caller requirement). -5. Update the README capability table with the current→target-accurate rows (§4). +Separate commits on one branch/PR, one per work item (per the no-stacked-PRs rule), each carrying its caller: -Per [[feedback_no_stacked_prs_for_design_docs]]: land as separate commits on one branch, one per work item — not a stacked-PR chain. +1. `Simulatable` v2 in place (trait reshape + `SimProfile`/`RandomWalkParams` + `.simulate()` ext, `enabled` and `std_rng` dropped), stations migrated, CI `check-no-sim` guard. +2. Core `signal_gauge` surface + `Observable` kernel reshape + `.observe()`/`.log()` ext, hub migrated, README `Observable` row. +3. `LinkableRegistrarExt` + `#[derive(Linkable)]`, mqtt demo migrated. +4. `aimdb-sync` `set_value` family + `settable` feature gate + tests. +5. README verb/tier tables + CHANGELOG sweep. -**Out of scope / recorded for later** (each needs its own caller-in-PR): the `Linkable` and `Settable` targets from §4; the **runtime sim↔real failover** multiplexer (§3.3 note) — a single source future that switches live between the real feed and `T::simulate(...)` on sensor staleness, kept single-writer-safe. +**Out of scope / recorded for later:** runtime sim↔real failover (a single source future multiplexing on sensor staleness — still single-writer-safe, own doc); `CodecError` alignment across `Linkable` and the connector (de)serializer signatures; AimX set-by-primitive verb (second `Settable` consumer); signal-metrics history/percentiles beyond last/min/max/mean; auto-migration on the `linked_from` path (§3.6). From 497f964d4f762ae9f5b04f0d62dd5338683a1c4d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Mon, 6 Jul 2026 19:58:17 +0000 Subject: [PATCH 03/31] =?UTF-8?q?feat(data-contracts):=20Simulatable=20v2?= =?UTF-8?q?=20=E2=80=94=20compile-time-only=20sim=20+=20.simulate()=20verb?= =?UTF-8?q?=20(design=20041=20=C2=A73.1)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reshape Simulatable to the one-verb-per-contract model and make simulation a compile-time fact, not a runtime flag: - Trait: add `type Params`; `simulate(params, previous, rng, timestamp_ms)`. Delete `SimulationConfig` (incl. `enabled`) and `SimulationParams`; add `SimProfile

` + off-the-shelf `RandomWalkParams`. - New `SimulatableRegistrarExt::simulate(profile, rng)` installs a source loop over `ctx.time()` (runtime-neutral, single-writer enforced by build()). - Cargo: `simulatable = ["rand", "aimdb-core"]`; `rand` drops `std_rng` (caller-supplied RNG; SmallRng needs no feature). D1 callers: - weather-mesh-common temp/humidity/location impls → `type Params = RandomWalkParams`. - weather-station-beta (std) + gamma (no_std): `sim` feature gates `.simulate()`; default (prod/flash) build reads a hardware source and is rand-free — the canonical §3.1.4 `#[cfg]` sim-to-real pattern. CI: `make check-no-sim` proves each station's production graph carries no `rand` (the tracer) and that `simulatable` is never a default feature; wired into the CI makefile-build job. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/ci.yml | 3 + Cargo.lock | 1 - Makefile | 27 +- aimdb-data-contracts/Cargo.toml | 8 +- aimdb-data-contracts/src/lib.rs | 27 +- aimdb-data-contracts/src/simulatable.rs | 245 +++++++++++------- .../weather-mesh-common/Cargo.toml | 6 +- .../src/contracts/humidity.rs | 20 +- .../src/contracts/location.rs | 18 +- .../src/contracts/temperature.rs | 20 +- .../weather-station-beta/Cargo.toml | 15 +- .../weather-station-beta/src/main.rs | 163 +++++++----- .../weather-station-gamma/Cargo.toml | 22 +- .../weather-station-gamma/src/main.rs | 112 ++++---- 14 files changed, 407 insertions(+), 280 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 848ab89..a9823d6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -75,6 +75,9 @@ jobs: - name: Check codegen template drift run: make codegen-drift + - name: Prove production is simulation-free + run: make check-no-sim + # Embedded target testing embedded-targets: name: Embedded Cross-compilation diff --git a/Cargo.lock b/Cargo.lock index d25be32..569ca1f 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4432,7 +4432,6 @@ dependencies = [ "aimdb-data-contracts", "aimdb-mqtt-connector", "aimdb-tokio-adapter", - "chrono", "rand 0.10.1", "tokio", "tracing", diff --git a/Makefile b/Makefile index bddfbe8..736b869 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # AimDB Makefile # Simple automation for common development tasks -.PHONY: help build test clean clean-embedded fmt fmt-check clippy doc all check test-embedded test-wasm wasm wasm-test examples deny audit security publish publish-check readme-check codegen-drift +.PHONY: help build test clean clean-embedded fmt fmt-check clippy doc all check test-embedded test-wasm wasm wasm-test examples deny audit security publish publish-check readme-check codegen-drift check-no-sim .DEFAULT_GOAL := help # Separate target dir for embedded checks so an interrupted example build @@ -535,8 +535,31 @@ codegen-drift: @printf "$(GREEN)Checking codegen templates against the workspace API...$(NC)\n" ./tools/scripts/codegen-drift-check.sh +# Prove that simulation code (design 041, dev tier) never reaches a production +# binary. `rand` is the tracer: it is reachable iff `simulatable` is enabled +# (aimdb-data-contracts/src/simulatable.rs). `cargo tree -i rand` exits non-zero +# when `rand` is absent from the resolved graph, so a *successful* lookup here is +# a failure. Also asserts `simulatable` is not a default feature of the contracts +# crate (which would pull `rand` into its own default graph). +SIM_EXAMPLES := weather-station-beta weather-station-gamma +check-no-sim: + @printf "$(GREEN)Proving production graphs are simulation-free...$(NC)\n" + @for bin in $(SIM_EXAMPLES); do \ + if cargo tree -p $$bin -e normal -i rand >/dev/null 2>&1; then \ + printf "$(RED)✗ '$$bin' (default/production, no sim) pulls in 'rand' — simulation code leaked into production$(NC)\n"; \ + exit 1; \ + fi; \ + printf "$(BLUE)✓ $$bin production graph is rand-free$(NC)\n"; \ + done + @if cargo tree -p aimdb-data-contracts -e normal -i rand >/dev/null 2>&1; then \ + printf "$(RED)✗ aimdb-data-contracts pulls 'rand' with default features — 'simulatable' must never be a default feature$(NC)\n"; \ + exit 1; \ + fi + @printf "$(BLUE)✓ 'simulatable' is not a default feature of aimdb-data-contracts$(NC)\n" + @printf "$(GREEN)✓ Production is simulation-free$(NC)\n" + ## Convenience commands -check: fmt-check clippy test test-embedded test-wasm deny readme-check codegen-drift +check: fmt-check clippy test test-embedded test-wasm deny readme-check codegen-drift check-no-sim @printf "$(GREEN)Comprehensive development checks completed!$(NC)\n" @printf "$(BLUE)✓ Code formatting verified$(NC)\n" @printf "$(BLUE)✓ Linter passed$(NC)\n" diff --git a/aimdb-data-contracts/Cargo.toml b/aimdb-data-contracts/Cargo.toml index 1153576..7eb995d 100644 --- a/aimdb-data-contracts/Cargo.toml +++ b/aimdb-data-contracts/Cargo.toml @@ -15,7 +15,10 @@ default = ["std"] alloc = ["serde/alloc"] std = ["alloc", "serde/std", "serde_json?/std"] linkable = ["alloc", "serde_json"] -simulatable = ["rand"] +# Dev-tier — never a default feature. The ext trait needs core's registrar, +# same coupling `observable` has. `rand` is the tracer CI uses to prove a +# production graph is sim-free (design 041 §3.1.1). +simulatable = ["rand", "aimdb-core"] migratable = ["alloc", "serde_json", "dep:aimdb-derive"] observable = ["alloc", "aimdb-core"] @@ -33,11 +36,12 @@ aimdb-derive = { version = "0.2.0", path = "../aimdb-derive", optional = true } aimdb-core = { version = "1.1.0", path = "../aimdb-core", optional = true, default-features = false, features = [ "alloc", ] } +# RNG is caller-supplied, so we need only the `Rng`/`RngExt` traits and the +# always-available `SmallRng` — no `std_rng` (design 041 §3.1.1, decision 2). [dependencies.rand] version = "0.10.1" optional = true default-features = false -features = ["std_rng"] [dev-dependencies] serde_json = "1.0" diff --git a/aimdb-data-contracts/src/lib.rs b/aimdb-data-contracts/src/lib.rs index e8792f6..17960b9 100644 --- a/aimdb-data-contracts/src/lib.rs +++ b/aimdb-data-contracts/src/lib.rs @@ -84,7 +84,7 @@ mod simulatable; mod migratable; #[cfg(feature = "simulatable")] -pub use simulatable::{SimulationConfig, SimulationParams}; +pub use simulatable::{RandomWalkParams, SimProfile, Simulatable, SimulatableRegistrarExt}; #[cfg(feature = "migratable")] pub use migratable::{MigrationChain, MigrationError, MigrationStep}; @@ -134,31 +134,6 @@ pub trait SchemaType: Sized { const VERSION: u32 = 1; } -// ═══════════════════════════════════════════════════════════════════ -// SIMULATABLE SUPPORT (feature = "simulatable") -// ═══════════════════════════════════════════════════════════════════ - -/// Generate realistic test/simulation data. -/// -/// This is an intrinsic capability of the schema type itself, -/// not a policy decision. If a type can be simulated, implement this. -#[cfg(feature = "simulatable")] -pub trait Simulatable: SchemaType { - /// Generate a new sample with optional reference to previous value. - /// - /// # Parameters - /// - `config`: Simulation parameters (type-specific) - /// - `previous`: Optional reference to last generated value (for random walks, trends) - /// - `rng`: Random number generator - /// - `timestamp`: Unix timestamp in milliseconds - fn simulate( - config: &SimulationConfig, - previous: Option<&Self>, - rng: &mut R, - timestamp: u64, - ) -> Self; -} - /// Construct a schema instance from its primary value. /// /// This defines the canonical way to create a new reading/measurement. diff --git a/aimdb-data-contracts/src/simulatable.rs b/aimdb-data-contracts/src/simulatable.rs index 0614b15..070519f 100644 --- a/aimdb-data-contracts/src/simulatable.rs +++ b/aimdb-data-contracts/src/simulatable.rs @@ -1,82 +1,77 @@ -//! Simulation configuration for data contracts. +//! Simulation capability for data contracts (dev-tier, `feature = "simulatable"`). //! -//! These structures configure how `Simulatable` types generate test data. +//! Implementing [`Simulatable`] unlocks exactly one verb — +//! [`SimulatableRegistrarExt::simulate`] — which installs a source that emits +//! synthetic samples on a timer. This is the **dev tier**: it must never ship in +//! a production binary. Sim-to-real selection is a compile-time `#[cfg]` in the +//! application (design 041 §3.1.4), never a runtime flag, and `rand` is the +//! tracer CI uses to prove production dependency graphs are sim-free. use serde::{Deserialize, Serialize}; +use aimdb_core::typed_api::RecordRegistrar; + +use crate::SchemaType; + // ═══════════════════════════════════════════════════════════════════ -// SIMULATION CONFIG +// SIMULATABLE TRAIT // ═══════════════════════════════════════════════════════════════════ -/// Configuration for data simulation. +/// Dev-only capability: generate realistic synthetic data. /// -/// This is passed to `Simulatable::simulate()` to control how -/// test data is generated. -#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] -pub struct SimulationConfig { - /// Whether simulation is active - #[serde(default = "default_true")] - pub enabled: bool, - - /// Interval between simulated samples (milliseconds) - #[serde(default = "default_interval")] - pub interval_ms: u64, - - /// Type-specific parameters (schema implementations interpret these) - #[serde(default)] - pub params: SimulationParams, +/// This is an intrinsic capability of the schema type itself, not a policy +/// decision. If a type can be simulated, implement this — then call +/// [`SimulatableRegistrarExt::simulate`] on the registrar. +pub trait Simulatable: SchemaType { + /// Type-specific generation parameters (a temperature defines walk bounds, + /// a GPS track defines waypoints, …). Use [`RandomWalkParams`] for scalar + /// walks, or define your own. + type Params: Clone + Send + Sync + Default + 'static; + + /// Generate the next sample. + /// + /// - `params`: type-specific generation parameters. + /// - `previous`: last generated value, enabling walks/trends. + /// - `rng`: caller-supplied RNG (bring [`rand::RngExt`] into scope for + /// `.random()`). + /// - `timestamp_ms`: Unix millis supplied by the driving loop. + fn simulate( + params: &Self::Params, + previous: Option<&Self>, + rng: &mut R, + timestamp_ms: u64, + ) -> Self; } -fn default_true() -> bool { - true -} - -fn default_interval() -> u64 { - 1000 -} +// ═══════════════════════════════════════════════════════════════════ +// PROFILE + OFF-THE-SHELF PARAMS +// ═══════════════════════════════════════════════════════════════════ -impl Default for SimulationConfig { - fn default() -> Self { - Self { - enabled: true, - interval_ms: 1000, - params: SimulationParams::default(), - } - } +/// Loop policy + generation params for one record. +#[derive(Clone, Debug, Default)] +pub struct SimProfile

{ + /// Interval between samples (milliseconds). + pub interval_ms: u64, + /// Type-specific generation parameters. + pub params: P, } -/// Type-specific simulation parameters. -/// -/// Common parameters that many schema types use. Schema implementations -/// can interpret these as appropriate for their domain. +/// Off-the-shelf [`Simulatable::Params`] for scalar random walks — what the +/// pre-0.3 `SimulationParams` actually was. Existing impls migrate by setting +/// `type Params = RandomWalkParams`. #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] -pub struct SimulationParams { - /// Base/center value for the simulation - #[serde(default)] +pub struct RandomWalkParams { + /// Base/center value for the walk. pub base: f64, - - /// Maximum deviation from base (for random walks) - #[serde(default = "default_variation")] + /// Maximum deviation from base. pub variation: f64, - - /// Linear trend per sample (positive = increasing, negative = decreasing) - #[serde(default)] + /// Linear trend applied per sample (positive = increasing). pub trend: f64, - - /// Step size multiplier for random walk (0.0-1.0) - #[serde(default = "default_step")] + /// Step-size multiplier for the random walk (0.0–1.0). pub step: f64, } -fn default_variation() -> f64 { - 1.0 -} - -fn default_step() -> f64 { - 0.2 -} - -impl Default for SimulationParams { +impl Default for RandomWalkParams { fn default() -> Self { Self { base: 0.0, @@ -87,6 +82,57 @@ impl Default for SimulationParams { } } +// ═══════════════════════════════════════════════════════════════════ +// REGISTRAR EXTENSION: `.simulate(profile, rng)` +// ═══════════════════════════════════════════════════════════════════ + +/// Adds `.simulate(profile, rng)` to [`RecordRegistrar`] for [`Simulatable`] types. +pub trait SimulatableRegistrarExt<'a, T> +where + T: Simulatable + Send + Sync + Clone + core::fmt::Debug + 'static, +{ + /// Install a source that emits `T::simulate(...)` every `interval_ms`, + /// driving the caller-supplied RNG (OS entropy on std, seeded PRNG on + /// no_std, fixed seed in tests). + /// + /// Installs a **source**, so single-writer-per-key is enforced by `build()` + /// exactly as for a hardware `.source()` — the two are mutually exclusive by + /// the app's `#[cfg]`, never both present in one binary. + fn simulate(&mut self, profile: SimProfile, rng: R) -> &mut RecordRegistrar<'a, T> + where + R: rand::Rng + Send + 'static; +} + +impl<'a, T> SimulatableRegistrarExt<'a, T> for RecordRegistrar<'a, T> +where + T: Simulatable + Send + Sync + Clone + core::fmt::Debug + 'static, +{ + fn simulate( + &mut self, + profile: SimProfile, + mut rng: R, + ) -> &mut RecordRegistrar<'a, T> + where + R: rand::Rng + Send + 'static, + { + self.source(move |ctx, producer| async move { + let mut prev: Option = None; + loop { + let now_ms = ctx + .time() + .unix_time() + .map(|(s, ns)| s.saturating_mul(1000) + (ns / 1_000_000) as u64) + .unwrap_or(0); + let next = T::simulate(&profile.params, prev.as_ref(), &mut rng, now_ms); + producer.produce(next.clone()); + prev = Some(next); + ctx.time().sleep_millis(profile.interval_ms.max(1)).await; + } + }) + .with_name("simulate") + } +} + // ═══════════════════════════════════════════════════════════════════ // TESTS // ═══════════════════════════════════════════════════════════════════ @@ -94,40 +140,61 @@ impl Default for SimulationParams { #[cfg(test)] mod tests { use super::*; + use rand::{RngExt, SeedableRng}; + + #[derive(Clone, Debug, PartialEq)] + struct Scalar(f64); + + impl SchemaType for Scalar { + const NAME: &'static str = "scalar"; + } + + impl Simulatable for Scalar { + type Params = RandomWalkParams; + + fn simulate( + params: &Self::Params, + previous: Option<&Self>, + rng: &mut R, + _timestamp_ms: u64, + ) -> Self { + let base = previous.map(|p| p.0).unwrap_or(params.base); + let delta = (rng.random::() - 0.5) * params.variation * params.step + params.trend; + Scalar(base + delta) + } + } #[test] - fn test_simulation_config_defaults() { - let config = SimulationConfig::default(); - assert!(config.enabled); - assert_eq!(config.interval_ms, 1000); - assert_eq!(config.params.base, 0.0); - assert_eq!(config.params.variation, 1.0); - assert_eq!(config.params.step, 0.2); + fn random_walk_defaults() { + let p = RandomWalkParams::default(); + assert_eq!(p.variation, 1.0); + assert_eq!(p.step, 0.2); + assert_eq!(p.base, 0.0); + assert_eq!(p.trend, 0.0); } #[test] - fn test_simulation_config_json_roundtrip() { - let json = r#"{ - "enabled": true, - "interval_ms": 500, - "params": { - "base": 22.0, - "variation": 3.0, - "trend": 0.1, - "step": 0.1 - } - }"#; - - let config: SimulationConfig = serde_json::from_str(json).unwrap(); - assert!(config.enabled); - assert_eq!(config.interval_ms, 500); - assert_eq!(config.params.base, 22.0); - assert_eq!(config.params.variation, 3.0); - assert_eq!(config.params.trend, 0.1); - - // Roundtrip - let serialized = serde_json::to_string(&config).unwrap(); - let deserialized: SimulationConfig = serde_json::from_str(&serialized).unwrap(); - assert_eq!(config, deserialized); + fn sim_profile_default_is_zero_interval() { + let profile: SimProfile = SimProfile::default(); + assert_eq!(profile.interval_ms, 0); + assert_eq!(profile.params, RandomWalkParams::default()); + } + + #[test] + fn simulate_is_deterministic_for_fixed_seed() { + let params = RandomWalkParams::default(); + // SmallRng is seedable and available with `default-features = false`. + let mut a = rand::rngs::SmallRng::seed_from_u64(42); + let mut b = rand::rngs::SmallRng::seed_from_u64(42); + + let mut prev_a: Option = None; + let mut prev_b: Option = None; + for ts in 0..5 { + let sa = Scalar::simulate(¶ms, prev_a.as_ref(), &mut a, ts); + let sb = Scalar::simulate(¶ms, prev_b.as_ref(), &mut b, ts); + assert_eq!(sa, sb, "same seed must yield the same walk"); + prev_a = Some(sa); + prev_b = Some(sb); + } } } diff --git a/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml b/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml index 13ed7d0..dac5446 100644 --- a/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml +++ b/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml @@ -21,6 +21,6 @@ aimdb-core = { path = "../../../aimdb-core", default-features = false, features aimdb-data-contracts = { path = "../../../aimdb-data-contracts", default-features = false } serde = { version = "1.0", default-features = false, features = ["derive"] } serde_json = { workspace = true, optional = true } -rand = { version = "0.10.1", optional = true, default-features = false, features = [ - "std_rng", -] } +# RNG is caller-supplied at the station; the contract impls only use the +# `Rng`/`RngExt` traits, so no `std_rng` here (design 041 §3.1.1). +rand = { version = "0.10.1", optional = true, default-features = false } diff --git a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/humidity.rs b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/humidity.rs index 43437a1..572747d 100644 --- a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/humidity.rs +++ b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/humidity.rs @@ -9,7 +9,7 @@ use serde::{Deserialize, Serialize}; use aimdb_data_contracts::Linkable; #[cfg(feature = "simulatable")] -use aimdb_data_contracts::{Simulatable, SimulationConfig}; +use aimdb_data_contracts::{RandomWalkParams, Simulatable}; #[cfg(feature = "simulatable")] use rand::RngExt; @@ -51,23 +51,25 @@ impl Observable for Humidity { #[cfg(feature = "simulatable")] impl Simulatable for Humidity { + type Params = RandomWalkParams; + /// Simulate humidity readings with random walk behavior. /// - /// # Config params interpretation + /// # Params interpretation /// - `base`: Center humidity value (default: 50.0%) /// - `variation`: Maximum deviation from base (default: 10.0%) /// - `step`: Random walk step multiplier (default: 0.2) /// - `trend`: Linear trend per sample (default: 0.0) fn simulate( - config: &SimulationConfig, + params: &Self::Params, previous: Option<&Self>, rng: &mut R, - timestamp: u64, + timestamp_ms: u64, ) -> Self { - let base = config.params.base as f32; - let variation = config.params.variation as f32; - let step = config.params.step as f32; - let trend = config.params.trend as f32; + let base = params.base as f32; + let variation = params.variation as f32; + let step = params.step as f32; + let trend = params.trend as f32; // Random walk: small delta from previous value, clamped to valid range let current = match previous { @@ -82,7 +84,7 @@ impl Simulatable for Humidity { Humidity { percent: current, - timestamp, + timestamp: timestamp_ms, } } } diff --git a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/location.rs b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/location.rs index 5938e41..41059a2 100644 --- a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/location.rs +++ b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/location.rs @@ -9,7 +9,7 @@ use serde::{Deserialize, Serialize}; use aimdb_data_contracts::Linkable; #[cfg(feature = "simulatable")] -use aimdb_data_contracts::{Simulatable, SimulationConfig}; +use aimdb_data_contracts::{RandomWalkParams, Simulatable}; #[cfg(feature = "simulatable")] use rand::RngExt; @@ -73,24 +73,26 @@ impl Observable for GpsLocation { #[cfg(feature = "simulatable")] impl Simulatable for GpsLocation { + type Params = RandomWalkParams; + /// Simulate GPS readings with random walk behavior around a base location. /// - /// # Config params interpretation + /// # Params interpretation /// - `base`: Base latitude (default: 48.2082 - Vienna) /// - `variation`: Maximum wander radius in degrees (default: 0.001 ≈ 111m) /// - `step`: Random walk step multiplier (default: 0.2) /// - `trend`: Not used for GPS fn simulate( - config: &SimulationConfig, + params: &Self::Params, previous: Option<&Self>, rng: &mut R, - timestamp: u64, + timestamp_ms: u64, ) -> Self { // Use base as latitude, and a fixed longitude offset - let base_lat = config.params.base; + let base_lat = params.base; let base_lon = 16.3738; // Vienna longitude as default - let max_delta = config.params.variation; - let step = config.params.step; + let max_delta = params.variation; + let step = params.step; // Random walk from previous position or start near base let (lat, lon) = match previous { @@ -115,7 +117,7 @@ impl Simulatable for GpsLocation { longitude: lon, altitude: Some(200.0 + rng.random::() * 10.0), accuracy: Some(5.0 + rng.random::() * 10.0), - timestamp, + timestamp: timestamp_ms, } } } diff --git a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/temperature.rs b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/temperature.rs index 74b09d0..0f850a5 100644 --- a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/temperature.rs +++ b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/temperature.rs @@ -20,7 +20,7 @@ use serde::{Deserialize, Serialize}; use aimdb_data_contracts::Linkable; #[cfg(feature = "simulatable")] -use aimdb_data_contracts::{Simulatable, SimulationConfig}; +use aimdb_data_contracts::{RandomWalkParams, Simulatable}; #[cfg(feature = "simulatable")] use rand::RngExt; @@ -189,23 +189,25 @@ impl Observable for TemperatureV2 { #[cfg(feature = "simulatable")] impl Simulatable for TemperatureV2 { + type Params = RandomWalkParams; + /// Simulate temperature readings with random walk behavior. /// - /// # Config params interpretation + /// # Params interpretation /// - `base`: Center temperature value (default: 22.0°C) /// - `variation`: Maximum deviation from base (default: 3.0°C) /// - `step`: Random walk step multiplier (default: 0.2) /// - `trend`: Linear trend per sample (default: 0.0) fn simulate( - config: &SimulationConfig, + params: &Self::Params, previous: Option<&Self>, rng: &mut R, - timestamp: u64, + timestamp_ms: u64, ) -> Self { - let base = config.params.base as f32; - let variation = config.params.variation as f32; - let step = config.params.step as f32; - let trend = config.params.trend as f32; + let base = params.base as f32; + let variation = params.variation as f32; + let step = params.step as f32; + let trend = params.trend as f32; // Random walk: small delta from previous value, clamped to range let current = match previous { @@ -219,7 +221,7 @@ impl Simulatable for TemperatureV2 { TemperatureV2 { schema_version: 2, celsius: current, - timestamp, + timestamp: timestamp_ms, } } } diff --git a/examples/weather-mesh-demo/weather-station-beta/Cargo.toml b/examples/weather-mesh-demo/weather-station-beta/Cargo.toml index c88c767..70846f1 100644 --- a/examples/weather-mesh-demo/weather-station-beta/Cargo.toml +++ b/examples/weather-mesh-demo/weather-station-beta/Cargo.toml @@ -10,16 +10,24 @@ publish = false name = "weather-station-beta" path = "src/main.rs" +[features] +# `sim` is opt-in and dev-only (design 041 §3.1.4): it pulls the `Simulatable` +# impls + `rand` and wires `.simulate()`. The DEFAULT (production) build reads a +# hardware source and contains no `rand` at all — `make check-no-sim` proves it. +sim = [ + "weather-mesh-common/simulatable", + "aimdb-data-contracts/simulatable", + "dep:rand", +] + [dependencies] weather-mesh-common = { path = "../weather-mesh-common", features = [ - "simulatable", "linkable", "migratable", ] } aimdb-core = { path = "../../../aimdb-core" } aimdb-tokio-adapter = { path = "../../../aimdb-tokio-adapter" } aimdb-data-contracts = { path = "../../../aimdb-data-contracts", features = [ - "simulatable", "linkable", "migratable", ] } @@ -30,5 +38,4 @@ aimdb-mqtt-connector = { path = "../../../aimdb-mqtt-connector", features = [ tokio = { version = "1", features = ["full"] } tracing = "0.1" tracing-subscriber = { version = "0.3", features = ["env-filter"] } -rand = "0.10.1" -chrono = "0.4" +rand = { version = "0.10.1", optional = true } diff --git a/examples/weather-mesh-demo/weather-station-beta/src/main.rs b/examples/weather-mesh-demo/weather-station-beta/src/main.rs index a6095c6..26cdfb3 100644 --- a/examples/weather-mesh-demo/weather-station-beta/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-beta/src/main.rs @@ -5,14 +5,58 @@ //! without external dependencies. use aimdb_core::{buffer::BufferCfg, AimDbBuilder, RecordKey}; -use aimdb_data_contracts::{Linkable, Simulatable, SimulationConfig, SimulationParams}; +use aimdb_data_contracts::Linkable; +#[cfg(feature = "sim")] +use aimdb_data_contracts::{RandomWalkParams, SimProfile, SimulatableRegistrarExt}; use aimdb_mqtt_connector::MqttConnector; use aimdb_tokio_adapter::{TokioAdapter, TokioRecordRegistrarExt}; +#[cfg(feature = "sim")] use rand::SeedableRng; use std::sync::Arc; use tracing::info; use weather_mesh_common::{DewPoint, DewPointKey, Humidity, HumidityKey, TempKey, Temperature}; +/// A Send RNG for one simulated record (fresh per source; OS-seeded on std). +#[cfg(feature = "sim")] +fn sim_rng() -> rand::rngs::StdRng { + rand::rngs::StdRng::from_rng(&mut rand::rng()) +} + +/// Placeholder "hardware" temperature source for the production (non-sim) build. +/// +/// A real deployment reads an I2C/SPI sensor here. The demo emits a fixed +/// reading on the same 5 s cadence so the record still has exactly one writer +/// and the production dependency graph stays sim-free (no `rand`). +#[cfg(not(feature = "sim"))] +async fn read_temperature(ctx: aimdb_core::RuntimeContext, producer: aimdb_core::Producer) { + loop { + let now = unix_millis(&ctx); + producer.produce(Temperature::new(20.0, now)); + ctx.time().sleep_millis(5000).await; + } +} + +/// Placeholder "hardware" humidity source for the production (non-sim) build. +#[cfg(not(feature = "sim"))] +async fn read_humidity(ctx: aimdb_core::RuntimeContext, producer: aimdb_core::Producer) { + loop { + let now = unix_millis(&ctx); + producer.produce(Humidity { + percent: 45.0, + timestamp: now, + }); + ctx.time().sleep_millis(5000).await; + } +} + +#[cfg(not(feature = "sim"))] +fn unix_millis(ctx: &aimdb_core::RuntimeContext) -> u64 { + ctx.time() + .unix_time() + .map(|(s, ns)| s.saturating_mul(1000) + (ns / 1_000_000) as u64) + .unwrap_or(0) +} + #[tokio::main] async fn main() -> Result<(), Box> { // Initialize logging @@ -42,8 +86,26 @@ async fn main() -> Result<(), Box> { // Configure temperature record let temp_topic = TempKey::Beta.link_address().unwrap(); builder.configure::(TempKey::Beta, |reg| { - reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) - .link_to(temp_topic) + reg.buffer(BufferCfg::SpmcRing { capacity: 10 }); + + // Sim-to-real is a compile-time `#[cfg]`: exactly one producer installed. + #[cfg(feature = "sim")] + reg.simulate( + SimProfile { + interval_ms: 5000, + params: RandomWalkParams { + base: 20.0, // Indoor: ~20°C + variation: 3.0, // ±3°C + trend: 0.0, + step: 0.2, + }, + }, + sim_rng(), + ); + #[cfg(not(feature = "sim"))] + reg.source(read_temperature); + + reg.link_to(temp_topic) .with_serializer(|ctx, t: &Temperature| { ctx.log().info(&format!( "Serializing temp {:.1}°C @ tick {:?}", @@ -59,8 +121,25 @@ async fn main() -> Result<(), Box> { // Configure humidity record let humidity_topic = HumidityKey::Beta.link_address().unwrap(); builder.configure::(HumidityKey::Beta, |reg| { - reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) - .link_to(humidity_topic) + reg.buffer(BufferCfg::SpmcRing { capacity: 10 }); + + #[cfg(feature = "sim")] + reg.simulate( + SimProfile { + interval_ms: 5000, + params: RandomWalkParams { + base: 45.0, // Indoor: ~45% + variation: 10.0, // ±10% + trend: 0.0, + step: 0.2, + }, + }, + sim_rng(), + ); + #[cfg(not(feature = "sim"))] + reg.source(read_humidity); + + reg.link_to(humidity_topic) .with_serializer(|ctx, h: &Humidity| { ctx.log().info(&format!( "Serializing humidity {:.1}% @ tick {:?}", @@ -114,79 +193,23 @@ async fn main() -> Result<(), Box> { .finish(); }); - let (db, runner) = builder.build().await?; + let (_db, runner) = builder.build().await?; tokio::spawn(runner.run()); info!("✅ Database initialized with 3 record types"); - info!(" - Temperature: {} (synthetic)", temp_topic); - info!(" - Humidity: {} (synthetic)", humidity_topic); + #[cfg(feature = "sim")] + let mode = "synthetic"; + #[cfg(not(feature = "sim"))] + let mode = "hardware"; + info!(" - Temperature: {} ({})", temp_topic, mode); + info!(" - Humidity: {} ({})", humidity_topic, mode); info!( " - DewPoint: {} (derived via transform_join)", dew_point_topic ); - // Get producers - let temp_producer = db.producer::(TempKey::Beta.as_str())?; - let humidity_producer = db.producer::(HumidityKey::Beta.as_str())?; - - // Spawn synthetic data producer - info!("🎲 Starting synthetic data producer..."); - tokio::spawn(async move { - // Simulation configuration for indoor conditions - let temp_config = SimulationConfig { - enabled: true, - interval_ms: 5000, - params: SimulationParams { - base: 20.0, // Indoor: ~20°C - variation: 3.0, // ±3°C - step: 0.2, // Smooth random walk - trend: 0.0, // No trend - }, - }; - - let humidity_config = SimulationConfig { - enabled: true, - interval_ms: 5000, - params: SimulationParams { - base: 45.0, // Indoor: ~45% - variation: 10.0, // ±10% - step: 0.2, // Smooth random walk - trend: 0.0, // No trend - }, - }; - - // Create RNG (StdRng is Send, ThreadRng is not) - let mut rng = rand::rngs::StdRng::from_rng(&mut rand::rng()); - let mut prev_temp: Option = None; - let mut prev_humidity: Option = None; - - loop { - let now = chrono::Utc::now().timestamp_millis() as u64; - - // Generate synthetic readings using Simulatable trait - let temp = Temperature::simulate(&temp_config, prev_temp.as_ref(), &mut rng, now); - let humidity = - Humidity::simulate(&humidity_config, prev_humidity.as_ref(), &mut rng, now); - - // Produce the readings - temp_producer.produce(temp.clone()); - - humidity_producer.produce(humidity.clone()); - - tracing::info!( - "📊 Published: {:.1}°C, {:.1}%", - temp.celsius, - humidity.percent - ); - - // Store for next iteration (random walk) - prev_temp = Some(temp); - prev_humidity = Some(humidity); - - // Update every 5 seconds for smooth demo - tokio::time::sleep(tokio::time::Duration::from_secs(5)).await; - } - }); + // Producers are installed inside `configure()` (a `.simulate()` source in the + // `sim` build, a hardware `.source()` otherwise) — no manual spawn needed. info!(""); info!("🎯 Weather Station Beta ready!"); diff --git a/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml b/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml index 312b37b..f8a92ce 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml +++ b/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml @@ -9,15 +9,22 @@ publish = false [features] default = ["embassy-runtime"] embassy-runtime = [] +# `sim` is opt-in and dev-only (design 041 §3.1.4): "develop against sim, flash +# against silicon". It pulls the `Simulatable` impls + `rand` and wires +# `.simulate()`; the default (flash) build reads a hardware source and has no +# `rand` in its graph. +sim = [ + "weather-mesh-common/simulatable", + "aimdb-data-contracts/simulatable", + "dep:rand", +] # Declare but don't enable these features to silence cfg warnings tokio-runtime = [] std = [] [dependencies] # AimDB dependencies -weather-mesh-common = { path = "../weather-mesh-common", default-features = false, features = [ - "simulatable", -] } +weather-mesh-common = { path = "../weather-mesh-common", default-features = false } aimdb-core = { path = "../../../aimdb-core", default-features = false, features = [ "alloc", ] } @@ -25,9 +32,7 @@ aimdb-embassy-adapter = { path = "../../../aimdb-embassy-adapter", default-featu "alloc", "embassy-runtime", ] } -aimdb-data-contracts = { path = "../../../aimdb-data-contracts", default-features = false, features = [ - "simulatable", -] } +aimdb-data-contracts = { path = "../../../aimdb-data-contracts", default-features = false } aimdb-mqtt-connector = { path = "../../../aimdb-mqtt-connector", default-features = false, features = [ "embassy-runtime", ] } @@ -85,8 +90,9 @@ micromath = { workspace = true } stm32-fmc = { workspace = true } embedded-alloc = { version = "0.6", features = ["llff"] } -# Random number generation for simulation -rand = { version = "0.10.1", default-features = false } +# Random number generation for simulation — only in the `sim` build. The +# network-stack seed uses the STM32 HAL RNG (embassy_stm32), not this crate. +rand = { version = "0.10.1", default-features = false, optional = true } [package.metadata.embassy] diff --git a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs index f230423..2887638 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs @@ -24,8 +24,9 @@ extern crate alloc; -use aimdb_core::{AimDbBuilder, Producer, RecordKey, RuntimeContext}; -use aimdb_data_contracts::{Simulatable, SimulationConfig, SimulationParams}; +use aimdb_core::{AimDbBuilder, RecordKey}; +#[cfg(feature = "sim")] +use aimdb_data_contracts::{RandomWalkParams, SimProfile, SimulatableRegistrarExt}; use aimdb_embassy_adapter::{EmbassyAdapter, EmbassyBufferType, EmbassyRecordRegistrarExtCustom}; use aimdb_mqtt_connector::embassy_client::MqttConnectorBuilder; use defmt::*; @@ -37,6 +38,7 @@ use embassy_stm32::peripherals::ETH; use embassy_stm32::rng::Rng; use embassy_stm32::{bind_interrupts, eth, peripherals, rng, Config}; use embassy_time::{Duration, Timer}; +#[cfg(feature = "sim")] use rand::SeedableRng; use static_cell::StaticCell; use weather_mesh_common::{DewPoint, DewPointKey, Humidity, HumidityKey, TempKey, Temperature}; @@ -64,66 +66,61 @@ async fn net_task(mut runner: embassy_net::Runner<'static, Device>) -> ! { runner.run().await } -/// Temperature producer - generates synthetic data -async fn temperature_producer(ctx: RuntimeContext, producer: Producer) { - let log = ctx.log(); - log.info("🌡️ Starting temperature producer..."); - - let config = SimulationConfig { - enabled: true, +/// Sim profile for temperature (dev-tier `sim` build only). +#[cfg(feature = "sim")] +fn temperature_profile() -> SimProfile { + SimProfile { interval_ms: 5000, - params: SimulationParams { + params: RandomWalkParams { base: 22.0, // Portable: ~22°C variation: 5.0, // ±5°C (more variation) step: 0.3, // Random walk trend: 0.0, }, - }; - - let mut rng = rand::rngs::SmallRng::from_seed([42; 32]); - let mut prev: Option = None; - - loop { - let now = ctx.time().now() / 1_000_000; - let temp = Temperature::simulate(&config, prev.as_ref(), &mut rng, now); - - log.info(&alloc::format!("📊 Temp: {:.1}°C", temp.celsius)); - - producer.produce(temp.clone()); - - prev = Some(temp); - Timer::after(Duration::from_secs(5)).await; } } -/// Humidity producer - generates synthetic data -async fn humidity_producer(ctx: RuntimeContext, producer: Producer) { - let log = ctx.log(); - log.info("💧 Starting humidity producer..."); - - let config = SimulationConfig { - enabled: true, +/// Sim profile for humidity (dev-tier `sim` build only). +#[cfg(feature = "sim")] +fn humidity_profile() -> SimProfile { + SimProfile { interval_ms: 5000, - params: SimulationParams { + params: RandomWalkParams { base: 55.0, // Portable: ~55% variation: 15.0, // ±15% step: 0.3, // Random walk trend: 0.0, }, - }; - - let mut rng = rand::rngs::SmallRng::from_seed([84; 32]); - let mut prev: Option = None; + } +} +/// Placeholder "hardware" temperature source for the production (flash) build. +/// +/// A real MCU deployment reads an ADC/I2C sensor here. The demo emits a fixed +/// reading so the record still has exactly one writer and the flashed image +/// carries no `rand` (design 041 §3.1.4). +#[cfg(not(feature = "sim"))] +async fn read_temperature(ctx: aimdb_core::RuntimeContext, producer: aimdb_core::Producer) { + let log = ctx.log(); + log.info("🌡️ Starting temperature sensor..."); loop { let now = ctx.time().now() / 1_000_000; - let humidity = Humidity::simulate(&config, prev.as_ref(), &mut rng, now); - - log.info(&alloc::format!("📊 Humidity: {:.1}%", humidity.percent)); - - producer.produce(humidity.clone()); + producer.produce(Temperature::new(22.0, now)); + Timer::after(Duration::from_secs(5)).await; + } +} - prev = Some(humidity); +/// Placeholder "hardware" humidity source for the production (flash) build. +#[cfg(not(feature = "sim"))] +async fn read_humidity(ctx: aimdb_core::RuntimeContext, producer: aimdb_core::Producer) { + let log = ctx.log(); + log.info("💧 Starting humidity sensor..."); + loop { + let now = ctx.time().now() / 1_000_000; + producer.produce(Humidity { + percent: 55.0, + timestamp: now, + }); Timer::after(Duration::from_secs(5)).await; } } @@ -257,9 +254,18 @@ async fn main(spawner: Spawner) { // Configure temperature record let temp_topic = TempKey::Gamma.link_address().unwrap(); builder.configure::(TempKey::Gamma, |reg| { - reg.buffer_sized::<16, 2>(EmbassyBufferType::SpmcRing) - .source(temperature_producer) - .link_to(temp_topic) + reg.buffer_sized::<16, 2>(EmbassyBufferType::SpmcRing); + + // Sim-to-real is a compile-time `#[cfg]`: exactly one producer installed. + #[cfg(feature = "sim")] + reg.simulate( + temperature_profile(), + rand::rngs::SmallRng::from_seed([42; 32]), + ); + #[cfg(not(feature = "sim"))] + reg.source(read_temperature); + + reg.link_to(temp_topic) .with_serializer(|_ctx, t: &Temperature| { // Manual JSON serialization for no_std let whole = t.celsius as i32; @@ -279,9 +285,17 @@ async fn main(spawner: Spawner) { // Configure humidity record let humidity_topic = HumidityKey::Gamma.link_address().unwrap(); builder.configure::(HumidityKey::Gamma, |reg| { - reg.buffer_sized::<16, 2>(EmbassyBufferType::SpmcRing) - .source(humidity_producer) - .link_to(humidity_topic) + reg.buffer_sized::<16, 2>(EmbassyBufferType::SpmcRing); + + #[cfg(feature = "sim")] + reg.simulate( + humidity_profile(), + rand::rngs::SmallRng::from_seed([84; 32]), + ); + #[cfg(not(feature = "sim"))] + reg.source(read_humidity); + + reg.link_to(humidity_topic) .with_serializer(|_ctx, h: &Humidity| { // Manual JSON serialization for no_std let whole = h.percent as i32; From be4a156e40befb28c66506ca0b959cbf9f26f8f6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Mon, 6 Jul 2026 20:40:55 +0000 Subject: [PATCH 04/31] Refactor observability features in AimDB - Removed `ICON` and `format_log` from the `Observable` trait, simplifying the interface. - Introduced `SignalGaugeHandle` for managing signal statistics, allowing for last/min/max/mean tracking. - Added `SignalStats` struct to encapsulate signal statistics with atomic operations for thread safety. - Updated `RecordProfilingMetrics` to include signal gauges and their statistics. - Enhanced `RecordMetadata` to optionally include signal statistics when the observability feature is enabled. - Modified `ObservableRegistrarExt` to support signal observation and logging. - Updated various schemas (e.g., `Temperature`, `Humidity`, `DewPoint`, `GpsLocation`) to align with the new observability model. - Adjusted example applications to utilize the new signal gauge functionality for metrics and logging. --- aimdb-codegen/src/rust.rs | 42 +--- aimdb-core/src/lib.rs | 9 +- aimdb-core/src/profiling/info.rs | 48 ++++- aimdb-core/src/profiling/mod.rs | 6 +- aimdb-core/src/profiling/record_profiling.rs | 40 +++- aimdb-core/src/profiling/signal_stats.rs | 185 ++++++++++++++++++ aimdb-core/src/remote/metadata.rs | 19 ++ aimdb-core/src/signal.rs | 49 +++++ aimdb-core/src/typed_api.rs | 27 +++ aimdb-core/src/typed_record.rs | 4 + aimdb-data-contracts/src/lib.rs | 52 ++--- aimdb-data-contracts/src/observable.rs | 153 ++++++++------- aimdb-data-contracts/src/simulatable.rs | 6 +- .../weather-mesh-demo/weather-hub/src/main.rs | 18 +- .../src/contracts/dew_point.rs | 12 -- .../src/contracts/humidity.rs | 12 -- .../src/contracts/location.rs | 23 --- .../src/contracts/temperature.rs | 12 -- .../weather-station-beta/src/main.rs | 5 +- 19 files changed, 500 insertions(+), 222 deletions(-) create mode 100644 aimdb-core/src/profiling/signal_stats.rs create mode 100644 aimdb-core/src/signal.rs diff --git a/aimdb-codegen/src/rust.rs b/aimdb-codegen/src/rust.rs index 00be677..37479ba 100644 --- a/aimdb-codegen/src/rust.rs +++ b/aimdb-codegen/src/rust.rs @@ -1009,55 +1009,19 @@ fn emit_observable_impl(rec: &RecordDef) -> Option { let signal_type: syn::Type = syn::parse_str(&signal_field.field_type).ok()?; let signal_ident = format_ident!("{}", obs.signal_field); - let icon = &obs.icon; let unit = &obs.unit; - // Timestamp heuristic: first u64 field named timestamp/computed_at/fetched_at - let timestamp_names = ["timestamp", "computed_at", "fetched_at"]; - let timestamp_field = rec - .fields - .iter() - .find(|f| f.field_type == "u64" && timestamp_names.contains(&f.name.as_str())); - - let format_log_body = if let Some(ts) = timestamp_field { - let ts_ident = format_ident!("{}", ts.name); - quote! { - alloc::format!( - "{} [{}] {}: {:.1}{} at {}", - Self::ICON, - node_id, - Self::NAME, - self.signal(), - Self::UNIT, - self.#ts_ident, - ) - } - } else { - quote! { - alloc::format!( - "{} [{}] {}: {:.1}{}", - Self::ICON, - node_id, - Self::NAME, - self.signal(), - Self::UNIT, - ) - } - }; - + // Observable is now a kernel-only trait (design 041 §3.2): the numeric + // projection + UNIT label. `SIGNAL` defaults to the schema name; presentation + // (`ICON`, `format_log`) moved out of the trait, so we no longer emit them. Some(quote! { impl Observable for #struct_name { type Signal = #signal_type; - const ICON: &'static str = #icon; const UNIT: &'static str = #unit; fn signal(&self) -> #signal_type { self.#signal_ident } - - fn format_log(&self, node_id: &str) -> alloc::string::String { - #format_log_body - } } }) } diff --git a/aimdb-core/src/lib.rs b/aimdb-core/src/lib.rs index 846e528..1074f01 100644 --- a/aimdb-core/src/lib.rs +++ b/aimdb-core/src/lib.rs @@ -39,6 +39,7 @@ pub mod remote; pub mod router; #[cfg(feature = "connector-session")] pub mod session; +pub mod signal; pub mod transform; pub mod transport; pub mod typed_api; @@ -83,9 +84,15 @@ pub use session::{ SessionLimits, Source, TransportError, TransportResult, }; +// Signal gauge handle (always available; inert without `observability`) +pub use signal::SignalGaugeHandle; + // Stage profiling exports (feature-gated) #[cfg(feature = "observability")] -pub use profiling::{RecordProfilingMetrics, StageMetrics, StageProfilingInfo}; +pub use profiling::{ + RecordProfilingMetrics, SignalGauge, SignalStats, SignalStatsInfo, StageMetrics, + StageProfilingInfo, +}; // Connector Infrastructure exports pub use connector::TopicProvider; diff --git a/aimdb-core/src/profiling/info.rs b/aimdb-core/src/profiling/info.rs index 359b2e3..921a64a 100644 --- a/aimdb-core/src/profiling/info.rs +++ b/aimdb-core/src/profiling/info.rs @@ -4,7 +4,7 @@ use alloc::{string::String, vec::Vec}; use serde::{Deserialize, Serialize}; -use crate::profiling::{RecordProfilingMetrics, StageEntry}; +use crate::profiling::{RecordProfilingMetrics, SignalGauge, StageEntry}; /// A point-in-time snapshot of one execution stage's timing metrics. /// @@ -31,6 +31,44 @@ pub struct StageProfilingInfo { pub max_time_ns: u64, } +/// A point-in-time snapshot of one record's signal gauge (`.observe()`). +/// +/// Carried in `RecordMetadata::signal_stats` so a record's live domain signal +/// (last/min/max/mean) surfaces on `record.list`/`record.get` and the MCP tools. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct SignalStatsInfo { + /// Signal label (`Observable::SIGNAL`, defaults to the schema name). + pub signal: String, + /// Unit label (`Observable::UNIT`), omitted when empty. + #[serde(skip_serializing_if = "String::is_empty")] + pub unit: String, + /// Number of samples observed. + pub count: u64, + /// Most recent sample. + pub last: f64, + /// Smallest sample observed. + pub min: f64, + /// Largest sample observed. + pub max: f64, + /// Mean of all samples. + pub mean: f64, +} + +impl SignalStatsInfo { + fn from_gauge(gauge: &SignalGauge) -> Self { + let s = &gauge.stats; + Self { + signal: gauge.name.clone(), + unit: gauge.unit.clone(), + count: s.count(), + last: s.last(), + min: s.min(), + max: s.max(), + mean: s.mean(), + } + } +} + impl StageProfilingInfo { fn from_entry(stage_type: &str, index: usize, entry: &StageEntry) -> Self { let m = &entry.metrics; @@ -64,4 +102,12 @@ impl RecordProfilingMetrics { } out } + + /// Returns a serializable snapshot of every registered signal gauge. + pub fn signal_snapshot(&self) -> Vec { + self.signals() + .iter() + .map(SignalStatsInfo::from_gauge) + .collect() + } } diff --git a/aimdb-core/src/profiling/mod.rs b/aimdb-core/src/profiling/mod.rs index 02b8baa..2f0b9fb 100644 --- a/aimdb-core/src/profiling/mod.rs +++ b/aimdb-core/src/profiling/mod.rs @@ -17,10 +17,12 @@ mod info; mod record_profiling; +mod signal_stats; mod stage_metrics; -pub use info::StageProfilingInfo; -pub use record_profiling::{RecordProfilingMetrics, StageEntry}; +pub use info::{SignalStatsInfo, StageProfilingInfo}; +pub use record_profiling::{RecordProfilingMetrics, SignalGauge, StageEntry}; +pub use signal_stats::SignalStats; pub use stage_metrics::StageMetrics; use alloc::{boxed::Box, sync::Arc}; diff --git a/aimdb-core/src/profiling/record_profiling.rs b/aimdb-core/src/profiling/record_profiling.rs index b524c4e..cf1840d 100644 --- a/aimdb-core/src/profiling/record_profiling.rs +++ b/aimdb-core/src/profiling/record_profiling.rs @@ -2,7 +2,7 @@ use alloc::{string::String, sync::Arc, vec::Vec}; -use crate::profiling::StageMetrics; +use crate::profiling::{SignalStats, StageMetrics}; use crate::StageKind; /// One registered stage: its shared metrics plus an optional human-readable name @@ -24,6 +24,18 @@ impl StageEntry { } } +/// One registered signal gauge: its label/unit plus the shared statistics that +/// `.observe()` folds `Observable::signal()` into (design 041 §3.2). +#[derive(Debug)] +pub struct SignalGauge { + /// Signal label (`Observable::SIGNAL`, defaults to the schema name). + pub name: String, + /// Unit label (`Observable::UNIT`, e.g. `"°C"`). + pub unit: String, + /// Shared cumulative statistics (last/min/max/mean). + pub stats: Arc, +} + /// All stage profiling metrics for a single record, indexed by registration order /// within each stage kind (`sources[0]` is the first `.source()`, `taps[1]` the /// second `.tap()`, etc.). @@ -33,6 +45,7 @@ pub struct RecordProfilingMetrics { taps: Vec, links: Vec, transforms: Vec, + signals: Vec, } impl RecordProfilingMetrics { @@ -41,12 +54,13 @@ impl RecordProfilingMetrics { Self::default() } - /// `true` if no stages have been registered. + /// `true` if no stages and no signal gauges have been registered. pub fn is_empty(&self) -> bool { self.sources.is_empty() && self.taps.is_empty() && self.links.is_empty() && self.transforms.is_empty() + && self.signals.is_empty() } /// Registers a new source stage; returns its index and shared metrics handle. @@ -120,6 +134,23 @@ impl RecordProfilingMetrics { &self.transforms } + /// Registers a new signal gauge; returns its shared stats handle. Called by + /// `RecordRegistrar::signal_gauge` (which `.observe()` builds on). + pub fn push_signal_gauge(&mut self, name: &str, unit: &str) -> Arc { + let stats = Arc::new(SignalStats::new()); + self.signals.push(SignalGauge { + name: String::from(name), + unit: String::from(unit), + stats: stats.clone(), + }); + stats + } + + /// All registered signal gauges, in registration order. + pub fn signals(&self) -> &[SignalGauge] { + &self.signals + } + /// Assigns a name to a previously registered stage. No-op if `idx` is out of range. pub fn set_stage_name(&mut self, kind: StageKind, idx: usize, name: &str) { let vec = match kind { @@ -133,7 +164,7 @@ impl RecordProfilingMetrics { } } - /// Resets every stage's counters. + /// Resets every stage's counters and signal-gauge statistics. pub fn reset_all(&self) { for e in self .sources @@ -144,6 +175,9 @@ impl RecordProfilingMetrics { { e.metrics.reset(); } + for g in &self.signals { + g.stats.reset(); + } } } diff --git a/aimdb-core/src/profiling/signal_stats.rs b/aimdb-core/src/profiling/signal_stats.rs new file mode 100644 index 0000000..0b1a6f5 --- /dev/null +++ b/aimdb-core/src/profiling/signal_stats.rs @@ -0,0 +1,185 @@ +//! Per-record signal-gauge statistics (feature `observability`). +//! +//! Where [`StageMetrics`](crate::profiling::StageMetrics) times *how long* a +//! stage runs, `SignalStats` folds the *domain value* a record carries — +//! `Observable::signal()` fed in via `.observe()` — into last/min/max/count/mean. +//! It surfaces on the same introspection paths (`record.list`/`record.get`, +//! stage profiling) so a temperature's live °C shows up next to its timing. + +use core::sync::atomic::Ordering; +use portable_atomic::AtomicU64; + +/// Cumulative statistics for one domain signal. +/// +/// Values are `f64`; each field is an `AtomicU64` holding the IEEE-754 bits, so +/// updates are lock-free on every target (no `AtomicF64` feature needed). +/// Updated with `Ordering::Relaxed` — these are diagnostics, not synchronization +/// primitives; concurrent `.observe()` taps (should there be more than one) may +/// call [`update`](Self::update) at once. +#[derive(Debug)] +pub struct SignalStats { + count: AtomicU64, + last_bits: AtomicU64, + min_bits: AtomicU64, + max_bits: AtomicU64, + sum_bits: AtomicU64, +} + +impl Default for SignalStats { + fn default() -> Self { + Self { + count: AtomicU64::new(0), + last_bits: AtomicU64::new(0), + // Seeded to ±∞ so the first real sample always wins the min/max CAS. + min_bits: AtomicU64::new(f64::INFINITY.to_bits()), + max_bits: AtomicU64::new(f64::NEG_INFINITY.to_bits()), + sum_bits: AtomicU64::new(0), + } + } +} + +impl SignalStats { + /// Creates empty stats (count 0). + pub fn new() -> Self { + Self::default() + } + + /// Records one signal sample. + pub fn update(&self, value: f64) { + self.last_bits.store(value.to_bits(), Ordering::Relaxed); + self.count.fetch_add(1, Ordering::Relaxed); + Self::accumulate(&self.sum_bits, |cur| cur + value); + Self::keep_if(&self.min_bits, value, |value, cur| value < cur); + Self::keep_if(&self.max_bits, value, |value, cur| value > cur); + } + + /// CAS loop applying `f` to the stored `f64`. + fn accumulate(cell: &AtomicU64, f: impl Fn(f64) -> f64) { + let mut cur = cell.load(Ordering::Relaxed); + loop { + let next = f(f64::from_bits(cur)).to_bits(); + match cell.compare_exchange_weak(cur, next, Ordering::Relaxed, Ordering::Relaxed) { + Ok(_) => break, + Err(actual) => cur = actual, + } + } + } + + /// CAS loop storing `value` while `keep(value, current)` holds. + fn keep_if(cell: &AtomicU64, value: f64, keep: impl Fn(f64, f64) -> bool) { + let mut cur = cell.load(Ordering::Relaxed); + while keep(value, f64::from_bits(cur)) { + match cell.compare_exchange_weak( + cur, + value.to_bits(), + Ordering::Relaxed, + Ordering::Relaxed, + ) { + Ok(_) => break, + Err(actual) => cur = actual, + } + } + } + + /// Number of samples recorded. + pub fn count(&self) -> u64 { + self.count.load(Ordering::Relaxed) + } + + /// Most recent sample (0.0 if none). + pub fn last(&self) -> f64 { + f64::from_bits(self.last_bits.load(Ordering::Relaxed)) + } + + /// Smallest sample seen (0.0 if none). + pub fn min(&self) -> f64 { + if self.count() == 0 { + 0.0 + } else { + f64::from_bits(self.min_bits.load(Ordering::Relaxed)) + } + } + + /// Largest sample seen (0.0 if none). + pub fn max(&self) -> f64 { + if self.count() == 0 { + 0.0 + } else { + f64::from_bits(self.max_bits.load(Ordering::Relaxed)) + } + } + + /// Mean of all samples (0.0 if none). + pub fn mean(&self) -> f64 { + let count = self.count(); + if count == 0 { + 0.0 + } else { + f64::from_bits(self.sum_bits.load(Ordering::Relaxed)) / count as f64 + } + } + + /// Clears all statistics back to their initial state. + pub fn reset(&self) { + self.count.store(0, Ordering::Relaxed); + self.last_bits.store(0, Ordering::Relaxed); + self.min_bits + .store(f64::INFINITY.to_bits(), Ordering::Relaxed); + self.max_bits + .store(f64::NEG_INFINITY.to_bits(), Ordering::Relaxed); + self.sum_bits.store(0, Ordering::Relaxed); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn empty_is_zero() { + let s = SignalStats::new(); + assert_eq!(s.count(), 0); + assert_eq!(s.last(), 0.0); + assert_eq!(s.min(), 0.0); + assert_eq!(s.max(), 0.0); + assert_eq!(s.mean(), 0.0); + } + + #[test] + fn tracks_last_min_max_mean() { + let s = SignalStats::new(); + for v in [10.0, 30.0, 20.0] { + s.update(v); + } + assert_eq!(s.count(), 3); + assert_eq!(s.last(), 20.0); + assert_eq!(s.min(), 10.0); + assert_eq!(s.max(), 30.0); + assert_eq!(s.mean(), 20.0); + } + + #[test] + fn handles_negative_values() { + let s = SignalStats::new(); + for v in [-5.0, -20.0, -1.0] { + s.update(v); + } + assert_eq!(s.min(), -20.0); + assert_eq!(s.max(), -1.0); + assert_eq!(s.last(), -1.0); + } + + #[test] + fn reset_clears_then_reusable() { + let s = SignalStats::new(); + s.update(5.0); + s.update(7.0); + s.reset(); + assert_eq!(s.count(), 0); + assert_eq!(s.min(), 0.0); + s.update(3.0); + assert_eq!(s.count(), 1); + assert_eq!(s.min(), 3.0); + assert_eq!(s.max(), 3.0); + } +} diff --git a/aimdb-core/src/remote/metadata.rs b/aimdb-core/src/remote/metadata.rs index 01c322e..fb2c842 100644 --- a/aimdb-core/src/remote/metadata.rs +++ b/aimdb-core/src/remote/metadata.rs @@ -86,6 +86,14 @@ pub struct RecordMetadata { #[cfg(feature = "observability")] #[serde(skip_serializing_if = "Option::is_none")] pub stage_profiling: Option>, + + // ===== Signal gauges (feature-gated) ===== + /// Per-record domain-signal statistics (last/min/max/mean) fed by + /// `Observable::observe()`, if the `observability` feature is enabled and any + /// gauge has been registered. + #[cfg(feature = "observability")] + #[serde(skip_serializing_if = "Option::is_none")] + pub signal_stats: Option>, } impl RecordMetadata { @@ -139,6 +147,8 @@ impl RecordMetadata { occupancy: None, #[cfg(feature = "observability")] stage_profiling: None, + #[cfg(feature = "observability")] + signal_stats: None, } } @@ -169,6 +179,15 @@ impl RecordMetadata { } self } + + /// Attaches a signal-gauge snapshot (observability feature only). + #[cfg(feature = "observability")] + pub fn with_signal_stats(mut self, signals: Vec) -> Self { + if !signals.is_empty() { + self.signal_stats = Some(signals); + } + self + } } #[cfg(test)] diff --git a/aimdb-core/src/signal.rs b/aimdb-core/src/signal.rs new file mode 100644 index 0000000..7827a1d --- /dev/null +++ b/aimdb-core/src/signal.rs @@ -0,0 +1,49 @@ +//! Signal gauge handle for `Observable::observe()` (design 041 §3.2). +//! +//! `RecordRegistrar::signal_gauge` hands back a [`SignalGaugeHandle`]. Feeding +//! values in with [`update`](SignalGaugeHandle::update) folds them into the +//! record's `SignalStats` (last/min/max/mean), which surface on `record.list` / +//! `record.get` and stage-profiling output. +//! +//! Like [`with_name`](crate::typed_api::RecordRegistrar::with_name), the handle +//! is **always available**: when the `observability` feature is off (or no gauge +//! was registered) it is inert and `update` is a no-op, so callers — including +//! the `.observe()` extension in `aimdb-data-contracts` — never `#[cfg]` on +//! core's features. + +/// A handle to a per-record signal gauge. +/// +/// Cheap to clone (an `Arc` bump, or nothing when inert). Obtained from +/// [`RecordRegistrar::signal_gauge`](crate::typed_api::RecordRegistrar::signal_gauge). +#[derive(Clone)] +pub struct SignalGaugeHandle { + #[cfg(feature = "observability")] + stats: Option>, +} + +impl SignalGaugeHandle { + /// An inert handle that records nothing (feature off, or no gauge registered). + #[cfg_attr(feature = "observability", allow(dead_code))] + pub(crate) fn inert() -> Self { + Self { + #[cfg(feature = "observability")] + stats: None, + } + } + + /// A live handle backed by shared statistics. + #[cfg(feature = "observability")] + pub(crate) fn live(stats: alloc::sync::Arc) -> Self { + Self { stats: Some(stats) } + } + + /// Records one signal sample. No-op on an inert handle. + pub fn update(&self, value: f64) { + #[cfg(feature = "observability")] + if let Some(stats) = &self.stats { + stats.update(value); + } + #[cfg(not(feature = "observability"))] + let _ = value; + } +} diff --git a/aimdb-core/src/typed_api.rs b/aimdb-core/src/typed_api.rs index ce8d35f..1fc8413 100644 --- a/aimdb-core/src/typed_api.rs +++ b/aimdb-core/src/typed_api.rs @@ -440,6 +440,33 @@ where self } + /// Registers a signal gauge on this record and returns a handle to feed it. + /// + /// Values pushed via [`SignalGaugeHandle::update`](crate::SignalGaugeHandle::update) + /// fold into per-record last/min/max/mean statistics that surface on + /// `record.list` / `record.get` and stage profiling. This is the core hook + /// behind `Observable::observe()` (design 041 §3.2). + /// + /// Always available, mirroring [`with_name`](Self::with_name): when the + /// `observability` feature is disabled it returns an inert handle whose + /// `update` is a no-op, so callers never `#[cfg]` on core's features. + pub fn signal_gauge( + &mut self, + name: &'static str, + unit: &'static str, + ) -> crate::signal::SignalGaugeHandle { + #[cfg(feature = "observability")] + { + let stats = self.rec.profiling_mut().push_signal_gauge(name, unit); + crate::signal::SignalGaugeHandle::live(stats) + } + #[cfg(not(feature = "observability"))] + { + let _ = (name, unit); + crate::signal::SignalGaugeHandle::inert() + } + } + /// Registers a producer service for this record type. /// /// The closure receives the [`RuntimeContext`](crate::RuntimeContext) diff --git a/aimdb-core/src/typed_record.rs b/aimdb-core/src/typed_record.rs index 3a503ad..a2ffc33 100644 --- a/aimdb-core/src/typed_record.rs +++ b/aimdb-core/src/typed_record.rs @@ -1183,6 +1183,10 @@ impl AnyRecord for TypedRecord { #[cfg(feature = "observability")] let metadata = metadata.with_stage_profiling(self.profiling.snapshot()); + // Attach signal-gauge statistics (from `.observe()`) when enabled. + #[cfg(feature = "observability")] + let metadata = metadata.with_signal_stats(self.profiling.signal_snapshot()); + metadata } diff --git a/aimdb-data-contracts/src/lib.rs b/aimdb-data-contracts/src/lib.rs index 17960b9..8140b56 100644 --- a/aimdb-data-contracts/src/lib.rs +++ b/aimdb-data-contracts/src/lib.rs @@ -75,7 +75,7 @@ mod linkable; mod observable; #[cfg(feature = "observable")] -pub use observable::log_tap; +pub use observable::{log_tap, ObservableRegistrarExt}; #[cfg(feature = "simulatable")] mod simulatable; @@ -153,52 +153,34 @@ pub trait Settable: SchemaType { // OBSERVABLE SUPPORT // ═══════════════════════════════════════════════════════════════════ -/// Extract a signal value for observation. +/// Project a schema type onto a numeric domain signal. /// -/// Implement this trait to enable threshold checking, alerting, -/// and other signal-based operations on your schema type. +/// The trait's kernel is the numeric projection: implement it, call +/// [`ObservableRegistrarExt::observe`](observable::ObservableRegistrarExt::observe), +/// and the signal is folded into live last/min/max/mean statistics that surface +/// on `record.list` / `record.get` and stage profiling. The signal can also feed +/// threshold checks, alerting, and aggregation. /// -/// The extracted signal can be used by node implementations to: -/// - Check against configured thresholds -/// - Trigger alerts when bounds are exceeded -/// - Compute aggregations (mean, min, max) -/// - Feed into monitoring systems -/// - Format log output with `format_log()` +/// Presentation lives elsewhere: `.log(node_id)` (also on the ext trait) formats +/// a human-readable line from `Debug` + [`SIGNAL`](Observable::SIGNAL)/[`UNIT`](Observable::UNIT), +/// and demo emoji live in the demo. pub trait Observable: SchemaType { /// The numeric type of the signal (e.g., `f32`, `f64`, `i32`). /// - /// Must be comparable and copyable for threshold checks. + /// Must be comparable and copyable for threshold checks. Bound + /// `Into` at the `.observe()` call site (not here) so exotic signals + /// can still implement `Observable` and write their own tap. type Signal: PartialOrd + Copy; - /// Icon/emoji for log output (e.g., "🌡️", "💧", "📊") - /// - /// Override this to provide a visual indicator for your data type. - const ICON: &'static str = "📊"; + /// What the signal means, for metrics/UI labels (e.g. `"celsius"`). + /// Defaults to the schema name. + const SIGNAL: &'static str = Self::NAME; - /// Unit label for the signal (e.g., "°C", "%", "hPa") - /// - /// Override this to display the appropriate unit in log output. + /// Unit label for the signal (e.g. `"°C"`, `"%"`, `"hPa"`). const UNIT: &'static str = ""; /// Extract the signal value from this instance. fn signal(&self) -> Self::Signal; - - /// Format a log entry for this observation. - /// - /// The default implementation uses `Debug` formatting. Override this - /// for prettier, human-readable output. - /// - /// # Example output - /// ```text - /// 🌡️ [alpha] Temperature: 22.5°C at 1704326400000 - /// 💧 [beta] Humidity: 65.3% at 1704326400000 - /// ``` - fn format_log(&self, node_id: &str) -> alloc::string::String - where - Self: core::fmt::Debug, - { - alloc::format!("{} [{}] {:?}", Self::ICON, node_id, self) - } } // ═══════════════════════════════════════════════════════════════════ diff --git a/aimdb-data-contracts/src/observable.rs b/aimdb-data-contracts/src/observable.rs index 105c35c..15c22e4 100644 --- a/aimdb-data-contracts/src/observable.rs +++ b/aimdb-data-contracts/src/observable.rs @@ -1,55 +1,94 @@ -//! Observable helper functions for data contracts. +//! Observable registrar extension + logging tap for data contracts. //! -//! Provides tap functions for logging observable data types. +//! Implementing [`Observable`](crate::Observable) unlocks one verb — +//! [`ObservableRegistrarExt::observe`] — which feeds the record's domain signal +//! into core signal-gauge metrics (design 041 §3.2). `.log(node_id)` is the +//! human-readable companion for console watching. extern crate alloc; +use aimdb_core::typed_api::RecordRegistrar; + use crate::Observable; // ═══════════════════════════════════════════════════════════════════ -// LOG TAP (feature = "observable") +// LOG TAP // ═══════════════════════════════════════════════════════════════════ -/// Generic logging tap for Observable types. -/// -/// This function can be used with any type implementing `Observable` to -/// log observations as they flow through the mesh. It uses `format_log()` -/// to produce human-readable output. -/// -/// # Example +/// Generic logging tap for [`Observable`] types. /// -/// ```no_run -/// use aimdb_data_contracts::log_tap; -/// # use aimdb_core::AimDbBuilder; -/// # use aimdb_data_contracts::{Observable, SchemaType}; -/// # #[derive(Clone, Debug)] -/// # struct Temperature { celsius: f32 } -/// # impl SchemaType for Temperature { const NAME: &'static str = "temperature"; } -/// # impl Observable for Temperature { -/// # type Signal = f32; -/// # fn signal(&self) -> f32 { self.celsius } -/// # } -/// # fn wire(builder: &mut AimDbBuilder) { -/// builder.configure::("node.alpha", |reg| { -/// // .buffer(BufferCfg::SingleLatest) — via your runtime adapter's ext trait -/// reg.tap(|ctx, consumer| log_tap(ctx, consumer, "alpha")); -/// }); -/// # } -/// ``` -#[cfg(feature = "observable")] +/// Formats each value from `Debug` plus the schema's +/// [`SIGNAL`](Observable::SIGNAL)/[`UNIT`](Observable::UNIT) labels. Prefer +/// [`ObservableRegistrarExt::log`]; this free function remains for hand-wired +/// `.tap()` calls. pub async fn log_tap( ctx: aimdb_core::RuntimeContext, consumer: aimdb_core::typed_api::Consumer, - node_id: &'static str, + node_id: &str, ) where T: Observable + Send + Sync + Clone + core::fmt::Debug + 'static, { let log = ctx.log(); - let mut reader = consumer.subscribe(); while let Ok(value) = reader.recv().await { - log.info(&value.format_log(node_id)); + log.info(&alloc::format!( + "[{}] {}: {:?}{}", + node_id, + T::SIGNAL, + value, + T::UNIT + )); + } +} + +// ═══════════════════════════════════════════════════════════════════ +// REGISTRAR EXTENSION: `.observe()` / `.log(node_id)` +// ═══════════════════════════════════════════════════════════════════ + +/// Adds `.observe()` and `.log(node_id)` to [`RecordRegistrar`] for +/// [`Observable`] types. +pub trait ObservableRegistrarExt<'a, T> +where + T: Observable + Send + Sync + Clone + core::fmt::Debug + 'static, +{ + /// Feed `T::signal()` into the record's signal gauge (last/min/max/mean), + /// visible via `record.list` / `record.get` / stage profiling. + /// + /// Bounded `T::Signal: Into` here (not on the trait), so `f32`/`i32`/ + /// `u32` signals qualify while a type with an exotic signal can still + /// implement `Observable` and write its own tap. + fn observe(&mut self) -> &mut RecordRegistrar<'a, T> + where + T::Signal: Into; + + /// Log each value to the runtime log, formatted from `Debug` + + /// `SIGNAL`/`UNIT`. For humans watching a console; `.observe()` is the + /// metrics path. + fn log(&mut self, node_id: &'static str) -> &mut RecordRegistrar<'a, T>; +} + +impl<'a, T> ObservableRegistrarExt<'a, T> for RecordRegistrar<'a, T> +where + T: Observable + Send + Sync + Clone + core::fmt::Debug + 'static, +{ + fn observe(&mut self) -> &mut RecordRegistrar<'a, T> + where + T::Signal: Into, + { + let gauge = self.signal_gauge(T::SIGNAL, T::UNIT); + self.tap(move |_ctx, consumer| async move { + let mut reader = consumer.subscribe(); + while let Ok(value) = reader.recv().await { + gauge.update(value.signal().into()); + } + }) + .with_name("observe") + } + + fn log(&mut self, node_id: &'static str) -> &mut RecordRegistrar<'a, T> { + self.tap(move |ctx, consumer| log_tap(ctx, consumer, node_id)) + .with_name("log") } } @@ -71,6 +110,7 @@ mod tests { impl Observable for TestSensor { type Signal = f32; + const UNIT: &'static str = "°C"; fn signal(&self) -> f32 { self.value @@ -78,54 +118,21 @@ mod tests { } #[test] - fn test_signal_extraction() { + fn signal_extraction() { let sensor = TestSensor { value: 42.5 }; assert_eq!(sensor.signal(), 42.5); } #[test] - fn test_threshold_comparison() { - let sensor = TestSensor { value: 25.0 }; - - // Above threshold - assert!(sensor.signal() > 20.0); - - // Below threshold - assert!(sensor.signal() < 30.0); - - // In range - let s = sensor.signal(); - assert!((20.0..=30.0).contains(&s)); - } - - #[test] - fn test_default_icon_and_unit() { - assert_eq!(TestSensor::ICON, "📊"); - assert_eq!(TestSensor::UNIT, ""); + fn signal_label_defaults_to_schema_name() { + assert_eq!(TestSensor::SIGNAL, "test_sensor"); + assert_eq!(TestSensor::UNIT, "°C"); } #[test] - fn test_format_log_default() { - #[derive(Debug)] - struct DebugSensor { - value: f32, - } - - impl SchemaType for DebugSensor { - const NAME: &'static str = "debug_sensor"; - } - - impl Observable for DebugSensor { - type Signal = f32; - fn signal(&self) -> f32 { - self.value - } - } - - let sensor = DebugSensor { value: 42.5 }; - let log = sensor.format_log("node1"); - assert!(log.contains("📊")); - assert!(log.contains("[node1]")); - assert!(log.contains("42.5")); + fn signal_is_into_f64() { + let sensor = TestSensor { value: 25.0 }; + let as_f64: f64 = sensor.signal().into(); + assert_eq!(as_f64, 25.0); } } diff --git a/aimdb-data-contracts/src/simulatable.rs b/aimdb-data-contracts/src/simulatable.rs index 070519f..53068b1 100644 --- a/aimdb-data-contracts/src/simulatable.rs +++ b/aimdb-data-contracts/src/simulatable.rs @@ -98,7 +98,11 @@ where /// Installs a **source**, so single-writer-per-key is enforced by `build()` /// exactly as for a hardware `.source()` — the two are mutually exclusive by /// the app's `#[cfg]`, never both present in one binary. - fn simulate(&mut self, profile: SimProfile, rng: R) -> &mut RecordRegistrar<'a, T> + fn simulate( + &mut self, + profile: SimProfile, + rng: R, + ) -> &mut RecordRegistrar<'a, T> where R: rand::Rng + Send + 'static; } diff --git a/examples/weather-mesh-demo/weather-hub/src/main.rs b/examples/weather-mesh-demo/weather-hub/src/main.rs index 20c3b39..94aefd8 100644 --- a/examples/weather-mesh-demo/weather-hub/src/main.rs +++ b/examples/weather-mesh-demo/weather-hub/src/main.rs @@ -1,5 +1,5 @@ use aimdb_core::{buffer::BufferCfg, AimDbBuilder, RecordKey}; -use aimdb_data_contracts::{log_tap, Linkable}; +use aimdb_data_contracts::{Linkable, ObservableRegistrarExt}; use aimdb_tokio_adapter::{TokioAdapter, TokioRecordRegistrarExt}; use weather_mesh_common::{Humidity, HumidityKey, TempKey, Temperature}; @@ -32,9 +32,12 @@ async fn main() -> aimdb_core::DbResult<()> { let topic = key.link_address().unwrap().to_string(); builder.configure::(key, |reg| { - reg.buffer(BufferCfg::SpmcRing { capacity: 100 }) - .tap(move |ctx, consumer| log_tap(ctx, consumer, key.as_str())) - .link_from(&topic) + reg.buffer(BufferCfg::SpmcRing { capacity: 100 }); + // Metrics: fold °C into the record's signal gauge (record.list/get). + reg.observe(); + // Console: one human-readable log line per value (the demo's point). + reg.log(key.as_str()); + reg.link_from(&topic) .with_deserializer(|ctx, data: &[u8]| { ctx.log() .debug("Deserializing temperature from MQTT payload"); @@ -55,9 +58,10 @@ async fn main() -> aimdb_core::DbResult<()> { let topic = key.link_address().unwrap().to_string(); builder.configure::(key, |reg| { - reg.buffer(BufferCfg::SpmcRing { capacity: 100 }) - .tap(move |ctx, consumer| log_tap(ctx, consumer, key.as_str())) - .link_from(&topic) + reg.buffer(BufferCfg::SpmcRing { capacity: 100 }); + // Metrics only for humidity; temperature keeps the console `.log()`. + reg.observe(); + reg.link_from(&topic) .with_deserializer(|ctx, data: &[u8]| { ctx.log().debug("Deserializing humidity from MQTT payload"); let humidity = Humidity::from_bytes(data)?; diff --git a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/dew_point.rs b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/dew_point.rs index d6124aa..c24dbb2 100644 --- a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/dew_point.rs +++ b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/dew_point.rs @@ -33,23 +33,11 @@ impl Streamable for DewPoint {} impl Observable for DewPoint { type Signal = f32; - const ICON: &'static str = "🌫️"; const UNIT: &'static str = "°C"; fn signal(&self) -> f32 { self.celsius } - - fn format_log(&self, node_id: &str) -> alloc::string::String { - alloc::format!( - "{} [{}] DewPoint: {:.1}{} at {}", - Self::ICON, - node_id, - self.celsius, - Self::UNIT, - self.timestamp - ) - } } #[cfg(feature = "linkable")] diff --git a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/humidity.rs b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/humidity.rs index 572747d..87e5c01 100644 --- a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/humidity.rs +++ b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/humidity.rs @@ -30,23 +30,11 @@ impl Streamable for Humidity {} impl Observable for Humidity { type Signal = f32; - const ICON: &'static str = "💧"; const UNIT: &'static str = "%"; fn signal(&self) -> f32 { self.percent } - - fn format_log(&self, node_id: &str) -> alloc::string::String { - alloc::format!( - "{} [{}] Humidity: {:.1}{} at {}", - Self::ICON, - node_id, - self.percent, - Self::UNIT, - self.timestamp - ) - } } #[cfg(feature = "simulatable")] diff --git a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/location.rs b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/location.rs index 41059a2..94926f1 100644 --- a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/location.rs +++ b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/location.rs @@ -41,34 +41,11 @@ impl Observable for GpsLocation { /// Ordering is lexicographic (lat first, then lon). type Signal = (f64, f64); - const ICON: &'static str = "📍"; const UNIT: &'static str = "°"; fn signal(&self) -> Self::Signal { (self.latitude, self.longitude) } - - fn format_log(&self, node_id: &str) -> alloc::string::String { - let alt_str = match self.altitude { - Some(alt) => alloc::format!(" alt={:.1}m", alt), - None => alloc::string::String::new(), - }; - let acc_str = match self.accuracy { - Some(acc) => alloc::format!(" acc=±{:.1}m", acc), - None => alloc::string::String::new(), - }; - alloc::format!( - "{} [{}] GpsLocation: {:.6}{}, {:.6}{}{}{}", - Self::ICON, - node_id, - self.latitude, - Self::UNIT, - self.longitude, - Self::UNIT, - alt_str, - acc_str - ) - } } #[cfg(feature = "simulatable")] diff --git a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/temperature.rs b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/temperature.rs index 0f850a5..43243ec 100644 --- a/examples/weather-mesh-demo/weather-mesh-common/src/contracts/temperature.rs +++ b/examples/weather-mesh-demo/weather-mesh-common/src/contracts/temperature.rs @@ -168,23 +168,11 @@ aimdb_data_contracts::migration_chain! { impl Observable for TemperatureV2 { type Signal = f32; - const ICON: &'static str = "🌡️"; const UNIT: &'static str = "°C"; fn signal(&self) -> f32 { self.celsius } - - fn format_log(&self, node_id: &str) -> alloc::string::String { - alloc::format!( - "{} [{}] Temperature: {:.1}{} at {}", - Self::ICON, - node_id, - self.celsius, - Self::UNIT, - self.timestamp - ) - } } #[cfg(feature = "simulatable")] diff --git a/examples/weather-mesh-demo/weather-station-beta/src/main.rs b/examples/weather-mesh-demo/weather-station-beta/src/main.rs index 26cdfb3..ed7a7a3 100644 --- a/examples/weather-mesh-demo/weather-station-beta/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-beta/src/main.rs @@ -28,7 +28,10 @@ fn sim_rng() -> rand::rngs::StdRng { /// reading on the same 5 s cadence so the record still has exactly one writer /// and the production dependency graph stays sim-free (no `rand`). #[cfg(not(feature = "sim"))] -async fn read_temperature(ctx: aimdb_core::RuntimeContext, producer: aimdb_core::Producer) { +async fn read_temperature( + ctx: aimdb_core::RuntimeContext, + producer: aimdb_core::Producer, +) { loop { let now = unix_millis(&ctx); producer.produce(Temperature::new(20.0, now)); From e226a318c337bc74acd4f42fa9c5a25eba2cff6c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Tue, 7 Jul 2026 03:47:23 +0000 Subject: [PATCH 05/31] =?UTF-8?q?feat(data-contracts):=20LinkableRegistrar?= =?UTF-8?q?Ext=20+=20#[derive(Linkable)]=20(design=20041=20=C2=A73.3)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implementing `Linkable` today changed nothing — every example hand-wired `with_deserializer`/`with_serializer` closures. Fix: - `LinkableRegistrarExt::linked_from`/`linked_to` install `.link_from()`/ `.link_to()` with the codec defaulted to `T::from_bytes`/`T::to_bytes`; the raw builders remain the escape hatch for per-link options (QoS, topic providers/resolvers). - `linkable` feature now depends on `aimdb-core` (same wiring as `observable`) for the registrar/connector-builder types the ext needs. - `#[derive(Linkable)]` in `aimdb-derive` emits `serde_json::to_vec`/ `from_slice` — no_std + alloc compatible (serde_json here is alloc-only, no `std` required), removing hand-written JSON boilerplate. D1 caller: tokio-mqtt-connector-demo's TempOutdoor/TempServerRoom (outbound) and CommandKey::TempOutdoor (inbound) now use `#[derive(Linkable)]` + `.linked_to`/`.linked_from`; TempIndoor keeps the raw builder since it needs QoS/retain options the ext doesn't cover. mqtt-connector-demo-common gains an opt-in `data-contracts` feature (alloc-only) so the embassy demo's hand-rolled no_std JSON path stays untouched — it doesn't enable the new feature. Co-Authored-By: Claude Opus 4.8 --- Cargo.lock | 2 + aimdb-data-contracts/Cargo.toml | 15 +++-- aimdb-data-contracts/src/lib.rs | 33 +++++++---- aimdb-data-contracts/src/linkable.rs | 51 ++++++++++++++++- aimdb-data-contracts/tests/linkable_derive.rs | 49 ++++++++++++++++ aimdb-derive/src/lib.rs | 56 +++++++++++++++++++ .../mqtt-connector-demo-common/Cargo.toml | 10 ++++ .../mqtt-connector-demo-common/src/types.rs | 15 +++++ examples/tokio-mqtt-connector-demo/Cargo.toml | 6 ++ .../tokio-mqtt-connector-demo/src/main.rs | 29 +++++----- 10 files changed, 230 insertions(+), 36 deletions(-) create mode 100644 aimdb-data-contracts/tests/linkable_derive.rs diff --git a/Cargo.lock b/Cargo.lock index 569ca1f..e3844eb 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2516,6 +2516,7 @@ name = "mqtt-connector-demo-common" version = "0.1.0" dependencies = [ "aimdb-core", + "aimdb-data-contracts", "defmt 1.0.1", "serde", "serde_json", @@ -3711,6 +3712,7 @@ name = "tokio-mqtt-connector-demo" version = "1.1.0" dependencies = [ "aimdb-core", + "aimdb-data-contracts", "aimdb-mqtt-connector", "aimdb-tokio-adapter", "mqtt-connector-demo-common", diff --git a/aimdb-data-contracts/Cargo.toml b/aimdb-data-contracts/Cargo.toml index 7eb995d..03e127c 100644 --- a/aimdb-data-contracts/Cargo.toml +++ b/aimdb-data-contracts/Cargo.toml @@ -14,7 +14,9 @@ categories = ["embedded", "no-std"] default = ["std"] alloc = ["serde/alloc"] std = ["alloc", "serde/std", "serde_json?/std"] -linkable = ["alloc", "serde_json"] +# The ext trait needs core's registrar/connector-builder types (same wiring as +# `observable`); `dep:aimdb-derive` powers `#[derive(Linkable)]`. +linkable = ["alloc", "serde_json", "aimdb-core", "dep:aimdb-derive"] # Dev-tier — never a default feature. The ext trait needs core's registrar, # same coupling `observable` has. `rand` is the tracer CI uses to prove a # production graph is sim-free (design 041 §3.1.1). @@ -26,13 +28,14 @@ observable = ["alloc", "aimdb-core"] serde = { version = "1.0", default-features = false, features = ["derive"] } serde_json = { workspace = true, optional = true } -# migration_chain! proc-macro (build-time only; no target/runtime/no_std -# impact — see aimdb-derive::migration_chain). No cycle: aimdb-derive emits -# token paths to this crate but does not depend on it. +# migration_chain! + #[derive(Linkable)] proc-macros (build-time only; no +# target/runtime/no_std impact — see aimdb-derive). No cycle: aimdb-derive +# emits token paths to this crate but does not depend on it. aimdb-derive = { version = "0.2.0", path = "../aimdb-derive", optional = true } -# Optional dependencies for observable feature (log_tap function) -# Note: aimdb-core requires alloc feature for core functionality +# Optional: shared by the `linkable`, `observable`, and `simulatable` registrar +# ext traits (RecordRegistrar/RuntimeContext). Note: aimdb-core requires the +# alloc feature for core functionality. aimdb-core = { version = "1.1.0", path = "../aimdb-core", optional = true, default-features = false, features = [ "alloc", ] } diff --git a/aimdb-data-contracts/src/lib.rs b/aimdb-data-contracts/src/lib.rs index 8140b56..4101c9a 100644 --- a/aimdb-data-contracts/src/lib.rs +++ b/aimdb-data-contracts/src/lib.rs @@ -71,6 +71,18 @@ pub use streamable::Streamable; #[cfg(feature = "linkable")] mod linkable; +#[cfg(feature = "linkable")] +pub use linkable::LinkableRegistrarExt; + +/// `#[derive(Linkable)]` — see [`Linkable`] and `aimdb_derive::Linkable`'s docs. +/// +/// Re-exported under the same name as the trait, following the trait+derive +/// pairing convention (`serde::Serialize`, `aimdb_derive::RecordKey`) — proc-macro +/// derive names live in a separate namespace from traits, so this is not a +/// conflict. +#[cfg(feature = "linkable")] +pub use aimdb_derive::Linkable; + #[cfg(feature = "observable")] mod observable; @@ -189,26 +201,25 @@ pub trait Observable: SchemaType { /// Types that can be serialized/deserialized for connector links. /// -/// Implement this trait to enable `link_from` and `link_to` operations -/// in AimDB connectors (MQTT, KNX, etc.). This provides the wire format -/// for transporting schema types across network boundaries. +/// Implement this trait, then call +/// [`LinkableRegistrarExt::linked_from`](linkable::LinkableRegistrarExt::linked_from) / +/// [`linked_to`](linkable::LinkableRegistrarExt::linked_to) to wire `link_from` +/// / `link_to` in AimDB connectors (MQTT, KNX, etc.) with the codec defaulted to +/// `from_bytes`/`to_bytes`. This provides the wire format for transporting +/// schema types across network boundaries. /// /// # Example /// -/// Not compiled: the snippet needs `aimdb-core`'s builder, which this crate -/// only depends on under the `observable` feature — `linkable` alone has no -/// core dependency. +/// Not compiled: the snippet needs `aimdb-core`'s builder types in scope. /// /// ```rust,ignore -/// use aimdb_data_contracts::Linkable; +/// use aimdb_data_contracts::{Linkable, LinkableRegistrarExt}; /// use my_app::Temperature; // user-defined type implementing Linkable /// /// // In connector configuration: /// builder.configure::(NODE_ID, |reg| { -/// reg.buffer(BufferCfg::SingleLatest) -/// .link_from("mqtt://sensors/temperature") -/// .with_deserializer(Temperature::from_bytes) -/// .finish(); +/// reg.buffer(BufferCfg::SingleLatest); +/// reg.linked_from("mqtt://sensors/temperature"); /// }); /// ``` #[cfg(feature = "linkable")] diff --git a/aimdb-data-contracts/src/linkable.rs b/aimdb-data-contracts/src/linkable.rs index 17a8d09..ed19373 100644 --- a/aimdb-data-contracts/src/linkable.rs +++ b/aimdb-data-contracts/src/linkable.rs @@ -1,7 +1,52 @@ -//! Linkable trait implementations and tests. +//! Linkable registrar extension: one-line link verbs for connector wiring. //! -//! This module provides the wire format support for transporting schema types -//! across connector links (MQTT, KNX, etc.). +//! Implementing [`Linkable`](crate::Linkable) unlocks two verbs — +//! [`LinkableRegistrarExt::linked_from`] / [`linked_to`](LinkableRegistrarExt::linked_to) +//! — that install the raw `.link_from()`/`.link_to()` builders with the codec +//! defaulted to `T::from_bytes`/`T::to_bytes` (design 041 §3.3). The raw builders +//! remain the escape hatch for per-link options (QoS, topic providers/resolvers). + +use aimdb_core::connector::SerializeError; +use aimdb_core::typed_api::RecordRegistrar; + +use crate::Linkable; + +/// Adds `.linked_from(url)` and `.linked_to(url)` to [`RecordRegistrar`] for +/// [`Linkable`] types. +pub trait LinkableRegistrarExt<'a, T> +where + T: Linkable + Send + Sync + Clone + core::fmt::Debug + 'static, +{ + /// `.link_from(url)` with the codec defaulted to `T::from_bytes`. + fn linked_from(&mut self, url: &str) -> &mut RecordRegistrar<'a, T>; + + /// `.link_to(url)` with the codec defaulted to `T::to_bytes`. + /// + /// `Linkable::to_bytes`'s `String` error is mapped to + /// `SerializeError::InvalidData` — the connector layer's serializer error + /// type has no string detail (see design 041 §3.3, `CodecError` alignment + /// recorded as a follow-up). + fn linked_to(&mut self, url: &str) -> &mut RecordRegistrar<'a, T>; +} + +impl<'a, T> LinkableRegistrarExt<'a, T> for RecordRegistrar<'a, T> +where + T: Linkable + Send + Sync + Clone + core::fmt::Debug + 'static, +{ + fn linked_from(&mut self, url: &str) -> &mut RecordRegistrar<'a, T> { + self.link_from(url) + .with_deserializer(|_ctx, bytes| T::from_bytes(bytes)) + .finish() + } + + fn linked_to(&mut self, url: &str) -> &mut RecordRegistrar<'a, T> { + self.link_to(url) + .with_serializer(|_ctx, value: &T| { + value.to_bytes().map_err(|_| SerializeError::InvalidData) + }) + .finish() + } +} #[cfg(test)] mod tests { diff --git a/aimdb-data-contracts/tests/linkable_derive.rs b/aimdb-data-contracts/tests/linkable_derive.rs new file mode 100644 index 0000000..9343406 --- /dev/null +++ b/aimdb-data-contracts/tests/linkable_derive.rs @@ -0,0 +1,49 @@ +//! Behavioral coverage for `#[derive(Linkable)]`. +//! +//! Lives under `tests/` (a separate crate), not `#[cfg(test)] mod tests` +//! inside `src/linkable.rs`: the derive emits absolute +//! `::aimdb_data_contracts::...` paths (matching the `migration_chain!` / +//! `RecordKey` precedent), which only resolve from outside the defining crate. + +#![cfg(feature = "linkable")] + +use aimdb_data_contracts::{Linkable, SchemaType}; +use serde::{Deserialize, Serialize}; + +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Linkable)] +struct DerivedTemperature { + celsius: f32, + timestamp: u64, +} + +impl SchemaType for DerivedTemperature { + const NAME: &'static str = "derived_temperature"; +} + +#[test] +fn roundtrips_through_json() { + let temp = DerivedTemperature { + celsius: 22.5, + timestamp: 1704326400000, + }; + + let bytes = temp.to_bytes().expect("serialization should succeed"); + let restored = DerivedTemperature::from_bytes(&bytes).expect("deserialization should succeed"); + + assert_eq!(temp, restored); +} + +#[test] +fn parses_hand_written_json() { + let json = br#"{"celsius": 25.0, "timestamp": 1704326400000}"#; + let temp = DerivedTemperature::from_bytes(json).expect("should parse valid JSON"); + + assert_eq!(temp.celsius, 25.0); + assert_eq!(temp.timestamp, 1704326400000); +} + +#[test] +fn rejects_invalid_json() { + let invalid = b"not valid json"; + assert!(DerivedTemperature::from_bytes(invalid).is_err()); +} diff --git a/aimdb-derive/src/lib.rs b/aimdb-derive/src/lib.rs index 421fbd1..bf02fec 100644 --- a/aimdb-derive/src/lib.rs +++ b/aimdb-derive/src/lib.rs @@ -41,6 +41,62 @@ pub fn migration_chain(input: TokenStream) -> TokenStream { migration_chain::migration_chain(input) } +/// Derive `Linkable` (JSON codec) for a schema type. +/// +/// Emits `from_bytes`/`to_bytes` via `serde_json::from_slice`/`to_vec` — the +/// boilerplate every hand-written `impl Linkable` for a JSON-wire type repeats +/// (design 041 §3.3). Requires `T: Serialize + DeserializeOwned` (a normal +/// compile error surfaces if it's missing) and the `linkable` feature of +/// `aimdb-data-contracts` (for the `Linkable` trait and its `__private::serde_json` +/// re-export). Binary formats (e.g. KNX DPT codecs) still implement `Linkable` +/// by hand — this derive is JSON-only. +/// +/// # Example +/// +/// Illustrative (not compiled: see the crate-level note — compiled +/// integration tests live in `aimdb-data-contracts`). +/// +/// ```rust,ignore +/// use aimdb_data_contracts::Linkable; +/// use serde::{Deserialize, Serialize}; +/// +/// #[derive(Clone, Debug, Serialize, Deserialize, Linkable)] +/// struct Temperature { +/// celsius: f32, +/// timestamp: u64, +/// } +/// ``` +#[proc_macro_derive(Linkable)] +pub fn derive_linkable(input: TokenStream) -> TokenStream { + let input = parse_macro_input!(input as DeriveInput); + let name = &input.ident; + + let expanded = quote! { + impl ::aimdb_data_contracts::Linkable for #name { + fn from_bytes( + data: &[u8], + ) -> Result { + ::aimdb_data_contracts::__private::serde_json::from_slice(data).map_err(|e| { + ::aimdb_data_contracts::__private::alloc::string::ToString::to_string(&e) + }) + } + + fn to_bytes( + &self, + ) -> Result< + ::aimdb_data_contracts::__private::alloc::vec::Vec, + ::aimdb_data_contracts::__private::alloc::string::String, + > { + ::aimdb_data_contracts::__private::serde_json::to_vec(self).map_err(|e| { + ::aimdb_data_contracts::__private::alloc::string::ToString::to_string(&e) + }) + } + } + }; + + expanded.into() +} + /// Derive the `RecordKey` trait for an enum /// /// Each variant must have a `#[key = "..."]` attribute specifying its string key. diff --git a/examples/mqtt-connector-demo-common/Cargo.toml b/examples/mqtt-connector-demo-common/Cargo.toml index b4a8fde..d715918 100644 --- a/examples/mqtt-connector-demo-common/Cargo.toml +++ b/examples/mqtt-connector-demo-common/Cargo.toml @@ -15,6 +15,12 @@ derive = ["aimdb-core/derive"] # needed: the no_std JSON paths are hand-rolled), so an embassy demo can expose # these records over a remote-access (serial) server. serde = ["dep:serde"] +# `SchemaType` + `#[derive(Linkable)]` (design 041 §3.3). `no_std + alloc` +# compatible (aimdb-data-contracts's `linkable` feature only needs `alloc` — +# its `serde_json` dep is alloc-only, no `std`). Not enabled by default: the +# embassy demo's hand-rolled no_std JSON path is an intentional illustration, +# left untouched; only the tokio demo opts into this feature. +data-contracts = ["alloc", "serde", "dep:aimdb-data-contracts"] # Embassy-specific features (no_std) defmt = ["dep:defmt"] @@ -25,6 +31,10 @@ aimdb-core = { path = "../../aimdb-core", default-features = false } # Serialization (std only) serde = { workspace = true, features = ["derive"], optional = true } serde_json = { workspace = true, optional = true } +# Optional: SchemaType + #[derive(Linkable)] (feature `data-contracts`, no_std + alloc) +aimdb-data-contracts = { path = "../../aimdb-data-contracts", default-features = false, optional = true, features = [ + "linkable", +] } # Optional defmt for embedded logging defmt = { workspace = true, optional = true } diff --git a/examples/mqtt-connector-demo-common/src/types.rs b/examples/mqtt-connector-demo-common/src/types.rs index a1c3fac..76df37b 100644 --- a/examples/mqtt-connector-demo-common/src/types.rs +++ b/examples/mqtt-connector-demo-common/src/types.rs @@ -9,6 +9,9 @@ use alloc::string::String; #[cfg(feature = "serde")] use serde::{Deserialize, Serialize}; +#[cfg(feature = "data-contracts")] +use aimdb_data_contracts::{Linkable, SchemaType}; + // ============================================================================ // TEMPERATURE READING // ============================================================================ @@ -18,6 +21,7 @@ use serde::{Deserialize, Serialize}; /// Represents a temperature measurement that can be published to MQTT. #[derive(Clone, Debug)] #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))] +#[cfg_attr(feature = "data-contracts", derive(Linkable))] pub struct Temperature { /// Sensor identifier (e.g., "indoor-001", "outdoor-001") pub sensor_id: String, @@ -25,6 +29,11 @@ pub struct Temperature { pub celsius: f32, } +#[cfg(feature = "data-contracts")] +impl SchemaType for Temperature { + const NAME: &'static str = "mqtt_demo.temperature"; +} + impl Temperature { /// Create a new temperature reading pub fn new(sensor_id: &str, celsius: f32) -> Self { @@ -81,6 +90,7 @@ impl Temperature { /// Command for controlling temperature sensors (inbound from MQTT) #[derive(Clone, Debug)] #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))] +#[cfg_attr(feature = "data-contracts", derive(Linkable))] pub struct TemperatureCommand { /// Action to perform (e.g., "read", "calibrate", "reset") pub action: String, @@ -88,6 +98,11 @@ pub struct TemperatureCommand { pub sensor_id: String, } +#[cfg(feature = "data-contracts")] +impl SchemaType for TemperatureCommand { + const NAME: &'static str = "mqtt_demo.temperature_command"; +} + impl TemperatureCommand { /// Create a new temperature command pub fn new(action: &str, sensor_id: &str) -> Self { diff --git a/examples/tokio-mqtt-connector-demo/Cargo.toml b/examples/tokio-mqtt-connector-demo/Cargo.toml index ec64d16..e0e364a 100644 --- a/examples/tokio-mqtt-connector-demo/Cargo.toml +++ b/examples/tokio-mqtt-connector-demo/Cargo.toml @@ -23,6 +23,12 @@ aimdb-tokio-adapter = { path = "../../aimdb-tokio-adapter", features = [ mqtt-connector-demo-common = { path = "../mqtt-connector-demo-common", features = [ "std", "derive", + "data-contracts", +] } + +# LinkableRegistrarExt / #[derive(Linkable)] verb (design 041 §3.3) +aimdb-data-contracts = { path = "../../aimdb-data-contracts", features = [ + "linkable", ] } # MQTT connector diff --git a/examples/tokio-mqtt-connector-demo/src/main.rs b/examples/tokio-mqtt-connector-demo/src/main.rs index d6dfbb0..b9d0344 100644 --- a/examples/tokio-mqtt-connector-demo/src/main.rs +++ b/examples/tokio-mqtt-connector-demo/src/main.rs @@ -36,6 +36,7 @@ use aimdb_core::buffer::BufferCfg; use aimdb_core::{AimDbBuilder, DbResult, Producer, RecordKey, RuntimeContext}; +use aimdb_data_contracts::LinkableRegistrarExt; use aimdb_mqtt_connector::{MqttLinkExt, MqttOutboundLinkExt}; use aimdb_tokio_adapter::{TokioAdapter, TokioRecordRegistrarExt}; use std::sync::Arc; @@ -139,9 +140,9 @@ async fn main() -> DbResult<()> { ); // Temperature sensors (outbound to MQTT) - using link_address() from key metadata. - // The MQTT knobs come from the connector crate's MqttLinkExt/ - // MqttOutboundLinkExt extension traits; QoS 1 / no - // retain matches the connector defaults. + // TempIndoor needs per-link QoS/retain, so it stays on the raw builder (the + // escape hatch, design 041 §3.3); TempOutdoor/TempServerRoom need no extra + // options, so `.linked_to()` (from `#[derive(Linkable)]`) is the 80% path. builder.configure::(SensorKey::TempIndoor, |reg| { reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) .source(indoor_temp_producer) @@ -156,22 +157,20 @@ async fn main() -> DbResult<()> { builder.configure::(SensorKey::TempOutdoor, |reg| { reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) .source(outdoor_temp_producer) - .tap(temperature_logger) - .link_to(SensorKey::TempOutdoor.link_address().unwrap()) - .with_serializer(|_ctx, temp: &Temperature| Ok(temp.to_json_vec())) - .finish(); + .tap(temperature_logger); + reg.linked_to(SensorKey::TempOutdoor.link_address().unwrap()); }); builder.configure::(SensorKey::TempServerRoom, |reg| { reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) .source(server_room_temp_producer) - .tap(temperature_logger) - .link_to(SensorKey::TempServerRoom.link_address().unwrap()) - .with_serializer(|_ctx, temp: &Temperature| Ok(temp.to_json_vec())) - .finish(); + .tap(temperature_logger); + reg.linked_to(SensorKey::TempServerRoom.link_address().unwrap()); }); - // Command consumers (inbound from MQTT) - using link_address() from key metadata + // Command consumers (inbound from MQTT) - using link_address() from key metadata. + // TempIndoor needs subscribe QoS, so it stays on the raw builder; TempOutdoor + // uses `.linked_from()`. builder.configure::(CommandKey::TempIndoor, |reg| { reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) .tap(command_consumer) @@ -183,10 +182,8 @@ async fn main() -> DbResult<()> { builder.configure::(CommandKey::TempOutdoor, |reg| { reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) - .tap(command_consumer) - .link_from(CommandKey::TempOutdoor.link_address().unwrap()) - .with_deserializer(|_ctx, data: &[u8]| TemperatureCommand::from_json(data)) - .finish(); + .tap(command_consumer); + reg.linked_from(CommandKey::TempOutdoor.link_address().unwrap()); }); println!("Subscribe: mosquitto_sub -h localhost -t 'sensors/#' -v"); From 55b7d38eba663561c190fa298af706309c5b3b0e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Tue, 7 Jul 2026 04:23:45 +0000 Subject: [PATCH 06/31] =?UTF-8?q?feat(sync):=20Settable=20->=20SyncProduce?= =?UTF-8?q?r::set=5Fvalue=20family=20(design=20041=20=C2=A73.4)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `Settable` was built for the sync bridge, but `aimdb-sync` never referenced it — `SyncProducer::set(value: T)` takes a fully constructed record, so every outside-the-thread caller hand-assembled the struct. - `Settable` moves behind a `settable` feature in `aimdb-data-contracts` for tier symmetry with the other wire contracts (pure trait, no dependencies of its own). weather-mesh-common enables it unconditionally (used by all three contracts); codegen's `generate_cargo_toml`/import emission gained the same `has_settable` detection already used for `has_observable`. - `aimdb-sync` gains a `data-contracts` feature -> optional `aimdb-data-contracts/settable` dep (dependency direction stays contracts -> sync, no `sync` feature added to the contracts crate). `SyncProducer::set_value`/`try_set_value`/`set_value_at` construct via `T::set(value, timestamp)` and send; `set_value`/ `try_set_value` stamp the caller's `SystemTime`, `set_value_at` takes an explicit timestamp for replay/testing. - Doctest + integration tests (aimdb-sync/tests/settable_integration.rs) exercise `set_value`/`try_set_value`/`set_value_at` end-to-end (construct -> produce -> consume). Co-Authored-By: Claude Opus 4.8 --- Cargo.lock | 1 + aimdb-codegen/src/rust.rs | 7 + aimdb-data-contracts/Cargo.toml | 5 +- aimdb-data-contracts/src/lib.rs | 5 +- aimdb-sync/Cargo.toml | 10 ++ aimdb-sync/src/producer.rs | 75 ++++++++++ aimdb-sync/tests/settable_integration.rs | 140 ++++++++++++++++++ .../weather-mesh-common/Cargo.toml | 4 +- 8 files changed, 244 insertions(+), 3 deletions(-) create mode 100644 aimdb-sync/tests/settable_integration.rs diff --git a/Cargo.lock b/Cargo.lock index e3844eb..8393bdb 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -282,6 +282,7 @@ name = "aimdb-sync" version = "0.5.0" dependencies = [ "aimdb-core", + "aimdb-data-contracts", "aimdb-tokio-adapter", "serde", "serde_json", diff --git a/aimdb-codegen/src/rust.rs b/aimdb-codegen/src/rust.rs index 37479ba..b208eb8 100644 --- a/aimdb-codegen/src/rust.rs +++ b/aimdb-codegen/src/rust.rs @@ -132,11 +132,18 @@ pub fn generate_cargo_toml(state: &ArchitectureState) -> String { .iter() .any(|r| r.serialization.as_ref() == Some(&SerializationType::Postcard)); let has_observable = state.records.iter().any(|r| r.observable.is_some()); + let has_settable = state + .records + .iter() + .any(|r| r.fields.iter().any(|f| f.settable)); let mut data_contracts_features = Vec::new(); if has_non_custom_ser { data_contracts_features.push("\"linkable\""); } + if has_settable { + data_contracts_features.push("\"settable\""); + } let dc_features_str = if data_contracts_features.is_empty() { String::new() diff --git a/aimdb-data-contracts/Cargo.toml b/aimdb-data-contracts/Cargo.toml index 03e127c..7fcaf7d 100644 --- a/aimdb-data-contracts/Cargo.toml +++ b/aimdb-data-contracts/Cargo.toml @@ -6,7 +6,7 @@ authors.workspace = true license.workspace = true repository.workspace = true homepage.workspace = true -description = "Trait definitions for AimDB data contracts: SchemaType, Streamable, Observable, Linkable, Simulatable, Migratable" +description = "Trait definitions for AimDB data contracts: SchemaType, Streamable, Observable, Linkable, Simulatable, Migratable, Settable" keywords = ["aimdb", "iot", "edge", "schema", "contracts"] categories = ["embedded", "no-std"] @@ -23,6 +23,9 @@ linkable = ["alloc", "serde_json", "aimdb-core", "dep:aimdb-derive"] simulatable = ["rand", "aimdb-core"] migratable = ["alloc", "serde_json", "dep:aimdb-derive"] observable = ["alloc", "aimdb-core"] +# Moved behind a feature for tier symmetry with the other wire contracts +# (design 041 §3.4). A pure trait — no dependencies of its own. +settable = [] [dependencies] serde = { version = "1.0", default-features = false, features = ["derive"] } diff --git a/aimdb-data-contracts/src/lib.rs b/aimdb-data-contracts/src/lib.rs index 4101c9a..474411f 100644 --- a/aimdb-data-contracts/src/lib.rs +++ b/aimdb-data-contracts/src/lib.rs @@ -148,7 +148,10 @@ pub trait SchemaType: Sized { /// Construct a schema instance from its primary value. /// -/// This defines the canonical way to create a new reading/measurement. +/// This defines the canonical way to create a new reading/measurement — the +/// write counterpart to [`Observable::signal`]'s read projection. The verb it +/// unlocks is `SyncProducer::set_value` in `aimdb-sync` (design 041 §3.4). +#[cfg(feature = "settable")] pub trait Settable: SchemaType { /// The primary value type (e.g., `f32` for temperature) type Value; diff --git a/aimdb-sync/Cargo.toml b/aimdb-sync/Cargo.toml index 671c113..b567ced 100644 --- a/aimdb-sync/Cargo.toml +++ b/aimdb-sync/Cargo.toml @@ -20,6 +20,13 @@ tokio = { version = "1.40", features = ["sync", "rt", "time", "macros"] } # Error handling thiserror = "1.0" +# Optional: Settable::set (feature `data-contracts`) for the `set_value` family. +# Dependency direction stays contracts -> sync; the contracts crate gains no +# `sync` feature (design 041 §3.4). +aimdb-data-contracts = { path = "../aimdb-data-contracts", version = "0.2.0", optional = true, default-features = false, features = [ + "settable", +] } + # For type erasure and Any [dev-dependencies] # Testing dependencies @@ -33,6 +40,9 @@ default = [] # Enable tracing for debugging tracing = ["aimdb-core/tracing"] +# SyncProducer::set_value / try_set_value / set_value_at (design 041 §3.4) +data-contracts = ["dep:aimdb-data-contracts"] + [package.metadata.docs.rs] all-features = true rustdoc-args = ["--cfg", "docsrs"] diff --git a/aimdb-sync/src/producer.rs b/aimdb-sync/src/producer.rs index 93efef1..33f4042 100644 --- a/aimdb-sync/src/producer.rs +++ b/aimdb-sync/src/producer.rs @@ -220,6 +220,81 @@ where } } +/// `Settable`-powered set-by-primitive verb (design 041 §3.4, feature `data-contracts`). +/// +/// `set()` takes a fully constructed `T`, so every outside-the-thread caller had +/// to hand-assemble the struct. `set_value` closes that gap: construct via +/// `T::set(value, timestamp)` and send, in one call. Distinct from AimX's +/// `record.set {name, value}` (full JSON value through `JsonCodec`) — this is +/// set-by-primitive. +#[cfg(feature = "data-contracts")] +impl SyncProducer +where + T: aimdb_data_contracts::Settable + Send + 'static + Debug + Clone, +{ + /// Construct via `T::set(value, now)` and send. Blocking, like [`set`](Self::set). + /// + /// Stamps with the *caller's* `SystemTime` (sample time at the edge), not + /// the engine's `ctx.time()` — use [`set_value_at`](Self::set_value_at) for + /// explicit-timestamp control (replay, testing). + /// + /// # Example + /// + /// ```no_run + /// # #[cfg(feature = "data-contracts")] + /// # fn main() -> Result<(), Box> { + /// use aimdb_core::AimDbBuilder; + /// use aimdb_data_contracts::{SchemaType, Settable}; + /// use aimdb_sync::AimDbBuilderSyncExt; + /// use aimdb_tokio_adapter::TokioAdapter; + /// use std::sync::Arc; + /// + /// #[derive(Debug, Clone)] + /// struct Temperature { celsius: f32, timestamp: u64 } + /// + /// impl SchemaType for Temperature { + /// const NAME: &'static str = "temperature"; + /// } + /// + /// impl Settable for Temperature { + /// type Value = f32; + /// fn set(value: f32, timestamp: u64) -> Self { + /// Temperature { celsius: value, timestamp } + /// } + /// } + /// + /// let handle = AimDbBuilder::new().runtime(Arc::new(TokioAdapter)).attach()?; + /// let producer = handle.producer::("temperature")?; + /// producer.set_value(22.5)?; // constructs Temperature::set(22.5, now_ms) and sends + /// # Ok(()) + /// # } + /// # #[cfg(not(feature = "data-contracts"))] + /// # fn main() {} + /// ``` + pub fn set_value(&self, value: T::Value) -> SyncResult<()> { + self.set(T::set(value, unix_now_ms())) + } + + /// Non-blocking variant, like [`try_set`](Self::try_set). + pub fn try_set_value(&self, value: T::Value) -> SyncResult<()> { + self.try_set(T::set(value, unix_now_ms())) + } + + /// Explicit-timestamp variant (replay, testing). + pub fn set_value_at(&self, value: T::Value, timestamp_ms: u64) -> SyncResult<()> { + self.set(T::set(value, timestamp_ms)) + } +} + +/// Current wall-clock time as Unix milliseconds (caller-side clock). +#[cfg(feature = "data-contracts")] +fn unix_now_ms() -> u64 { + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap_or_default() + .as_millis() as u64 +} + impl Clone for SyncProducer where T: Send + 'static + Debug + Clone, diff --git a/aimdb-sync/tests/settable_integration.rs b/aimdb-sync/tests/settable_integration.rs new file mode 100644 index 0000000..f6ebb6e --- /dev/null +++ b/aimdb-sync/tests/settable_integration.rs @@ -0,0 +1,140 @@ +//! Integration coverage for `SyncProducer::set_value` (feature `data-contracts`, +//! design 041 §3.4): construct via `Settable::set`, produce, and consume +//! end-to-end through the real sync bridge. + +#![cfg(feature = "data-contracts")] + +use aimdb_core::{buffer::BufferCfg, AimDbBuilder}; +use aimdb_data_contracts::{SchemaType, Settable}; +use aimdb_sync::AimDbBuilderSyncExt; +use aimdb_tokio_adapter::{TokioAdapter, TokioRecordRegistrarExt}; +use std::sync::Arc; +use std::thread; +use std::time::Duration; + +#[derive(Debug, Clone, PartialEq)] +struct Temperature { + celsius: f32, + timestamp: u64, +} + +impl SchemaType for Temperature { + const NAME: &'static str = "sync_settable.temperature"; +} + +impl Settable for Temperature { + type Value = f32; + + fn set(value: Self::Value, timestamp: u64) -> Self { + Temperature { + celsius: value, + timestamp, + } + } +} + +#[test] +fn set_value_constructs_produces_and_is_consumed() { + let adapter = Arc::new(TokioAdapter); + let mut builder = AimDbBuilder::new().runtime(adapter); + + builder.configure::("temperature", |reg| { + reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) + .tap(|_ctx, _consumer| async move {}); + }); + + let handle = builder.attach().expect("failed to attach"); + let producer = handle + .producer::("temperature") + .expect("failed to create producer"); + let consumer = handle + .consumer::("temperature") + .expect("failed to create consumer"); + + producer.set_value(22.5).expect("set_value should succeed"); + + let received = consumer + .get_with_timeout(Duration::from_secs(2)) + .expect("failed to consume"); + + assert_eq!(received.celsius, 22.5); + assert!(received.timestamp > 0, "timestamp should be stamped"); + + handle.detach().expect("failed to detach"); +} + +#[test] +fn try_set_value_is_non_blocking_and_produces() { + let adapter = Arc::new(TokioAdapter); + let mut builder = AimDbBuilder::new().runtime(adapter); + + builder.configure::("temperature", |reg| { + reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) + .tap(|_ctx, _consumer| async move {}); + }); + + let handle = builder.attach().expect("failed to attach"); + let producer = handle + .producer::("temperature") + .expect("failed to create producer"); + let consumer = handle + .consumer::("temperature") + .expect("failed to create consumer"); + + producer + .try_set_value(18.0) + .expect("try_set_value should succeed"); + + thread::sleep(Duration::from_millis(100)); + + let received = consumer + .get_with_timeout(Duration::from_secs(2)) + .expect("failed to consume"); + assert_eq!(received.celsius, 18.0); + + handle.detach().expect("failed to detach"); +} + +#[test] +fn set_value_at_stamps_the_explicit_timestamp() { + let adapter = Arc::new(TokioAdapter); + let mut builder = AimDbBuilder::new().runtime(adapter); + + builder.configure::("temperature", |reg| { + reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) + .tap(|_ctx, _consumer| async move {}); + }); + + let handle = builder.attach().expect("failed to attach"); + let producer = handle + .producer::("temperature") + .expect("failed to create producer"); + let consumer = handle + .consumer::("temperature") + .expect("failed to create consumer"); + + // Explicit timestamp, not wall-clock — deterministic regardless of when the + // test runs (replay/testing use case, design 041 §3.4). + producer + .set_value_at(22.5, 1_700_000_000_000) + .expect("set_value_at should succeed"); + + let received = consumer + .get_with_timeout(Duration::from_secs(2)) + .expect("failed to consume"); + + assert_eq!( + received, + Temperature { + celsius: 22.5, + timestamp: 1_700_000_000_000, + } + ); +} + +#[test] +fn settable_set_is_a_pure_deterministic_constructor() { + let a = Temperature::set(22.5, 1_700_000_000_000); + let b = Temperature::set(22.5, 1_700_000_000_000); + assert_eq!(a, b); +} diff --git a/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml b/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml index dac5446..5845976 100644 --- a/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml +++ b/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml @@ -18,7 +18,9 @@ aimdb-core = { path = "../../../aimdb-core", default-features = false, features "derive", "alloc", ] } -aimdb-data-contracts = { path = "../../../aimdb-data-contracts", default-features = false } +aimdb-data-contracts = { path = "../../../aimdb-data-contracts", default-features = false, features = [ + "settable", +] } serde = { version = "1.0", default-features = false, features = ["derive"] } serde_json = { workspace = true, optional = true } # RNG is caller-supplied at the station; the contract impls only use the From bbc447783464a5c5d08a3db747f3e3803b465c19 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Tue, 7 Jul 2026 04:56:26 +0000 Subject: [PATCH 07/31] =?UTF-8?q?docs:=20README=20verb/tier=20table=20+=20?= =?UTF-8?q?CHANGELOG=20sweep=20+=20version=20bumps=20(design=20041=20?= =?UTF-8?q?=C2=A76-7)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - README: the 4-trait capability table becomes the §2 verb/tier table (Contract / Implement when / Verb / Tier), documents the `Simulatable` dev-tier "never ships in prod" call-out with the `#[cfg]` sim-to-real pattern, and adds the `Settable` row. - CHANGELOG entries for all four touched crates (aimdb-data-contracts, aimdb-core, aimdb-derive, aimdb-sync) describing the trait reshapes, new registrar ext traits, the signal-gauge surface, and the derive macro. - Version bumps: aimdb-data-contracts 0.2.0 -> 0.3.0 (breaking: Simulatable/ Observable reshaped, Settable feature-gated), aimdb-core 1.1.0 -> 1.2.0 (additive: signal_gauge surface), aimdb-derive 0.2.0 -> 0.3.0 (additive: #[derive(Linkable)]), aimdb-sync 0.5.0 -> 0.6.0 (additive: set_value family). Dependents pinning an exact version (aimdb-sync, aimdb-wasm- adapter, aimdb-websocket-connector -> data-contracts; aimdb-core, aimdb-data-contracts -> aimdb-derive) updated to match; aimdb-core's own 0.x-style dependents needed no change (^1.1.0 already permits 1.2.0). Verified: `make check` and `make build` green in aimdb; `make check`/`make all` green in aimdb-pro against the bumped versions. Co-Authored-By: Claude Opus 4.8 --- Cargo.lock | 8 ++++---- README.md | 29 +++++++++++++++++++++------- aimdb-core/CHANGELOG.md | 7 ++++++- aimdb-core/Cargo.toml | 4 ++-- aimdb-data-contracts/CHANGELOG.md | 15 +++++++++++++- aimdb-data-contracts/Cargo.toml | 12 +++++++----- aimdb-derive/CHANGELOG.md | 1 + aimdb-derive/Cargo.toml | 2 +- aimdb-sync/CHANGELOG.md | 7 ++++++- aimdb-sync/Cargo.toml | 4 ++-- aimdb-wasm-adapter/Cargo.toml | 2 +- aimdb-websocket-connector/Cargo.toml | 2 +- 12 files changed, 67 insertions(+), 26 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 8393bdb..95910e3 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -96,7 +96,7 @@ dependencies = [ [[package]] name = "aimdb-core" -version = "1.1.0" +version = "1.2.0" dependencies = [ "aimdb-derive", "anyhow", @@ -119,7 +119,7 @@ dependencies = [ [[package]] name = "aimdb-data-contracts" -version = "0.2.0" +version = "0.3.0" dependencies = [ "aimdb-core", "aimdb-derive", @@ -131,7 +131,7 @@ dependencies = [ [[package]] name = "aimdb-derive" -version = "0.2.0" +version = "0.3.0" dependencies = [ "proc-macro2", "quote", @@ -279,7 +279,7 @@ dependencies = [ [[package]] name = "aimdb-sync" -version = "0.5.0" +version = "0.6.0" dependencies = [ "aimdb-core", "aimdb-data-contracts", diff --git a/README.md b/README.md index 97573ab..9370cc1 100644 --- a/README.md +++ b/README.md @@ -128,14 +128,29 @@ docker compose up | [**SingleLatest**](examples/hello-single-latest-async) | Only the current value matters | Feature flags, config, UI state | | [**Mailbox**](examples/hello-mailbox) / [**async Mailbox**](examples/hello-mailbox-async)| Latest instruction wins | Device commands, actuation, RPC | -**Four capability traits** — opt-in, type-checked: +**Capability traits** — opt-in, type-checked. Each one is a promise: implement it, and exactly one verb becomes available. The **tier** column says whether the capability may exist in a production binary. -| Trait | What it unlocks | Runtimes | -| --- | --- | --- | -| [`Streamable`](https://aimdb.dev/blog/streamable-crossing-boundaries) | Crossing WASM / WebSocket / CLI boundaries | std, no_std | -| [`Migratable`](https://aimdb.dev/blog/schema-migration-without-ceremony) | Typed schema evolution across deployed fleets | std, no_std | -| `Observable` | Automatic per-record metrics | std, no_std | -| [`Linkable`](https://aimdb.dev/blog/connectors-where-aimdb-meets-the-real-world) | Wire-format connectors | std, no_std | +| Contract | Implement when… | Verb it unlocks | Tier | +| --- | --- | --- | --- | +| [`Linkable`](https://aimdb.dev/blog/connectors-where-aimdb-meets-the-real-world) | the type crosses a per-URL byte boundary (MQTT/KNX/serial/UDS) | `.linked_from(url)` / `.linked_to(url)` (`#[derive(Linkable)]` for JSON) | wire (prod) | +| [`Streamable`](https://aimdb.dev/blog/streamable-crossing-boundaries) | the type streams as schema-named JSON (browser/WASM) | ws-connector `.register::()` | wire (prod) | +| [`Migratable`](https://aimdb.dev/blog/schema-migration-without-ceremony) | the schema evolved across versions | `migration_chain!` | wire (prod) | +| `Settable` | callers outside the AimDB thread set the record from a primitive | `SyncProducer::set_value(v)` | wire (prod) | +| `Observable` | the type carries a domain signal worth watching | `.observe()` → live signal metrics (last/min/max/mean on `record.list`/`record.get`) | introspection (prod, optional) | +| `Simulatable` | the type can generate realistic synthetic data | `.simulate(profile, rng)` | **dev-only — never ships in prod** | + +`Simulatable` is the one exception to "opt-in, always safe": it lives behind the `simulatable` feature (never a default), and sim-to-real selection is a compile-time `#[cfg]` in your app — + +```rust +builder.configure::(KEY, |reg| { + #[cfg(feature = "sim")] + reg.simulate(profile, rng); + #[cfg(not(feature = "sim"))] + reg.source(read_hardware); +}); +``` + +— so a production build with `sim` off carries zero simulation code: no `T::simulate` impls, no `rand`, nothing to audit. CI proves it (`rand` is the tracer: `cargo tree -e normal -i rand` on the production feature set must come up empty). **One async API across runtimes.** Tokio, Embassy, WASM — swap the runtime adapter, keep the code. → [How the runtime abstraction works](https://aimdb.dev/blog/building-aimdb-one-async-api) diff --git a/aimdb-core/CHANGELOG.md b/aimdb-core/CHANGELOG.md index 1f6e504..9e50392 100644 --- a/aimdb-core/CHANGELOG.md +++ b/aimdb-core/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **`RecordRegistrar::signal_gauge(name, unit) -> SignalGaugeHandle`** (design 041 §3.2) — the core hook behind `aimdb-data-contracts`'s `Observable::observe()`. Feeding values via `SignalGaugeHandle::update(f64)` folds them into per-record `SignalStats` (last/min/max/mean), surfaced through `RecordMetadata::signal_stats` on `record.list`/`record.get`. Mirrors the `with_name` precedent: always callable, an inert no-op handle when the `observability` feature is off, so callers never `#[cfg]` on core's features. New public types: `SignalGaugeHandle` (always available), `SignalStats`/`SignalGauge`/`SignalStatsInfo` (feature `observability`). + ### Fixed - **AimX protocol doc rot cleaned up; `remote::PROTOCOL_VERSION` corrected to `"2.0"` and exported.** The `remote` module docs claimed "AimX v1" and linked a spec file that no longer exists; they now describe the v2 NDJSON tagged-frame wire and point at `crate::session::aimx` / `docs/design/remote-access-via-connectors.md`. The AimX dispatch's Welcome uses the constant instead of a hardcoded `"2.0"` (same bytes on the wire). The dead, never-exported v1 `Message` untagged envelope and its helpers were removed from `remote::protocol`. Also de-advertised Kafka/HTTP connector semantics from `ConnectorUrl` docs (the parser is scheme-agnostic; those connectors never existed) and updated the `connector` module docs from the removed `.link()` API to `.link_to()`/`.link_from()`. @@ -452,7 +456,8 @@ warning fires if you exceed 1000 interned keys. --- -[Unreleased]: https://github.com/aimdb-dev/aimdb/compare/v1.1.0...HEAD +[Unreleased]: https://github.com/aimdb-dev/aimdb/compare/v1.2.0...HEAD +[1.2.0]: https://github.com/aimdb-dev/aimdb/compare/v1.1.0...v1.2.0 [1.1.0]: https://github.com/aimdb-dev/aimdb/compare/v1.0.0...v1.1.0 [1.0.0]: https://github.com/aimdb-dev/aimdb/compare/v0.5.0...v1.0.0 [0.5.0]: https://github.com/aimdb-dev/aimdb/compare/v0.4.0...v0.5.0 diff --git a/aimdb-core/Cargo.toml b/aimdb-core/Cargo.toml index 8d5bd2a..407999d 100644 --- a/aimdb-core/Cargo.toml +++ b/aimdb-core/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "aimdb-core" -version = "1.1.0" +version = "1.2.0" edition = "2021" license.workspace = true repository.workspace = true @@ -68,7 +68,7 @@ test-utils = ["std"] [dependencies] # Derive macros (optional) -aimdb-derive = { version = "0.2.0", path = "../aimdb-derive", optional = true } +aimdb-derive = { version = "0.3.0", path = "../aimdb-derive", optional = true } # Stream trait for bidirectional connectors (minimal, no_std compatible) futures-core = { version = "0.3", default-features = false } diff --git a/aimdb-data-contracts/CHANGELOG.md b/aimdb-data-contracts/CHANGELOG.md index 13362c8..c4d41cb 100644 --- a/aimdb-data-contracts/CHANGELOG.md +++ b/aimdb-data-contracts/CHANGELOG.md @@ -7,7 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -### Changed (breaking) +### Changed (breaking) — Design 041: capability traits as first-class verbs + +- **`Simulatable` reshaped for compile-time-only simulation.** `simulate(config: &SimulationConfig, ...)` → `simulate(params: &Self::Params, ...)` with a new associated `type Params`. `SimulationConfig` (including its runtime `enabled` gate) and `SimulationParams` are removed; `SimProfile

{ interval_ms, params }` and off-the-shelf `RandomWalkParams` replace them (`type Params = RandomWalkParams` is the migration path for existing scalar-walk impls). `simulatable` now also requires `aimdb-core` (for the new `SimulatableRegistrarExt`) and `rand` drops the `std_rng` feature (RNG is always caller-supplied). +- **`Observable` loses `ICON` and `format_log()`.** The trait is now numeric-projection-only: `type Signal`, `const SIGNAL` (defaults to `Self::NAME`), `const UNIT`, `fn signal()`. Presentation moved to `ObservableRegistrarExt::log(node_id)`. +- **`Settable` moves behind a new `settable` feature** (was compiled unconditionally) for tier symmetry with the other wire contracts. + +### Added + +- `SimulatableRegistrarExt::simulate(profile, rng)` — installs a `.source()` loop emitting `T::simulate(...)` on a timer. +- `ObservableRegistrarExt::observe()` — feeds `T::signal()` into a core signal gauge (last/min/max/mean), surfaced on `record.list`/`record.get`/stage profiling; `ObservableRegistrarExt::log(node_id)` for the console-logging path (replaces the old `format_log`). +- `LinkableRegistrarExt::linked_from(url)` / `linked_to(url)` — one-line `.link_from()`/`.link_to()` wiring defaulted to `T::from_bytes`/`T::to_bytes`. `linkable` now also requires `aimdb-core`. +- `#[derive(Linkable)]` (via `aimdb-derive`) — emits a JSON `Linkable` impl (`serde_json::to_vec`/`from_slice`), `no_std + alloc` compatible. + +### Changed - **`migratable` no longer requires `std`.** `migratable = ["alloc", "serde_json"]` (was `["std", "serde_json"]`); the crate's `serde_json` dependency now follows the workspace pin (`default-features = false, features = ["alloc"]`) instead of pulling in `serde_json/std` by default. `migration_chain!` and both `Linkable`-with-migration patterns now compile on `no_std + alloc` (e.g. `thumbv7em-none-eabihf`). A crate that enabled only `migratable` and relied on it transitively activating `std` will need to enable `std` explicitly. In-repo consumers are unaffected (they enable `std` by default). - `log_tap(ctx, consumer, node_id)` now takes `Consumer` instead of `Consumer` — `R` is still on `RuntimeContext` (Design 029, M14). User-visible signature shrink only; behaviour unchanged. diff --git a/aimdb-data-contracts/Cargo.toml b/aimdb-data-contracts/Cargo.toml index 7fcaf7d..874ea54 100644 --- a/aimdb-data-contracts/Cargo.toml +++ b/aimdb-data-contracts/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "aimdb-data-contracts" -version = "0.2.0" +version = "0.3.0" edition.workspace = true authors.workspace = true license.workspace = true @@ -34,12 +34,14 @@ serde_json = { workspace = true, optional = true } # migration_chain! + #[derive(Linkable)] proc-macros (build-time only; no # target/runtime/no_std impact — see aimdb-derive). No cycle: aimdb-derive # emits token paths to this crate but does not depend on it. -aimdb-derive = { version = "0.2.0", path = "../aimdb-derive", optional = true } +aimdb-derive = { version = "0.3.0", path = "../aimdb-derive", optional = true } # Optional: shared by the `linkable`, `observable`, and `simulatable` registrar -# ext traits (RecordRegistrar/RuntimeContext). Note: aimdb-core requires the -# alloc feature for core functionality. -aimdb-core = { version = "1.1.0", path = "../aimdb-core", optional = true, default-features = false, features = [ +# ext traits (RecordRegistrar/RuntimeContext). 1.2.0 is the true minimum: the +# `observable` feature's `.observe()` calls `RecordRegistrar::signal_gauge`, +# added in 1.2.0. Note: aimdb-core requires the alloc feature for core +# functionality. +aimdb-core = { version = "1.2.0", path = "../aimdb-core", optional = true, default-features = false, features = [ "alloc", ] } # RNG is caller-supplied, so we need only the `Rng`/`RngExt` traits and the diff --git a/aimdb-derive/CHANGELOG.md b/aimdb-derive/CHANGELOG.md index efd24cb..fb61963 100644 --- a/aimdb-derive/CHANGELOG.md +++ b/aimdb-derive/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **`#[derive(Linkable)]` proc-macro** (design 041 §3.3). Emits a JSON `aimdb_data_contracts::Linkable` impl (`serde_json::to_vec`/`from_slice` via the `__private` re-export, same foreign-crate-path pattern as `migration_chain!`) — replaces the hand-written `from_bytes`/`to_bytes` boilerplate every JSON-wire type repeated. `no_std + alloc` compatible; binary wire formats (e.g. KNX DPT codecs) still implement `Linkable` by hand. - **`migration_chain!` proc-macro.** Variable-arity replacement for the 3-arm `macro_rules!` previously hand-unrolled in `aimdb-data-contracts`; re-exported as `aimdb_data_contracts::migration_chain!` with the same grammar and call path. Generated dispatch is `O(N)` in code size regardless of chain length (one `__up_k`/`__down_k` helper per step). Emits foreign-crate paths into `aimdb_data_contracts` without depending on it (same pattern as `RecordKey` → `aimdb_core`) — a build-time-only dependency with no target/runtime/`no_std` impact. - **Tree-free version probe in `migrate_from_bytes`.** The generated dispatch no longer parses the payload into a full `serde_json::Value` tree to read the version field. It now scans a small `#[derive(serde::Deserialize)]` probe struct for just the version, then parses the same bytes a second time directly into the matched concrete type — peak allocation drops from O(payload tree) to O(concrete struct). `serde_json::Error::is_data()` preserves the existing `MissingVersion` vs `DeserializationFailed` split (missing/wrong-type field vs malformed JSON). Wire behavior is unchanged (same errors for missing/unknown versions). diff --git a/aimdb-derive/Cargo.toml b/aimdb-derive/Cargo.toml index 8835399..00d27d8 100644 --- a/aimdb-derive/Cargo.toml +++ b/aimdb-derive/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "aimdb-derive" -version = "0.2.0" +version = "0.3.0" edition.workspace = true authors.workspace = true license.workspace = true diff --git a/aimdb-sync/CHANGELOG.md b/aimdb-sync/CHANGELOG.md index a66e8ea..e37d2bf 100644 --- a/aimdb-sync/CHANGELOG.md +++ b/aimdb-sync/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **`SyncProducer::set_value`/`try_set_value`/`set_value_at`** (design 041 §3.4, feature `data-contracts`). `set()` took a fully constructed `T`, so every outside-the-thread caller hand-assembled the struct; `set_value(value)` constructs via `T::set(value, timestamp)` and sends in one call — blocking (`set_value`), non-blocking (`try_set_value`), or with an explicit timestamp for replay/testing (`set_value_at`). `set_value`/`try_set_value` stamp the caller's `SystemTime` (crate is std-only). New optional dependency: `aimdb-data-contracts` (feature `settable`), behind the new `data-contracts` feature — the contracts crate gains no `sync` feature (dependency direction unchanged). + ### Changed (breaking) - **Issue #131:** `AimDbSyncExt` extends the non-generic `aimdb_core::AimDb`; internal handles drop the `TokioAdapter` type parameter. @@ -60,7 +64,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 --- -[Unreleased]: https://github.com/aimdb-dev/aimdb/compare/v0.5.0...HEAD +[Unreleased]: https://github.com/aimdb-dev/aimdb/compare/v0.6.0...HEAD +[0.6.0]: https://github.com/aimdb-dev/aimdb/compare/v0.5.0...v0.6.0 [0.5.0]: https://github.com/aimdb-dev/aimdb/compare/v0.4.0...v0.5.0 [0.4.0]: https://github.com/aimdb-dev/aimdb/compare/v0.3.0...v0.4.0 [0.3.0]: https://github.com/aimdb-dev/aimdb/compare/v0.2.0...v0.3.0 diff --git a/aimdb-sync/Cargo.toml b/aimdb-sync/Cargo.toml index b567ced..8576093 100644 --- a/aimdb-sync/Cargo.toml +++ b/aimdb-sync/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "aimdb-sync" -version = "0.5.0" +version = "0.6.0" edition = "2021" authors.workspace = true license.workspace = true @@ -23,7 +23,7 @@ thiserror = "1.0" # Optional: Settable::set (feature `data-contracts`) for the `set_value` family. # Dependency direction stays contracts -> sync; the contracts crate gains no # `sync` feature (design 041 §3.4). -aimdb-data-contracts = { path = "../aimdb-data-contracts", version = "0.2.0", optional = true, default-features = false, features = [ +aimdb-data-contracts = { path = "../aimdb-data-contracts", version = "0.3.0", optional = true, default-features = false, features = [ "settable", ] } diff --git a/aimdb-wasm-adapter/Cargo.toml b/aimdb-wasm-adapter/Cargo.toml index 9209a88..9b516f7 100644 --- a/aimdb-wasm-adapter/Cargo.toml +++ b/aimdb-wasm-adapter/Cargo.toml @@ -35,7 +35,7 @@ aimdb-core = { version = "1.1.0", path = "../aimdb-core", default-features = fal aimdb-ws-protocol = { version = "0.1.0", path = "../aimdb-ws-protocol" } # Data contracts (alloc only — no std) -aimdb-data-contracts = { version = "0.2.0", path = "../aimdb-data-contracts", default-features = false, features = [ +aimdb-data-contracts = { version = "0.3.0", path = "../aimdb-data-contracts", default-features = false, features = [ "alloc", ] } diff --git a/aimdb-websocket-connector/Cargo.toml b/aimdb-websocket-connector/Cargo.toml index 7005c15..fe7a47a 100644 --- a/aimdb-websocket-connector/Cargo.toml +++ b/aimdb-websocket-connector/Cargo.toml @@ -33,7 +33,7 @@ tracing = ["dep:tracing"] [dependencies] aimdb-core = { version = "1.1.0", path = "../aimdb-core", default-features = false } -aimdb-data-contracts = { version = "0.2.0", path = "../aimdb-data-contracts", default-features = false } +aimdb-data-contracts = { version = "0.3.0", path = "../aimdb-data-contracts", default-features = false } aimdb-ws-protocol = { version = "0.1.0", path = "../aimdb-ws-protocol" } # Async runtime From a874372cc8cfa809360060214d880275ab9d4783 Mon Sep 17 00:00:00 2001 From: "sounds.like.lx" <147444674+lxsaah@users.noreply.github.com> Date: Tue, 7 Jul 2026 18:49:44 +0200 Subject: [PATCH 08/31] Claude/design doc 41 review s7hguj (#169) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(review): SignalStatsInfo serde symmetry, live hub gauge, gamma fmt Review follow-ups on the design 041 implementation: - SignalStatsInfo.unit: add #[serde(default)] to match its skip_serializing_if. Observable::UNIT defaults to "", so a gauge without a unit serialized record.list/record.get JSON that aimdb-client/aimdb-mcp (built with observability) could not deserialize back (missing field `unit`). Regression test added. - weather-hub: enable aimdb-tokio-adapter/observability so .observe() gets a live gauge instead of the inert handle — the demo now actually surfaces signal stats on record.list/record.get as its comments and design 041 §6 claim. - weather-station-gamma: rustfmt the non-sim read_temperature signature (make fmt-check was failing). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_017sJzkN61UVQboWuHVHwSKP * ci: make check-no-sim fail closed + tracer positive control The guard treated ANY non-zero 'cargo tree -i rand' exit as 'rand is absent'. An unrelated cargo failure — missing _external submodules, a typo'd package in SIM_EXAMPLES, registry trouble — therefore passed the check vacuously (observed: with submodules uninitialized, all four assertions 'passed' while cargo was erroring on embassy-executor). Now 'rand-free' is accepted only when cargo tree fails with its specific 'did not match any packages' error; any other failure aborts the check and prints cargo's output. A positive control per example additionally asserts the '--features sim' graph DOES find rand, so a silently broken tracer (e.g. rand dropped from the simulatable feature) can no longer turn the whole guard into a no-op. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_017sJzkN61UVQboWuHVHwSKP --------- Co-authored-by: Claude --- Makefile | 42 ++++++++++++------- aimdb-core/src/profiling/info.rs | 34 ++++++++++++++- .../weather-mesh-demo/weather-hub/Cargo.toml | 7 +++- .../weather-station-gamma/src/main.rs | 5 ++- 4 files changed, 71 insertions(+), 17 deletions(-) diff --git a/Makefile b/Makefile index 736b869..79cb0df 100644 --- a/Makefile +++ b/Makefile @@ -537,26 +537,40 @@ codegen-drift: # Prove that simulation code (design 041, dev tier) never reaches a production # binary. `rand` is the tracer: it is reachable iff `simulatable` is enabled -# (aimdb-data-contracts/src/simulatable.rs). `cargo tree -i rand` exits non-zero -# when `rand` is absent from the resolved graph, so a *successful* lookup here is -# a failure. Also asserts `simulatable` is not a default feature of the contracts -# crate (which would pull `rand` into its own default graph). +# (aimdb-data-contracts/src/simulatable.rs). The guard must not fail open: +# "rand is absent" is accepted only when `cargo tree -i rand` fails with its +# specific "did not match any packages" error — any other failure (missing +# submodule, typo'd package name, registry trouble) aborts the check instead of +# passing it vacuously. A positive control per example asserts the sim build +# DOES find `rand`, proving the tracer still traces. Also asserts `simulatable` +# is not a default feature of the contracts crate (which would pull `rand` into +# its own default graph). SIM_EXAMPLES := weather-station-beta weather-station-gamma check-no-sim: @printf "$(GREEN)Proving production graphs are simulation-free...$(NC)\n" - @for bin in $(SIM_EXAMPLES); do \ - if cargo tree -p $$bin -e normal -i rand >/dev/null 2>&1; then \ - printf "$(RED)✗ '$$bin' (default/production, no sim) pulls in 'rand' — simulation code leaked into production$(NC)\n"; \ + @tree_rand() { cargo tree -p "$$1" $$2 -e normal -i rand 2>&1; }; \ + assert_rand_free() { \ + if out=$$(tree_rand "$$1" "$$2"); then \ + printf "$(RED)✗ $$3$(NC)\n"; \ + exit 1; \ + elif ! printf '%s\n' "$$out" | grep -q 'did not match any packages'; then \ + printf "$(RED)✗ cargo tree for '$$1' failed for a reason other than 'rand is absent' — refusing to pass vacuously:$(NC)\n"; \ + printf '%s\n' "$$out"; \ exit 1; \ fi; \ + }; \ + for bin in $(SIM_EXAMPLES); do \ + assert_rand_free "$$bin" "" "'$$bin' (default/production, no sim) pulls in 'rand' — simulation code leaked into production"; \ printf "$(BLUE)✓ $$bin production graph is rand-free$(NC)\n"; \ - done - @if cargo tree -p aimdb-data-contracts -e normal -i rand >/dev/null 2>&1; then \ - printf "$(RED)✗ aimdb-data-contracts pulls 'rand' with default features — 'simulatable' must never be a default feature$(NC)\n"; \ - exit 1; \ - fi - @printf "$(BLUE)✓ 'simulatable' is not a default feature of aimdb-data-contracts$(NC)\n" - @printf "$(GREEN)✓ Production is simulation-free$(NC)\n" + if ! tree_rand "$$bin" "--features sim" >/dev/null; then \ + printf "$(RED)✗ positive control failed: '$$bin --features sim' does not pull 'rand' — the tracer no longer traces, so the rand-free results above prove nothing$(NC)\n"; \ + exit 1; \ + fi; \ + printf "$(BLUE)✓ $$bin sim graph finds rand (tracer positive control)$(NC)\n"; \ + done; \ + assert_rand_free "aimdb-data-contracts" "" "aimdb-data-contracts pulls 'rand' with default features — 'simulatable' must never be a default feature"; \ + printf "$(BLUE)✓ 'simulatable' is not a default feature of aimdb-data-contracts$(NC)\n"; \ + printf "$(GREEN)✓ Production is simulation-free$(NC)\n" ## Convenience commands check: fmt-check clippy test test-embedded test-wasm deny readme-check codegen-drift check-no-sim diff --git a/aimdb-core/src/profiling/info.rs b/aimdb-core/src/profiling/info.rs index 921a64a..b6a422c 100644 --- a/aimdb-core/src/profiling/info.rs +++ b/aimdb-core/src/profiling/info.rs @@ -40,7 +40,11 @@ pub struct SignalStatsInfo { /// Signal label (`Observable::SIGNAL`, defaults to the schema name). pub signal: String, /// Unit label (`Observable::UNIT`), omitted when empty. - #[serde(skip_serializing_if = "String::is_empty")] + /// + /// `default` matches the skip: `Observable::UNIT` defaults to `""`, so a + /// snapshot without a unit must deserialize (aimdb-client/aimdb-mcp read + /// this back from `record.list`/`record.get`). + #[serde(default, skip_serializing_if = "String::is_empty")] pub unit: String, /// Number of samples observed. pub count: u64, @@ -111,3 +115,31 @@ impl RecordProfilingMetrics { .collect() } } + +#[cfg(test)] +mod tests { + use super::*; + use alloc::string::ToString; + + /// `unit` is skipped when empty (`Observable::UNIT` defaults to `""`), so + /// the emitted JSON must deserialize back without it — aimdb-client and + /// aimdb-mcp read these snapshots from `record.list`/`record.get`. + #[test] + fn empty_unit_roundtrips() { + let info = SignalStatsInfo { + signal: "temperature".to_string(), + unit: String::new(), + count: 3, + last: 1.0, + min: 0.5, + max: 2.0, + mean: 1.2, + }; + + let json = serde_json::to_string(&info).unwrap(); + assert!(!json.contains("unit"), "empty unit must be omitted: {json}"); + + let back: SignalStatsInfo = serde_json::from_str(&json).unwrap(); + assert_eq!(back, info); + } +} diff --git a/examples/weather-mesh-demo/weather-hub/Cargo.toml b/examples/weather-mesh-demo/weather-hub/Cargo.toml index bec2f6d..e9a3e79 100644 --- a/examples/weather-mesh-demo/weather-hub/Cargo.toml +++ b/examples/weather-mesh-demo/weather-hub/Cargo.toml @@ -16,7 +16,12 @@ weather-mesh-common = { path = "../weather-mesh-common", features = [ "migratable", ] } aimdb-core = { path = "../../../aimdb-core", features = ["derive"] } -aimdb-tokio-adapter = { path = "../../../aimdb-tokio-adapter" } +# `observability` makes `.observe()`'s signal gauge live (design 041 §3.2); +# without it core hands back an inert handle and no signal stats surface on +# record.list/record.get. +aimdb-tokio-adapter = { path = "../../../aimdb-tokio-adapter", features = [ + "observability", +] } aimdb-data-contracts = { path = "../../../aimdb-data-contracts", features = [ "linkable", "observable", diff --git a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs index 2887638..98fe016 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs @@ -100,7 +100,10 @@ fn humidity_profile() -> SimProfile { /// reading so the record still has exactly one writer and the flashed image /// carries no `rand` (design 041 §3.1.4). #[cfg(not(feature = "sim"))] -async fn read_temperature(ctx: aimdb_core::RuntimeContext, producer: aimdb_core::Producer) { +async fn read_temperature( + ctx: aimdb_core::RuntimeContext, + producer: aimdb_core::Producer, +) { let log = ctx.log(); log.info("🌡️ Starting temperature sensor..."); loop { From 7dc703eba1ab708ec1f2fb3532e825d97d72d8c6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Tue, 7 Jul 2026 17:22:44 +0000 Subject: [PATCH 09/31] feat(tests): add serde_json as a dev-dependency for roundtrip tests --- aimdb-core/Cargo.toml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/aimdb-core/Cargo.toml b/aimdb-core/Cargo.toml index 407999d..45353a9 100644 --- a/aimdb-core/Cargo.toml +++ b/aimdb-core/Cargo.toml @@ -116,6 +116,9 @@ spin = { version = "0.9", default-features = false, features = ["mutex", "spin_m hashbrown = { version = "0.15", default-features = false, features = ["default-hasher"] } [dev-dependencies] +# Serde-attribute roundtrip tests (src/profiling/info.rs) must run in every +# feature config, including ones where the optional `serde_json` dep is off. +serde_json = { workspace = true } # For no_std testing heapless = "0.9.1" # For async testing (`sync` covers tokio::sync::mpsc in tests/session_engine.rs, From 145d321809a03982720de27ca6107c26afa11a97 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Tue, 7 Jul 2026 17:52:51 +0000 Subject: [PATCH 10/31] Refactor documentation and comments across multiple modules - Updated comments in `linkable.rs`, `observable.rs`, and `simulatable.rs` to improve clarity and remove references to design documents. - Enhanced documentation in `lib.rs` of `aimdb-derive` to clarify the derive functionality for `Linkable`. - Revised comments in `Cargo.toml` files across various examples to remove design references and improve readability. - Adjusted comments in `producer.rs` and integration tests to clarify the functionality of `SyncProducer` and its methods. - Updated design documentation to reflect changes in the implementation and clarify the purpose of various traits and features. - Removed unnecessary design references in example projects to streamline the documentation and focus on functionality. --- Makefile | 7 +- aimdb-codegen/src/rust.rs | 6 +- aimdb-core/src/profiling/record_profiling.rs | 2 +- aimdb-core/src/signal.rs | 2 +- aimdb-core/src/typed_api.rs | 2 +- aimdb-data-contracts/Cargo.toml | 8 +- aimdb-data-contracts/src/lib.rs | 16 ++-- aimdb-data-contracts/src/linkable.rs | 7 +- aimdb-data-contracts/src/observable.rs | 4 +- aimdb-data-contracts/src/simulatable.rs | 10 +- aimdb-derive/src/lib.rs | 4 +- aimdb-sync/Cargo.toml | 6 +- aimdb-sync/src/producer.rs | 11 +-- aimdb-sync/tests/settable_integration.rs | 8 +- docs/design/041-data-contracts-integration.md | 94 +++++++++---------- .../mqtt-connector-demo-common/Cargo.toml | 2 +- examples/tokio-mqtt-connector-demo/Cargo.toml | 2 +- .../tokio-mqtt-connector-demo/src/main.rs | 4 +- .../weather-mesh-demo/weather-hub/Cargo.toml | 5 +- .../weather-mesh-common/Cargo.toml | 2 +- .../weather-station-beta/Cargo.toml | 2 +- .../weather-station-gamma/Cargo.toml | 2 +- .../weather-station-gamma/src/main.rs | 2 +- 23 files changed, 102 insertions(+), 106 deletions(-) diff --git a/Makefile b/Makefile index 1637bd7..598e656 100644 --- a/Makefile +++ b/Makefile @@ -555,9 +555,10 @@ codegen-drift: @printf "$(GREEN)Checking codegen templates against the workspace API...$(NC)\n" ./tools/scripts/codegen-drift-check.sh -# Prove that simulation code (design 041, dev tier) never reaches a production -# binary. `rand` is the tracer: it is reachable iff `simulatable` is enabled -# (aimdb-data-contracts/src/simulatable.rs). The guard must not fail open: +# Prove that simulation code (the dev-tier `simulatable` contract) never +# reaches a production binary. `rand` is the tracer: it is reachable iff +# `simulatable` is enabled (aimdb-data-contracts/src/simulatable.rs). +# The guard must not fail open: # "rand is absent" is accepted only when `cargo tree -i rand` fails with its # specific "did not match any packages" error — any other failure (missing # submodule, typo'd package name, registry trouble) aborts the check instead of diff --git a/aimdb-codegen/src/rust.rs b/aimdb-codegen/src/rust.rs index b208eb8..ffb95cc 100644 --- a/aimdb-codegen/src/rust.rs +++ b/aimdb-codegen/src/rust.rs @@ -1018,9 +1018,9 @@ fn emit_observable_impl(rec: &RecordDef) -> Option { let unit = &obs.unit; - // Observable is now a kernel-only trait (design 041 §3.2): the numeric - // projection + UNIT label. `SIGNAL` defaults to the schema name; presentation - // (`ICON`, `format_log`) moved out of the trait, so we no longer emit them. + // Observable is a kernel-only trait: the numeric projection + UNIT label. + // `SIGNAL` defaults to the schema name; presentation (icons, log formatting) + // is not part of the trait, so nothing else is emitted. Some(quote! { impl Observable for #struct_name { type Signal = #signal_type; diff --git a/aimdb-core/src/profiling/record_profiling.rs b/aimdb-core/src/profiling/record_profiling.rs index cf1840d..5be34e1 100644 --- a/aimdb-core/src/profiling/record_profiling.rs +++ b/aimdb-core/src/profiling/record_profiling.rs @@ -25,7 +25,7 @@ impl StageEntry { } /// One registered signal gauge: its label/unit plus the shared statistics that -/// `.observe()` folds `Observable::signal()` into (design 041 §3.2). +/// `.observe()` folds `Observable::signal()` into. #[derive(Debug)] pub struct SignalGauge { /// Signal label (`Observable::SIGNAL`, defaults to the schema name). diff --git a/aimdb-core/src/signal.rs b/aimdb-core/src/signal.rs index 7827a1d..4e6a9d6 100644 --- a/aimdb-core/src/signal.rs +++ b/aimdb-core/src/signal.rs @@ -1,4 +1,4 @@ -//! Signal gauge handle for `Observable::observe()` (design 041 §3.2). +//! Signal gauge handle: per-record domain-signal statistics. //! //! `RecordRegistrar::signal_gauge` hands back a [`SignalGaugeHandle`]. Feeding //! values in with [`update`](SignalGaugeHandle::update) folds them into the diff --git a/aimdb-core/src/typed_api.rs b/aimdb-core/src/typed_api.rs index 1fc8413..136f0dc 100644 --- a/aimdb-core/src/typed_api.rs +++ b/aimdb-core/src/typed_api.rs @@ -445,7 +445,7 @@ where /// Values pushed via [`SignalGaugeHandle::update`](crate::SignalGaugeHandle::update) /// fold into per-record last/min/max/mean statistics that surface on /// `record.list` / `record.get` and stage profiling. This is the core hook - /// behind `Observable::observe()` (design 041 §3.2). + /// behind `aimdb-data-contracts`' `Observable::observe()`. /// /// Always available, mirroring [`with_name`](Self::with_name): when the /// `observability` feature is disabled it returns an inert handle whose diff --git a/aimdb-data-contracts/Cargo.toml b/aimdb-data-contracts/Cargo.toml index 874ea54..f8e5ff6 100644 --- a/aimdb-data-contracts/Cargo.toml +++ b/aimdb-data-contracts/Cargo.toml @@ -19,12 +19,12 @@ std = ["alloc", "serde/std", "serde_json?/std"] linkable = ["alloc", "serde_json", "aimdb-core", "dep:aimdb-derive"] # Dev-tier — never a default feature. The ext trait needs core's registrar, # same coupling `observable` has. `rand` is the tracer CI uses to prove a -# production graph is sim-free (design 041 §3.1.1). +# production graph is sim-free (`make check-no-sim`). simulatable = ["rand", "aimdb-core"] migratable = ["alloc", "serde_json", "dep:aimdb-derive"] observable = ["alloc", "aimdb-core"] -# Moved behind a feature for tier symmetry with the other wire contracts -# (design 041 §3.4). A pure trait — no dependencies of its own. +# Feature-gated for tier symmetry with the other wire contracts. A pure trait — +# no dependencies of its own. settable = [] [dependencies] @@ -45,7 +45,7 @@ aimdb-core = { version = "1.2.0", path = "../aimdb-core", optional = true, defau "alloc", ] } # RNG is caller-supplied, so we need only the `Rng`/`RngExt` traits and the -# always-available `SmallRng` — no `std_rng` (design 041 §3.1.1, decision 2). +# always-available `SmallRng` — no `std_rng`. [dependencies.rand] version = "0.10.1" optional = true diff --git a/aimdb-data-contracts/src/lib.rs b/aimdb-data-contracts/src/lib.rs index 474411f..763bb11 100644 --- a/aimdb-data-contracts/src/lib.rs +++ b/aimdb-data-contracts/src/lib.rs @@ -149,8 +149,8 @@ pub trait SchemaType: Sized { /// Construct a schema instance from its primary value. /// /// This defines the canonical way to create a new reading/measurement — the -/// write counterpart to [`Observable::signal`]'s read projection. The verb it -/// unlocks is `SyncProducer::set_value` in `aimdb-sync` (design 041 §3.4). +/// write counterpart to [`Observable::signal`]'s read projection. Implementing +/// it unlocks the `SyncProducer::set_value` family in `aimdb-sync`. #[cfg(feature = "settable")] pub trait Settable: SchemaType { /// The primary value type (e.g., `f32` for temperature) @@ -176,15 +176,15 @@ pub trait Settable: SchemaType { /// on `record.list` / `record.get` and stage profiling. The signal can also feed /// threshold checks, alerting, and aggregation. /// -/// Presentation lives elsewhere: `.log(node_id)` (also on the ext trait) formats -/// a human-readable line from `Debug` + [`SIGNAL`](Observable::SIGNAL)/[`UNIT`](Observable::UNIT), -/// and demo emoji live in the demo. +/// Presentation is not part of the trait: `.log(node_id)` (also on the ext +/// trait) formats a human-readable line from `Debug` + +/// [`SIGNAL`](Observable::SIGNAL)/[`UNIT`](Observable::UNIT). pub trait Observable: SchemaType { /// The numeric type of the signal (e.g., `f32`, `f64`, `i32`). /// - /// Must be comparable and copyable for threshold checks. Bound - /// `Into` at the `.observe()` call site (not here) so exotic signals - /// can still implement `Observable` and write their own tap. + /// Must be comparable and copyable for threshold checks. `.observe()` + /// additionally requires `Signal: Into` at its call site; a type with + /// an exotic signal can still implement `Observable` and write its own tap. type Signal: PartialOrd + Copy; /// What the signal means, for metrics/UI labels (e.g. `"celsius"`). diff --git a/aimdb-data-contracts/src/linkable.rs b/aimdb-data-contracts/src/linkable.rs index ed19373..162a567 100644 --- a/aimdb-data-contracts/src/linkable.rs +++ b/aimdb-data-contracts/src/linkable.rs @@ -3,8 +3,8 @@ //! Implementing [`Linkable`](crate::Linkable) unlocks two verbs — //! [`LinkableRegistrarExt::linked_from`] / [`linked_to`](LinkableRegistrarExt::linked_to) //! — that install the raw `.link_from()`/`.link_to()` builders with the codec -//! defaulted to `T::from_bytes`/`T::to_bytes` (design 041 §3.3). The raw builders -//! remain the escape hatch for per-link options (QoS, topic providers/resolvers). +//! defaulted to `T::from_bytes`/`T::to_bytes`. The raw builders remain the +//! escape hatch for per-link options (QoS, topic providers/resolvers). use aimdb_core::connector::SerializeError; use aimdb_core::typed_api::RecordRegistrar; @@ -24,8 +24,7 @@ where /// /// `Linkable::to_bytes`'s `String` error is mapped to /// `SerializeError::InvalidData` — the connector layer's serializer error - /// type has no string detail (see design 041 §3.3, `CodecError` alignment - /// recorded as a follow-up). + /// type carries no string detail. fn linked_to(&mut self, url: &str) -> &mut RecordRegistrar<'a, T>; } diff --git a/aimdb-data-contracts/src/observable.rs b/aimdb-data-contracts/src/observable.rs index 15c22e4..7e0248e 100644 --- a/aimdb-data-contracts/src/observable.rs +++ b/aimdb-data-contracts/src/observable.rs @@ -2,8 +2,8 @@ //! //! Implementing [`Observable`](crate::Observable) unlocks one verb — //! [`ObservableRegistrarExt::observe`] — which feeds the record's domain signal -//! into core signal-gauge metrics (design 041 §3.2). `.log(node_id)` is the -//! human-readable companion for console watching. +//! into core signal-gauge metrics. `.log(node_id)` is the human-readable +//! companion for console watching. extern crate alloc; diff --git a/aimdb-data-contracts/src/simulatable.rs b/aimdb-data-contracts/src/simulatable.rs index 53068b1..eaba29d 100644 --- a/aimdb-data-contracts/src/simulatable.rs +++ b/aimdb-data-contracts/src/simulatable.rs @@ -4,8 +4,8 @@ //! [`SimulatableRegistrarExt::simulate`] — which installs a source that emits //! synthetic samples on a timer. This is the **dev tier**: it must never ship in //! a production binary. Sim-to-real selection is a compile-time `#[cfg]` in the -//! application (design 041 §3.1.4), never a runtime flag, and `rand` is the -//! tracer CI uses to prove production dependency graphs are sim-free. +//! application, never a runtime flag, and `rand` is the tracer CI uses to prove +//! production dependency graphs are sim-free (see `make check-no-sim`). use serde::{Deserialize, Serialize}; @@ -56,9 +56,9 @@ pub struct SimProfile

{ pub params: P, } -/// Off-the-shelf [`Simulatable::Params`] for scalar random walks — what the -/// pre-0.3 `SimulationParams` actually was. Existing impls migrate by setting -/// `type Params = RandomWalkParams`. +/// Off-the-shelf [`Simulatable::Params`] for scalar signals that wander around +/// a base value: set `type Params = RandomWalkParams` and derive the next +/// sample from `previous` plus a bounded random step. #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] pub struct RandomWalkParams { /// Base/center value for the walk. diff --git a/aimdb-derive/src/lib.rs b/aimdb-derive/src/lib.rs index bf02fec..d65a51b 100644 --- a/aimdb-derive/src/lib.rs +++ b/aimdb-derive/src/lib.rs @@ -44,8 +44,8 @@ pub fn migration_chain(input: TokenStream) -> TokenStream { /// Derive `Linkable` (JSON codec) for a schema type. /// /// Emits `from_bytes`/`to_bytes` via `serde_json::from_slice`/`to_vec` — the -/// boilerplate every hand-written `impl Linkable` for a JSON-wire type repeats -/// (design 041 §3.3). Requires `T: Serialize + DeserializeOwned` (a normal +/// boilerplate every hand-written `impl Linkable` for a JSON-wire type repeats. +/// Requires `T: Serialize + DeserializeOwned` (a normal /// compile error surfaces if it's missing) and the `linkable` feature of /// `aimdb-data-contracts` (for the `Linkable` trait and its `__private::serde_json` /// re-export). Binary formats (e.g. KNX DPT codecs) still implement `Linkable` diff --git a/aimdb-sync/Cargo.toml b/aimdb-sync/Cargo.toml index 8576093..1bc8f3e 100644 --- a/aimdb-sync/Cargo.toml +++ b/aimdb-sync/Cargo.toml @@ -21,8 +21,8 @@ tokio = { version = "1.40", features = ["sync", "rt", "time", "macros"] } thiserror = "1.0" # Optional: Settable::set (feature `data-contracts`) for the `set_value` family. -# Dependency direction stays contracts -> sync; the contracts crate gains no -# `sync` feature (design 041 §3.4). +# aimdb-sync depends on the contracts crate, never the reverse — the contracts +# crate has no `sync` feature. aimdb-data-contracts = { path = "../aimdb-data-contracts", version = "0.3.0", optional = true, default-features = false, features = [ "settable", ] } @@ -40,7 +40,7 @@ default = [] # Enable tracing for debugging tracing = ["aimdb-core/tracing"] -# SyncProducer::set_value / try_set_value / set_value_at (design 041 §3.4) +# SyncProducer::set_value / try_set_value / set_value_at for Settable types data-contracts = ["dep:aimdb-data-contracts"] [package.metadata.docs.rs] diff --git a/aimdb-sync/src/producer.rs b/aimdb-sync/src/producer.rs index 33f4042..e66c9f3 100644 --- a/aimdb-sync/src/producer.rs +++ b/aimdb-sync/src/producer.rs @@ -220,13 +220,12 @@ where } } -/// `Settable`-powered set-by-primitive verb (design 041 §3.4, feature `data-contracts`). +/// Set-by-primitive verbs for `Settable` types (feature `data-contracts`). /// -/// `set()` takes a fully constructed `T`, so every outside-the-thread caller had -/// to hand-assemble the struct. `set_value` closes that gap: construct via -/// `T::set(value, timestamp)` and send, in one call. Distinct from AimX's -/// `record.set {name, value}` (full JSON value through `JsonCodec`) — this is -/// set-by-primitive. +/// Where [`set`](Self::set) takes a fully constructed `T`, `set_value` +/// constructs it via `T::set(value, timestamp)` and sends, in one call. +/// Distinct from AimX's `record.set {name, value}` (full JSON value through +/// `JsonCodec`) — this is set-by-primitive. #[cfg(feature = "data-contracts")] impl SyncProducer where diff --git a/aimdb-sync/tests/settable_integration.rs b/aimdb-sync/tests/settable_integration.rs index f6ebb6e..815885b 100644 --- a/aimdb-sync/tests/settable_integration.rs +++ b/aimdb-sync/tests/settable_integration.rs @@ -1,6 +1,6 @@ -//! Integration coverage for `SyncProducer::set_value` (feature `data-contracts`, -//! design 041 §3.4): construct via `Settable::set`, produce, and consume -//! end-to-end through the real sync bridge. +//! Integration coverage for `SyncProducer::set_value` (feature `data-contracts`): +//! construct via `Settable::set`, produce, and consume end-to-end through the +//! real sync bridge. #![cfg(feature = "data-contracts")] @@ -114,7 +114,7 @@ fn set_value_at_stamps_the_explicit_timestamp() { .expect("failed to create consumer"); // Explicit timestamp, not wall-clock — deterministic regardless of when the - // test runs (replay/testing use case, design 041 §3.4). + // test runs (the replay/testing use case). producer .set_value_at(22.5, 1_700_000_000_000) .expect("set_value_at should succeed"); diff --git a/docs/design/041-data-contracts-integration.md b/docs/design/041-data-contracts-integration.md index 4659b0f..5759d8b 100644 --- a/docs/design/041-data-contracts-integration.md +++ b/docs/design/041-data-contracts-integration.md @@ -1,6 +1,6 @@ -# 041 — Data contracts as first-class capabilities (v2) +# 041 — Data contracts as first-class capabilities -**Status:** Proposed 2026-07-06, **v2**. Supersedes v1 (same date, same number). v1's two anchors — "no changes to the contract trait definitions" and the `SourceMode` build-time sim selector (§3.3 of v1) — were overturned by owner steer on 2026-07-06: simulation must be a **compile-time** decision (no sim code in production binaries, at all), `Observable` must earn the name (metrics, not logging), and `Streamable`/`Linkable`/`Settable` need to become intuitive. Follow-up to design-038 **D1/D12** (tracking issue [#161](https://github.com/aimdb-dev/aimdb/issues/161)). +**Status:** ✅ Implemented. Follow-up to design-038 **D1/D12** (tracking issue [#161](https://github.com/aimdb-dev/aimdb/issues/161)). **Scope:** `aimdb-data-contracts` (trait reshapes + three registrar ext traits; `Simulatable` stays here behind its feature), `aimdb-sync` (consumes `Settable`), one small `aimdb-core` observability surface (signal gauges), the README capability table, the weather-mesh example, and a CI guard. @@ -10,11 +10,11 @@ ## 1. Problem -Design 038 D1/D12: the capability traits are advertised as a headline feature, but core consumes almost none of them. v1 of this doc fixed that by *adding consumers* without touching the traits. Three problems survived v1: +Design 038 D1/D12: the capability traits were advertised as a headline feature, but core consumed almost none of them. Three specific gaps: -1. **Simulation was still a build-time value, not a compile-time fact.** v1 §3.3's `source_or_simulate(mode, cfg, rng, real)` selected the producer with a runtime-valued `SourceMode` — which means **both arms compile into the binary**. Even env-var selection ships the sim loop, every `T::simulate` impl, `SimulationConfig`, and `rand` in the production image. The owner requirement is stronger: production binaries must contain **zero** simulation code, verifiable from the dependency graph. -2. **`Observable` remained logging.** v1 resolved the false README claim ("automatic per-record metrics") by *rewording downward* to "built-in logging tap". The right fix is upward: make the trait actually feed metrics, then the strong claim is true. -3. **`Streamable`/`Linkable`/`Settable` are unintuitive** because a user cannot answer *"what does implementing this let me write?"* `Linkable` exists but every example still hand-writes `with_deserializer` closures; `Settable` was built for `aimdb-sync` but sync never references it (`SyncProducer::set` takes a full `T` — [`producer.rs:136`](../../aimdb-sync/src/producer.rs)); `Streamable` works but nothing says its verb is the ws-connector's `.register::()`. +1. **Simulation was a runtime value, not a compile-time fact.** `SimulationConfig.enabled` gated simulation with a runtime flag, so the sim loop, every `T::simulate` impl, `SimulationConfig`, and `rand` all shipped in the production image even with simulation "off". A build-time *value* selector (e.g. an env-var-driven mode enum) fails the same test: both arms still compile into the binary. The requirement is stronger — production binaries must contain **zero** simulation code, verifiable from the dependency graph. +2. **`Observable` was logging.** The README claimed "automatic per-record metrics", but the trait only powered a console tap (`format_log`). Rewording the claim downward would fix the dishonesty; making the trait actually feed metrics fixes the feature. +3. **`Streamable`/`Linkable`/`Settable` were unintuitive** because a user could not answer *"what does implementing this let me write?"* `Linkable` existed but every example still hand-wrote `with_deserializer` closures; `Settable` was built for `aimdb-sync` but sync never referenced it (`SyncProducer::set` took a full `T`); `Streamable` worked but nothing said its verb is the ws-connector's `.register::()`. ## 2. Organizing principle: one verb per contract, tiered by deployment role @@ -30,9 +30,9 @@ A capability trait is a compile-time promise about a schema type. Each promise u | `Observable` | the type carries a domain signal worth watching | `.observe()` → live signal metrics (§3.2) | introspection surface | introspection (prod, optional) | | `Simulatable` | the type can generate realistic synthetic data | `.simulate(profile, rng)` (§3.1) | `simulatable` feature ext | **dev-only — never in prod** | -Mechanism (unchanged from v1, copied from `.persist()` — [`aimdb-persistence/src/ext.rs`](../../aimdb-persistence/src/ext.rs)): extension traits over `RecordRegistrar`/`SyncProducer` in the crate that owns the contract, installing plain `.source()`/`.tap()`/`.link_*()` stages. `.source()`/`.tap()` stay unbounded; `aimdb-core` never learns the contracts exist; dependency direction is always *contracts → core*, never the reverse. +Mechanism (the `.persist()` precedent — [`aimdb-persistence/src/ext.rs`](../../aimdb-persistence/src/ext.rs)): extension traits over `RecordRegistrar`/`SyncProducer` in the crate that owns the contract, installing plain `.source()`/`.tap()`/`.link_*()` stages. `.source()`/`.tap()` stay unbounded; `aimdb-core` never learns the contracts exist; dependency direction is always *contracts → core*, never the reverse. -The tier column is the second half of the fix: *wire* contracts ship in production, *introspection* is prod-optional, and the *dev* tier must be **structurally excludable** — which drives §3.1's crate split. +The tier column is the second half of the fix: *wire* contracts ship in production, *introspection* is prod-optional, and the *dev* tier must be **structurally excludable** — which drives §3.1's compile-time-only design. ## 3. Design @@ -40,17 +40,17 @@ The tier column is the second half of the fix: *wire* contracts ship in producti #### 3.1.1 Stays in `aimdb-data-contracts`, as the dev-tier feature -`Simulatable` remains in the contracts crate behind the existing `simulatable` feature. (A separate `aimdb-simulation` crate was considered for the structural guarantee and **rejected 2026-07-06** — owner call: not worth another crate in the monorepo.) The compile-time guarantee never depended on a crate boundary; it rests on three enforceable properties: +`Simulatable` lives in the contracts crate behind the `simulatable` feature. A separate `aimdb-simulation` crate was considered for the structural guarantee and rejected — another crate in the monorepo buys nothing, because the guarantee never depended on a crate boundary. It rests on three enforceable properties: -1. **`simulatable` is not (and never becomes) a default feature.** Already true today ([`Cargo.toml:14`](../../aimdb-data-contracts/Cargo.toml)); CI asserts it stays that way (§3.1.5). +1. **`simulatable` is not (and never becomes) a default feature** ([`Cargo.toml`](../../aimdb-data-contracts/Cargo.toml)); CI asserts it stays that way (§3.1.5). 2. **Production binaries resolve features per-package.** The workspace uses `resolver = "2"` ([`Cargo.toml:45`](../../Cargo.toml)), so `cargo build -p --release` unifies features only over that binary's own graph — a sim-enabled example elsewhere in the workspace cannot leak the feature into the release artifact. Corollary, worth one sentence in CONTRIBUTING: release artifacts are built per-package, never lifted out of a `--workspace` build (where unification does cross targets). 3. **`rand` is the tracer.** `Simulatable::simulate` makes `rand` reachable iff the feature is on, so an inverse-dependency check on `rand` proves sim absence from a production graph (§3.1.5). -Feature wiring: `simulatable = ["rand", "aimdb-core"]` — the ext trait needs `RecordRegistrar`/`Producer`/`RuntimeContext`, the same core coupling `observable` already has and `linkable` gains in §3.3. `rand` stays `default-features = false` and **loses `std_rng`** (the RNG is caller-supplied, carried over from v1 §5.1). The feature is `no_std`-compatible: the "develop against sim, flash against silicon" story is embedded-first. +Feature wiring: `simulatable = ["rand", "aimdb-core"]` — the ext trait needs `RecordRegistrar`/`Producer`/`RuntimeContext`, the same core coupling `observable` has and `linkable` gains in §3.3. `rand` is `default-features = false` without `std_rng`: the RNG is always caller-supplied (§5.2). The feature is `no_std`-compatible: the "develop against sim, flash against silicon" story is embedded-first. #### 3.1.2 Trait reshape: domain params, no runtime gate -The current `SimulationParams` (base/variation/trend/step — [`simulatable.rs:53`](../../aimdb-data-contracts/src/simulatable.rs)) is a random-walk config pretending to be universal, and `SimulationConfig.enabled` is a runtime gate that contradicts the compile-time stance. Both go: +The previous `SimulationParams` (base/variation/trend/step) was a random-walk config pretending to be universal, and `SimulationConfig.enabled` was a runtime gate that contradicts the compile-time stance. Their replacement: ```rust // aimdb-data-contracts/src/simulatable.rs (feature = "simulatable") @@ -79,9 +79,8 @@ pub struct SimProfile

{ pub params: P, } -/// Off-the-shelf `Params` for scalar random walks — what today's -/// `SimulationParams` actually was. Existing impls migrate by setting -/// `type Params = RandomWalkParams`. +/// Off-the-shelf `Params` for scalar random walks. Impls written against the +/// old `SimulationParams` migrate by setting `type Params = RandomWalkParams`. #[derive(Clone, Debug, serde::Serialize, serde::Deserialize)] pub struct RandomWalkParams { pub base: f64, @@ -91,11 +90,11 @@ pub struct RandomWalkParams { } ``` -There is **no `enabled` field**. "Sim but off" is not a state: production excludes the crate (§3.1.4), and a std dev binary that wants sim conditionally simply branches before calling `.simulate()`. +There is **no `enabled` field**. "Sim but off" is not a state: production excludes the feature (§3.1.4), and a std dev binary that wants sim conditionally simply branches before calling `.simulate()`. #### 3.1.3 `SimulatableRegistrarExt::simulate(profile, rng)` -The v1 §3.1 loop, minus the `enabled` branch, over the reshaped types: +The verb installs a plain `.source()` loop over the reshaped types: ```rust pub trait SimulatableRegistrarExt<'a, T> @@ -139,9 +138,9 @@ where Runtime-neutral (`ctx.time()` for clock and delay, like `.persist()`'s cleanup loop), installs a **source** so writer-exclusivity is enforced by `build()` unchanged ([`typed_record.rs:570`](../../aimdb-core/src/typed_record.rs)). -#### 3.1.4 Sim-to-real is a `#[cfg]`, not an API — `SourceMode` is deleted +#### 3.1.4 Sim-to-real is a `#[cfg]`, not an API -v1 §3.3 (`SourceMode` + `source_or_simulate`) is **removed without replacement API**. The sim-to-real selection is the app's Cargo feature, and this doc makes that pattern canonical instead of wrapping it: +There is deliberately no selector API — no mode enum, no `source_or_simulate(mode, …)` wrapper. Any selector that takes the choice as a *value* (even one read from the environment at build time) compiles both arms into the binary, which is exactly what the dev tier forbids. The sim-to-real selection is the app's Cargo feature, and that pattern is canonical rather than wrapped: ```toml # app Cargo.toml @@ -161,17 +160,17 @@ builder.configure::(KEY, |reg| { }); ``` -The app's contracts crate gates its impls the same way (`#[cfg(feature = "sim")] impl Simulatable for Temperature { … }`). With `sim` off there is nothing to audit: no impls, no `SimProfile`, no `rand` anywhere in the binary's graph. Exactly one producer is installed on either path, so single-writer-per-key holds by construction — same argument as v1, now with the selection resolved by the compiler instead of a value. +The app's contracts crate gates its impls the same way (`#[cfg(feature = "sim")] impl Simulatable for Temperature { … }`). With `sim` off there is nothing to audit: no impls, no `SimProfile`, no `rand` anywhere in the binary's graph. Exactly one producer is installed on either path, so single-writer-per-key holds by construction, with the selection resolved by the compiler instead of a value. #### 3.1.5 CI guard: prove production is sim-free -A CI step (Makefile target `check-no-sim`) builds the production configuration of each example that has a `sim` feature and asserts the exclusion from the resolved graph — `cargo tree -p -e normal -i rand` must report no matching package when `sim` is off (`rand` is the tracer, §3.1.1) — and additionally asserts that `simulatable` never appears in `aimdb-data-contracts`' default features. This turns "at all cost" from a convention into a gate. +A CI step (Makefile target `check-no-sim`) builds the production configuration of each example that has a `sim` feature and asserts the exclusion from the resolved graph — `cargo tree -p -e normal -i rand` must report no matching package when `sim` is off (`rand` is the tracer, §3.1.1) — and additionally asserts that `simulatable` never appears in `aimdb-data-contracts`' default features. This turns "never ships in production" from a convention into a gate. ### 3.2 `Observable` → the signal, not the log line #### 3.2.1 Trait reshape: keep the kernel, drop the cosmetics -The trait's valuable kernel is the numeric projection (`Signal`, `signal()`, `UNIT`); `ICON` and `format_log()` are presentation and made the whole trait read as a log formatter ([`lib.rs:192-227`](../../aimdb-data-contracts/src/lib.rs)). Reshape: +The trait's valuable kernel is the numeric projection (`Signal`, `signal()`, `UNIT`); `ICON` and `format_log()` were presentation and made the whole trait read as a log formatter. The kernel-only trait: ```rust pub trait Observable: SchemaType { @@ -190,14 +189,14 @@ pub trait Observable: SchemaType { } ``` -`ICON` and `format_log()` are deleted from the trait. The logging tap (below) formats from `Debug` + `UNIT`; demo emoji live in the demo. +`ICON` and `format_log()` are gone: the logging tap (§3.2.3) formats from `Debug` + `UNIT`, and demo-specific decoration (emoji) belongs to the demo. #### 3.2.2 `.observe()` installs a signal-metrics tap -`.observe()` folds `signal()` into per-record **signal stats** (last/min/max/count/mean as `f64`) that surface on the existing introspection paths. Mechanism — a small, feature-honest core addition mirroring `with_name` ([`typed_api.rs:433`](../../aimdb-core/src/typed_api.rs)): +`.observe()` folds `signal()` into per-record **signal stats** (last/min/max/count/mean as `f64`) that surface on the existing introspection paths. Mechanism — a small, feature-honest core addition mirroring `with_name`: -- `aimdb-core` gains `RecordRegistrar::signal_gauge(name: &'static str, unit: &'static str) -> SignalGaugeHandle`. Under the `observability` feature it registers atomic `SignalStats` on the record's profiling state and returns a live handle; without the feature it is always-available but returns an inert handle (the `with_name` precedent — callers never `cfg` on core's features). -- Exposure: `RecordMetadata` ([`builder.rs:191`](../../aimdb-core/src/builder.rs) `list_records`) gains an optional signal-stats field, and the AimX `record.get` response carries it — so the signal shows up in `record.list`/`record.get`, the MCP tools built on them, and stage-profiling output. *Implement `Observable`, call `.observe()`, and your domain signal is live on every introspection surface.* That is the README claim, and after this change it is true. +- `aimdb-core` provides `RecordRegistrar::signal_gauge(name: &'static str, unit: &'static str) -> SignalGaugeHandle`. Under the `observability` feature it registers atomic `SignalStats` on the record's profiling state and returns a live handle; without the feature it is still callable but returns an inert handle (the `with_name` precedent — callers never `cfg` on core's features). +- Exposure: `RecordMetadata` gains an optional signal-stats field, and the AimX `record.get` response carries it — so the signal shows up in `record.list`/`record.get`, the MCP tools built on them, and stage-profiling output. *Implement `Observable`, call `.observe()`, and your domain signal is live on every introspection surface.* That is the README claim, and the gauge is what makes it true. ```rust // aimdb-data-contracts/src/observable.rs (feature = "observable") @@ -232,7 +231,6 @@ where Notes: - `T::Signal: Into` is bounded **at the consumption site**, not in the trait — `f32`/`i32`/`u32` signals qualify; a type with an exotic signal can still implement `Observable` and write its own tap. - `.observe()` takes **no `node_id`** — the record key already identifies the gauge; `node_id` was a logging concern. -- Landing order inside the work item: the core `signal_gauge` surface and the ext land together; if the core PR needs to decouple, an interim `Extensions`-TypeMap registry owned by the contracts crate is acceptable, but the gauge API is the target and the README claim waits for it. #### 3.2.3 Logging stays, honestly named @@ -240,7 +238,7 @@ Notes: ### 3.3 `Linkable` → the default codec that `.link_*` actually consumes -Implementing `Linkable` today changes nothing — every example still hand-wires closures ([`embassy-mqtt-connector-demo/src/main.rs:350-393`](../../examples/embassy-mqtt-connector-demo/src/main.rs)). Fix: one-line link verbs, ext trait in the contracts crate (dependency direction preserved): +Before this design, implementing `Linkable` changed nothing — every example hand-wired `with_deserializer`/`with_serializer` closures. The fix: one-line link verbs, ext trait in the contracts crate (dependency direction preserved): ```rust // aimdb-data-contracts/src/linkable.rs (feature = "linkable") @@ -275,10 +273,10 @@ where ``` Notes: -- **Inbound matches exactly** (`with_deserializer` takes `Result` — [`typed_api.rs:872`](../../aimdb-core/src/typed_api.rs)). **Outbound needs one lossy mapping**: `with_serializer` returns `Result, SerializeError>` ([`typed_api.rs:661`](../../aimdb-core/src/typed_api.rs)), so the `String` detail is dropped to `SerializeError::InvalidData`. Aligning `Linkable`'s error type with the connector layer (a shared `CodecError` instead of `String` — which also removes an alloc on embedded) is a recorded follow-up (§7); it touches core connector signatures and doesn't block this verb. +- **Inbound matches exactly** (`with_deserializer` takes `Result`). **Outbound needs one lossy mapping**: `with_serializer` returns `Result, SerializeError>`, so the `String` detail is dropped to `SerializeError::InvalidData`. Aligning `Linkable`'s error type with the connector layer (a shared `CodecError` instead of `String` — which also removes an alloc on embedded) is a recorded follow-up (§7); it touches core connector signatures and doesn't block this verb. - **The raw builders remain the escape hatch** for per-link options (`with_config`, QoS ext traits, topic providers/resolvers). `.linked_from`/`.linked_to` are the 80% path. -- **JSON boilerplate gets a derive:** `#[derive(Linkable)]` in `aimdb-derive` emitting `serde_json::to_vec`/`from_slice` (JSON is the default format; binary formats implement by hand, as the KNX DPT codecs rightly do). D1 caller: the derive replaces at least one hand-written JSON impl in the same commit. -- **Coupling note (the v1 §5.3 move, now for `linkable`):** the `linkable` feature currently has no `aimdb-core` dependency; the ext adds one (`default-features = false, features = ["alloc"]`, same wiring as `observable`). +- **JSON boilerplate gets a derive:** `#[derive(Linkable)]` in `aimdb-derive` emitting `serde_json::to_vec`/`from_slice` (JSON is the default format; binary formats implement by hand, as the KNX DPT codecs rightly do). Per the D1 rule, the derive replaces a hand-written JSON impl in the same change. +- **Coupling:** the `linkable` feature gains an `aimdb-core` dependency for the ext trait (`default-features = false, features = ["alloc"]`, same wiring as `observable`). - **Three serialization stories, stated once** so users stop guessing: | Boundary | Mechanism | Contract | @@ -289,7 +287,7 @@ Notes: ### 3.4 `Settable` → consumed by `aimdb-sync` -`Settable` was built for the sync bridge, but `aimdb-sync` never references it: `SyncProducer::set(value: T)` takes a fully constructed record, so every outside-the-thread caller hand-assembles the struct. The verb it was named for is the missing ten lines — and note this is distinct from AimX's existing `record.set {name, value}` (full JSON value through `JsonCodec`); `Settable` is **set-by-primitive**: +`Settable` was built for the sync bridge, but `aimdb-sync` never referenced it: `SyncProducer::set(value: T)` took a fully constructed record, so every outside-the-thread caller hand-assembled the struct. The verb the trait was named for is a small inherent impl — and note it is distinct from AimX's existing `record.set {name, value}` (full JSON value through `JsonCodec`); `Settable` is **set-by-primitive**: ```rust // aimdb-sync/src/producer.rs (inherent impl, feature = "data-contracts") @@ -318,8 +316,8 @@ Notes: - **Dependency direction:** `aimdb-sync` gains `aimdb-data-contracts = { optional = true, default-features = false, features = ["settable"] }` behind a `data-contracts` feature. The contracts crate does **not** grow a `sync` feature. - **Clock semantics (documented, one sentence):** `set_value` stamps with the *caller's* `SystemTime` (sample time at the edge), not the engine's `ctx.time()`; `set_value_at` exists for callers that need control. `aimdb-sync` is std-only, so `SystemTime` is always available. - **Single-writer untouched:** `set_value` flows through the same channel as `set()`; the sync bridge remains the record's one producer, and concurrent `SyncProducer` clones already arbitrate through that one request stream. -- **Feature gate:** `Settable` is currently compiled unconditionally in the contracts crate; it moves behind a `settable` feature for tier symmetry (codegen templates that emit `impl Settable` enable it). -- A future AimX **set-by-primitive** verb (`aimdb set temperature 22.5` from CLI/MCP) becomes a second consumer of the same trait — recorded in §7, not in scope. The read/write symmetry is now explicit: `Observable::signal()` is the generic *read* projection, `Settable::set()` the generic *write* injection. +- **Feature gate:** `Settable` sits behind a `settable` feature for tier symmetry with the other wire contracts (codegen templates that emit `impl Settable` enable it). +- A future AimX **set-by-primitive** verb (`aimdb set temperature 22.5` from CLI/MCP) becomes a second consumer of the same trait — recorded in §7, not in scope. The read/write symmetry is explicit: `Observable::signal()` is the generic *read* projection, `Settable::set()` the generic *write* injection. ### 3.5 `Streamable` — unchanged @@ -329,7 +327,7 @@ Already consumed (ws-connector registry bound). Its fix is the verb table (§2) Consumed since design 039, works no_std. Stretch (recorded, not in scope): auto-apply the migration chain on the `linked_from` deserialize path so connectors migrate old wire data without a manual call. -## 4. Crate & feature layout after this doc +## 4. Crate & feature layout ``` aimdb-data-contracts @@ -347,30 +345,30 @@ aimdb-core `aimdb-data-contracts` bumps 0.2.0 → 0.3.0 (breaking: `Simulatable` reshaped — `type Params`, `SimulationConfig`/`SimulationParams` replaced by `SimProfile`/`RandomWalkParams`; `Observable` loses `ICON`/`format_log`; `Settable` gains a feature gate). -## 5. Decisions (resolved 2026-07-06, v2) +## 5. Decisions -1. **Simulation is compile-time.** `SourceMode`/`source_or_simulate` deleted; `enabled` deleted; `#[cfg]` branch is the canonical sim-to-real pattern; CI guard (`rand` tracer + never-a-default-feature assert). `Simulatable` **stays in `aimdb-data-contracts`** behind `simulatable` — a separate `aimdb-simulation` crate was considered and rejected (2026-07-06, owner: keep the monorepo lean); the guarantee rests on §3.1.1's three properties, not on a crate boundary. (Overturns v1 §3.3/§5.4.) -2. **Caller-supplied RNG stands** (v1 §5.1 carried over); `rand` loses `std_rng` everywhere. -3. **`Simulatable::Params` is an associated type**; `RandomWalkParams` ships as the migration path for today's scalar walks. -4. **`Observable` claims metrics because it delivers metrics**: kernel-only trait, `.observe()` → core signal gauges surfaced via `record.list`/`record.get`/profiling. (Overturns v1 §4.1's reword-downward.) -5. **`Linkable` ext is in scope** (was a v1 follow-up): `.linked_from`/`.linked_to` + `#[derive(Linkable)]`; `String`→`SerializeError::InvalidData` mapping accepted for now. +1. **Simulation is compile-time.** No runtime `enabled` flag and no selector API (a `source_or_simulate(mode, …)` wrapper was considered and rejected — both arms compile into the binary); the `#[cfg]` branch is the canonical sim-to-real pattern, enforced by the CI guard (`rand` tracer + never-a-default-feature assert). `Simulatable` **stays in `aimdb-data-contracts`** behind `simulatable` — a separate `aimdb-simulation` crate was rejected to keep the monorepo lean; the guarantee rests on §3.1.1's three properties, not on a crate boundary. +2. **The RNG is caller-supplied**; `rand` loses `std_rng` everywhere. +3. **`Simulatable::Params` is an associated type**; `RandomWalkParams` ships as the migration path for existing scalar walks. +4. **`Observable` claims metrics because it delivers metrics**: kernel-only trait, `.observe()` → core signal gauges surfaced via `record.list`/`record.get`/profiling. +5. **`Linkable` gets its verbs in the same design**: `.linked_from`/`.linked_to` + `#[derive(Linkable)]`; the `String` → `SerializeError::InvalidData` mapping is accepted until the `CodecError` alignment (§7). 6. **`Settable` is wired to its original purpose**: `SyncProducer::set_value` family in `aimdb-sync`, caller-side clock, `settable` feature gate. -7. **No trait bounds on `.source()`/`.tap()`/`.transform()`; no core→contracts dependency** — unchanged from v1. +7. **No trait bounds on `.source()`/`.tap()`/`.transform()`; no core→contracts dependency.** ## 6. Migration & examples (the D1 callers) -- **weather-station-beta/gamma:** delete the manual RNG-loop producers; `reg.simulate(SimProfile { interval_ms, params: RandomWalkParams { … } }, StdRng::…)` under the app's `sim` feature; contracts impls move to `type Params = RandomWalkParams` and `#[cfg(feature = "sim")]`. The station is the §3.1.4 pattern's reference implementation (with the CI guard building its prod configuration). -- **weather-hub:** `.observe()` replaces the manual `log_tap` tap; keep one `.log(node_id)` where console output is the point of the demo. Hub verifies the signal appears in `record.list`/`record.get`. +- **weather-station-beta/gamma:** the manual RNG-loop producers are deleted; `reg.simulate(SimProfile { interval_ms, params: RandomWalkParams { … } }, StdRng::…)` under the app's `sim` feature; contracts impls use `type Params = RandomWalkParams` and `#[cfg(feature = "sim")]`. The stations are the §3.1.4 pattern's reference implementation (with the CI guard building their prod configuration). +- **weather-hub:** `.observe()` replaces the manual `log_tap` tap; one `.log(node_id)` stays where console output is the point of the demo. The hub verifies the signal appears in `record.list`/`record.get`. - **one connector demo (tokio-mqtt):** `#[derive(Linkable)]` + `.linked_from`/`.linked_to` replace the hand-written closures. - **aimdb-sync doctest/integration test:** `producer.set_value(22.5)` end-to-end (construct → produce → consume), plus `set_value_at` determinism in a unit test. -- **README:** capability table becomes the §2 verb table (Contract / Implement when / Verb / Tier); the `observability` feature keeps the buffer-statistics story; the `simulatable` feature gets a "dev tier — never ships" call-out with the `#[cfg]` pattern. +- **README:** the capability table is the §2 verb table (Contract / Implement when / Verb / Tier); the `observability` feature keeps the buffer-statistics story; the `simulatable` feature gets a "dev tier — never ships" call-out with the `#[cfg]` pattern. - **CHANGELOG:** entries for all four crates touched. -## 7. Sequencing +## 7. Work items -Separate commits on one branch/PR, one per work item (per the no-stacked-PRs rule), each carrying its caller: +One branch/PR; each work item lands as its own commit carrying its caller: -1. `Simulatable` v2 in place (trait reshape + `SimProfile`/`RandomWalkParams` + `.simulate()` ext, `enabled` and `std_rng` dropped), stations migrated, CI `check-no-sim` guard. +1. `Simulatable` reshape (trait + `SimProfile`/`RandomWalkParams` + `.simulate()` ext, `enabled` and `std_rng` dropped), stations migrated, CI `check-no-sim` guard. 2. Core `signal_gauge` surface + `Observable` kernel reshape + `.observe()`/`.log()` ext, hub migrated, README `Observable` row. 3. `LinkableRegistrarExt` + `#[derive(Linkable)]`, mqtt demo migrated. 4. `aimdb-sync` `set_value` family + `settable` feature gate + tests. diff --git a/examples/mqtt-connector-demo-common/Cargo.toml b/examples/mqtt-connector-demo-common/Cargo.toml index d715918..cb583cc 100644 --- a/examples/mqtt-connector-demo-common/Cargo.toml +++ b/examples/mqtt-connector-demo-common/Cargo.toml @@ -15,7 +15,7 @@ derive = ["aimdb-core/derive"] # needed: the no_std JSON paths are hand-rolled), so an embassy demo can expose # these records over a remote-access (serial) server. serde = ["dep:serde"] -# `SchemaType` + `#[derive(Linkable)]` (design 041 §3.3). `no_std + alloc` +# `SchemaType` + `#[derive(Linkable)]`. `no_std + alloc` # compatible (aimdb-data-contracts's `linkable` feature only needs `alloc` — # its `serde_json` dep is alloc-only, no `std`). Not enabled by default: the # embassy demo's hand-rolled no_std JSON path is an intentional illustration, diff --git a/examples/tokio-mqtt-connector-demo/Cargo.toml b/examples/tokio-mqtt-connector-demo/Cargo.toml index e0e364a..eb5fd4b 100644 --- a/examples/tokio-mqtt-connector-demo/Cargo.toml +++ b/examples/tokio-mqtt-connector-demo/Cargo.toml @@ -26,7 +26,7 @@ mqtt-connector-demo-common = { path = "../mqtt-connector-demo-common", features "data-contracts", ] } -# LinkableRegistrarExt / #[derive(Linkable)] verb (design 041 §3.3) +# LinkableRegistrarExt / #[derive(Linkable)] verb aimdb-data-contracts = { path = "../../aimdb-data-contracts", features = [ "linkable", ] } diff --git a/examples/tokio-mqtt-connector-demo/src/main.rs b/examples/tokio-mqtt-connector-demo/src/main.rs index b9d0344..28b64ec 100644 --- a/examples/tokio-mqtt-connector-demo/src/main.rs +++ b/examples/tokio-mqtt-connector-demo/src/main.rs @@ -141,8 +141,8 @@ async fn main() -> DbResult<()> { // Temperature sensors (outbound to MQTT) - using link_address() from key metadata. // TempIndoor needs per-link QoS/retain, so it stays on the raw builder (the - // escape hatch, design 041 §3.3); TempOutdoor/TempServerRoom need no extra - // options, so `.linked_to()` (from `#[derive(Linkable)]`) is the 80% path. + // escape hatch for per-link options); TempOutdoor/TempServerRoom need no + // extra options, so `.linked_to()` (from `#[derive(Linkable)]`) suffices. builder.configure::(SensorKey::TempIndoor, |reg| { reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) .source(indoor_temp_producer) diff --git a/examples/weather-mesh-demo/weather-hub/Cargo.toml b/examples/weather-mesh-demo/weather-hub/Cargo.toml index e9a3e79..73bb9c1 100644 --- a/examples/weather-mesh-demo/weather-hub/Cargo.toml +++ b/examples/weather-mesh-demo/weather-hub/Cargo.toml @@ -16,9 +16,8 @@ weather-mesh-common = { path = "../weather-mesh-common", features = [ "migratable", ] } aimdb-core = { path = "../../../aimdb-core", features = ["derive"] } -# `observability` makes `.observe()`'s signal gauge live (design 041 §3.2); -# without it core hands back an inert handle and no signal stats surface on -# record.list/record.get. +# `observability` makes `.observe()`'s signal gauge live; without it core hands +# back an inert handle and no signal stats surface on record.list/record.get. aimdb-tokio-adapter = { path = "../../../aimdb-tokio-adapter", features = [ "observability", ] } diff --git a/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml b/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml index 5845976..6a8d694 100644 --- a/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml +++ b/examples/weather-mesh-demo/weather-mesh-common/Cargo.toml @@ -24,5 +24,5 @@ aimdb-data-contracts = { path = "../../../aimdb-data-contracts", default-feature serde = { version = "1.0", default-features = false, features = ["derive"] } serde_json = { workspace = true, optional = true } # RNG is caller-supplied at the station; the contract impls only use the -# `Rng`/`RngExt` traits, so no `std_rng` here (design 041 §3.1.1). +# `Rng`/`RngExt` traits, so no `std_rng` here. rand = { version = "0.10.1", optional = true, default-features = false } diff --git a/examples/weather-mesh-demo/weather-station-beta/Cargo.toml b/examples/weather-mesh-demo/weather-station-beta/Cargo.toml index 70846f1..55fc6f1 100644 --- a/examples/weather-mesh-demo/weather-station-beta/Cargo.toml +++ b/examples/weather-mesh-demo/weather-station-beta/Cargo.toml @@ -11,7 +11,7 @@ name = "weather-station-beta" path = "src/main.rs" [features] -# `sim` is opt-in and dev-only (design 041 §3.1.4): it pulls the `Simulatable` +# `sim` is opt-in and dev-only: it pulls the `Simulatable` # impls + `rand` and wires `.simulate()`. The DEFAULT (production) build reads a # hardware source and contains no `rand` at all — `make check-no-sim` proves it. sim = [ diff --git a/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml b/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml index f8a92ce..b534144 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml +++ b/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml @@ -9,7 +9,7 @@ publish = false [features] default = ["embassy-runtime"] embassy-runtime = [] -# `sim` is opt-in and dev-only (design 041 §3.1.4): "develop against sim, flash +# `sim` is opt-in and dev-only: "develop against sim, flash # against silicon". It pulls the `Simulatable` impls + `rand` and wires # `.simulate()`; the default (flash) build reads a hardware source and has no # `rand` in its graph. diff --git a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs index 98fe016..6c0eeb4 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs @@ -98,7 +98,7 @@ fn humidity_profile() -> SimProfile { /// /// A real MCU deployment reads an ADC/I2C sensor here. The demo emits a fixed /// reading so the record still has exactly one writer and the flashed image -/// carries no `rand` (design 041 §3.1.4). +/// carries no `rand`. #[cfg(not(feature = "sim"))] async fn read_temperature( ctx: aimdb_core::RuntimeContext, From 39e2a4d95c42f0d3be07da1ba305b3bbe0cc127d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Tue, 7 Jul 2026 18:45:54 +0000 Subject: [PATCH 11/31] docs: update next development areas section to reference GitHub issues --- CONTRIBUTING.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9314ee3..a23b225 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -235,12 +235,7 @@ examples/ # Demo applications ## Next Development Areas -See `.github/copilot-instructions.md` for current implementation status and priorities: - -- **Kafka Connector** - Kafka integration using `rdkafka` -- **DDS Connector** - DDS protocol support -- **CLI Tools** - Introspection, monitoring, debugging commands -- **Performance** - Benchmarks and profiling infrastructure +See the [GitHub issues](https://github.com/aimdb-dev/aimdb/issues) for planned work and open feature requests. ## Getting Help From 219b709e266344c7784e644a2fb4cac2c9b87ef7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Tue, 7 Jul 2026 19:49:55 +0000 Subject: [PATCH 12/31] feat(docs): enhance README with evidence of data contracts and capabilities --- README.md | 44 +++++++++++++++++++++++++++++++------------- 1 file changed, 31 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 1c9223b..77d0e6e 100644 --- a/README.md +++ b/README.md @@ -45,6 +45,16 @@ Distributed systems spend most of their complexity budget translating between la - **The buffer defines how data moves.** No manual queue wiring, no separate transport config. - **No untyped boundaries.** Capabilities, like streaming, migration, observability and connectors, are unlocked by traits. +### The Evidence + +Everything here is backed by code, tests or committed benchmarks — not roadmap: + +- **Data contracts as schema** → the Rust type *is* the wire contract. No IDL, no codegen, [CI cross-compiles](Makefile) unchanged from Cortex-M to WASM. +- **Typed data migrations** → [`migration_chain!`](aimdb-data-contracts/src/migratable.rs) const-validates the chain and works `no_std`, so old and new nodes coexist on the wire. +- **Zero allocation per message** → [allocation baselines](aimdb-bench/data/baselines) across Tokio, Embassy and WASM. +- **Non-blocking producers** → synchronous [`produce()`](aimdb-core/src/typed_api.rs) calls and overwrite semantics on all buffers. +- **Identical buffer contracts across runtimes** → [shared conformance suite](aimdb-core/src/buffer/test_support.rs) validates SPMC Ring, SingleLatest and Mailbox on every adapter. + [The Next Era of Software Architecture Is Data-First](https://aimdb.dev/blog/data-driven-design) --- @@ -128,7 +138,19 @@ docker compose up | [**SingleLatest**](examples/hello-single-latest-async) | Only the current value matters | Feature flags, config, UI state | | [**Mailbox**](examples/hello-mailbox) / [**async Mailbox**](examples/hello-mailbox-async)| Latest instruction wins | Device commands, actuation, RPC | -**Capability traits** — opt-in, type-checked. Each one is a promise: implement it, and exactly one verb becomes available. The **tier** column says whether the capability may exist in a production binary. +**One async API across runtimes.** Tokio, Embassy, WASM — swap the runtime adapter, keep the code. → [How the runtime abstraction works](https://aimdb.dev/blog/building-aimdb-one-async-api) + +**Connectors that ship today:** MQTT, KNX, WebSocket, TCP, serial (COBS-framed UART) and Unix domain sockets. Writing your own is one trait impl. → [Connector status](#connectors) + +**Optional persistence.** The core is an in-memory data plane; [`aimdb-persistence`](aimdb-persistence) adds `.persist()` with a SQLite backend ([`aimdb-persistence-sqlite`](aimdb-persistence-sqlite)) for records whose history must survive restarts. + +Deep dives: [source/tap/transform](https://aimdb.dev/blog/source-tap-transform) · [schema migration](https://aimdb.dev/blog/schema-migration-without-ceremony) · [reactive pipelines](https://aimdb.dev/blog/reactive-pipelines) + +--- + +## Data contracts: one method per capability + +A capability trait is a compile-time promise about a schema type. Each one is opt-in and type-checked: implement it, and **exactly one method** becomes available on the registrar. The **tier** column says whether the capability may exist in a production binary. | Contract | Implement when… | Verb it unlocks | Tier | | --- | --- | --- | --- | @@ -152,11 +174,9 @@ builder.configure::(KEY, |reg| { — so a production build with `sim` off carries zero simulation code: no `T::simulate` impls, no `rand`, nothing to audit. CI proves it (`rand` is the tracer: `cargo tree -e normal -i rand` on the production feature set must come up empty). -**One async API across runtimes.** Tokio, Embassy, WASM — swap the runtime adapter, keep the code. → [How the runtime abstraction works](https://aimdb.dev/blog/building-aimdb-one-async-api) - -**Connectors that ship today:** MQTT, KNX, WebSocket. Writing your own is one trait impl. +**Old and new nodes coexist.** Migration steps are typed and bidirectional and `migration_chain!` validates the whole chain at compile time. The [roundtrip tests](aimdb-data-contracts/tests/migration_roundtrip.rs) exercise upgrade *and* downgrade across multi-step chains. This works `no_std`: an MCU can ingest a payload one schema version behind its own contract or downgrade its output for an older peer. -Deep dives: [data contracts](https://aimdb.dev/blog/data-contracts-deep-dive) · [source/tap/transform](https://aimdb.dev/blog/source-tap-transform) · [schema migration](https://aimdb.dev/blog/schema-migration-without-ceremony) · [reactive pipelines](https://aimdb.dev/blog/reactive-pipelines) +Deep dive: [data contracts](https://aimdb.dev/blog/data-contracts-deep-dive) --- @@ -172,7 +192,7 @@ A record is written by a `Source`, lands in a typed `Buffer` and fans out to in- Source ───► │ Buffer │ (typed) │ SPMC / SL / │ ───► Tap (another subscriber) │ Mailbox │ - └──────────────┘ ───► Link ──► MQTT / KNX / WebSocket + └──────────────┘ ───► Link ──► MQTT / KNX / WS / UDS / serial ``` The Rust type system enforces correctness at compile time, buffer semantics enforce flow guarantees at runtime and connectors wire to your infrastructure without an integration layer. The same code compiles for MCU, edge, cloud or browser — see [Platform Support](#platform-support) below. @@ -222,10 +242,14 @@ See the [MCP server docs](tools/aimdb-mcp/) for Claude Desktop and other editors | **MQTT** — `aimdb-mqtt-connector` | ✅ Ready | std, no_std | | **KNX** — `aimdb-knx-connector` | ✅ Ready | std, no_std | | **WebSocket** — `aimdb-websocket-connector` | ✅ Ready | std, wasm | +| **Serial (COBS/UART)** — `aimdb-serial-connector` | ✅ Ready | std, no_std | +| **Unix domain socket** — `aimdb-uds-connector` | ✅ Ready | std | | **TCP** — `aimdb-tcp-connector` | ✅ Ready | std, no_std client | | **Kafka** | 📋 Planned | std | | **Modbus** | 📋 Planned | std, no_std | +The serial, TCP and UDS connectors carry both record mirroring and the AimX remote-access protocol used by the CLI and MCP server. Typed records over multiple transports, from bare metal to cloud. + --- ### Platform Support @@ -243,14 +267,8 @@ See the [MCP server docs](tools/aimdb-mcp/) for Claude Desktop and other editors We're a small team building something ambitious. The fastest way to help is to take on a scoped piece of it. Each of these is sized for a few hours and includes file pointers, acceptance criteria and a place to ask questions: -- [#92 — `no_std` `Display` for `DbError` should include numeric fields](https://github.com/aimdb-dev/aimdb/issues/92) · 2–3h · core · embedded - [#93 — Minimal example: `hello-single-latest`](https://github.com/aimdb-dev/aimdb/issues/93) · 2–3h · docs -- [#95 — CLI: add `aimdb instance ping` subcommand](https://github.com/aimdb-dev/aimdb/issues/95) · 3–4h · cli -- [#96 — CI: fail on broken rustdoc links](https://github.com/aimdb-dev/aimdb/issues/96) · 1–2h · docs -- [#97 — Doctests for `BufferCfg` variants](https://github.com/aimdb-dev/aimdb/issues/97) · 2–3h · core · docs -- [#99 — Async example: `hello-mailbox-async`](https://github.com/aimdb-dev/aimdb/issues/99) · 2–3h · docs -- [#100 — Async example: `hello-single-latest-async`](https://github.com/aimdb-dev/aimdb/issues/100) · 2–3h · docs -- [#101 — Async example: `hello-spmc-ring-async`](https://github.com/aimdb-dev/aimdb/issues/101) · 2–3h · docs +- [#101 — Minimal example: `hello-spmc-ring-async`](https://github.com/aimdb-dev/aimdb/issues/101) · 2–3h · docs [See all good first issues →](https://github.com/aimdb-dev/aimdb/labels/good%20first%20issue) From acb842fec4787d9fb444dbdf287288db90b7ea73 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Wed, 8 Jul 2026 19:07:38 +0000 Subject: [PATCH 13/31] refactor(examples): simplify comments in main.rs for clarity --- README.md | 29 ++++++++++++-------------- examples/readme-quickstart/src/main.rs | 5 +---- 2 files changed, 14 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 77d0e6e..b3fe526 100644 --- a/README.md +++ b/README.md @@ -39,15 +39,15 @@ AimDB is not a storage engine. It's a typed data plane where the Rust type *is* ## Why AimDB exists -Distributed systems spend most of their complexity budget translating between layers. IDLs, codegen, serialization, schema registries and glue services. AimDB removes that layer by making **the Rust type the contract**: defined once, compiled unchanged from a `no_std` microcontroller to the browser. +Most of a distributed system's complexity is translation: IDLs, codegen, serialization glue, schema registries. AimDB deletes that layer by making **the Rust type the contract** — defined once, compiled unchanged from a `no_std` microcontroller to the browser. - **One type, every tier.** The same struct compiles for firmware and cloud. No conversion layer between them. - **The buffer defines how data moves.** No manual queue wiring, no separate transport config. - **No untyped boundaries.** Capabilities, like streaming, migration, observability and connectors, are unlocked by traits. -### The Evidence +### The receipts -Everything here is backed by code, tests or committed benchmarks — not roadmap: +None of this is roadmap. Every claim is backed by code, tests or committed benchmarks: - **Data contracts as schema** → the Rust type *is* the wire contract. No IDL, no codegen, [CI cross-compiles](Makefile) unchanged from Cortex-M to WASM. - **Typed data migrations** → [`migration_chain!`](aimdb-data-contracts/src/migratable.rs) const-validates the chain and works `no_std`, so old and new nodes coexist on the wire. @@ -103,10 +103,7 @@ async fn main() -> Result<(), Box> { }); }); - // `.run()` builds the database, collects every producer/consumer/transform - // future, and drives them all on a single `FuturesUnordered`. It blocks - // until shutdown. For programmatic access to the `AimDb` handle, call - // `.build().await?` directly — it returns `(AimDb, AimDbRunner)`. + // Build the db and drive every source/tap future until shutdown. builder.run().await?; Ok(()) } @@ -150,18 +147,18 @@ Deep dives: [source/tap/transform](https://aimdb.dev/blog/source-tap-transform) ## Data contracts: one method per capability -A capability trait is a compile-time promise about a schema type. Each one is opt-in and type-checked: implement it, and **exactly one method** becomes available on the registrar. The **tier** column says whether the capability may exist in a production binary. +Every capability is an opt-in trait on your schema type: implement it and **exactly one method** appears on the registrar. | Contract | Implement when… | Verb it unlocks | Tier | | --- | --- | --- | --- | -| [`Linkable`](https://aimdb.dev/blog/connectors-where-aimdb-meets-the-real-world) | the type crosses a per-URL byte boundary (MQTT/KNX/serial/UDS) | `.linked_from(url)` / `.linked_to(url)` (`#[derive(Linkable)]` for JSON) | wire (prod) | -| [`Streamable`](https://aimdb.dev/blog/streamable-crossing-boundaries) | the type streams as schema-named JSON (browser/WASM) | ws-connector `.register::()` | wire (prod) | +| [`Linkable`](https://aimdb.dev/blog/connectors-where-aimdb-meets-the-real-world) | the record is mirrored to/from an endpoint (MQTT, KNX, serial, UDS…) | `.linked_from(url)` / `.linked_to(url)` (`#[derive(Linkable)]` for JSON) | wire (prod) | +| [`Streamable`](https://aimdb.dev/blog/streamable-crossing-boundaries) | the record streams to browsers as schema-named JSON | ws-connector `.register::()` | wire (prod) | | [`Migratable`](https://aimdb.dev/blog/schema-migration-without-ceremony) | the schema evolved across versions | `migration_chain!` | wire (prod) | -| `Settable` | callers outside the AimDB thread set the record from a primitive | `SyncProducer::set_value(v)` | wire (prod) | -| `Observable` | the type carries a domain signal worth watching | `.observe()` → live signal metrics (last/min/max/mean on `record.list`/`record.get`) | introspection (prod, optional) | +| `Settable` | sync code outside AimDB sets the value | `SyncProducer::set_value(v)` | wire (prod) | +| `Observable` | the value is worth watching in production | `.observe()` → live last/min/max/mean on `record.list`/`record.get` | introspection (prod, optional) | | `Simulatable` | the type can generate realistic synthetic data | `.simulate(profile, rng)` | **dev-only — never ships in prod** | -`Simulatable` is the one exception to "opt-in, always safe": it lives behind the `simulatable` feature (never a default), and sim-to-real selection is a compile-time `#[cfg]` in your app — +`Simulatable` is the odd one out: it lives behind the `simulatable` feature (never a default) and switching from simulated to real data is one `#[cfg]` in your app: ```rust builder.configure::(KEY, |reg| { @@ -172,9 +169,9 @@ builder.configure::(KEY, |reg| { }); ``` -— so a production build with `sim` off carries zero simulation code: no `T::simulate` impls, no `rand`, nothing to audit. CI proves it (`rand` is the tracer: `cargo tree -e normal -i rand` on the production feature set must come up empty). +Build with `sim` off and the simulation code is *gone* — no `T::simulate` impls, no `rand`, nothing to audit. -**Old and new nodes coexist.** Migration steps are typed and bidirectional and `migration_chain!` validates the whole chain at compile time. The [roundtrip tests](aimdb-data-contracts/tests/migration_roundtrip.rs) exercise upgrade *and* downgrade across multi-step chains. This works `no_std`: an MCU can ingest a payload one schema version behind its own contract or downgrade its output for an older peer. +**Old and new nodes coexist.** Migration steps are typed and bidirectional and `migration_chain!` checks the whole chain at compile time — [roundtrip tests](aimdb-data-contracts/tests/migration_roundtrip.rs) cover upgrade *and* downgrade across multi-step chains. It's `no_std` too: even an MCU can accept a payload one schema version behind or downgrade its output for an older peer. Deep dive: [data contracts](https://aimdb.dev/blog/data-contracts-deep-dive) @@ -195,7 +192,7 @@ A record is written by a `Source`, lands in a typed `Buffer` and fans out to in- └──────────────┘ ───► Link ──► MQTT / KNX / WS / UDS / serial ``` -The Rust type system enforces correctness at compile time, buffer semantics enforce flow guarantees at runtime and connectors wire to your infrastructure without an integration layer. The same code compiles for MCU, edge, cloud or browser — see [Platform Support](#platform-support) below. +Types check correctness at compile time, buffers enforce flow semantics at runtime and connectors bridge to your infrastructure, with no integration layer in between. The same code compiles for MCU, edge, cloud or browser — see [Platform Support](#platform-support) below. --- diff --git a/examples/readme-quickstart/src/main.rs b/examples/readme-quickstart/src/main.rs index 7bae8ab..b136040 100644 --- a/examples/readme-quickstart/src/main.rs +++ b/examples/readme-quickstart/src/main.rs @@ -29,10 +29,7 @@ async fn main() -> Result<(), Box> { }); }); - // `.run()` builds the database, collects every producer/consumer/transform - // future, and drives them all on a single `FuturesUnordered`. It blocks - // until shutdown. For programmatic access to the `AimDb` handle, call - // `.build().await?` directly — it returns `(AimDb, AimDbRunner)`. + // Build the db and drive every source/tap future until shutdown. builder.run().await?; Ok(()) } From d6a66bca33e12237a0facb4b871263d9652af545 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Wed, 8 Jul 2026 19:15:15 +0000 Subject: [PATCH 14/31] feat(observable): enhance documentation on feature layering and recording behavior --- aimdb-data-contracts/src/lib.rs | 19 ++++++++++++++++++- aimdb-data-contracts/src/observable.rs | 5 +++++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/aimdb-data-contracts/src/lib.rs b/aimdb-data-contracts/src/lib.rs index 763bb11..f6f5c57 100644 --- a/aimdb-data-contracts/src/lib.rs +++ b/aimdb-data-contracts/src/lib.rs @@ -171,7 +171,7 @@ pub trait Settable: SchemaType { /// Project a schema type onto a numeric domain signal. /// /// The trait's kernel is the numeric projection: implement it, call -/// [`ObservableRegistrarExt::observe`](observable::ObservableRegistrarExt::observe), +/// [`ObservableRegistrarExt::observe`], /// and the signal is folded into live last/min/max/mean statistics that surface /// on `record.list` / `record.get` and stage profiling. The signal can also feed /// threshold checks, alerting, and aggregation. @@ -179,6 +179,23 @@ pub trait Settable: SchemaType { /// Presentation is not part of the trait: `.log(node_id)` (also on the ext /// trait) formats a human-readable line from `Debug` + /// [`SIGNAL`](Observable::SIGNAL)/[`UNIT`](Observable::UNIT). +/// +/// # Feature layering +/// +/// Three layers, each useful without the next: +/// +/// 1. **This trait** — always compiled, no features. [`signal()`](Observable::signal) +/// plus the `SIGNAL`/`UNIT` labels are enough for hand-wired `.tap()`s, +/// threshold checks, and generic code over `T: Observable`. +/// 2. **`observable` (this crate)** — unlocks the registrar verbs `.observe()` +/// and `.log()`. Gated only because the ext trait needs `alloc` and +/// `aimdb-core`. +/// 3. **`observability` (`aimdb-core`)** — the metrics backend. When it is off, +/// `.observe()` still compiles and runs but its gauge is inert: updates are +/// no-ops and nothing surfaces on `record.list` / `record.get` (see +/// `RecordRegistrar::signal_gauge`). `.log()` is unaffected. This lets +/// constrained targets ship `Observable` contracts while compiling the +/// metrics cost away. pub trait Observable: SchemaType { /// The numeric type of the signal (e.g., `f32`, `f64`, `i32`). /// diff --git a/aimdb-data-contracts/src/observable.rs b/aimdb-data-contracts/src/observable.rs index 7e0248e..503b6a3 100644 --- a/aimdb-data-contracts/src/observable.rs +++ b/aimdb-data-contracts/src/observable.rs @@ -55,6 +55,11 @@ where /// Feed `T::signal()` into the record's signal gauge (last/min/max/mean), /// visible via `record.list` / `record.get` / stage profiling. /// + /// Recording requires `aimdb-core`'s `observability` feature: without it + /// [`RecordRegistrar::signal_gauge`] hands back an inert gauge and this + /// tap compiles and runs but records nothing. `.log()` does not depend on + /// that feature. + /// /// Bounded `T::Signal: Into` here (not on the trait), so `f32`/`i32`/ /// `u32` signals qualify while a type with an exotic signal can still /// implement `Observable` and write its own tap. From 2c24ec3380ed12503f96a398c4fd698a8549ea60 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 9 Jul 2026 11:07:53 +0000 Subject: [PATCH 15/31] =?UTF-8?q?docs(design):=20add=20042=20=E2=80=94=20p?= =?UTF-8?q?ublic=20weather=20mesh=20flagship=20spec?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Specifies how the weather-mesh demo becomes a public, joinable flagship instance: string-keyed slot pool in the hub (no core changes), EMQX Cloud broker with per-station credentials/ACLs, self-serve admission via GitHub device flow in a new generic 'aimdb join' CLI subcommand, an open provisioning endpoint format, privacy/retention policy, and the open Embassy TLS question. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01BFw7Pz4cjF6dUT9Zv2T8t9 --- .../042-public-weather-mesh-flagship.md | 240 ++++++++++++++++++ 1 file changed, 240 insertions(+) create mode 100644 docs/design/042-public-weather-mesh-flagship.md diff --git a/docs/design/042-public-weather-mesh-flagship.md b/docs/design/042-public-weather-mesh-flagship.md new file mode 100644 index 0000000..de3aca0 --- /dev/null +++ b/docs/design/042-public-weather-mesh-flagship.md @@ -0,0 +1,240 @@ +# 042 — Public weather mesh flagship + +**Status:** 📝 Proposed + +**Scope:** `examples/weather-mesh-demo` (hub slot pool, station config), `tools/aimdb-cli` (new `join` subcommand), a small closed-source provisioning service (private ops repo), EMQX Cloud broker setup, and the public landing/dashboard pages. Possible follow-up in `aimdb-mqtt-connector` (Embassy TLS, §8.1). + +**Goal:** turn the three-station weather-mesh demo into a single public, hosted mesh that anyone can join with one command — a live demonstration of typed data contracts across MCU → edge → cloud, and a source of real propose/admit traffic for dogfooding. + +--- + +## 1. Context + +`examples/weather-mesh-demo` is a working three-tier demo: typed, versioned contracts (`Temperature`, `Humidity`, `DewPoint`, `GpsLocation`, including a `TemperatureV1 → V2` migration), an Open-Meteo-fed station (`alpha`), an Embassy/STM32H563 hardware station (`gamma`), and a Tokio hub. A hosted instance already streams live to a browser dashboard, and the mesh is queryable by any LLM agent via the public MCP endpoint (`aimdb.dev/mcp`, over the `transport-tcp` client transport — [`aimdb-client/src/endpoint.rs`](../../aimdb-client/src/endpoint.rs)). + +What separates this from "a public mesh anyone can join" is membership: the hub configures exactly three stations via closed `TempKey`/`HumidityKey`/`DewPointKey` enums with compile-time `#[link_address]` topics ([`weather-mesh-common/src/lib.rs`](../../examples/weather-mesh-demo/weather-mesh-common/src/lib.rs)). Admitting a fourth station today means a new enum variant and a recompile. The demo broker config is also demo-grade: [`docker/mosquitto.conf`](../../examples/weather-mesh-demo/docker/mosquitto.conf) allows anonymous, unlimited connections. + +This spec closes both gaps and defines the join UX end to end. + +## 2. Decisions + +| # | Decision | Rationale | +|---|---|---| +| D1 | **One flagship instance**, operated by the project — not a multi-tenant, self-serve platform. | Scope control; the value is a live artifact, not a product. | +| D2 | Hub membership becomes a **bounded slot pool of N string-keyed records** (`StringKey::intern`), configured at startup. | No core changes, no enum surgery; N doubles as the abuse cap. See §4. | +| D3 | Genuinely dynamic (post-`run()`) registration is **out of scope** — deferred to the knowledge-graph layer work, which faces the same static/dynamic-plane problem. Solve it once there. | Avoid solving the hard problem twice. | +| D4 | Broker is **EMQX Cloud** (managed, EU region), replacing self-hosted Mosquitto for the public instance. | Real auth/ACLs/rate limits with zero broker ops; keeps public write ingress off the web host. See §5. | +| D5 | Admission is **self-serve, gated by GitHub identity** (OAuth device flow in the CLI), not a human queue. | Identity substitutes for approval: one active slot per GitHub account, minimum account age, global cap N, post-hoc revocation, and a kill switch to pause admissions. Joining is one command; the maintainer moderates after the fact instead of gating before. See §6. | +| D6 | Credentials are issued by a **small closed-source provisioning service** (server-side, private ops repo). Everything that runs on a contributor's machine stays **open source**. | Client-side secrets are extractable — closed-source client code is obfuscation, not security, and closed-source Rust is impractical for `no_std` targets anyway. Server-side is the only place the EMQX API key and abuse control can actually live. See §6.2. | +| D7 | The join client is a new **`aimdb join` subcommand** in the existing CLI, generic over a provisioning endpoint URL — not a mesh-specific tool, and never a direct client of the EMQX management API. | One tool joins the mesh *and* verifies data arrival over AimX; "how do new edge nodes get admitted" is a product question, not a demo hack. See §7. | +| D8 | Station GPS is **coarsened to 2 decimal places (~1 km)** before publication; precise location is never collected. | The dashboard publishes contributor home locations otherwise. See §9. | +| D9 | Retention: **30 days of live data**; slots silent for 30 days are recycled; per-station credentials are individually revocable. | Policy stated up front on the join page. Storage cost is negligible at weather cadence — these are policy choices, not cost choices. | +| D10 | Slot cap **N = 64** initially (env-configurable, raised by restart). | Large enough that "mesh full" is unlikely at launch; small enough to bound broker connections and dashboard fan-out. | + +## 3. Non-goals + +- Multi-tenant / bring-your-own-cloud hosting. +- Runtime (post-startup) record registration in `aimdb-core` (D3). +- Any closed-source code in the client path — connector, station, or CLI (D6). +- Proprietary application logic in this deployment. The private ops repo holds deployment configuration and the provisioning shim only; the hub and contracts are the OSS crates. +- Topic validation or schema negotiation at admission time — a station that publishes malformed payloads is rejected by the existing contract deserialization at the hub, visibly (§9). + +## 4. Hub: the slot pool + +The closed enums are a property of the *example*, not the engine. `aimdb-core` already provides everything needed: + +- [`StringKey::intern`](../../aimdb-core/src/record_id.rs) exists precisely for runtime-constructed keys. +- `reg.link_from(&topic)` already takes a runtime string (the current hub passes `key.link_address()` through a `String`). +- Inbound runtime topic resolution is covered by design [018](./018-M7-dynamic-mqtt-topics.md). + +The hub change is therefore a loop: + +```rust +let n: u16 = std::env::var("MESH_SLOTS") + .ok().and_then(|v| v.parse().ok()).unwrap_or(64); + +for slot in 0..n { + let key = StringKey::intern(format!("station.{slot}.temperature")); + let topic = format!("mqtt://station/{slot}/temperature"); + builder.configure::(key, |reg| { + reg.buffer(BufferCfg::SpmcRing { capacity: 100 }); + reg.observe(); + reg.link_from(&topic) + .with_deserializer(|_ctx, data: &[u8]| Temperature::from_bytes(data)) + .finish(); + }); +} +// … same loop for Humidity; DewPoint derived per slot as today. +``` + +Properties: + +- **No recompile per station.** Admission assigns an unused slot; the record and subscription already exist. +- **Slot count is fixed per process start.** Raising N is a hub restart, which is acceptable for a single flagship instance (D1). +- **N is the abuse cap.** The broker ACL (§5) is the enforcement point; N bounds worst-case hub state and dashboard fan-out. +- The dashboard filters to slots with recent data, so empty slots are invisible rather than a wall of grey tiles. + +Topic scheme: `station/{slot}/{temperature|humidity|dew_point}` — flat, slot-scoped, and matching the ACL pattern below. The existing `sensors/{alpha,beta,gamma}/…` topics remain in the local docker-compose demo, which is unchanged by this spec. + +The typed-contract story is unaffected: schema types stay typed end to end; only the *key namespace* moves from a closed enum to bounded strings. The three-enum version remains in the repo as the local demo, where compile-time keys are the point. + +## 5. Broker: EMQX Cloud + +The public instance uses a managed EMQX Cloud deployment (EU region) instead of self-hosted Mosquitto/EMQX: + +- **Isolation:** the public write ingress no longer shares a host with the project website; broker abuse cannot starve the web server. +- **Auth & ACL:** one credential per station, created at admission via the EMQX Cloud API, restricted to publish on `station/{slot}/#` only. The hub uses a separate subscribe-only credential. +- **Limits:** per-client rate and connection limits configured at the broker — the only place abuse control is enforceable (D6). +- **Revocation:** deleting a station's credential evicts it without touching any other station (D9). + +The hub connects **outbound** to the broker over `mqtts://…:8883` — supported today by the Tokio client ([`tokio_client.rs`](../../aimdb-mqtt-connector/src/tokio_client.rs), `use-native-tls`). The serverless tier is TLS-only, which is the right default for internet-facing stations but creates one open problem for the MCU path — see §8.1 before committing to a tier. + +## 6. Admission and provisioning + +### 6.1 Flow + +``` +contributor CLI GitHub provisioning svc EMQX Cloud + │ aimdb join │ │ │ + │── device-code request ─────────▶│ │ │ + │◀─ user code + verify URL ───────│ │ │ + │ (user confirms code at │ │ │ + │ github.com/login/device) │ │ │ + │◀─ access token (poll) ──────────│ │ │ + │ prompts: station name, city │ │ │ + │── POST /v1/join {token, app} ──────────────────────────── ▶│ │ + │ │◀── verify identity ──────│ │ + │ │ check admission rules │── create user + ACL ▶│ + │ │ │ allocate slot │ + │◀────────────── station profile (station.toml) ──────────── │ │ +``` + +- **GitHub device flow** replaces the issue/Action/email plumbing: the CLI prints a short code, the user confirms it at `github.com/login/device`, and the CLI polls for the access token. The OAuth app's client ID is public by design — nothing secret ships in the CLI. No scopes beyond identity are requested. +- **Station metadata** (name, city-level location) is collected by interactive CLI prompts and sent in the join request — no form, no issue template. +- **Admission rules are enforced server-side**, where they cannot be bypassed: one active slot per GitHub account, minimum account age (e.g. 90 days), global cap N (D10), and an admissions kill switch. Mass abuse thus requires mass *aged* GitHub accounts, and N bounds the blast radius regardless. +- **Moderation is post-hoc**, proportionate to the exposure: payloads are schema-validated at the hub, each station can only write its own `station/{slot}/#` topics, and revoking one credential evicts one station (D9). The worst case is garbage numbers in the offender's own slot until revoked. +- The public roster of who's on the mesh is the station pages (§8) — no admission queue needed for transparency. + +### 6.2 Provisioning service + +A thin HTTPS shim, closed source, in the private ops repo. It is the *only* holder of the EMQX Cloud API key. Responsibilities: verify GitHub identities, enforce the admission rules (§6.1), allocate slots, create/delete broker credentials and ACLs, return station profiles, recycle silent slots (D9). State is one small table: `github_account ↔ slot ↔ broker credential`. + +What it is **not**: it is not in the data path (stations speak MQTT to the broker directly), and it is not a client-side component. The boundary is the same one the deployment already draws — ops/config private, contracts/connectors/examples open. + +### 6.3 Provisioning endpoint format (open) + +The request/response format is documented openly so `aimdb join` (§7) is generic, not flagship-specific. Version 1: + +``` +POST /v1/join +Content-Type: application/json + +{ + "auth": { "kind": "github", "token": "gho_…" }, + "app": { "name": "graz-balcony", "location": "Graz" } +} +``` + +`auth.kind` keeps the format deployment-agnostic: the flagship accepts `github`; a private deployment can accept `claim-token` (pre-issued one-time tokens, no public self-serve) with the same envelope. + +``` +200 OK +{ + "profile_version": 1, + "station_id": "slot-17", + "broker": { + "url": "mqtts://xxxx.eu-central-1.emqx.cloud:8883", + "username": "station-17", + "password": "…" + }, + "app": { + "name": "graz-balcony", + "lat": 47.07, + "lon": 15.44 + } +} +``` + +- `app` is an opaque, deployment-specific map: the CLI collects it via prompts (driven by the deployment's docs) and passes it through; the service validates, coarsens the location (D8), and echoes the final values back in the profile. Other AimDB deployments define their own `app` contents. +- Errors: `403` (identity rejected: account too new, admissions paused), `409` (account already holds a slot — body points at the existing profile), `503` (mesh full). Bodies carry a human-readable `message` the CLI prints as-is. +- `profile_version` is versioned and backward-compatible from day one: the CLI ships on the workspace release cadence and must not need a release in lockstep with mesh-side changes. + +## 7. `aimdb join` + +New subcommand in [`tools/aimdb-cli`](../../tools/aimdb-cli) (binary is already named `aimdb`, clap-based): + +``` +aimdb join [--token ] [--out station.toml] +``` + +- Default path: runs the GitHub device flow, prompts for the `app` fields, calls `/v1/join` (§6.3), and writes the profile to `station.toml` (or `--out`). `--token` selects the `claim-token` auth kind instead, for deployments without public self-serve. +- **Generic by construction:** the endpoint URL is an argument, the `app` map is passed through untouched, and no weather-mesh types appear in the CLI. `join` is a product capability — "how a new edge node gets admitted to a deployment" — that debuts on the flagship. +- **Never talks to the EMQX management API.** Doing so would require the admin key in a public binary — the same client-side-secret problem D6 exists to prevent, except leaking the key that can mint credentials for the whole broker. +- One new dependency (`reqwest` with rustls), gated behind a `join` cargo feature, on by default in released binaries. + +The three-command loop this enables is the demo: + +``` +$ aimdb join https://mesh.aimdb.dev + → To authorize, enter code WDJB-MJHT at https://github.com/login/device + ✓ authorized as @alexschnoerch + Station name: graz-balcony + Location (city): Graz + ✓ slot-17 "graz-balcony" — credentials written to station.toml + +$ cargo run -p weather-station -- --config station.toml & + +$ aimdb record list --url tcp://aimdb.dev:7433 + ✓ station.17.temperature 21.4 °C 2s ago +``` + +Join, run, and verify through the hub's own remote-access protocol — one tool, with the last command quietly demonstrating AimX introspection. + +## 8. Station UX + +Three tiers, ordered by commitment: + +- **Tier 0 — look (30 s):** `aimdb.dev/mesh` — live map + charts, and a copy-pasteable MCP snippet ("point your agent at `aimdb.dev/mcp` and ask which station is coldest"). Already running; landing-page work only. +- **Tier 1 — run locally (5 min, no admission):** the existing docker-compose demo, unchanged. Gives impatient visitors something immediate and shows the contracts are ordinary, readable Rust. +- **Tier 2 — join the public mesh:** §6 flow, then §7 commands. Target: **under 10 minutes from `aimdb join` to dot on the map** — with self-serve admission there is no waiting step left in the funnel. On first successful publish, the station prints the direct URL to its live chart and a ready-made MCP prompt for its own data — the shareable moment. + +**Hardware path:** same `station.toml`, consumed at firmware build time and flashed via probe-rs as `weather-station-gamma` does today — contingent on §8.1. + +**Station identity pages** (`aimdb.dev/mesh/`): live chart, uptime, last reading. Gives contributors a link to share and a reason to keep stations running — which sustains the propose/admit traffic this exists to generate. + +### 8.1 Open problem: Embassy TLS + +EMQX Cloud serverless is TLS-only, and [`embassy_client.rs`](../../aimdb-mqtt-connector/src/embassy_client.rs) has no TLS support — so the MCU station cannot currently connect to the public broker directly. Options: + +1. **Add TLS to the Embassy MQTT client** (e.g. `embedded-tls`). Real work, but a valuable connector feature in its own right, and it keeps the "MCU speaks directly to a managed cloud broker" claim true. **Recommended.** +2. EMQX Cloud dedicated tier with a plain-TCP listener. Solves it with money and reopens the unencrypted-public-broker question this migration closes. +3. A local bridge/gateway in front of the MCU. Cheapest; quietly breaks the direct-to-cloud story. + +This is the one place the broker choice touches the product. Decide before committing to a broker tier; option 1 can land after launch if the hardware station joins later than the cloud-fed path. + +## 9. Privacy, lifecycle, and failure UX + +- **Location:** only city-level location is collected at admission; published coordinates are rounded to 2 decimals (~1 km) (D8). A short privacy note on the join page states exactly what station metadata is published and for how long. +- **Retention:** 30 days of live data; optionally, downsampled aggregates kept indefinitely (D9). +- **Slot lifecycle:** silent for 30 days → credential revoked, slot recycled, stated up front (D9). +- **Schema errors are a feature:** a malformed payload is rejected by contract deserialization at the hub; the station's own logs say *what contract was violated and what was expected*. The differentiator, demonstrated at the exact moment a user is paying attention. +- **Failure paths:** rejected identity or full mesh → the CLI prints the service's `message` as-is (§6.3); revoked slot → the station says so instead of retrying forever; re-running `aimdb join` with an account that already holds a slot points at the existing profile rather than erroring opaquely. + +## 10. Sequencing + +1. **Hub slot pool** (§4) — OSS, `examples/weather-mesh-demo`, independent of everything else. +2. **EMQX Cloud setup** (§5) — EU region; resolve §8.1 before choosing the tier. +3. **Provisioning service** (§6.2) — private ops repo; includes the GitHub OAuth app registration, identity/admission checks, and slot-recycling job. +4. **`aimdb join`** (§7) + the open endpoint format doc (§6.3) — OSS. +5. **Landing page, station pages, privacy note** (§8, §9). +6. Launch; Embassy TLS (§8.1 option 1) may follow as the hardware-station on-ramp. + +Steps 1 and 4 are pure OSS and unblocked today. Step 3 is the only new operational component, and it is a thin shim in front of the EMQX Cloud API. + +## 11. References + +- [016 — RecordKey trait](./016-M6-record-key-trait.md) (`StringKey`, `intern`) +- [018 — Dynamic MQTT topics](./018-M7-dynamic-mqtt-topics.md) (runtime topic resolution) +- [041 — Data contracts as first-class capabilities](./041-data-contracts-integration.md) (contract tiers; `Simulatable` stays dev-only) +- [`examples/weather-mesh-demo`](../../examples/weather-mesh-demo) — the demo this spec extends +- [`tools/aimdb-cli`](../../tools/aimdb-cli), [`aimdb-client`](../../aimdb-client) — AimX client surface, `transport-tcp` From 2249dacbb55382977bfdff058da67a2c8c1d9ca4 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 9 Jul 2026 12:25:17 +0000 Subject: [PATCH 16/31] docs(design): add MCU join path (Embassy) to weather mesh flagship spec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add §8.2 describing the hardware-station user flow: aimdb join on the flashing host, build-time config embedding via MESH_CONFIG in build.rs with local-demo fallback, and the trust model for credentials in the firmware image. Ground §8.1 option 1 with the concrete TLS work (TRNG entropy, SNTP time source) and update sequencing. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01BFw7Pz4cjF6dUT9Zv2T8t9 --- .../042-public-weather-mesh-flagship.md | 34 +++++++++++++++++-- 1 file changed, 31 insertions(+), 3 deletions(-) diff --git a/docs/design/042-public-weather-mesh-flagship.md b/docs/design/042-public-weather-mesh-flagship.md index de3aca0..5628492 100644 --- a/docs/design/042-public-weather-mesh-flagship.md +++ b/docs/design/042-public-weather-mesh-flagship.md @@ -198,7 +198,7 @@ Three tiers, ordered by commitment: - **Tier 1 — run locally (5 min, no admission):** the existing docker-compose demo, unchanged. Gives impatient visitors something immediate and shows the contracts are ordinary, readable Rust. - **Tier 2 — join the public mesh:** §6 flow, then §7 commands. Target: **under 10 minutes from `aimdb join` to dot on the map** — with self-serve admission there is no waiting step left in the funnel. On first successful publish, the station prints the direct URL to its live chart and a ready-made MCP prompt for its own data — the shareable moment. -**Hardware path:** same `station.toml`, consumed at firmware build time and flashed via probe-rs as `weather-station-gamma` does today — contingent on §8.1. +**Hardware path:** same `station.toml`, consumed at firmware build time and flashed via probe-rs — see §8.2. **Station identity pages** (`aimdb.dev/mesh/`): live chart, uptime, last reading. Gives contributors a link to share and a reason to keep stations running — which sustains the propose/admit traffic this exists to generate. @@ -206,12 +206,40 @@ Three tiers, ordered by commitment: EMQX Cloud serverless is TLS-only, and [`embassy_client.rs`](../../aimdb-mqtt-connector/src/embassy_client.rs) has no TLS support — so the MCU station cannot currently connect to the public broker directly. Options: -1. **Add TLS to the Embassy MQTT client** (e.g. `embedded-tls`). Real work, but a valuable connector feature in its own right, and it keeps the "MCU speaks directly to a managed cloud broker" claim true. **Recommended.** +1. **Add TLS to the Embassy MQTT client** (e.g. `embedded-tls`). Real work — the TLS session itself, entropy (the STM32H5 on-chip TRNG, which `weather-station-gamma` already binds via the `rng` interrupt), and a time source for certificate validation (e.g. an SNTP task, since the board has no RTC battery) — but a valuable connector feature in its own right, and it keeps the "MCU speaks directly to a managed cloud broker" claim true. **Recommended.** 2. EMQX Cloud dedicated tier with a plain-TCP listener. Solves it with money and reopens the unencrypted-public-broker question this migration closes. 3. A local bridge/gateway in front of the MCU. Cheapest; quietly breaks the direct-to-cloud story. This is the one place the broker choice touches the product. Decide before committing to a broker tier; option 1 can land after launch if the hardware station joins later than the cloud-fed path. +### 8.2 MCU join path (Embassy) + +Premise: the `aimdb` CLI runs on the **host machine used for flashing** — the same laptop that drives probe-rs, exactly `weather-station-gamma`'s workflow today. The reference board (Nucleo STM32H563ZI) uses Ethernet with DHCP, so there is **zero on-device network configuration** — no Wi-Fi credentials, nothing to type on the device. + +The flow is the cloud-station flow with the flash step swapped in for "run the binary": + +``` +$ aimdb join https://mesh.aimdb.dev --out station.toml # §6 — same self-serve flow, on the host + +$ cd examples/weather-mesh-demo/weather-station-gamma +$ MESH_CONFIG=../station.toml cargo run --release # build → probe-rs flash → defmt logs + +$ aimdb record list --url tcp://aimdb.dev:7433 # §7 — same verification, same tool +``` + +**Mechanism — build-time config embedding.** Today gamma hardcodes `MQTT_BROKER_IP`/`MQTT_BROKER_PORT` as consts with an "update this" comment. Replace that with the station profile consumed at build time: [`build.rs`](../../examples/weather-mesh-demo/weather-station-gamma/build.rs) reads the profile at the path in `MESH_CONFIG`, generates a `station_config.rs` into `OUT_DIR` (`include!`d by `main.rs`), and declares `cargo:rerun-if-env-changed=MESH_CONFIG` / `cargo:rerun-if-changed=`. With `MESH_CONFIG` unset, the generated config falls back to the local docker-compose demo defaults, so the existing tier-1 demo stays zero-setup. + +Why build-time rather than a config flash partition or serial provisioning: + +- Per-station firmware builds are the norm for this audience — the crate already pins its toolchain (`rust-toolchain.toml`) and builds locally; there is no identical-binary distribution problem to solve. +- It reuses the exact workflow embedded Rust developers already know: `cargo run` under the existing probe-rs runner *is* flash + RTT/defmt streaming. The §8 success banner (station chart URL + ready-made MCP prompt) prints over defmt, so the dot-on-the-map moment happens in the same terminal that flashed the board. +- The split build/flash workflow ([`flash.sh`](../../examples/weather-mesh-demo/weather-station-gamma/flash.sh): build in the dev container, flash from the host) keeps working unchanged — `MESH_CONFIG` matters only at build time. +- A config partition (probe-rs writing a config blob at a fixed flash address, firmware reading it at boot) is the natural v2 if prebuilt-image distribution ever becomes desirable; it isn't needed for one flagship's contributors. + +**Trust model:** the slot-scoped credential ends up inside the firmware image in `target/`, which is equivalent to `station.toml` sitting on the same disk — it is the user's own credential, scoped to their own `station/{slot}/#` topics. Built images shouldn't be shared; revocation (D9) covers mistakes. + +**Contingency:** this path reaches the public broker only once §8.1 is resolved — until then, `MESH_CONFIG`-built firmware can target the local demo broker only. + ## 9. Privacy, lifecycle, and failure UX - **Location:** only city-level location is collected at admission; published coordinates are rounded to 2 decimals (~1 km) (D8). A short privacy note on the join page states exactly what station metadata is published and for how long. @@ -227,7 +255,7 @@ This is the one place the broker choice touches the product. Decide before commi 3. **Provisioning service** (§6.2) — private ops repo; includes the GitHub OAuth app registration, identity/admission checks, and slot-recycling job. 4. **`aimdb join`** (§7) + the open endpoint format doc (§6.3) — OSS. 5. **Landing page, station pages, privacy note** (§8, §9). -6. Launch; Embassy TLS (§8.1 option 1) may follow as the hardware-station on-ramp. +6. Launch; the hardware-station on-ramp — Embassy TLS (§8.1 option 1) plus the gamma build-time config plumbing (§8.2) — may follow after launch. The `build.rs` embedding in §8.2 is small and independent, so it can also land with step 1. Steps 1 and 4 are pure OSS and unblocked today. Step 3 is the only new operational component, and it is a thin shim in front of the EMQX Cloud API. From ea5b5d653944202885848de952a713b84551d7f4 Mon Sep 17 00:00:00 2001 From: "sounds.like.lx" <147444674+lxsaah@users.noreply.github.com> Date: Thu, 9 Jul 2026 19:05:22 +0200 Subject: [PATCH 17/31] docs(plan): add implementation plan for 042 public weather mesh flagship (#172) Translates the 042 design spec into sequenced work packages with file-level detail for the OSS items (hub slot pool, aimdb join, generic cloud station, gamma build-time config, Embassy TLS) and checklists for the external ones (EMQX Cloud, provisioning service, website). Claude-Session: https://claude.ai/code/session_012h2fegrHHQqRFT48dW8oZW Co-authored-by: Claude --- docs/plan/042-public-weather-mesh-flagship.md | 243 ++++++++++++++++++ 1 file changed, 243 insertions(+) create mode 100644 docs/plan/042-public-weather-mesh-flagship.md diff --git a/docs/plan/042-public-weather-mesh-flagship.md b/docs/plan/042-public-weather-mesh-flagship.md new file mode 100644 index 0000000..a71954c --- /dev/null +++ b/docs/plan/042-public-weather-mesh-flagship.md @@ -0,0 +1,243 @@ +# Implementation plan — 042 Public weather mesh flagship + +**Design:** [042 — Public weather mesh flagship](../design/042-public-weather-mesh-flagship.md) + +**Status:** 📝 Planned + +This plan translates the design's sequencing (§10) into concrete work packages +(WPs). Each WP is marked by where the work happens: + +- **OSS** — this repository; file-level detail below. +- **External** — the private ops repo, the EMQX Cloud console, or the + aimdb.dev website. Tracked here as checklists so the launch dependency + graph is complete, but implemented elsewhere. + +Codebase facts this plan relies on (verified against the current tree): + +- `StringKey::intern` ([`aimdb-core/src/record_id.rs`](../../aimdb-core/src/record_id.rs)) + and `reg.link_from(&topic)` with a runtime string already work — the hub + already passes `key.link_address()` through a `String`. The slot pool + needs **no `aimdb-core` changes**, as the design claims (D2). +- The Tokio MQTT client supports `mqtts://` and URL-embedded credentials + ([`tokio_client.rs`](../../aimdb-mqtt-connector/src/tokio_client.rs), + `use-native-tls`; credentials are parsed from the connector URL), so hub + and cloud stations can authenticate to EMQX Cloud today. The Embassy + client has no TLS — design §8.1, WP7. +- The CLI ([`tools/aimdb-cli`](../../tools/aimdb-cli), binary `aimdb`) is + clap-based, one file per command in `src/commands/`; it has no HTTP + client dependency yet. +- `weather-station-gamma` hardcodes `MQTT_BROKER_IP`/`MQT_BROKER_PORT` + consts in `main.rs`; its `build.rs` is currently just linker args — the + §8.2 config generation slots in there. + +--- + +## WP1 — Hub slot pool (OSS) — design §4 + +**Crate:** `examples/weather-mesh-demo/weather-hub` + +The hub gains an env-gated *mesh mode* alongside the existing three-enum +demo mode. The local docker-compose demo must remain byte-for-byte +unchanged in behaviour (design §4: "the existing `sensors/{alpha,beta,gamma}/…` +topics remain in the local docker-compose demo"). + +Changes in [`weather-hub/src/main.rs`](../../examples/weather-mesh-demo/weather-hub/src/main.rs): + +1. **Mode switch:** if `MESH_SLOTS` is set, configure the slot pool; if + unset, run the current `TempKey`/`HumidityKey` enum configuration + untouched. +2. **Slot pool loop:** for `slot in 0..n` (`n` = `MESH_SLOTS`, default 64, + D10): + - `StringKey::intern(format!("station.{slot}.temperature"))`, buffer + `SpmcRing { capacity: 100 }`, `.observe()`, and + `.link_from("mqtt://station/{slot}/temperature")` with the existing + `Temperature::from_bytes` deserializer — same for `Humidity`. + - `DewPoint` derived per slot via `transform_join` on that slot's + Temperature + Humidity, reusing the derivation already written in + [`weather-station-alpha/src/main.rs`](../../examples/weather-mesh-demo/weather-station-alpha/src/main.rs) + (the `t.celsius - (100.0 - h.percent) / 5.0` join). Deriving at the + hub keeps the dashboard supplied even if a station publishes only + temperature/humidity. + - Drop the per-value `.log()` console lines in mesh mode — 64 slots of + per-message logging is noise; `.observe()` metrics remain. +3. **Broker URL:** accept a full connector URL via `MQTT_URL` + (e.g. `mqtts://hub-sub:…@xxxx.eu-central-1.emqx.cloud:8883`), falling + back to the current `MQTT_BROKER` host-only var for the local demo. The + hub uses the subscribe-only EMQX credential (design §5). Enable the + `use-native-tls` feature of `aimdb-mqtt-connector` in + `weather-hub/Cargo.toml`. + +**Verification:** local mosquitto + `MESH_SLOTS=4 cargo run -p weather-hub`; +`mosquitto_pub -t station/2/temperature -m `; confirm +`station.2.temperature` updates via `aimdb record list` and that a malformed +payload is rejected with a contract error (design §9). Unset `MESH_SLOTS` +and confirm the docker-compose demo behaves exactly as before. + +## WP2 — EMQX Cloud setup (external) — design §5 + +Checklist, in the ops repo / EMQX console: + +- [ ] Decide the tier — **blocked on the §8.1 decision** (serverless is + TLS-only; see WP7). Per design, option 1 (Embassy TLS) is + recommended, which permits the serverless tier. +- [ ] Deployment in EU region. +- [ ] Hub credential: subscribe-only on `station/#`. +- [ ] Station ACL template: publish-only on `station/{slot}/#`. +- [ ] Per-client rate and connection limits. +- [ ] API key for the provisioning service (never leaves the service, D6). + +## WP3 — Provisioning service (external) + open endpoint doc (OSS) — design §6 + +The service itself is a thin closed-source HTTPS shim in the private ops +repo: GitHub identity verification, admission rules (one slot per account, +minimum account age, cap N, kill switch), slot allocation, EMQX credential +create/delete, 30-day silent-slot recycling, one +`github_account ↔ slot ↔ credential` table. Includes registering the GitHub +OAuth app (device flow; client ID is public). + +**OSS artifact in this WP:** the open endpoint format doc — +`docs/design/043-join-endpoint-v1.md` (next free number) — specifying +`POST /v1/join` v1 exactly as design §6.3 defines it: the +`auth.kind = github | claim-token` envelope, the opaque `app` map, the +`profile_version`-ed response, and the `403/409/503` error bodies with +human-readable `message`. This doc is the contract `aimdb join` (WP4) is +built against, so it lands **before or with** WP4. + +## WP4 — `aimdb join` (OSS) — design §7 + +**Crate:** `tools/aimdb-cli` + +- New `src/commands/join.rs`, wired as `Command::Join(JoinCommand)` in + `src/main.rs` following the existing per-command pattern: + + ``` + aimdb join [--token ] [--out station.toml] + ``` + +- Default auth path: GitHub device flow — POST to + `github.com/login/device/code` with the compiled-in public client ID, + print the user code + verification URL, poll + `github.com/login/oauth/access_token` respecting `interval`/`slow_down`. + `--token` switches the envelope to `claim-token` and skips the flow. +- Prompt on stdin for the deployment's `app` fields (v1: station name, + city) and pass them through untouched — no weather-mesh types in the CLI. +- `POST /v1/join` per the 043 doc; on `403/409/503` print the body's + `message` as-is and exit non-zero; on success write the profile as TOML + to `--out` (default `station.toml`, mode 0600) and print the assigned + slot + station name. +- Dependencies, gated behind a `join` cargo feature (default **on**): + `reqwest` (rustls, no default features) and `toml`. Never a client of + the EMQX management API (D6/D7). + +**Verification:** unit tests for profile serialization and error-body +handling against a mock server (`tools/aimdb-cli` already has +`tokio-test`/`tempfile` dev-deps); manual device-flow run against the real +GitHub endpoint with the flagship OAuth app. + +## WP5 — Generic cloud station (OSS) — design §7/§8 + +**New crate:** `examples/weather-mesh-demo/weather-station` (workspace +member), the binary in the design's demo loop: +`cargo run -p weather-station -- --config station.toml`. + +- Parses `station.toml` (`serde` + `toml`): broker URL/credentials, + `station_id` (slot), `app.name`, coarsened `app.lat`/`app.lon`. +- Configures `Temperature`/`Humidity` with `StringKey::intern` + (`station.{slot}.temperature`, …) and `.link_to("mqtt://station/{slot}/temperature")`, + plus the derived `DewPoint` — a config-driven port of alpha's setup, + reusing alpha's Open-Meteo client at the profile's lat/lon. +- First-successful-publish banner: direct URL to the station's live chart + (`aimdb.dev/mesh/`) and a ready-made MCP prompt — the shareable + moment (design §8 tier 2). +- Revoked-credential failure UX: on MQTT auth failure, say the slot was + likely revoked and point at the join page instead of retrying forever + (design §9). +- Alpha, beta, and the docker-compose demo remain untouched. + +**Verification:** hand-written `station.toml` against local mosquitto + +WP1 hub in mesh mode; then the full three-command loop of design §7 once +WP2/WP3 are live. + +## WP6 — Gamma build-time config (OSS) — design §8.2 + +**Crate:** `examples/weather-mesh-demo/weather-station-gamma` + +- [`build.rs`](../../examples/weather-mesh-demo/weather-station-gamma/build.rs): + read the profile at `$MESH_CONFIG` (same `station.toml`), generate + `station_config.rs` into `OUT_DIR` (broker host/port/credentials, slot, + name); with `MESH_CONFIG` unset, emit the current local-demo defaults + (`192.168.1.3:1883`, `sensors/gamma/…` topics) so tier 1 stays + zero-setup. Declare `cargo:rerun-if-env-changed=MESH_CONFIG` and + `cargo:rerun-if-changed=`. +- `main.rs`: replace the `MQTT_BROKER_IP`/`MQTT_BROKER_PORT` consts with + `include!(concat!(env!("OUT_DIR"), "/station_config.rs"))`; success + banner over defmt. +- Small and independent — can land with WP1. **Contingency:** until WP7, + `MESH_CONFIG`-built firmware can only target the local demo broker + (no TLS on Embassy). + +**Verification:** build with and without `MESH_CONFIG` and diff the +generated file; flash via the existing `cargo run` probe-rs runner and via +`flash.sh` (split build/flash must keep working). + +## WP7 — Embassy TLS (OSS, post-launch acceptable) — design §8.1 option 1 + +**Crate:** `aimdb-mqtt-connector` (`embassy_client.rs`) + +Real connector work, to be scoped in its own design doc before +implementation: `embedded-tls` session over the Embassy TCP socket, entropy +from the STM32H5 TRNG (already bound via the `rng` interrupt in gamma), and +an SNTP task as the time source for certificate validation (no RTC +battery). Keeps the "MCU speaks directly to a managed cloud broker" claim +true. Blocks only the *hardware* on-ramp to the public mesh; the cloud +path (WP1–WP5) launches without it. + +## WP8 — Landing, station pages, privacy note (external) — design §8/§9 + +Website repo checklist: + +- [ ] `aimdb.dev/mesh` landing: live map + charts, MCP snippet, join + instructions (tier 0/2 funnel). +- [ ] Station identity pages `aimdb.dev/mesh/`: live chart, uptime, + last reading. +- [ ] Dashboard filters to slots with recent data (empty slots invisible, + design §4). +- [ ] Privacy note: what metadata is published, 2-decimal coordinate + rounding (D8), 30-day retention and slot recycling (D9). + +--- + +## Sequencing and dependencies + +``` +WP1 hub slot pool ──────────────┐ +WP3(OSS) 043 endpoint doc ─► WP4 aimdb join ─┐ +WP5 weather-station ────────────┤ ├─► launch (cloud path) +WP6 gamma build config ─────────┘ │ +WP2 EMQX setup ◄─ tier decision (§8.1) ───────┤ +WP3(ext) provisioning service ────────────────┘ +WP7 Embassy TLS ─► hardware on-ramp (post-launch OK) +WP8 website ──────► launch +``` + +- **Unblocked today (pure OSS):** WP1, WP3's 043 doc, WP4, WP5, WP6. +- **WP2** waits only on the §8.1 tier decision (recommended: serverless + + WP7 later). +- **WP3 (service)** is the only new operational component and gates the + live end-to-end flow. +- **Launch =** WP1 + WP2 + WP3 + WP4 + WP5 + WP8. **WP6/WP7** complete the + hardware story and may follow. + +## End-to-end verification + +1. **Local drill (no cloud dependencies):** mosquitto + WP1 hub with + `MESH_SLOTS=8` + WP5 station with a hand-written `station.toml` → + `aimdb record list --url tcp://localhost:7433` shows the slot updating; + a deliberately malformed publish surfaces a contract error at the hub. +2. **Hosted smoke test:** the design §7 three-command loop against the real + stack — `aimdb join https://mesh.aimdb.dev`, run the station, verify via + `aimdb record list --url tcp://aimdb.dev:7433` — measured against the + design's target of **under 10 minutes** from join to dot on the map. +3. **Regression:** `docker compose up` in `examples/weather-mesh-demo` + behaves exactly as before (tier 1 unchanged), and the gamma local-demo + build with `MESH_CONFIG` unset is byte-identical in behaviour. From ae42fca2dc45ef21388b2fbcfb3e14c1eb29d35b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 18:19:57 +0000 Subject: [PATCH 18/31] fix(mqtt-connector): select TLS transport for mqtts:// URLs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ConnectorUrl parsing already recognised the mqtts scheme (port default 8883, URL-embedded credentials), but rumqttc defaults to plain TCP unless a TLS transport is set explicitly — an mqtts:// broker URL silently spoke cleartext to port 8883 and failed the handshake. Select Transport::Tls with the native-tls default configuration (system trust roots) when the scheme is mqtts, which the public weather mesh hub relies on to reach EMQX Cloud (design 042 §5). Co-Authored-By: Claude Fable 5 --- aimdb-mqtt-connector/src/tokio_client.rs | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/aimdb-mqtt-connector/src/tokio_client.rs b/aimdb-mqtt-connector/src/tokio_client.rs index 923e0ce..e4b2ca0 100644 --- a/aimdb-mqtt-connector/src/tokio_client.rs +++ b/aimdb-mqtt-connector/src/tokio_client.rs @@ -177,6 +177,12 @@ impl MqttConnectorImpl { mqtt_opts.set_credentials(username, password); } + // mqtts:// selects the TLS transport (system trust roots via + // native-tls); rumqttc otherwise speaks plain TCP regardless of port. + if connector_url.scheme == "mqtts" { + mqtt_opts.set_transport(rumqttc::Transport::Tls(rumqttc::TlsConfiguration::Native)); + } + // Wrap router early so we can count topics for capacity calculation let router_arc = Arc::new(router); let topic_count = router_arc.resource_ids().len(); @@ -364,4 +370,18 @@ mod tests { let connector = MqttConnectorImpl::build_internal("not-a-valid-url", None, router).await; assert!(connector.is_err()); } + + #[tokio::test] + async fn test_connector_mqtts_url_with_credentials() { + // mqtts:// with URL-embedded credentials must parse and build; the TLS + // handshake itself only happens once the event loop is polled. + let router = RouterBuilder::new().build(); + let connector = MqttConnectorImpl::build_internal( + "mqtts://hub-sub:secret@broker.example.com:8883", + None, + router, + ) + .await; + assert!(connector.is_ok()); + } } From 5affb384a10cf3ea486fc21986b95323d7ee4508 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 18:19:57 +0000 Subject: [PATCH 19/31] feat(weather-mesh): hub slot pool behind MESH_SLOTS (042 WP1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MESH_SLOTS switches the hub into public-mesh mode (design 042 §4): a bounded pool of N string-keyed slot records (station.{slot}.temperature/humidity, default 64) subscribed to station/{slot}/… topics, with DewPoint derived per slot at the hub via transform_join so the dashboard stays supplied even when a station publishes only temperature/humidity. Slot records exist from startup, so admission assigns a slot number instead of requiring an enum variant and a recompile. MQTT_URL accepts a full connector URL (mqtts://hub-sub:…@…:8883 for the EMQX Cloud deployment, credentials redacted from logs); unset, the hub falls back to the local demo's host-only MQTT_BROKER variable. With MESH_SLOTS unset the three-enum docker-compose demo configuration is unchanged. Mesh mode drops the per-value console .log() lines (64 slots of per-message logging is noise); .observe() signal gauges remain on record list/get. Co-Authored-By: Claude Fable 5 --- .../weather-mesh-demo/weather-hub/src/main.rs | 110 ++++++++++++++++-- 1 file changed, 102 insertions(+), 8 deletions(-) diff --git a/examples/weather-mesh-demo/weather-hub/src/main.rs b/examples/weather-mesh-demo/weather-hub/src/main.rs index 94aefd8..015c8bb 100644 --- a/examples/weather-mesh-demo/weather-hub/src/main.rs +++ b/examples/weather-mesh-demo/weather-hub/src/main.rs @@ -1,7 +1,7 @@ -use aimdb_core::{buffer::BufferCfg, AimDbBuilder, RecordKey}; +use aimdb_core::{buffer::BufferCfg, AimDbBuilder, RecordKey, StringKey}; use aimdb_data_contracts::{Linkable, ObservableRegistrarExt}; use aimdb_tokio_adapter::{TokioAdapter, TokioRecordRegistrarExt}; -use weather_mesh_common::{Humidity, HumidityKey, TempKey, Temperature}; +use weather_mesh_common::{DewPoint, Humidity, HumidityKey, TempKey, Temperature}; #[tokio::main] async fn main() -> aimdb_core::DbResult<()> { @@ -15,11 +15,15 @@ async fn main() -> aimdb_core::DbResult<()> { tracing::info!("🚀 Starting Weather Hub"); - // Configuration from environment - let mqtt_broker = std::env::var("MQTT_BROKER").unwrap_or_else(|_| "localhost".to_string()); - let mqtt_url = format!("mqtt://{}", mqtt_broker); + // Broker: MQTT_URL takes a full connector URL (e.g. the public mesh broker, + // mqtts://hub-sub:…@xxxx.eu-central-1.emqx.cloud:8883); otherwise fall back + // to the local demo's host-only MQTT_BROKER variable. + let mqtt_url = std::env::var("MQTT_URL").unwrap_or_else(|_| { + let mqtt_broker = std::env::var("MQTT_BROKER").unwrap_or_else(|_| "localhost".to_string()); + format!("mqtt://{}", mqtt_broker) + }); - tracing::info!("📡 Connecting to MQTT broker: {}", mqtt_url); + tracing::info!("📡 Connecting to MQTT broker: {}", redact_url(&mqtt_url)); let runtime = std::sync::Arc::new(TokioAdapter::new()?); @@ -27,6 +31,25 @@ async fn main() -> aimdb_core::DbResult<()> { aimdb_mqtt_connector::MqttConnector::new(&mqtt_url).with_client_id("weather-hub"), ); + // MESH_SLOTS switches the hub into public-mesh mode (design 042 §4): a + // bounded pool of string-keyed slot records instead of the closed + // three-station enums. Unset = the local docker-compose demo, unchanged. + match std::env::var("MESH_SLOTS").ok() { + Some(v) => { + let slots: u16 = v.parse().unwrap_or(64); + tracing::info!("🕸️ Mesh mode: {} slots", slots); + configure_mesh(&mut builder, slots); + } + None => configure_demo(&mut builder), + } + + tracing::info!("✅ Weather Hub configured, starting..."); + + builder.run().await +} + +/// The local demo configuration: the three closed-enum stations. +fn configure_demo(builder: &mut AimDbBuilder) { // Configure Temperature records for all nodes with tap functions for key in [TempKey::Alpha, TempKey::Beta, TempKey::Gamma] { let topic = key.link_address().unwrap().to_string(); @@ -75,8 +98,79 @@ async fn main() -> aimdb_core::DbResult<()> { .finish(); }); } +} - tracing::info!("✅ Weather Hub configured, starting..."); +/// The public-mesh slot pool (design 042 §4). +/// +/// Every slot's records exist from startup; admission just assigns an unused +/// slot number, so joining never needs a hub recompile. A malformed payload is +/// rejected by `from_bytes` (contract deserialization) — that error surfacing +/// at the hub is the admission-time schema check. +/// +/// No per-value `.log()` here: 64 slots of per-message console lines is noise. +/// `.observe()` keeps the signal gauges on `record list`/`record get`. +fn configure_mesh(builder: &mut AimDbBuilder, slots: u16) { + for slot in 0..slots { + let temp_key = StringKey::intern(format!("station.{slot}.temperature")); + let temp_topic = format!("mqtt://station/{slot}/temperature"); + builder.configure::(temp_key, |reg| { + reg.buffer(BufferCfg::SpmcRing { capacity: 100 }); + reg.observe(); + reg.link_from(&temp_topic) + .with_deserializer(|_ctx, data: &[u8]| Temperature::from_bytes(data)) + .finish(); + }); - builder.run().await + let humidity_key = StringKey::intern(format!("station.{slot}.humidity")); + let humidity_topic = format!("mqtt://station/{slot}/humidity"); + builder.configure::(humidity_key, |reg| { + reg.buffer(BufferCfg::SpmcRing { capacity: 100 }); + reg.observe(); + reg.link_from(&humidity_topic) + .with_deserializer(|_ctx, data: &[u8]| Humidity::from_bytes(data)) + .finish(); + }); + + // Derived at the hub so the dashboard stays supplied even when a + // station publishes only temperature/humidity. + let dew_point_key = StringKey::intern(format!("station.{slot}.dew_point")); + builder.configure::(dew_point_key, |reg| { + reg.buffer(BufferCfg::SpmcRing { capacity: 100 }); + reg.observe(); + reg.transform_join(|b| { + b.input::(temp_key) + .input::(humidity_key) + .on_triggers(|mut rx, producer| async move { + let mut last_temp: Option = None; + let mut last_hum: Option = None; + while let Ok(trigger) = rx.recv().await { + match trigger.index() { + 0 => last_temp = trigger.as_input::().cloned(), + 1 => last_hum = trigger.as_input::().cloned(), + _ => {} + } + if let (Some(t), Some(h)) = (&last_temp, &last_hum) { + // Magnus approximation: T_dp ≈ T - (100 - RH) / 5 + let dew_point = t.celsius - (100.0 - h.percent) / 5.0; + let timestamp = t.timestamp.max(h.timestamp); + producer.produce(DewPoint { + celsius: dew_point, + timestamp, + }); + } + } + }) + }); + }); + } +} + +/// Strip URL-embedded credentials before logging. +fn redact_url(url: &str) -> String { + if let Some((scheme, rest)) = url.split_once("://") { + if let Some((_creds, host)) = rest.rsplit_once('@') { + return format!("{scheme}://…@{host}"); + } + } + url.to_string() } From 092597f50a45b2868da5e030ea21c364a59d64a7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 18:21:17 +0000 Subject: [PATCH 20/31] =?UTF-8?q?docs(design):=20add=20043=20=E2=80=94=20j?= =?UTF-8?q?oin=20endpoint=20format=20v1=20(042=20WP3)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The open contract for POST /v1/join that aimdb join is built against (design 042 §6.3): the auth.kind = github | claim-token identity envelope, the opaque deployment-specific app map, the profile_version-ed response with the scoped broker credential, the 403/409/503 error bodies whose message the CLI prints as-is, and the station.toml serialization consumed by weather-station and gamma's MESH_CONFIG build embedding. The server side stays in the private ops repo (042 D6); this doc keeps the format public so the CLI is deployment-agnostic and any AimDB deployment can implement admission. Co-Authored-By: Claude Fable 5 --- docs/design/043-join-endpoint-v1.md | 179 ++++++++++++++++++++++++++++ 1 file changed, 179 insertions(+) create mode 100644 docs/design/043-join-endpoint-v1.md diff --git a/docs/design/043-join-endpoint-v1.md b/docs/design/043-join-endpoint-v1.md new file mode 100644 index 0000000..e78a7bb --- /dev/null +++ b/docs/design/043-join-endpoint-v1.md @@ -0,0 +1,179 @@ +# 043 — Join endpoint format v1 + +**Status:** 📝 Proposed + +**Scope:** the open wire format of the provisioning endpoint `aimdb join` +speaks (`POST /v1/join`), and the station profile file the CLI writes. No +code in this repo implements the *server* side — the flagship's provisioning +service lives in the private ops repo ([042](./042-public-weather-mesh-flagship.md) +D6) — but the format is public so any AimDB deployment can implement it and +the CLI stays deployment-agnostic. + +**Consumers:** `aimdb join` ([`tools/aimdb-cli`](../../tools/aimdb-cli), +042 §7) is built against this document; `weather-station` +([`examples/weather-mesh-demo`](../../examples/weather-mesh-demo)) consumes +the profile file defined in §4. + +--- + +## 1. Overview + +A *provisioning endpoint* admits a new edge node to a deployment: it verifies +an identity, applies the deployment's admission rules, allocates whatever +resources a member needs (for the weather mesh flagship: a slot number and a +scoped MQTT credential), and returns a **station profile** the node runs +with. The CLI is generic over the endpoint URL: + +``` +aimdb join [--token ] [--out station.toml] +``` + +`aimdb join https://mesh.aimdb.dev` sends `POST https://mesh.aimdb.dev/v1/join`. +The path is fixed; the base URL is the deployment. TLS is required — the +request carries a bearer-equivalent token and the response carries a broker +credential. + +## 2. Request + +``` +POST /v1/join +Content-Type: application/json + +{ + "auth": { "kind": "github", "token": "gho_…" }, + "app": { "name": "graz-balcony", "location": "Graz" } +} +``` + +### 2.1 `auth` — identity envelope + +`auth.kind` keeps the format deployment-agnostic. v1 defines two kinds: + +| `kind` | `token` | Used by | +|---|---|---| +| `github` | A GitHub **user access token**, obtained by the CLI via the OAuth device flow (no scopes — public identity only). The service verifies it against the GitHub API and applies identity-based admission rules (one active slot per account, minimum account age, …). | The public flagship (self-serve). | +| `claim-token` | A pre-issued, one-time opaque token, distributed out of band by the deployment operator. | Private deployments without public self-serve. `aimdb join --token ` selects this kind. | + +Deployments accept the kinds they support and reject others with `403`. + +### 2.2 `app` — deployment-specific metadata + +`app` is an **opaque string-keyed map**: the CLI collects its fields via +interactive prompts (driven by the deployment's docs) and passes them through +untouched — no deployment-specific types exist in the CLI. The service +validates, normalizes, and echoes the final values back in the profile (§3). + +For the weather mesh flagship, v1 `app` fields are: + +| Field | Meaning | +|---|---| +| `name` | Station name — becomes the public identity page (`aimdb.dev/mesh/`). | +| `location` | City-level location. The service geocodes and **coarsens it to 2 decimal places (~1 km)** before it appears anywhere (042 D8); precise location is never collected. | + +## 3. Response + +### 3.1 Success — `200 OK` + +```json +{ + "profile_version": 1, + "station_id": "slot-17", + "broker": { + "url": "mqtts://xxxx.eu-central-1.emqx.cloud:8883", + "username": "station-17", + "password": "…" + }, + "app": { + "name": "graz-balcony", + "lat": 47.07, + "lon": 15.44 + } +} +``` + +| Field | Contract | +|---|---| +| `profile_version` | Format version of this response, `1` for this document. See §5. | +| `station_id` | The node's identity within the deployment — opaque to the CLI. The flagship uses `slot-`, where `` is the assigned slot in the hub's pool (042 §4); the station binary derives its topic/record namespace (`station//…`, `station..…`) from it. | +| `broker.url` | Connector URL of the data-plane broker, without credentials. `mqtts://` for the flagship (broker is TLS-only). | +| `broker.username`, `broker.password` | The per-station credential, scoped server-side to the station's own topics (flagship: publish-only on `station//#`). Individually revocable (042 D9). | +| `app` | The **final** deployment-specific values (post validation/coarsening) — clients must use these, not what they sent. The flagship echoes `name` and the coarsened `lat`/`lon` the station feeds to its weather source. | + +### 3.2 Errors + +Error responses are JSON with a human-readable `message` the CLI prints +**as-is** and exits non-zero — the service owns the failure UX text: + +```json +{ "message": "The mesh is full (64/64 slots) — try again later." } +``` + +| Status | Meaning | Flagship examples | +|---|---|---| +| `403` | Identity rejected. | Account too new; admissions paused (kill switch); unsupported `auth.kind`. | +| `409` | The identity already holds a membership. The `message` points at the existing profile (station name/id) rather than erroring opaquely. | "Your account already runs `graz-balcony` (slot-17). Revoke it first or reuse its profile." | +| `503` | Deployment full or temporarily not admitting. | Mesh at cap N. | + +Any other non-2xx status is unexpected; the CLI reports it with the body if +one is present. + +## 4. The station profile file (`station.toml`) + +`aimdb join` writes the §3.1 response as TOML (mode `0600` — it contains a +credential), default path `station.toml`, `--out` to override: + +```toml +profile_version = 1 +station_id = "slot-17" + +[broker] +url = "mqtts://xxxx.eu-central-1.emqx.cloud:8883" +username = "station-17" +password = "…" + +[app] +name = "graz-balcony" +lat = 47.07 +lon = 15.44 +``` + +The mapping is mechanical (JSON object → TOML table, `app` passed through); +consumers (`weather-station --config station.toml`, gamma's `MESH_CONFIG` +build embedding — 042 §8.2) read this file, never the HTTP response. + +## 5. Versioning and compatibility + +The CLI ships on the workspace release cadence and must not need a release +in lockstep with mesh-side changes (042 §6.3). Therefore: + +- **Unknown fields are ignored**, both in the response (CLI) and in the + profile (station binaries). Services may add fields without a version bump. +- `profile_version` is bumped only for **breaking** changes to the meaning + of existing fields. A client that receives a `profile_version` newer than + it knows writes the profile anyway and warns; a station binary that cannot + interpret a profile says which version it expected. +- The request envelope is versioned by the URL path (`/v1/join`). New auth + kinds may be added to v1 (a service rejects unknown kinds with `403`, which + the CLI reports verbatim). + +## 6. What this format deliberately is not + +- **Not a client of the broker's management API.** The response carries a + ready-made credential; how it was minted (EMQX Cloud API for the flagship) + is invisible to the client, so no admin key can end up in a public binary + (042 D6/D7). +- **Not a schema registry.** Payload validity is enforced by contract + deserialization at the hub (042 §9); join-time carries no topic lists or + schemas. +- **Not a session.** One request, one response, no state on the client + beyond the profile file. Re-joining with an identity that already holds a + membership is a `409`, not an idempotent re-issue — credential re-issue is + a service-side policy decision. + +## 7. References + +- [042 — Public weather mesh flagship](./042-public-weather-mesh-flagship.md) + §6 (admission flow), §7 (`aimdb join`), D5–D8 +- [Implementation plan — 042](../plan/042-public-weather-mesh-flagship.md) + WP3/WP4 +- [GitHub OAuth device flow](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/authorizing-oauth-apps#device-flow) From e0dcf9b7c731334d6cc1167d7ea63e086f51d1e7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 18:26:58 +0000 Subject: [PATCH 21/31] =?UTF-8?q?feat(cli):=20add=20aimdb=20join=20?= =?UTF-8?q?=E2=80=94=20self-serve=20deployment=20admission=20(042=20WP4)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New join subcommand speaking the open provisioning format of design 043: aimdb join [--token ] [--out station.toml] Default auth is the GitHub OAuth device flow (042 §6.1): print the user code + verification URL, poll for the access token respecting interval/slow_down, no scopes requested. The public client ID is compiled in via AIMDB_GITHUB_CLIENT_ID at build time (nothing secret ships in the CLI); without one the command points at --token. --token switches the envelope to the claim-token auth kind for deployments without public self-serve. The deployment's app fields (v1: station name, city) are collected via stdin prompts and passed through untouched — no deployment-specific types in the CLI. On 403/409/503 the service's message is printed as-is; on success the profile is written as TOML (mode 0600 — it carries the broker credential) and the assigned station id + name are printed. The CLI never talks to the broker management API (042 D6/D7). Dependencies (reqwest with rustls, toml) are gated behind a new join cargo feature, on by default. Unit tests cover profile JSON→TOML round-trip, unknown-field tolerance (043 §5), error-body handling, file permissions, and both paths against a one-shot mock HTTP server. Co-Authored-By: Claude Fable 5 --- Cargo.lock | 114 +++++++ tools/aimdb-cli/Cargo.toml | 14 +- tools/aimdb-cli/src/commands/join.rs | 479 +++++++++++++++++++++++++++ tools/aimdb-cli/src/commands/mod.rs | 2 + tools/aimdb-cli/src/main.rs | 9 + 5 files changed, 617 insertions(+), 1 deletion(-) create mode 100644 tools/aimdb-cli/src/commands/join.rs diff --git a/Cargo.lock b/Cargo.lock index 9eb299a..5a9bdf6 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -54,6 +54,7 @@ dependencies = [ "clap", "colored", "futures", + "reqwest", "serde", "serde_json", "serde_yaml", @@ -62,6 +63,7 @@ dependencies = [ "thiserror 1.0.69", "tokio", "tokio-test", + "toml 0.8.23", ] [[package]] @@ -1806,8 +1808,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "335ff9f135e4384c8150d6f27c6daed433577f86b4750418338c01a1a2528592" dependencies = [ "cfg-if", + "js-sys", "libc", "wasi", + "wasm-bindgen", ] [[package]] @@ -1817,9 +1821,11 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd" dependencies = [ "cfg-if", + "js-sys", "libc", "r-efi 5.3.0", "wasip2", + "wasm-bindgen", ] [[package]] @@ -2056,6 +2062,7 @@ dependencies = [ "tokio", "tokio-rustls", "tower-service", + "webpki-roots", ] [[package]] @@ -2422,6 +2429,12 @@ dependencies = [ "tracing-subscriber", ] +[[package]] +name = "lru-slab" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154" + [[package]] name = "mach2" version = "0.4.3" @@ -2813,6 +2826,61 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "quinn" +version = "0.11.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9e20a958963c291dc322d98411f541009df2ced7b5a4f2bd52337638cfccf20" +dependencies = [ + "bytes", + "cfg_aliases", + "pin-project-lite", + "quinn-proto", + "quinn-udp", + "rustc-hash", + "rustls", + "socket2", + "thiserror 2.0.17", + "tokio", + "tracing", + "web-time", +] + +[[package]] +name = "quinn-proto" +version = "0.11.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1906b49b0c3bc04b5fe5d86a77925ae6524a19b816ae38ce1e426255f1d8a31" +dependencies = [ + "bytes", + "getrandom 0.3.4", + "lru-slab", + "rand 0.9.4", + "ring", + "rustc-hash", + "rustls", + "rustls-pki-types", + "slab", + "thiserror 2.0.17", + "tinyvec", + "tracing", + "web-time", +] + +[[package]] +name = "quinn-udp" +version = "0.5.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "addec6a0dcad8a8d96a771f815f0eaf55f9d1805756410b39f5fa81332574cbd" +dependencies = [ + "cfg_aliases", + "libc", + "once_cell", + "socket2", + "tracing", + "windows-sys 0.52.0", +] + [[package]] name = "quote" version = "1.0.41" @@ -2973,6 +3041,8 @@ dependencies = [ "native-tls", "percent-encoding", "pin-project-lite", + "quinn", + "rustls", "rustls-pki-types", "serde", "serde_json", @@ -2980,6 +3050,7 @@ dependencies = [ "sync_wrapper", "tokio", "tokio-native-tls", + "tokio-rustls", "tower", "tower-http", "tower-service", @@ -2987,6 +3058,7 @@ dependencies = [ "wasm-bindgen", "wasm-bindgen-futures", "web-sys", + "webpki-roots", ] [[package]] @@ -3048,6 +3120,12 @@ dependencies = [ "smallvec", ] +[[package]] +name = "rustc-hash" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "357703d41365b4b27c590e3ed91eabb1b663f07c4c084095e60cbed4362dff0d" + [[package]] name = "rustc_version" version = "0.2.3" @@ -3077,6 +3155,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "758025cb5fccfd3bc2fd74708fd4682be41d99e5dff73c377c0646c6012c73a4" dependencies = [ "once_cell", + "ring", "rustls-pki-types", "rustls-webpki", "subtle", @@ -3089,6 +3168,7 @@ version = "1.13.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "94182ad936a0c91c324cd46c6511b9510ed16af436d7b5bab34beab0afd55f7a" dependencies = [ + "web-time", "zeroize", ] @@ -3679,6 +3759,21 @@ dependencies = [ "serde_json", ] +[[package]] +name = "tinyvec" +version = "1.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bfa5fdc3bce6191a1dbc8c02d5c8bffcf557bafa17c124c5264a458f1b0613fa" +dependencies = [ + "tinyvec_macros", +] + +[[package]] +name = "tinyvec_macros" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20" + [[package]] name = "tokio" version = "1.48.0" @@ -4500,6 +4595,25 @@ dependencies = [ "wasm-bindgen", ] +[[package]] +name = "web-time" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb" +dependencies = [ + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "webpki-roots" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "22cfaf3c063993ff62e73cb4311efde4db1efb31ab78a3e5c457939ad5cc0bed" +dependencies = [ + "rustls-pki-types", +] + [[package]] name = "winapi" version = "0.3.9" diff --git a/tools/aimdb-cli/Cargo.toml b/tools/aimdb-cli/Cargo.toml index f2399ff..fbba9d0 100644 --- a/tools/aimdb-cli/Cargo.toml +++ b/tools/aimdb-cli/Cargo.toml @@ -53,9 +53,21 @@ chrono = "0.4" # Optional: YAML output serde_yaml = { version = "0.9", optional = true } +# `join` only (design 043): HTTP client for the provisioning endpoint + the +# GitHub device flow, and TOML for the station profile. rustls so released +# binaries don't depend on a system OpenSSL. +reqwest = { version = "0.12", default-features = false, features = [ + "rustls-tls", + "json", +], optional = true } +toml = { version = "0.8", optional = true } + [features] -default = [] +default = ["join"] yaml = ["serde_yaml"] +# `aimdb join ` — admit this machine to a deployment +# (design 043). On by default in released binaries. +join = ["dep:reqwest", "dep:toml"] # Add the serial transport to the endpoint resolver, so `--connect serial://…` # works. Off by default (pulls tokio-serial → libudev on Linux). transport-serial = ["aimdb-client/transport-serial"] diff --git a/tools/aimdb-cli/src/commands/join.rs b/tools/aimdb-cli/src/commands/join.rs new file mode 100644 index 0000000..b3f6c31 --- /dev/null +++ b/tools/aimdb-cli/src/commands/join.rs @@ -0,0 +1,479 @@ +//! Join Command — admit this machine to an AimDB deployment +//! +//! Speaks the open provisioning format of design 043 (`POST /v1/join`): +//! authenticate (GitHub device flow, or a pre-issued claim token), pass the +//! deployment's `app` metadata through untouched, and write the returned +//! station profile as `station.toml`. Generic by construction — the endpoint +//! URL is an argument and no deployment-specific types appear here. + +use std::io::Write as _; +use std::path::PathBuf; +use std::time::Duration; + +use anyhow::{anyhow, Context}; +use clap::Args; +use serde::{Deserialize, Serialize}; + +use crate::error::CliResult; + +/// Public OAuth client ID of the flagship's GitHub App — public by design, +/// nothing secret ships in the CLI (design 042 §6.1). Deployments building +/// their own binaries bake their own app in at build time: +/// `AIMDB_GITHUB_CLIENT_ID=… cargo build`. +const GITHUB_CLIENT_ID: Option<&str> = option_env!("AIMDB_GITHUB_CLIENT_ID"); + +const GITHUB_DEVICE_CODE_URL: &str = "https://github.com/login/device/code"; +const GITHUB_ACCESS_TOKEN_URL: &str = "https://github.com/login/oauth/access_token"; +const GITHUB_USER_URL: &str = "https://api.github.com/user"; + +/// The station profile format this CLI understands (design 043 §3/§5). +const PROFILE_VERSION: u64 = 1; + +/// `app` fields collected via prompts (design 043 §2.2). The values are +/// passed through untouched; the service validates and echoes the final +/// values back in the profile. +const APP_PROMPTS: &[(&str, &str)] = &[("name", "Station name"), ("location", "Location (city)")]; + +/// Join an AimDB deployment via its provisioning endpoint +#[derive(Debug, Args)] +pub struct JoinCommand { + /// Base URL of the provisioning endpoint (the CLI POSTs to /v1/join) + provisioning_url: String, + + /// Pre-issued claim token — selects the `claim-token` auth kind and skips + /// the GitHub device flow (for deployments without public self-serve) + #[arg(long)] + token: Option, + + /// Where to write the station profile (contains the broker credential; + /// written with mode 0600) + #[arg(long, default_value = "station.toml")] + out: PathBuf, +} + +/// Identity envelope of the join request (design 043 §2.1). +#[derive(Debug, Serialize)] +struct AuthEnvelope { + kind: &'static str, + token: String, +} + +/// The station profile returned on success (design 043 §3.1). Unknown fields +/// are ignored on both ends so services can extend the format without a +/// version bump (043 §5). +#[derive(Debug, Serialize, Deserialize)] +struct StationProfile { + profile_version: u64, + station_id: String, + broker: BrokerProfile, + app: serde_json::Map, +} + +#[derive(Debug, Serialize, Deserialize)] +struct BrokerProfile { + url: String, + username: String, + password: String, +} + +impl JoinCommand { + pub async fn execute(self) -> CliResult<()> { + let client = reqwest::Client::builder() + .user_agent(concat!("aimdb-cli/", env!("CARGO_PKG_VERSION"))) + .timeout(Duration::from_secs(30)) + .build() + .context("failed to build HTTP client")?; + + let auth = match self.token { + Some(token) => AuthEnvelope { + kind: "claim-token", + token, + }, + None => AuthEnvelope { + kind: "github", + token: github_device_flow(&client).await?, + }, + }; + + let mut app = serde_json::Map::new(); + for (key, label) in APP_PROMPTS { + app.insert((*key).to_string(), prompt(label)?.into()); + } + + let profile = request_join(&client, &self.provisioning_url, &auth, &app).await?; + + if profile.profile_version > PROFILE_VERSION { + eprintln!( + "⚠ profile_version {} is newer than this CLI understands ({}); writing it anyway", + profile.profile_version, PROFILE_VERSION + ); + } + + write_profile(&self.out, &profile)?; + + let name = profile + .app + .get("name") + .and_then(|v| v.as_str()) + .unwrap_or("station"); + println!( + "✓ {} \"{}\" — credentials written to {}", + profile.station_id, + name, + self.out.display() + ); + Ok(()) + } +} + +/// GitHub OAuth device flow (design 042 §6.1): print a short code, the user +/// confirms it at github.com/login/device, poll for the access token. No +/// scopes are requested — public identity only. +async fn github_device_flow(client: &reqwest::Client) -> CliResult { + let client_id = GITHUB_CLIENT_ID.ok_or_else(|| { + anyhow!( + "this build has no GitHub OAuth client ID compiled in\n \ + Hint: use --token for claim-token deployments, or rebuild with \ + AIMDB_GITHUB_CLIENT_ID= for GitHub self-serve" + ) + })?; + + #[derive(Deserialize)] + struct DeviceCode { + device_code: String, + user_code: String, + verification_uri: String, + expires_in: u64, + interval: u64, + } + + let device: DeviceCode = client + .post(GITHUB_DEVICE_CODE_URL) + .header("Accept", "application/json") + .json(&serde_json::json!({ "client_id": client_id })) + .send() + .await + .context("device-code request to GitHub failed")? + .error_for_status() + .context("device-code request to GitHub failed")? + .json() + .await + .context("unexpected device-code response from GitHub")?; + + println!( + "→ To authorize, enter code {} at {}", + device.user_code, device.verification_uri + ); + + #[derive(Deserialize)] + struct TokenPoll { + access_token: Option, + error: Option, + interval: Option, + } + + let deadline = std::time::Instant::now() + Duration::from_secs(device.expires_in); + let mut interval = device.interval.max(1); + let token = loop { + tokio::time::sleep(Duration::from_secs(interval)).await; + if std::time::Instant::now() > deadline { + return Err( + anyhow!("device code expired before authorization — run join again").into(), + ); + } + + let poll: TokenPoll = client + .post(GITHUB_ACCESS_TOKEN_URL) + .header("Accept", "application/json") + .json(&serde_json::json!({ + "client_id": client_id, + "device_code": device.device_code, + "grant_type": "urn:ietf:params:oauth:grant-type:device_code", + })) + .send() + .await + .context("access-token poll to GitHub failed")? + .json() + .await + .context("unexpected access-token response from GitHub")?; + + if let Some(token) = poll.access_token { + break token; + } + match poll.error.as_deref() { + Some("authorization_pending") => {} + // GitHub asks us to back off; it may supply the new interval. + Some("slow_down") => interval = poll.interval.unwrap_or(interval + 5), + Some("expired_token") => { + return Err( + anyhow!("device code expired before authorization — run join again").into(), + ) + } + Some("access_denied") => { + return Err(anyhow!("authorization was denied on GitHub").into()) + } + other => { + return Err(anyhow!( + "GitHub device flow failed: {}", + other.unwrap_or("no error code in response") + ) + .into()) + } + } + }; + + // Cosmetic only — the service re-verifies the identity server-side. + #[derive(Deserialize)] + struct GithubUser { + login: String, + } + if let Ok(resp) = client + .get(GITHUB_USER_URL) + .header("Accept", "application/vnd.github+json") + .bearer_auth(&token) + .send() + .await + { + if let Ok(user) = resp.json::().await { + println!("✓ authorized as @{}", user.login); + } + } + + Ok(token) +} + +/// `POST /v1/join` and interpret the response per design 043. +async fn request_join( + client: &reqwest::Client, + base_url: &str, + auth: &AuthEnvelope, + app: &serde_json::Map, +) -> CliResult { + let url = format!("{}/v1/join", base_url.trim_end_matches('/')); + let response = client + .post(&url) + .json(&serde_json::json!({ "auth": auth, "app": app })) + .send() + .await + .with_context(|| format!("request to {url} failed"))?; + + let status = response.status().as_u16(); + let body = response + .text() + .await + .with_context(|| format!("failed to read response from {url}"))?; + parse_join_response(status, &body) +} + +/// Interpret a `/v1/join` response (design 043 §3). On 403/409/503 the body's +/// `message` is surfaced as-is — the service owns the failure UX text. +fn parse_join_response(status: u16, body: &str) -> CliResult { + #[derive(Deserialize)] + struct ErrorBody { + message: String, + } + + match status { + 200 => serde_json::from_str(body) + .context("malformed station profile in join response") + .map_err(Into::into), + 403 | 409 | 503 => { + let message = serde_json::from_str::(body) + .map(|e| e.message) + .unwrap_or_else(|_| format!("join rejected with status {status}")); + Err(anyhow!("{message}").into()) + } + other => Err(anyhow!( + "unexpected response from provisioning endpoint (status {other}): {}", + body.trim() + ) + .into()), + } +} + +/// Write the profile as TOML (design 043 §4), mode 0600 — it carries the +/// broker credential. +fn write_profile(path: &PathBuf, profile: &StationProfile) -> CliResult<()> { + let toml = + toml::to_string_pretty(profile).context("failed to serialize station profile as TOML")?; + + let mut options = std::fs::OpenOptions::new(); + options.write(true).create(true).truncate(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.mode(0o600); + } + let mut file = options + .open(path) + .with_context(|| format!("failed to open {}", path.display()))?; + file.write_all(toml.as_bytes()) + .with_context(|| format!("failed to write {}", path.display()))?; + Ok(()) +} + +/// Prompt on stdin for one `app` field; re-asks until non-empty. +fn prompt(label: &str) -> CliResult { + loop { + print!("{label}: "); + std::io::stdout().flush()?; + let mut line = String::new(); + std::io::stdin().read_line(&mut line)?; + let value = line.trim(); + if !value.is_empty() { + return Ok(value.to_string()); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + + /// The design 043 §3.1 sample response. + const PROFILE_JSON: &str = r#"{ + "profile_version": 1, + "station_id": "slot-17", + "broker": { + "url": "mqtts://xxxx.eu-central-1.emqx.cloud:8883", + "username": "station-17", + "password": "s3cret" + }, + "app": { "name": "graz-balcony", "lat": 47.07, "lon": 15.44 } + }"#; + + #[test] + fn profile_survives_json_to_toml_round_trip() { + let profile = parse_join_response(200, PROFILE_JSON).unwrap(); + let toml_str = toml::to_string_pretty(&profile).unwrap(); + + let reread: StationProfile = toml::from_str(&toml_str).unwrap(); + assert_eq!(reread.profile_version, 1); + assert_eq!(reread.station_id, "slot-17"); + assert_eq!(reread.broker.username, "station-17"); + assert_eq!(reread.broker.password, "s3cret"); + assert_eq!( + reread.app.get("name").and_then(|v| v.as_str()), + Some("graz-balcony") + ); + assert_eq!(reread.app.get("lat").and_then(|v| v.as_f64()), Some(47.07)); + } + + #[test] + fn unknown_profile_fields_are_ignored() { + // 043 §5: services may add fields without a version bump. + let body = PROFILE_JSON.replacen( + "\"profile_version\": 1,", + "\"profile_version\": 1, \"issued_at\": \"2026-07-09\",", + 1, + ); + let profile = parse_join_response(200, &body).unwrap(); + assert_eq!(profile.station_id, "slot-17"); + } + + #[test] + fn rejection_surfaces_service_message_as_is() { + for status in [403, 409, 503] { + let err = parse_join_response( + status, + r#"{ "message": "The mesh is full (64/64 slots) — try again later." }"#, + ) + .unwrap_err(); + assert_eq!( + err.to_string(), + "The mesh is full (64/64 slots) — try again later." + ); + } + } + + #[test] + fn rejection_without_json_body_still_reports_status() { + let err = parse_join_response(503, "upstream timeout").unwrap_err(); + assert!(err.to_string().contains("503"), "got: {err}"); + } + + #[test] + fn unexpected_status_is_an_error() { + let err = parse_join_response(500, "oops").unwrap_err(); + assert!(err.to_string().contains("500"), "got: {err}"); + } + + #[test] + fn malformed_success_body_is_an_error() { + let err = parse_join_response(200, r#"{ "hello": 1 }"#).unwrap_err(); + assert!(err.to_string().contains("profile"), "got: {err}"); + } + + #[test] + fn profile_written_with_owner_only_permissions() { + let profile = parse_join_response(200, PROFILE_JSON).unwrap(); + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("station.toml"); + write_profile(&path, &profile).unwrap(); + + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = std::fs::metadata(&path).unwrap().permissions().mode(); + assert_eq!(mode & 0o777, 0o600); + } + let reread: StationProfile = + toml::from_str(&std::fs::read_to_string(&path).unwrap()).unwrap(); + assert_eq!(reread.station_id, "slot-17"); + } + + /// One-shot HTTP server: accepts a single connection, reads the request, + /// answers with the canned status/body, then closes. + async fn mock_join_server(status_line: &'static str, body: &'static str) -> String { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let addr = listener.local_addr().unwrap(); + tokio::spawn(async move { + let (mut socket, _) = listener.accept().await.unwrap(); + // Drain the request headers + body far enough to respond politely. + let mut buf = [0u8; 4096]; + let _ = socket.read(&mut buf).await; + let response = format!( + "HTTP/1.1 {status_line}\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", + body.len() + ); + socket.write_all(response.as_bytes()).await.unwrap(); + }); + format!("http://{addr}") + } + + #[tokio::test] + async fn join_request_round_trips_against_mock_server() { + let base = mock_join_server("200 OK", PROFILE_JSON).await; + let client = reqwest::Client::new(); + let auth = AuthEnvelope { + kind: "claim-token", + token: "t".into(), + }; + let profile = request_join(&client, &base, &auth, &serde_json::Map::new()) + .await + .unwrap(); + assert_eq!(profile.station_id, "slot-17"); + } + + #[tokio::test] + async fn join_request_surfaces_mock_rejection_message() { + let base = mock_join_server( + "409 Conflict", + r#"{ "message": "Your account already runs graz-balcony (slot-17)." }"#, + ) + .await; + let client = reqwest::Client::new(); + let auth = AuthEnvelope { + kind: "claim-token", + token: "t".into(), + }; + let err = request_join(&client, &base, &auth, &serde_json::Map::new()) + .await + .unwrap_err(); + assert_eq!( + err.to_string(), + "Your account already runs graz-balcony (slot-17)." + ); + } +} diff --git a/tools/aimdb-cli/src/commands/mod.rs b/tools/aimdb-cli/src/commands/mod.rs index dfb501b..70264ba 100644 --- a/tools/aimdb-cli/src/commands/mod.rs +++ b/tools/aimdb-cli/src/commands/mod.rs @@ -5,6 +5,8 @@ pub mod generate; pub mod graph; pub mod instance; +#[cfg(feature = "join")] +pub mod join; pub mod record; pub mod watch; diff --git a/tools/aimdb-cli/src/main.rs b/tools/aimdb-cli/src/main.rs index 21f8d10..23240a5 100644 --- a/tools/aimdb-cli/src/main.rs +++ b/tools/aimdb-cli/src/main.rs @@ -4,6 +4,8 @@ //! AimDB instances via the AimX remote access protocol (v2 NDJSON wire). use clap::{Parser, Subcommand}; +#[cfg(feature = "join")] +use commands::join::JoinCommand; use commands::{ generate::GenerateCommand, graph::GraphCommand, instance::InstanceCommand, record::RecordCommand, watch::WatchCommand, @@ -54,6 +56,11 @@ enum Command { /// Generate architecture artefacts from state.toml #[command(name = "generate")] Generate(GenerateCommand), + + /// Join an AimDB deployment via its provisioning endpoint (design 043) + #[cfg(feature = "join")] + #[command(name = "join")] + Join(JoinCommand), } #[tokio::main] @@ -73,6 +80,8 @@ async fn main() { Command::Graph(cmd) => cmd.execute(endpoint).await, Command::Watch(cmd) => cmd.execute(endpoint).await, Command::Generate(cmd) => cmd.execute().await, + #[cfg(feature = "join")] + Command::Join(cmd) => cmd.execute().await, }; if let Err(e) = result { From 57eb4e7aaf04b827d84a8cd887ca130ae0b00a35 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 18:31:31 +0000 Subject: [PATCH 22/31] feat(weather-mesh): generic cloud station driven by station.toml (042 WP5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New weather-station crate — the binary in the design's three-command demo loop: aimdb join writes station.toml, then cargo run -p weather-station -- --config station.toml. The profile (design 043 §4) supplies everything: broker URL + per-slot credential (embedded into the connector URL; redacted in logs), the assigned slot (station_id = slot- → station..* records, station//… topics), the station name, and the service-coarsened lat/lon fed to the same Open-Meteo source as station alpha. DewPoint is derived on the station via transform_join exactly as alpha does. Failure/success UX per design §9/§8: a one-shot CONNECT pre-flight turns a revoked credential into a clear pointer back to aimdb join instead of the connector's silent retry-forever, and the first successful publish prints the station's live chart URL (aimdb.dev/mesh/) plus a ready-made MCP prompt — the shareable moment. Alpha, beta, and the docker-compose demo are untouched. Co-Authored-By: Claude Fable 5 --- Cargo.lock | 21 + Cargo.toml | 1 + Makefile | 6 +- .../weather-station/Cargo.toml | 42 ++ .../weather-station/src/main.rs | 479 ++++++++++++++++++ 5 files changed, 547 insertions(+), 2 deletions(-) create mode 100644 examples/weather-mesh-demo/weather-station/Cargo.toml create mode 100644 examples/weather-mesh-demo/weather-station/src/main.rs diff --git a/Cargo.lock b/Cargo.lock index 5a9bdf6..ef43745 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4519,6 +4519,27 @@ dependencies = [ "serde_json", ] +[[package]] +name = "weather-station" +version = "0.1.0" +dependencies = [ + "aimdb-core", + "aimdb-data-contracts", + "aimdb-mqtt-connector", + "aimdb-tokio-adapter", + "chrono", + "clap", + "reqwest", + "rumqttc", + "serde", + "serde_json", + "tokio", + "toml 0.8.23", + "tracing", + "tracing-subscriber", + "weather-mesh-common", +] + [[package]] name = "weather-station-alpha" version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml index e203272..45e56e4 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -32,6 +32,7 @@ members = [ "examples/remote-access-demo", "examples/weather-mesh-demo/weather-mesh-common", "examples/weather-mesh-demo/weather-hub", + "examples/weather-mesh-demo/weather-station", "examples/weather-mesh-demo/weather-station-alpha", "examples/weather-mesh-demo/weather-station-beta", "examples/weather-mesh-demo/weather-station-gamma", diff --git a/Makefile b/Makefile index 2999ff7..a3a56b3 100644 --- a/Makefile +++ b/Makefile @@ -178,7 +178,7 @@ test: fmt: @printf "$(GREEN)Formatting code (workspace members only)...$(NC)\n" - @for pkg in aimdb-derive aimdb-data-contracts aimdb-core aimdb-client aimdb-embassy-adapter aimdb-tokio-adapter aimdb-wasm-adapter aimdb-sync aimdb-persistence aimdb-persistence-sqlite aimdb-mqtt-connector aimdb-knx-connector aimdb-ws-protocol aimdb-websocket-connector aimdb-uds-connector aimdb-serial-connector aimdb-tcp-connector aimdb-codegen aimdb-cli aimdb-mcp sync-api-demo tokio-mqtt-connector-demo embassy-mqtt-connector-demo tokio-knx-connector-demo embassy-knx-connector-demo embassy-serial-connector-demo embassy-bench-stm32h5 weather-mesh-common weather-hub weather-station-alpha weather-station-beta hello-mailbox hello-mailbox-async hello-single-latest-async aimdb-bench; do \ + @for pkg in aimdb-derive aimdb-data-contracts aimdb-core aimdb-client aimdb-embassy-adapter aimdb-tokio-adapter aimdb-wasm-adapter aimdb-sync aimdb-persistence aimdb-persistence-sqlite aimdb-mqtt-connector aimdb-knx-connector aimdb-ws-protocol aimdb-websocket-connector aimdb-uds-connector aimdb-serial-connector aimdb-tcp-connector aimdb-codegen aimdb-cli aimdb-mcp sync-api-demo tokio-mqtt-connector-demo embassy-mqtt-connector-demo tokio-knx-connector-demo embassy-knx-connector-demo embassy-serial-connector-demo embassy-bench-stm32h5 weather-mesh-common weather-hub weather-station weather-station-alpha weather-station-beta hello-mailbox hello-mailbox-async hello-single-latest-async aimdb-bench; do \ printf "$(YELLOW) → Formatting $$pkg$(NC)\n"; \ cargo fmt -p $$pkg 2>/dev/null || true; \ done @@ -187,7 +187,7 @@ fmt: fmt-check: @printf "$(GREEN)Checking code formatting (workspace members only)...$(NC)\n" @FAILED=0; \ - for pkg in aimdb-derive aimdb-data-contracts aimdb-core aimdb-client aimdb-embassy-adapter aimdb-tokio-adapter aimdb-wasm-adapter aimdb-sync aimdb-persistence aimdb-persistence-sqlite aimdb-mqtt-connector aimdb-knx-connector aimdb-ws-protocol aimdb-websocket-connector aimdb-uds-connector aimdb-serial-connector aimdb-tcp-connector aimdb-codegen aimdb-cli aimdb-mcp sync-api-demo tokio-mqtt-connector-demo embassy-mqtt-connector-demo tokio-knx-connector-demo embassy-knx-connector-demo embassy-serial-connector-demo embassy-bench-stm32h5 weather-mesh-common weather-hub weather-station-alpha weather-station-beta hello-mailbox hello-mailbox-async hello-single-latest-async aimdb-bench; do \ + for pkg in aimdb-derive aimdb-data-contracts aimdb-core aimdb-client aimdb-embassy-adapter aimdb-tokio-adapter aimdb-wasm-adapter aimdb-sync aimdb-persistence aimdb-persistence-sqlite aimdb-mqtt-connector aimdb-knx-connector aimdb-ws-protocol aimdb-websocket-connector aimdb-uds-connector aimdb-serial-connector aimdb-tcp-connector aimdb-codegen aimdb-cli aimdb-mcp sync-api-demo tokio-mqtt-connector-demo embassy-mqtt-connector-demo tokio-knx-connector-demo embassy-knx-connector-demo embassy-serial-connector-demo embassy-bench-stm32h5 weather-mesh-common weather-hub weather-station weather-station-alpha weather-station-beta hello-mailbox hello-mailbox-async hello-single-latest-async aimdb-bench; do \ printf "$(YELLOW) → Checking $$pkg$(NC)\n"; \ if ! cargo fmt -p $$pkg -- --check 2>&1; then \ printf "$(RED)❌ Formatting check failed for $$pkg$(NC)\n"; \ @@ -398,6 +398,8 @@ examples: cargo build --package weather-station-alpha @printf "$(YELLOW) → Building weather-mesh-demo: weather-station-beta (edge, synthetic)$(NC)\n" cargo build --package weather-station-beta + @printf "$(YELLOW) → Building weather-mesh-demo: weather-station (public mesh, profile-driven)$(NC)\n" + cargo build --package weather-station @printf "$(YELLOW) → Building weather-station-gamma (embedded, embassy runtime)$(NC)\n" cargo build --package weather-station-gamma --target thumbv7em-none-eabihf @printf "$(YELLOW) → Building remote-access-demo (AimX server + client)$(NC)\n" diff --git a/examples/weather-mesh-demo/weather-station/Cargo.toml b/examples/weather-mesh-demo/weather-station/Cargo.toml new file mode 100644 index 0000000..567fbac --- /dev/null +++ b/examples/weather-mesh-demo/weather-station/Cargo.toml @@ -0,0 +1,42 @@ +[package] +name = "weather-station" +version = "0.1.0" +edition = "2021" +description = "Generic cloud weather station - joins the public mesh from a station.toml profile" +license = "MIT OR Apache-2.0" +publish = false + +[[bin]] +name = "weather-station" +path = "src/main.rs" + +[dependencies] +weather-mesh-common = { path = "../weather-mesh-common", features = [ + "linkable", + "migratable", +] } +aimdb-core = { path = "../../../aimdb-core" } +aimdb-tokio-adapter = { path = "../../../aimdb-tokio-adapter" } +aimdb-data-contracts = { path = "../../../aimdb-data-contracts", features = [ + "linkable", + "migratable", +] } +aimdb-mqtt-connector = { path = "../../../aimdb-mqtt-connector", features = [ + "tokio-runtime", +] } + +tokio = { version = "1", features = ["full"] } +tracing = "0.1" +tracing-subscriber = { version = "0.3", features = ["env-filter"] } +reqwest = { version = "0.12", features = ["json"] } +serde = { version = "1.0", features = ["derive"] } +serde_json = "1.0" +chrono = "0.4" +clap = { version = "4", features = ["derive"] } +toml = "0.8" +# Pre-flight CONNECT probe only (revoked-credential UX, design 042 §9); the +# data path goes through aimdb-mqtt-connector. Same feature set as the +# connector's own rumqttc. +rumqttc = { version = "0.25", default-features = false, features = [ + "use-native-tls", +] } diff --git a/examples/weather-mesh-demo/weather-station/src/main.rs b/examples/weather-mesh-demo/weather-station/src/main.rs new file mode 100644 index 0000000..fecad87 --- /dev/null +++ b/examples/weather-mesh-demo/weather-station/src/main.rs @@ -0,0 +1,479 @@ +//! # Weather Station (generic) +//! +//! The public-mesh station binary (design 042 §7/§8): configured entirely by +//! the `station.toml` profile that `aimdb join` writes — broker URL and +//! credentials, assigned slot, station name, and the (coarsened) location the +//! real weather data is fetched for. One profile, one command: +//! +//! ```bash +//! aimdb join https://mesh.aimdb.dev # writes station.toml +//! cargo run -p weather-station -- --config station.toml +//! ``` +//! +//! Same contracts, same Open-Meteo source as station alpha — only the key +//! namespace differs: this station publishes into its assigned slot +//! (`station/{slot}/…`) instead of a compile-time enum topic. + +use aimdb_core::{buffer::BufferCfg, AimDbBuilder, RecordKey, StringKey}; +use aimdb_data_contracts::Linkable; +use aimdb_mqtt_connector::MqttConnector; +use aimdb_tokio_adapter::{TokioAdapter, TokioRecordRegistrarExt}; +use clap::Parser; +use serde::Deserialize; +use std::sync::Arc; +use std::time::Duration; +use tracing::info; +use weather_mesh_common::{DewPoint, Humidity, Temperature}; + +/// Generic cloud weather station — runs from a `station.toml` profile +#[derive(Debug, Parser)] +#[command(name = "weather-station", version, about)] +struct Cli { + /// Path to the station profile written by `aimdb join` (design 043 §4) + #[arg(long)] + config: std::path::PathBuf, +} + +/// The station profile (design 043 §4). Unknown fields are ignored so the +/// service can extend the format without a version bump. +#[derive(Debug, Deserialize)] +struct StationProfile { + profile_version: u64, + station_id: String, + broker: BrokerProfile, + app: AppProfile, +} + +#[derive(Debug, Deserialize)] +struct BrokerProfile { + url: String, + username: String, + password: String, +} + +/// The flagship's `app` map: name + service-coarsened coordinates (042 D8). +#[derive(Debug, Deserialize)] +struct AppProfile { + name: String, + lat: f64, + lon: f64, +} + +#[tokio::main] +async fn main() -> Result<(), Box> { + // Initialize logging + tracing_subscriber::fmt() + .with_env_filter( + tracing_subscriber::EnvFilter::try_from_default_env() + .unwrap_or_else(|_| "weather_station=info,aimdb_core=info".into()), + ) + .init(); + + let cli = Cli::parse(); + let profile: StationProfile = toml::from_str(&std::fs::read_to_string(&cli.config)?)?; + + if profile.profile_version != 1 { + return Err(format!( + "unsupported profile_version {} (this station understands 1) — \ + re-run `aimdb join` with a current CLI", + profile.profile_version + ) + .into()); + } + + // The flagship assigns slot-scoped identities: station_id = "slot-" + // maps onto the hub's pool records station..* (design 042 §4). + let slot: u16 = profile + .station_id + .strip_prefix("slot-") + .and_then(|n| n.parse().ok()) + .ok_or_else(|| { + format!( + "station_id '{}' is not of the form slot- — \ + is this profile from a weather-mesh deployment?", + profile.station_id + ) + })?; + + info!("🚀 Starting Weather Station \"{}\"", profile.app.name); + info!( + "📡 Broker: {} (slot {})", + redact_url(&profile.broker.url), + slot + ); + info!( + "🌍 Weather location: {:.2}°N, {:.2}°E", + profile.app.lat, profile.app.lon + ); + + let client_id = format!("weather-station-{slot}"); + + // Fail fast on a dead credential instead of retrying forever: a revoked + // slot should say so at the moment the user is looking (design 042 §9). + preflight_broker_check(&profile.broker, &client_id).await?; + + let mqtt_url = url_with_credentials(&profile.broker)?; + + // Create runtime adapter + let adapter = Arc::new(TokioAdapter::new()?); + + // Build database with MQTT connector + let mut builder = AimDbBuilder::new() + .runtime(adapter) + .with_connector(MqttConnector::new(&mqtt_url).with_client_id(&client_id)); + + // Configure temperature record + let temp_key = StringKey::intern(format!("station.{slot}.temperature")); + let temp_topic = format!("mqtt://station/{slot}/temperature"); + builder.configure::(temp_key, |reg| { + reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) + .link_to(&temp_topic) + .with_serializer(|_ctx, t: &Temperature| { + t.to_bytes() + .map_err(|_| aimdb_core::connector::SerializeError::InvalidData) + }) + .finish(); + }); + + // Configure humidity record + let humidity_key = StringKey::intern(format!("station.{slot}.humidity")); + let humidity_topic = format!("mqtt://station/{slot}/humidity"); + builder.configure::(humidity_key, |reg| { + reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) + .link_to(&humidity_topic) + .with_serializer(|_ctx, h: &Humidity| { + h.to_bytes() + .map_err(|_| aimdb_core::connector::SerializeError::InvalidData) + }) + .finish(); + }); + + // Configure dew point record — derived from temperature and humidity + let dew_point_key = StringKey::intern(format!("station.{slot}.dew_point")); + let dew_point_topic = format!("mqtt://station/{slot}/dew_point"); + builder.configure::(dew_point_key, |reg| { + reg.buffer(BufferCfg::SpmcRing { capacity: 10 }) + .transform_join(|b| { + b.input::(temp_key) + .input::(humidity_key) + .on_triggers(|mut rx, producer| async move { + let mut last_temp: Option = None; + let mut last_hum: Option = None; + while let Ok(trigger) = rx.recv().await { + match trigger.index() { + 0 => last_temp = trigger.as_input::().cloned(), + 1 => last_hum = trigger.as_input::().cloned(), + _ => {} + } + if let (Some(t), Some(h)) = (&last_temp, &last_hum) { + // Magnus approximation: T_dp ≈ T - (100 - RH) / 5 + let dew_point = t.celsius - (100.0 - h.percent) / 5.0; + let timestamp = t.timestamp.max(h.timestamp); + producer.produce(DewPoint { + celsius: dew_point, + timestamp, + }); + } + } + }) + }) + .link_to(&dew_point_topic) + .with_serializer(|_ctx, d: &DewPoint| { + d.to_bytes() + .map_err(|_| aimdb_core::connector::SerializeError::InvalidData) + }) + .finish(); + }); + + let (db, runner) = builder.build().await?; + tokio::spawn(runner.run()); + + info!("✅ Database initialized with 3 record types"); + + // Get producers + let temp_producer = db.producer::(temp_key.as_str())?; + let humidity_producer = db.producer::(humidity_key.as_str())?; + + // Spawn weather data producer + info!("🌤️ Starting weather data producer..."); + let station_name = profile.app.name.clone(); + let (lat, lon) = (profile.app.lat, profile.app.lon); + tokio::spawn(async move { + let client = OpenMeteoClient::new(); + let mut first_publish = true; + + loop { + match client.fetch_current(lat, lon).await { + Ok(data) => { + let now = chrono::Utc::now().timestamp_millis() as u64; + + temp_producer.produce(Temperature::new(data.temperature as f32, now)); + humidity_producer.produce(Humidity { + percent: data.humidity as f32, + timestamp: now, + }); + + tracing::info!( + "📊 Published: {:.1}°C, {:.1}%", + data.temperature, + data.humidity + ); + + if first_publish { + first_publish = false; + print_live_banner(&station_name); + } + } + Err(e) => { + tracing::warn!("Failed to fetch weather data: {}", e); + } + } + + // OpenMeteo updates every 15 minutes, poll every 5 min + tokio::time::sleep(Duration::from_secs(300)).await; + } + }); + + info!(""); + info!("🎯 Weather Station \"{}\" ready!", profile.app.name); + info!("📡 Publishing to MQTT topics:"); + info!(" - {}", temp_topic); + info!(" - {}", humidity_topic); + info!(" - {}", dew_point_topic); + info!(""); + info!("Press Ctrl+C to stop"); + + // Keep running + tokio::signal::ctrl_c().await?; + info!("🛑 Shutting down..."); + + Ok(()) +} + +/// The shareable moment (design 042 §8, tier 2): after the first successful +/// publish, hand the user their station's live chart and a ready-made MCP +/// prompt for its own data. +fn print_live_banner(name: &str) { + println!(); + println!("🗺️ Your station is live: https://aimdb.dev/mesh/{name}"); + println!("🤖 Ask any MCP agent about it — point it at https://aimdb.dev/mcp and try:"); + println!(" \"What is the latest temperature and dew point at station {name}?\""); + println!(); +} + +/// One CONNECT → CONNACK round-trip before building the database. +/// +/// The connector's event loop retries connection errors forever (by design — +/// brokers come and go). For an admission-scoped credential that is the wrong +/// UX: a revoked slot would retry silently for eternity. Probe once and turn +/// an auth rejection into a human answer (design 042 §9). +async fn preflight_broker_check( + broker: &BrokerProfile, + client_id: &str, +) -> Result<(), Box> { + let (tls, host, port) = split_broker_url(&broker.url)?; + + let mut opts = rumqttc::MqttOptions::new(format!("{client_id}-preflight"), host, port); + opts.set_keep_alive(Duration::from_secs(10)); + opts.set_credentials(&broker.username, &broker.password); + if tls { + opts.set_transport(rumqttc::Transport::Tls(rumqttc::TlsConfiguration::Native)); + } + + let (client, mut event_loop) = rumqttc::AsyncClient::new(opts, 4); + let result = tokio::time::timeout(Duration::from_secs(15), async { + loop { + match event_loop.poll().await { + Ok(rumqttc::Event::Incoming(rumqttc::Packet::ConnAck(_))) => return Ok(()), + Ok(_) => continue, + Err(e) => return Err(e), + } + } + }) + .await; + let _ = client.disconnect().await; + + match result { + Ok(Ok(())) => { + info!("✅ Broker accepted the station credential"); + Ok(()) + } + Ok(Err(rumqttc::ConnectionError::ConnectionRefused(code))) => Err(format!( + "the broker rejected this station's credential ({code:?}).\n \ + The slot was likely revoked (silent for 30 days, or by the operator).\n \ + Re-join the mesh for a fresh slot: aimdb join " + ) + .into()), + Ok(Err(e)) => Err(format!( + "cannot reach the broker at {}: {e}", + redact_url(&broker.url) + ) + .into()), + Err(_) => Err(format!( + "timed out connecting to the broker at {}", + redact_url(&broker.url) + ) + .into()), + } +} + +/// Split a `mqtt[s]://host[:port]` broker URL into (tls, host, port). +fn split_broker_url(url: &str) -> Result<(bool, String, u16), String> { + let (scheme, rest) = url + .split_once("://") + .ok_or_else(|| format!("broker URL '{url}' has no scheme"))?; + let tls = match scheme { + "mqtt" => false, + "mqtts" => true, + other => return Err(format!("unsupported broker scheme '{other}'")), + }; + let authority = rest.split('/').next().unwrap_or(rest); + let (host, port) = match authority.rsplit_once(':') { + Some((host, port)) => ( + host.to_string(), + port.parse() + .map_err(|_| format!("invalid broker port in '{url}'"))?, + ), + None => (authority.to_string(), if tls { 8883 } else { 1883 }), + }; + Ok((tls, host, port)) +} + +/// Embed the profile's credential into the connector URL +/// (`mqtts://user:pass@host:port`), the form the MQTT connector parses. +fn url_with_credentials(broker: &BrokerProfile) -> Result { + let (scheme, rest) = broker + .url + .split_once("://") + .ok_or_else(|| format!("broker URL '{}' has no scheme", broker.url))?; + Ok(format!( + "{scheme}://{}:{}@{rest}", + broker.username, broker.password + )) +} + +/// Strip URL-embedded credentials before logging. +fn redact_url(url: &str) -> String { + if let Some((scheme, rest)) = url.split_once("://") { + if let Some((_creds, host)) = rest.rsplit_once('@') { + return format!("{scheme}://…@{host}"); + } + } + url.to_string() +} + +// --- OpenMeteo API Client (same source as station alpha) --- + +struct OpenMeteoClient { + client: reqwest::Client, +} + +#[derive(Debug, Deserialize)] +struct OpenMeteoResponse { + current: CurrentWeather, +} + +#[derive(Debug, Deserialize)] +struct CurrentWeather { + temperature_2m: f64, + relative_humidity_2m: f64, +} + +struct WeatherData { + temperature: f64, + humidity: f64, +} + +impl OpenMeteoClient { + fn new() -> Self { + Self { + client: reqwest::Client::new(), + } + } + + async fn fetch_current(&self, lat: f64, lon: f64) -> Result { + let url = format!( + "https://api.open-meteo.com/v1/forecast?latitude={}&longitude={}¤t=temperature_2m,relative_humidity_2m", + lat, lon + ); + + let resp: OpenMeteoResponse = self.client.get(&url).send().await?.json().await?; + + Ok(WeatherData { + temperature: resp.current.temperature_2m, + humidity: resp.current.relative_humidity_2m, + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn broker(url: &str) -> BrokerProfile { + BrokerProfile { + url: url.to_string(), + username: "station-17".to_string(), + password: "s3cret".to_string(), + } + } + + #[test] + fn profile_parses_the_043_sample() { + let profile: StationProfile = toml::from_str( + r#" + profile_version = 1 + station_id = "slot-17" + + [broker] + url = "mqtts://xxxx.eu-central-1.emqx.cloud:8883" + username = "station-17" + password = "s3cret" + + [app] + name = "graz-balcony" + lat = 47.07 + lon = 15.44 + "#, + ) + .unwrap(); + assert_eq!(profile.profile_version, 1); + assert_eq!(profile.station_id, "slot-17"); + assert_eq!(profile.app.name, "graz-balcony"); + } + + #[test] + fn broker_url_splits_with_scheme_defaults() { + assert_eq!( + split_broker_url("mqtts://broker.example.com:8883").unwrap(), + (true, "broker.example.com".to_string(), 8883) + ); + assert_eq!( + split_broker_url("mqtts://broker.example.com").unwrap(), + (true, "broker.example.com".to_string(), 8883) + ); + assert_eq!( + split_broker_url("mqtt://localhost").unwrap(), + (false, "localhost".to_string(), 1883) + ); + assert!(split_broker_url("http://x").is_err()); + assert!(split_broker_url("no-scheme").is_err()); + } + + #[test] + fn connector_url_carries_credentials() { + assert_eq!( + url_with_credentials(&broker("mqtts://broker.example.com:8883")).unwrap(), + "mqtts://station-17:s3cret@broker.example.com:8883" + ); + } + + #[test] + fn redaction_hides_credentials() { + assert_eq!( + redact_url("mqtts://station-17:s3cret@broker.example.com:8883"), + "mqtts://…@broker.example.com:8883" + ); + assert_eq!(redact_url("mqtt://localhost"), "mqtt://localhost"); + } +} From 400414f878ed4e1a00eb1fb0b604348db6ab102e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 18:39:17 +0000 Subject: [PATCH 23/31] feat(weather-mesh): gamma build-time station config via MESH_CONFIG (042 WP6) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit build.rs reads the station profile at $MESH_CONFIG (the same station.toml that aimdb join writes, design 043 §4) and generates station_config.rs into OUT_DIR — broker host/port, credentials, slot, station name, and the slot-scoped station//… topics — so the MCU join path is the cloud-station flow with the flash step swapped in: MESH_CONFIG=../station.toml cargo run --release (design 042 §8.2). rerun-if-env-changed/rerun-if-changed keep the generated file honest across builds. With MESH_CONFIG unset the generated defaults are exactly the previous hardcoded consts (192.168.1.3:1883, sensors/gamma/… topics), keeping the tier-1 local demo zero-setup; the split build/flash flow (flash.sh) is untouched since MESH_CONFIG matters only at build time. main.rs include!s the generated config, keeps the compile-time enum record keys, and prints the mesh success banner (live chart URL + MCP prompt) over defmt. An mqtts:// profile fails the build with a pointer at design 042 §8.1 — the Embassy client has no TLS yet (WP7), so MESH_CONFIG firmware can only target a plain-TCP broker until then. The generated credential consts are in place for WP7 to consume. Co-Authored-By: Claude Fable 5 --- Cargo.lock | 1 + .../weather-station-gamma/Cargo.toml | 4 + .../weather-station-gamma/build.rs | 147 ++++++++++++++++++ .../weather-station-gamma/src/main.rs | 60 +++++-- 4 files changed, 196 insertions(+), 16 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index ef43745..bbd9b04 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4603,6 +4603,7 @@ dependencies = [ "rand 0.10.1", "static_cell", "stm32-fmc 0.3.2", + "toml 0.8.23", "weather-mesh-common", ] diff --git a/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml b/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml index b534144..c73a46f 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml +++ b/examples/weather-mesh-demo/weather-station-gamma/Cargo.toml @@ -94,6 +94,10 @@ embedded-alloc = { version = "0.6", features = ["llff"] } # network-stack seed uses the STM32 HAL RNG (embassy_stm32), not this crate. rand = { version = "0.10.1", default-features = false, optional = true } +[build-dependencies] +# Parses the $MESH_CONFIG station profile (station.toml, design 042 §8.2) to +# generate station_config.rs. Host-side only — nothing lands in the firmware. +toml = "0.8" [package.metadata.embassy] build = [ diff --git a/examples/weather-mesh-demo/weather-station-gamma/build.rs b/examples/weather-mesh-demo/weather-station-gamma/build.rs index 8cd32d7..0f46d18 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/build.rs +++ b/examples/weather-mesh-demo/weather-station-gamma/build.rs @@ -1,5 +1,152 @@ +//! Linker args + build-time station configuration (design 042 §8.2). +//! +//! With `MESH_CONFIG` pointing at a station profile (`station.toml`, written +//! by `aimdb join` — design 043 §4), the broker address, slot topics, and +//! station name are generated into `OUT_DIR/station_config.rs` and baked into +//! the firmware: `MESH_CONFIG=../station.toml cargo run --release` is +//! build → flash → live. With `MESH_CONFIG` unset, the generated file carries +//! the local docker-compose demo defaults, so the tier-1 demo stays +//! zero-setup. + +use std::env; +use std::fs; +use std::path::PathBuf; + fn main() { println!("cargo:rustc-link-arg-bins=--nmagic"); println!("cargo:rustc-link-arg-bins=-Tlink.x"); println!("cargo:rustc-link-arg-bins=-Tdefmt.x"); + + println!("cargo:rerun-if-env-changed=MESH_CONFIG"); + + let config = match env::var("MESH_CONFIG") { + Ok(path) => { + println!("cargo:rerun-if-changed={path}"); + mesh_station_config(&path) + } + Err(_) => local_demo_config(), + }; + + let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap()); + fs::write(out_dir.join("station_config.rs"), config).unwrap(); +} + +/// Local docker-compose demo defaults (tier 1 — no profile, no mesh). +/// The topics mirror the `TempKey::Gamma`/…`::Gamma` link addresses in +/// `weather-mesh-common`. +fn local_demo_config() -> String { + station_config("192.168.1.3", 1883, None, None, "gamma", "sensors/gamma") +} + +/// Configuration from the station profile at `path` (design 043 §4). +fn mesh_station_config(path: &str) -> String { + let text = fs::read_to_string(path) + .unwrap_or_else(|e| panic!("MESH_CONFIG: cannot read '{path}': {e}")); + let profile: toml::Value = toml::from_str(&text) + .unwrap_or_else(|e| panic!("MESH_CONFIG: '{path}' is not valid TOML: {e}")); + + let version = profile + .get("profile_version") + .and_then(|v| v.as_integer()) + .unwrap_or_else(|| panic!("MESH_CONFIG: '{path}' has no profile_version")); + assert!( + version == 1, + "MESH_CONFIG: unsupported profile_version {version} (this firmware understands 1)" + ); + + let str_field = |table: &str, key: &str| -> &str { + profile + .get(table) + .and_then(|t| t.get(key)) + .and_then(|v| v.as_str()) + .unwrap_or_else(|| panic!("MESH_CONFIG: '{path}' has no [{table}] {key}")) + }; + + let broker_url = str_field("broker", "url"); + // The Embassy MQTT client has no TLS yet (design 042 §8.1) — a profile for + // the public broker (mqtts://) cannot work; only a plain-TCP broker (e.g. + // the local demo one) is reachable until that lands. + let rest = broker_url.strip_prefix("mqtt://").unwrap_or_else(|| { + panic!( + "MESH_CONFIG: broker URL '{broker_url}' is not mqtt:// — the Embassy \ + MQTT client has no TLS support yet (design 042 §8.1), so this \ + firmware can only target a plain-TCP broker" + ) + }); + let authority = rest.split('/').next().unwrap_or(rest); + let (host, port) = match authority.rsplit_once(':') { + Some((host, port)) => ( + host, + port.parse::() + .unwrap_or_else(|_| panic!("MESH_CONFIG: invalid broker port in '{broker_url}'")), + ), + None => (authority, 1883), + }; + + let station_id = profile + .get("station_id") + .and_then(|v| v.as_str()) + .unwrap_or_else(|| panic!("MESH_CONFIG: '{path}' has no station_id")); + let slot: u16 = station_id + .strip_prefix("slot-") + .and_then(|n| n.parse().ok()) + .unwrap_or_else(|| { + panic!("MESH_CONFIG: station_id '{station_id}' is not of the form slot-") + }); + + let name = str_field("app", "name"); + let credentials = Some(( + str_field("broker", "username"), + str_field("broker", "password"), + )); + + station_config( + host, + port, + credentials, + Some(slot), + name, + &format!("station/{slot}"), + ) +} + +/// Render the generated `station_config.rs`. String values go through `{:?}` +/// so arbitrary profile content stays a valid Rust literal. +fn station_config( + host: &str, + port: u16, + credentials: Option<(&str, &str)>, + slot: Option, + name: &str, + topic_prefix: &str, +) -> String { + let credential_const = |value: Option<&str>| match value { + Some(v) => format!("Some({v:?})"), + None => "None".to_string(), + }; + let username = credential_const(credentials.map(|(u, _)| u)); + let password = credential_const(credentials.map(|(_, p)| p)); + let slot = match slot { + Some(n) => format!("Some({n})"), + None => "None".to_string(), + }; + + format!( + "// Generated by build.rs — station configuration (design 042 §8.2).\n\ + // Source: $MESH_CONFIG station profile, or local demo defaults. Do not edit.\n\ + pub const MQTT_BROKER_HOST: &str = {host:?};\n\ + pub const MQTT_BROKER_PORT: u16 = {port};\n\ + /// Broker credential from the profile. Unused until the Embassy MQTT\n\ + /// client supports authentication + TLS (design 042 §8.1, WP7).\n\ + #[allow(dead_code)]\n\ + pub const MQTT_USERNAME: Option<&str> = {username};\n\ + #[allow(dead_code)]\n\ + pub const MQTT_PASSWORD: Option<&str> = {password};\n\ + /// Assigned mesh slot; `None` in the local demo.\n\ + pub const STATION_SLOT: Option = {slot};\n\ + pub const STATION_NAME: &str = {name:?};\n\ + pub const TEMPERATURE_TOPIC: &str = \"mqtt://{topic_prefix}/temperature\";\n\ + pub const HUMIDITY_TOPIC: &str = \"mqtt://{topic_prefix}/humidity\";\n\ + pub const DEW_POINT_TOPIC: &str = \"mqtt://{topic_prefix}/dew_point\";\n" + ) } diff --git a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs index 6c0eeb4..4af8534 100644 --- a/examples/weather-mesh-demo/weather-station-gamma/src/main.rs +++ b/examples/weather-mesh-demo/weather-station-gamma/src/main.rs @@ -13,18 +13,26 @@ //! //! ## Running //! -//! 1. Start MQTT broker on your network -//! 2. Update MQTT_BROKER_IP constant below -//! 3. Flash to target: +//! The broker address, topics, and station identity are generated at build +//! time by `build.rs` (design 042 §8.2): +//! +//! - **Local demo (default):** no setup — the generated config carries the +//! docker-compose demo defaults. Point it at your broker by editing the +//! defaults in `build.rs`, then: //! ```bash //! cd examples/weather-mesh-demo/weather-station-gamma -//! cargo build --release -//! cargo flash --release +//! cargo run --release # build → probe-rs flash → defmt logs +//! ``` +//! - **Public mesh:** feed it the profile written by `aimdb join`: +//! ```bash +//! MESH_CONFIG=../station.toml cargo run --release //! ``` +//! (Reaches the public broker only once the Embassy client speaks TLS — +//! design 042 §8.1; until then a profile must name a plain-TCP broker.) extern crate alloc; -use aimdb_core::{AimDbBuilder, RecordKey}; +use aimdb_core::AimDbBuilder; #[cfg(feature = "sim")] use aimdb_data_contracts::{RandomWalkParams, SimProfile, SimulatableRegistrarExt}; use aimdb_embassy_adapter::{EmbassyAdapter, EmbassyBufferType, EmbassyRecordRegistrarExtCustom}; @@ -48,9 +56,9 @@ use {defmt_rtt as _, panic_probe as _}; #[global_allocator] static ALLOCATOR: embedded_alloc::LlffHeap = embedded_alloc::LlffHeap::empty(); -// MQTT broker address - update this to match your network -const MQTT_BROKER_IP: &str = "192.168.1.3"; -const MQTT_BROKER_PORT: u16 = 1883; +// Station configuration generated by build.rs: the $MESH_CONFIG station +// profile, or the local demo defaults when unset (design 042 §8.2). +include!(concat!(env!("OUT_DIR"), "/station_config.rs")); bind_interrupts!(struct Irqs { ETH => eth::InterruptHandler; @@ -248,14 +256,17 @@ async fn main(spawner: Spawner) { // Build MQTT broker URL use alloc::format; - let broker_url = format!("mqtt://{}:{}", MQTT_BROKER_IP, MQTT_BROKER_PORT); + let broker_url = format!("mqtt://{}:{}", MQTT_BROKER_HOST, MQTT_BROKER_PORT); let mut builder = AimDbBuilder::new().runtime(runtime.clone()).with_connector( MqttConnectorBuilder::new(&broker_url, stack).with_client_id("weather-station-gamma"), ); - // Configure temperature record - let temp_topic = TempKey::Gamma.link_address().unwrap(); + // Configure temperature record. The record keys stay the compile-time + // enums (they name the records in *this* db); only the MQTT topics come + // from the generated config, so a mesh profile re-points the publishes to + // its assigned slot (station//…). + let temp_topic = TEMPERATURE_TOPIC; builder.configure::(TempKey::Gamma, |reg| { reg.buffer_sized::<16, 2>(EmbassyBufferType::SpmcRing); @@ -286,7 +297,7 @@ async fn main(spawner: Spawner) { }); // Configure humidity record - let humidity_topic = HumidityKey::Gamma.link_address().unwrap(); + let humidity_topic = HUMIDITY_TOPIC; builder.configure::(HumidityKey::Gamma, |reg| { reg.buffer_sized::<16, 2>(EmbassyBufferType::SpmcRing); @@ -319,7 +330,7 @@ async fn main(spawner: Spawner) { // // Showcases the transform_join task model: the closure owns its state and // can hold borrows across .await without Box::pin or manual cloning. - let dew_point_topic = DewPointKey::Gamma.link_address().unwrap(); + let dew_point_topic = DEW_POINT_TOPIC; builder.configure::(DewPointKey::Gamma, |reg| { reg.buffer_sized::<8, 1>(EmbassyBufferType::SpmcRing) .transform_join(|b| { @@ -378,7 +389,7 @@ async fn main(spawner: Spawner) { info!(" Temperature: {}", temp_topic); info!(" Humidity: {}", humidity_topic); info!(" DewPoint: {}", dew_point_topic); - info!(" Broker: {}:{}", MQTT_BROKER_IP, MQTT_BROKER_PORT); + info!(" Broker: {}:{}", MQTT_BROKER_HOST, MQTT_BROKER_PORT); info!(""); static DB_CELL: StaticCell = StaticCell::new(); @@ -386,7 +397,24 @@ async fn main(spawner: Spawner) { let _db = DB_CELL.init(db); info!("✅ Database running"); - info!("🎯 Weather Station Gamma ready!"); + // The dot-on-the-map moment, in the same terminal that flashed the board + // (design 042 §8.2): mesh builds print the station's live chart URL and a + // ready-made MCP prompt over defmt. + match STATION_SLOT { + Some(slot) => { + info!( + "🎯 Station \"{}\" (slot {}) is publishing to the mesh!", + STATION_NAME, slot + ); + info!("🗺️ Live chart: https://aimdb.dev/mesh/{}", STATION_NAME); + info!("🤖 Point an MCP agent at https://aimdb.dev/mcp and try:"); + info!( + " \"What is the latest temperature at station {}?\"", + STATION_NAME + ); + } + None => info!("🎯 Weather Station Gamma ready!"), + } // Drive the AimDB runner and LED blink concurrently. embassy_futures::join::join(db_runner.run(), async { From 4f4953e4976db4087c612de1eee6035a9224c9cd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 19:08:35 +0000 Subject: [PATCH 24/31] feat(weather-mesh): AimX endpoint + visible contract rejections in mesh hub (042 WP1 follow-up) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verification of the local drill (plan §End-to-end #1) surfaced three gaps between the slot pool and the design's demo loop: - Mesh mode now registers a read-only AimX TcpServer (AIMX_BIND, default 127.0.0.1:7433) — without it there is no server behind 'aimdb record list --url tcp://…:7433' (design 042 §7). The local demo path is unchanged (no listener). - Slot records register .with_remote_access(), installing the JSON codec that record.get/subscribe and the dashboard read through. - The mesh deserializers report contract violations through ctx.log() — design §9 makes 'a malformed payload is visibly rejected at the hub' a feature, and the router's own error path is compiled out without the tracing feature. The fallback env filter now includes the runtime-context log target (aimdb=info) so the reports actually show on bare cargo run; the docker demo always sets RUST_LOG explicitly and is unaffected. Verified against mosquitto: MESH_SLOTS=4, publishes to station/2/… surface as station.2.temperature/humidity signal gauges over tcp://127.0.0.1:7433, the per-slot join derives station.2.dew_point (21.4°C/55% → 12.4°C), and a malformed payload logs 'station.2.temperature: rejected payload: Migration error: …' while the slot keeps its last good value. Co-Authored-By: Claude Fable 5 --- Cargo.lock | 1 + .../weather-mesh-demo/weather-hub/Cargo.toml | 5 +++ .../weather-mesh-demo/weather-hub/src/main.rs | 41 +++++++++++++++++-- 3 files changed, 44 insertions(+), 3 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index bbd9b04..338c32c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4501,6 +4501,7 @@ dependencies = [ "aimdb-core", "aimdb-data-contracts", "aimdb-mqtt-connector", + "aimdb-tcp-connector", "aimdb-tokio-adapter", "tokio", "tracing", diff --git a/examples/weather-mesh-demo/weather-hub/Cargo.toml b/examples/weather-mesh-demo/weather-hub/Cargo.toml index 73bb9c1..ec44dc0 100644 --- a/examples/weather-mesh-demo/weather-hub/Cargo.toml +++ b/examples/weather-mesh-demo/weather-hub/Cargo.toml @@ -29,6 +29,11 @@ aimdb-data-contracts = { path = "../../../aimdb-data-contracts", features = [ aimdb-mqtt-connector = { path = "../../../aimdb-mqtt-connector", features = [ "tokio-runtime", ] } +# AimX server for mesh mode: `aimdb record list --url tcp://…:7433` +# (design 042 §7). Unused in the local demo path. +aimdb-tcp-connector = { path = "../../../aimdb-tcp-connector", features = [ + "tokio-runtime", +] } tokio = { version = "1", features = ["full"] } tracing = "0.1" diff --git a/examples/weather-mesh-demo/weather-hub/src/main.rs b/examples/weather-mesh-demo/weather-hub/src/main.rs index 015c8bb..38a7f77 100644 --- a/examples/weather-mesh-demo/weather-hub/src/main.rs +++ b/examples/weather-mesh-demo/weather-hub/src/main.rs @@ -6,10 +6,14 @@ use weather_mesh_common::{DewPoint, Humidity, HumidityKey, TempKey, Temperature} #[tokio::main] async fn main() -> aimdb_core::DbResult<()> { // Initialize logging + // `aimdb` is the runtime-context log target (ctx.log() via the adapter) — + // without it in the fallback filter, contract-violation reports from the + // mesh deserializers are invisible. The docker-compose demo always sets + // RUST_LOG explicitly, so this only affects bare `cargo run`. tracing_subscriber::fmt() .with_env_filter( tracing_subscriber::EnvFilter::try_from_default_env() - .unwrap_or_else(|_| "weather_hub=info,aimdb_core=info".into()), + .unwrap_or_else(|_| "weather_hub=info,aimdb_core=info,aimdb=info".into()), ) .init(); @@ -38,6 +42,16 @@ async fn main() -> aimdb_core::DbResult<()> { Some(v) => { let slots: u16 = v.parse().unwrap_or(64); tracing::info!("🕸️ Mesh mode: {} slots", slots); + + // AimX introspection endpoint — the `aimdb record list --url + // tcp://…:7433` half of the demo loop (design 042 §7). Read-only + // (the default policy); loopback unless AIMX_BIND says otherwise + // (the hosted deployment fronts it as aimdb.dev:7433). + let aimx_bind = + std::env::var("AIMX_BIND").unwrap_or_else(|_| "127.0.0.1:7433".to_string()); + tracing::info!("🔌 AimX endpoint: tcp://{}", aimx_bind); + builder = builder.with_connector(aimdb_tcp_connector::TcpServer::new(&aimx_bind)); + configure_mesh(&mut builder, slots); } None => configure_demo(&mut builder), @@ -116,8 +130,20 @@ fn configure_mesh(builder: &mut AimDbBuilder, slots: u16) { builder.configure::(temp_key, |reg| { reg.buffer(BufferCfg::SpmcRing { capacity: 100 }); reg.observe(); + // JSON codec for AimX record.get/subscribe — the dashboard and + // `aimdb record get` read the mesh through this. + reg.with_remote_access(); + let key_name = temp_key.as_str(); reg.link_from(&temp_topic) - .with_deserializer(|_ctx, data: &[u8]| Temperature::from_bytes(data)) + .with_deserializer(move |ctx, data: &[u8]| { + Temperature::from_bytes(data).map_err(|e| { + // Design 042 §9: schema errors are a feature — a + // malformed payload is visibly rejected at the hub. + ctx.log() + .error(&format!("{key_name}: rejected payload: {e}")); + e + }) + }) .finish(); }); @@ -126,8 +152,16 @@ fn configure_mesh(builder: &mut AimDbBuilder, slots: u16) { builder.configure::(humidity_key, |reg| { reg.buffer(BufferCfg::SpmcRing { capacity: 100 }); reg.observe(); + reg.with_remote_access(); + let key_name = humidity_key.as_str(); reg.link_from(&humidity_topic) - .with_deserializer(|_ctx, data: &[u8]| Humidity::from_bytes(data)) + .with_deserializer(move |ctx, data: &[u8]| { + Humidity::from_bytes(data).map_err(|e| { + ctx.log() + .error(&format!("{key_name}: rejected payload: {e}")); + e + }) + }) .finish(); }); @@ -137,6 +171,7 @@ fn configure_mesh(builder: &mut AimDbBuilder, slots: u16) { builder.configure::(dew_point_key, |reg| { reg.buffer(BufferCfg::SpmcRing { capacity: 100 }); reg.observe(); + reg.with_remote_access(); reg.transform_join(|b| { b.input::(temp_key) .input::(humidity_key) From 60d87673e9e93254ccc06dc57b3f980d2f19acff Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 19:08:35 +0000 Subject: [PATCH 25/31] fix(cli): keep observability metadata in record list output (042 WP4 follow-up) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The AimX server attaches signal_stats/stage_profiling/buffer metrics to record.list when built with observability, but those RecordMetadata fields are themselves cfg-gated — a client compiled without the feature silently drops them on deserialize, so 'record list --format json' showed no signal gauges even when the server sent them. Enable aimdb-core/observability for the CLI so the fields round-trip; this is also how the mesh drill confirms a slot is updating (ring buffers have no canonical latest for record.get). Co-Authored-By: Claude Fable 5 --- tools/aimdb-cli/Cargo.toml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tools/aimdb-cli/Cargo.toml b/tools/aimdb-cli/Cargo.toml index fbba9d0..4109ab7 100644 --- a/tools/aimdb-cli/Cargo.toml +++ b/tools/aimdb-cli/Cargo.toml @@ -19,9 +19,13 @@ path = "src/main.rs" aimdb-client = { version = "0.6.0", path = "../../aimdb-client" } aimdb-codegen = { version = "0.2.0", path = "../../aimdb-codegen" } -# Core dependencies - reuse protocol types from aimdb-core +# Core dependencies - reuse protocol types from aimdb-core. +# `observability` keeps the optional RecordMetadata fields (signal_stats, +# stage_profiling, buffer metrics) in the client-side struct — without it +# serde silently drops them from `record list --format json` output. aimdb-core = { version = "1.1.0", path = "../../aimdb-core", features = [ "std", + "observability", ] } serde = { version = "1", features = ["derive"] } serde_json = "1" From 6524f17eaa2fc004d68f4b49ee2c6f61ec6b30a5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 19:08:35 +0000 Subject: [PATCH 26/31] fix(weather-mesh): station failure UX prints via Display (042 WP5 follow-up) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Returning Err from main Debug-formats the message — the revoked-slot text came out as one line of escaped \n. Route run() errors through eprintln (Display) + exit(1) so the design §9 failure UX reads as written. Co-Authored-By: Claude Fable 5 --- .../weather-mesh-demo/weather-station/src/main.rs | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/examples/weather-mesh-demo/weather-station/src/main.rs b/examples/weather-mesh-demo/weather-station/src/main.rs index fecad87..d2ad005 100644 --- a/examples/weather-mesh-demo/weather-station/src/main.rs +++ b/examples/weather-mesh-demo/weather-station/src/main.rs @@ -60,7 +60,16 @@ struct AppProfile { } #[tokio::main] -async fn main() -> Result<(), Box> { +async fn main() { + // Display-format errors (revoked slot, bad profile, …) instead of the + // default Debug dump — these messages are the failure UX (design 042 §9). + if let Err(e) = run().await { + eprintln!("Error: {e}"); + std::process::exit(1); + } +} + +async fn run() -> Result<(), Box> { // Initialize logging tracing_subscriber::fmt() .with_env_filter( From 5d504d91c952de9cc0f1f93ca09680f698648cd1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 19:09:38 +0000 Subject: [PATCH 27/31] docs(weather-mesh): record 042 OSS implementation status + mesh-mode usage Plan 042 status now reflects the implemented OSS work packages (WP1, WP3's 043 doc, WP4, WP5, WP6 and the verification follow-ups); the demo README documents the opt-in public-mesh mode (MESH_SLOTS/MQTT_URL/AIMX_BIND, the weather-station join loop, gamma's MESH_CONFIG build embedding) alongside the unchanged local demo. Co-Authored-By: Claude Fable 5 --- docs/plan/042-public-weather-mesh-flagship.md | 2 +- examples/weather-mesh-demo/README.md | 28 +++++++++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/docs/plan/042-public-weather-mesh-flagship.md b/docs/plan/042-public-weather-mesh-flagship.md index a71954c..9f725d6 100644 --- a/docs/plan/042-public-weather-mesh-flagship.md +++ b/docs/plan/042-public-weather-mesh-flagship.md @@ -2,7 +2,7 @@ **Design:** [042 — Public weather mesh flagship](../design/042-public-weather-mesh-flagship.md) -**Status:** 📝 Planned +**Status:** 🚧 OSS work packages implemented 2026-07-09 (WP1, WP3's 043 doc, WP4, WP5, WP6 — plus the mqtts:// TLS-transport fix in the Tokio MQTT client the WP1 premise relied on, a read-only AimX TCP endpoint in the hub's mesh mode so the §7 `aimdb record list` loop has a server, and CLI observability metadata so signal gauges round-trip). External WPs (WP2 broker, WP3 provisioning service, WP8 website) and WP7 (Embassy TLS) remain open. This plan translates the design's sequencing (§10) into concrete work packages (WPs). Each WP is marked by where the work happens: diff --git a/examples/weather-mesh-demo/README.md b/examples/weather-mesh-demo/README.md index e794203..fee771a 100644 --- a/examples/weather-mesh-demo/README.md +++ b/examples/weather-mesh-demo/README.md @@ -27,6 +27,7 @@ A distributed weather monitoring mesh demonstrating AimDB's multi-tier architect | `weather-station-alpha` | Tokio (Linux Edge) | Real weather data (Open-Meteo API) | | `weather-station-beta` | Tokio (Linux Edge) | Synthetic sensor data | | `weather-station-gamma` | Embassy (STM32H563ZITx) | Portable/remote MCU sensor (hardware) | +| `weather-station` | Tokio (anywhere) | Public-mesh station, driven by a `station.toml` profile | | `weather-mesh-common` | no_std compatible | Shared contracts and configuration | | `mqtt` | Eclipse Mosquitto | Message bus for all nodes | @@ -110,6 +111,33 @@ cargo build --release probe-rs run --chip STM32H563ZITx target/thumbv8m.main-none-eabihf/release/weather-station-gamma ``` +## Public mesh mode (design 042) + +The same crates also run the hosted public mesh anyone can join +([design 042](../../docs/design/042-public-weather-mesh-flagship.md)). The +local demo above is unchanged; mesh mode is opt-in per environment variable: + +- **Hub** — `MESH_SLOTS=64 cargo run -p weather-hub` replaces the three-enum + configuration with a bounded pool of string-keyed slot records + (`station.{slot}.temperature/humidity/dew_point` from + `station/{slot}/…` topics; dew point derived at the hub). `MQTT_URL` takes + a full connector URL (e.g. `mqtts://hub-sub:…@…:8883` for EMQX Cloud), and + a read-only AimX endpoint is served at `AIMX_BIND` + (default `127.0.0.1:7433`) for `aimdb record list/get/watch`. +- **Station** — the three-command join loop + ([format doc 043](../../docs/design/043-join-endpoint-v1.md)): + + ```bash + aimdb join https://mesh.aimdb.dev # writes station.toml + cargo run -p weather-station -- --config station.toml + aimdb record list --connect tcp://aimdb.dev:7433 + ``` + +- **MCU station** — same profile, consumed at build time: + `MESH_CONFIG=../station.toml cargo run --release` inside + `weather-station-gamma` (plain-TCP brokers only until the Embassy MQTT + client speaks TLS — design 042 §8.1). + ## Inspecting with VS Code Copilot The weather mesh exposes an AimX API socket that Copilot can read through MCP (Model Context Protocol). Simply ask Copilot in natural language: From 5e728a8abb96aa5260ebe37f13c062acd65986e6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 19:22:35 +0000 Subject: [PATCH 28/31] fix(cli): join HTTP client uses system trust roots (042 WP4 follow-up) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit reqwest's rustls-tls feature bundles webpki-roots, whose CDLA-Permissive-2.0 license is not on the deny.toml allowlist — make check failed at cargo-deny. Switch to rustls-tls-native-roots (system trust store), which also matches how TLS trust works everywhere else in the workspace. cargo deny is green again; the join unit/mock tests are unaffected. Co-Authored-By: Claude Fable 5 --- Cargo.lock | 48 +++++++++++++++++++++++++++----------- tools/aimdb-cli/Cargo.toml | 6 +++-- 2 files changed, 39 insertions(+), 15 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 338c32c..ad700ea 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2059,10 +2059,10 @@ dependencies = [ "hyper", "hyper-util", "rustls", + "rustls-native-certs", "tokio", "tokio-rustls", "tower-service", - "webpki-roots", ] [[package]] @@ -2560,10 +2560,10 @@ dependencies = [ "libc", "log", "openssl", - "openssl-probe", + "openssl-probe 0.1.6", "openssl-sys", "schannel", - "security-framework", + "security-framework 2.11.1", "security-framework-sys", "tempfile", ] @@ -2678,6 +2678,12 @@ version = "0.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d05e27ee213611ffe7d6348b942e8f942b37114c00cc03cec254295a4a17852e" +[[package]] +name = "openssl-probe" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe" + [[package]] name = "openssl-sys" version = "0.9.116" @@ -3043,6 +3049,7 @@ dependencies = [ "pin-project-lite", "quinn", "rustls", + "rustls-native-certs", "rustls-pki-types", "serde", "serde_json", @@ -3058,7 +3065,6 @@ dependencies = [ "wasm-bindgen", "wasm-bindgen-futures", "web-sys", - "webpki-roots", ] [[package]] @@ -3162,6 +3168,18 @@ dependencies = [ "zeroize", ] +[[package]] +name = "rustls-native-certs" +version = "0.8.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dab5152771c58876a2146916e53e35057e1a4dfa2b9df0f0305b07f611fdea4d" +dependencies = [ + "openssl-probe 0.2.1", + "rustls-pki-types", + "schannel", + "security-framework 3.5.1", +] + [[package]] name = "rustls-pki-types" version = "1.13.0" @@ -3244,6 +3262,19 @@ dependencies = [ "security-framework-sys", ] +[[package]] +name = "security-framework" +version = "3.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b3297343eaf830f66ede390ea39da1d462b6b0c1b000f420d0a83f898bbbe6ef" +dependencies = [ + "bitflags 2.11.0", + "core-foundation 0.10.1", + "core-foundation-sys", + "libc", + "security-framework-sys", +] + [[package]] name = "security-framework-sys" version = "2.15.0" @@ -4628,15 +4659,6 @@ dependencies = [ "wasm-bindgen", ] -[[package]] -name = "webpki-roots" -version = "1.0.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "22cfaf3c063993ff62e73cb4311efde4db1efb31ab78a3e5c457939ad5cc0bed" -dependencies = [ - "rustls-pki-types", -] - [[package]] name = "winapi" version = "0.3.9" diff --git a/tools/aimdb-cli/Cargo.toml b/tools/aimdb-cli/Cargo.toml index 4109ab7..1ada495 100644 --- a/tools/aimdb-cli/Cargo.toml +++ b/tools/aimdb-cli/Cargo.toml @@ -59,9 +59,11 @@ serde_yaml = { version = "0.9", optional = true } # `join` only (design 043): HTTP client for the provisioning endpoint + the # GitHub device flow, and TOML for the station profile. rustls so released -# binaries don't depend on a system OpenSSL. +# binaries don't depend on a system OpenSSL; native roots (system trust +# store) rather than the bundled webpki-roots, whose CDLA license is not on +# the deny.toml allowlist. reqwest = { version = "0.12", default-features = false, features = [ - "rustls-tls", + "rustls-tls-native-roots", "json", ], optional = true } toml = { version = "0.8", optional = true } From bff922f4c992ede629371c51abc000156116156f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 20:55:53 +0000 Subject: [PATCH 29/31] fix(docs): update TCP connector description to include client support --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index aecab71..b3fe526 100644 --- a/README.md +++ b/README.md @@ -241,7 +241,7 @@ See the [MCP server docs](tools/aimdb-mcp/) for Claude Desktop and other editors | **WebSocket** — `aimdb-websocket-connector` | ✅ Ready | std, wasm | | **Serial (COBS/UART)** — `aimdb-serial-connector` | ✅ Ready | std, no_std | | **Unix domain socket** — `aimdb-uds-connector` | ✅ Ready | std | -| **TCP** — `aimdb-tcp-connector` | ✅ Ready | std, no_std | +| **TCP** — `aimdb-tcp-connector` | ✅ Ready | std, no_std client | | **Kafka** | 📋 Planned | std | | **Modbus** | 📋 Planned | std, no_std | From c3d2c3dea99ffefbfaf392d645f45c46a3dc10f0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 21:10:02 +0000 Subject: [PATCH 30/31] fix(deps): update quinn-proto to version 0.11.15 and windows-sys to version 0.60.2 --- Cargo.lock | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index ad700ea..10c7ab5 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2854,9 +2854,9 @@ dependencies = [ [[package]] name = "quinn-proto" -version = "0.11.13" +version = "0.11.15" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f1906b49b0c3bc04b5fe5d86a77925ae6524a19b816ae38ce1e426255f1d8a31" +checksum = "4fcb935c5bec503c2f0e306bdd3e58bb9029dcb14fa8d9ac76e3a5256ac0763e" dependencies = [ "bytes", "getrandom 0.3.4", @@ -2884,7 +2884,7 @@ dependencies = [ "once_cell", "socket2", "tracing", - "windows-sys 0.52.0", + "windows-sys 0.60.2", ] [[package]] From 213c6be88b2da0ab7b8f93937fb5648041392292 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Schn=C3=B6rch?= Date: Thu, 9 Jul 2026 21:51:50 +0000 Subject: [PATCH 31/31] fix(docs): update TCP connector description to remove client support mention --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b3fe526..aecab71 100644 --- a/README.md +++ b/README.md @@ -241,7 +241,7 @@ See the [MCP server docs](tools/aimdb-mcp/) for Claude Desktop and other editors | **WebSocket** — `aimdb-websocket-connector` | ✅ Ready | std, wasm | | **Serial (COBS/UART)** — `aimdb-serial-connector` | ✅ Ready | std, no_std | | **Unix domain socket** — `aimdb-uds-connector` | ✅ Ready | std | -| **TCP** — `aimdb-tcp-connector` | ✅ Ready | std, no_std client | +| **TCP** — `aimdb-tcp-connector` | ✅ Ready | std, no_std | | **Kafka** | 📋 Planned | std | | **Modbus** | 📋 Planned | std, no_std |