diff --git a/README.md b/README.md index 47a969f..cbe42a0 100644 --- a/README.md +++ b/README.md @@ -54,6 +54,17 @@ Discover our primary open-source initiatives below: --- +## 🧭 Start here: the build playbook + +### πŸ“˜ [building-the-new-internet](skills/building-the-new-internet/) +> **One skill that hands you the whole stack, ready to deploy onto the MATA distributed cloud.** + +* 🌟 **Focus:** What to use for identity, storage, media, AI, allocation and compression β€” plus the architecture, the deploy seams, and the measurement discipline that make an app mesh-ready on day one. +* πŸ“– **Read it:** as a human, or drop it into your coding agent's skills folder. +* πŸ“œ **License:** MIT + +--- + ## πŸ› οΈ Tech Stack & Toolkit While **Rust** sits at the absolute core of our system programming goals, we use a tailored stack to push our innovations natively across platforms and runtime targets: diff --git a/profile/README.md b/profile/README.md index 0863d10..776e648 100644 --- a/profile/README.md +++ b/profile/README.md @@ -59,6 +59,17 @@ Discover our primary open-source initiatives below: --- +## 🧭 Start here: the build playbook + +### πŸ“˜ [building-the-new-internet](https://github.com/Remade-With-Rust/.github/tree/main/skills/building-the-new-internet) +> **One skill that hands you the whole stack, ready to deploy onto the MATA distributed cloud.** + +* 🌟 **Focus:** What to use for identity, storage, media, AI, allocation and compression β€” plus the architecture, the deploy seams, and the measurement discipline that make an app mesh-ready on day one. +* πŸ“– **Read it:** as a human, or drop it into your coding agent's skills folder. +* πŸ“œ **License:** MIT + +--- + ## πŸ› οΈ Tech Stack & Toolkit While **Rust** sits at the absolute core of our system programming goals, we use a tailored stack to push our innovations natively across platforms and runtime targets: diff --git a/skills/README.md b/skills/README.md new file mode 100644 index 0000000..c2589ad --- /dev/null +++ b/skills/README.md @@ -0,0 +1,58 @@ +# Skills + +Agent skills for building on the Remade With Rust stack. Drop one into your agent's skills +directory and it loads automatically when the work matches. + +| Skill | What it's for | +|---|---| +| [`building-the-new-internet`](building-the-new-internet/) | Starting and shipping a Rust app, service or website that is ready to deploy onto the MATA distributed cloud the day it goes live. | + +## building-the-new-internet + +The single playbook for the whole stack. It answers **"what do we use for X"** β€” mID for +identity, SpaceDB for storage, remade_ffmpeg_rs for media, FFAI for AI, rusty_alloc for +allocation, rusty_zstd for compression, thoth for UI chrome, Deputy for the supply chain β€” and +then the parts that decide whether your app can actually run on a mesh: API-first architecture, +the per-entry CRDT law, the five deploy seams, the performance and measurement discipline, and +the build/validate workflow. + +``` +building-the-new-internet/ +β”œβ”€β”€ SKILL.md the stack table, the day-one scaffold, the pre-flight checklist +β”œβ”€β”€ stack.md every dependency decision, with versions, features and traps +β”œβ”€β”€ architecture.md general primitives, API-first, per-entry CRDT, the trust boundary +β”œβ”€β”€ deploy.md the road to the distributed cloud, and what is actually live today +β”œβ”€β”€ performance.md the five moves, the measurement bar, when unsafe is earned +β”œβ”€β”€ workflow.md compile gates, isolation harnesses, the curiosity discipline +└── ui.md Dioxus footguns and the glyph/token/a11y crates +``` + +### Install + +**Claude Code** β€” copy the directory into your skills folder: + +```sh +git clone https://github.com/Remade-With-Rust/.github rwr-meta +cp -r rwr-meta/skills/building-the-new-internet ~/.claude/skills/ +``` + +Per-project instead of global: copy it to `.claude/skills/` in the repo. + +**Any other agent** β€” the files are plain Markdown with YAML frontmatter. Point your tool at +`SKILL.md`; it links the six reference files by name. + +### Reading it as a human + +You don't need an agent. Start at `SKILL.md` β€” the stack table and the one-line test are the +whole contract in two screens β€” then read whichever reference file your task touches. + +### Keeping it honest + +Every status and version in these files is what was true when written, including the +unflattering ones: which services are Preview rather than GA, which crates are not on crates.io +yet, and where a C dependency still enters the tree. If you find one that has drifted, a PR +correcting it is the most useful contribution you can make here. + +## License + +MIT. diff --git a/skills/building-the-new-internet/SKILL.md b/skills/building-the-new-internet/SKILL.md new file mode 100644 index 0000000..3885a85 --- /dev/null +++ b/skills/building-the-new-internet/SKILL.md @@ -0,0 +1,186 @@ +--- +name: building-the-new-internet +description: >- + The single house playbook for starting and shipping a Rust app, service, or website that is + ready to deploy onto the MATA distributed cloud (disco) the day it goes live. Answers "what + do we use for X" with the fixed stack β€” mID for identity/access, SpaceDB for storage, + remade_ffmpeg_rs for media, FFAI for AI, rusty_alloc for allocation, rusty_zstd for + compression, thoth/rusty_tokens/rusty_symbols/rusty_a11y for UI chrome, Deputy for the + supply chain β€” plus the architecture (API-first, general primitives, per-entry CRDT, trust + boundary), the deploy-readiness seams, the performance doctrine, and the build/validate + workflow. Read BEFORE `cargo new`, before adding any dependency, before choosing a + crypto/storage/identity/AI/allocator stack, and before a PR that adds one. Consolidates the + former rusty-coding-requirements, rusty-dev, rusty-blazing-fast, rusty-memory-model, + rusty-unsafe-optimizations and rusty-async-internals skills into one. +--- + +# Building the New Internet + +You are starting, or growing, a Rust application, service, or website. This skill is the whole +contract: the stack you pick from, the shape you build in, and what "ready to deploy" means. + +The target is not a VPS. It is the **MATA distributed cloud** β€” a mesh of machines nobody owns +centrally, where your data lives near your users, every access is a signed capability, and the +node running your code is hardware you do not control. Build for that from day one and the +deploy is a command. Retrofit it later and it is a rewrite. + +--- + +## 0. The one-line test + +> **Could this ship as-is to a user who assumes their data is theirs alone, onto a machine you +> do not own, with no C toolchain anywhere in the build?** + +Every rule below is a corollary. When a decision is unclear, ask that sentence. + +Three failures it catches, in the order they usually happen: + +| You wrote | It fails because | Fix | +|---|---|---| +| A Postgres connection string | there is no data centre to connect to | SpaceDB replica (`stack.md` Β§2) | +| `openssl-sys` / `protoc` / `libgmp` in the tree | a mesh node has no C toolchain, and `wasm32` has none at all | the replacement table (`stack.md` Β§8) | +| A session cookie backed by a users table | the user's identity is a keypair they hold, not a row you own | mID (`stack.md` Β§1) | + +--- + +## 1. The stack β€” what we use for X + +Reach for the house crate **first**. Each exists to delete a C dependency, a data centre, or a +password. Full detail, versions and traps: **`stack.md`**. + +| Need | Use | Status today | +|---|---|---| +| **Identity, sign-in, access control** | **mID** β€” `mid-verify` (RP side), `mid-issuer` (wallet), `mata-cap` (authz) | published, MIT/Apache | +| **Storage β€” all of it** | **SpaceDB** β€” `spacedb-sdk` | `0.5.2` on crates.io | +| **Media convert / transcode / probe** | **remade_ffmpeg_rs** β€” the `rff` facade crate, **by git** | pre-1.0, git only β€” see the trap below | +| **AI β€” OCR, ASR/TTS, detection, VLM** | **FFAI** β€” `ffai-core` + the engine crate you need | published | +| **AI β€” LLM serving / tool calling** | **mistral.rs** on **candle**, behind one `ChatEngine` seam | β€” | +| **Memory allocation** | **rusty_alloc**, via a one-crate seam (or `rusty_alloc_default`) | `1.0.1` on crates.io | +| **Compression** | **rusty_zstd** | deploying β€” **not on crates.io yet**, git/path only | +| **UI chrome β€” glyphs, tokens, a11y** | **thoth** (or split: `rusty_symbols` / `rusty_tokens` / `rusty_a11y`) | published | +| **UI framework** | **Dioxus** β€” web, PWA, desktop, mobile from one codebase | β€” | +| **Supply chain** | **Deputy** β€” `cargo install deputy-cli` | published | +| **Crypto** | **RustCrypto** β€” Argon2id + AES-256-GCM | β€” | +| **TLS, system tooling** | **memorysafety.org** β€” rustls, sudo-rs, curl-rust | β€” | +| **Text shaping / layout / raster** | **OxiText** (never FreeType/HarfBuzz) | β€” | +| **Internal wire format** | **oxicode** (never on public APIs β€” those get JSON) | β€” | +| **Big / exact math** | **OxiNum** (never on secrets β€” not constant-time) | β€” | +| **Protobuf codegen** | **OxiProto** (no system `protoc`) | β€” | + +> **Two traps that cost real time.** +> **`rff` on crates.io is not ours** β€” it is an unrelated fuzzy text selector. Depend on +> remade_ffmpeg_rs **by git URL**, never by bare crate name. +> **`rusty_zstd` is not published yet.** Use a git or path dependency and pin a commit; when it +> lands on crates.io, switch the pin, don't switch the API. + +**Before anything not on this table**, walk the ladder: memorysafety.org β†’ RustCrypto β†’ +[Remade-With-Rust](https://github.com/orgs/Remade-With-Rust/repositories) β†’ cool-japan `Oxi*` β†’ +crates.io. The org ships new crates continuously β€” **check it before you write the thing +yourself** (`stack.md` Β§8 has the current inventory and how to re-check it). + +Adding a `*-sys` crate is a decision you state out loud in the PR, with the reason it could not +be avoided. + +--- + +## 2. Day one β€” the scaffold that is already deploy-ready + +``` +myapp/ +β”œβ”€β”€ Cargo.toml # workspace; pins live here, once +β”œβ”€β”€ crates/ +β”‚ β”œβ”€β”€ myapp-core/ # LIBRARY: the ops. No allocator, no UI, no I/O policy. +β”‚ β”œβ”€β”€ myapp-api/ # LIBRARY: typed ops over a Transport seam. API first. +β”‚ β”œβ”€β”€ myapp-cli/ # DELIVERABLE: allocator declared here +β”‚ β”œβ”€β”€ myapp-server/ # DELIVERABLE: allocator declared here +β”‚ β”œβ”€β”€ myapp-ui/ # DELIVERABLE (Dioxus): allocator declared here +β”‚ └── myapp-alloc/ # the allocator seam β€” one crate, one pin +``` + +Four rules the layout encodes, and why each is load-bearing: + +1. **`#[global_allocator]` lives in the deliverable, never a library.** A program may define + exactly one. A library that declares it forces the choice on every consumer and makes two + such libraries impossible to link together. In a diff, `#[global_allocator]` in a library + crate is a stop-the-review finding. (`stack.md` Β§5) +2. **The core knows bytes, CIDs, DIDs and capabilities β€” never a product type.** Litmus: *could + a developer who has never heard of your product use this op?* (`architecture.md` Β§1) +3. **Every capability is an op before it is a button.** The UI is consumer #1, never the only + consumer. If a behaviour only exists inside an event handler, it is not built yet. + (`architecture.md` Β§2) +4. **Every persisted type goes in per-entry, encrypted, under its own compound key.** No + single-blob formats, ever. (`architecture.md` Β§3 β€” this one is a law, not a preference) + +Copy-paste scaffold, manifests and the allocator seam: **`stack.md` Β§9**. + +--- + +## 3. The five seams that make deploy a command + +The distributed cloud does not ask you to port your app. It asks you to have built against five +seams it fills. Build against them locally on day one and `disco` takes over on launch day. + +| Seam | You build against | disco fills it with | +|---|---|---| +| **Identity** | `mid-verify` β€” verify a token, locally, no network | the mID roster + KMS resolver | +| **Storage** | `spacedb-sdk` β€” a local replica on your own box | `managed-db`, mesh-replicated | +| **Transport** | SpaceDB's `Transport` trait | iroh + relay across the mesh | +| **Placement** | SpaceDB's `ShardStore` | erasure shards, anti-affinity, self-repair | +| **Settlement** | SpaceDB's `Settlement` trait | Iron Bank, `$MATA` per-use metering | + +Then the go-live is: + +```sh +disco sites deploy ./dist --domain app.example.com +# seals -> chunks -> places -> replicates -> registers -> serves off the mesh +``` + +**Be honest about what is live.** `identity` is GA. `hosting`, `edge-functions`, +`object-storage`, `cdn`, `bandwidth`, `dns` and `managed-db` are **Preview** β€” the primitives +are built and the provideβ†’proveβ†’PAID loop closes, but the self-serve console path is still +being wired. Build against the seams now; the switch flips under you. +Full deploy path, op by op, plus what to run today: **`deploy.md`**. + +--- + +## 4. Reference files β€” read the one your task touches + +- **`stack.md`** β€” every dependency decision with versions, features and traps: mID, SpaceDB, + media, AI (the candle/mistral.rs two-layer rule), the allocator seam, compression, the UI + crates, Deputy, the Oxi* replacements, and how to check the org for what shipped since. +- **`architecture.md`** β€” general primitives / thin wiring, the opβ†’SDKβ†’gateway seam, API-first, + the per-entry CRDT law, the trust boundary, content-addressing, and *cores are sound, bugs + live at the seams*. +- **`deploy.md`** β€” the road to the distributed cloud: mID sign-in end to end, SpaceDB from + local replica to mesh, `disco sites deploy`, the resource SDK's six services, what is + Available vs Preview, and the deploy-readiness checklist. +- **`performance.md`** β€” making it fast without breaking it: the five high-leverage moves, the + measurement bar a number must clear before you act on it, when `unsafe` is earned and how to + fence it, and the memory model that explains why the rules are shaped this way. +- **`workflow.md`** β€” the build/validate/debug loop: compile-gate every target, validate the + mechanism in isolation, why a green test can test the wrong scenario, instrument before you + guess, and the curiosity discipline for when a result defies expectation. +- **`ui.md`** β€” Dioxus 0.7 footguns across web/desktop/wasm, the thoth glyph/token/a11y trio, + and the staged-progress pattern that turns "it silently does nothing" into a visible failure. + +--- + +## 5. Pre-flight β€” run this before you add a dependency or open a PR + +- [ ] Does it build for **every** target you ship, including `wasm32-unknown-unknown`, with no + system C library, no `protoc`, no CMake? +- [ ] Did I check the org (Β§1 ladder) before reaching for crates.io β€” including for something + that shipped since I last looked? +- [ ] Does every new persisted type go in **per-entry**, encrypted, under its own compound key? +- [ ] Is the capability reachable from an **op** β€” callable by a CLI, a test and an agent β€” not + only from the UI? +- [ ] Does any **library** crate in this diff declare `#[global_allocator]` or depend on + `rusty_alloc-api` directly instead of through the seam? **Stop the review.** +- [ ] ML change: right layer β€” mistral.rs for LLM serving, candle for everything else β€” and do + `candle-core` / `-nn` / `-transformers` still move in lockstep? +- [ ] Secrets: Argon2id-derived, AES-256-GCM at rest, never touched by non-constant-time math + (OxiNum), never sent to a remote model? +- [ ] Tests written **and run**, against the real deployment topology β€” not a green test of the + wrong scenario? +- [ ] `cargo check` on every target the change touches, before the push? +- [ ] No `unwrap()` on a path a user can reach; no "temporary" shim; no TODO left as the fix? diff --git a/skills/building-the-new-internet/architecture.md b/skills/building-the-new-internet/architecture.md new file mode 100644 index 0000000..888d3d1 --- /dev/null +++ b/skills/building-the-new-internet/architecture.md @@ -0,0 +1,231 @@ +# architecture.md β€” the shape that survives contact with a mesh + +Five patterns and two laws. They are not style preferences: each one is what lets the same code +run on a laptop, on a phone, in a browser, and on a node in the distributed cloud that you do +not own β€” without a rewrite at any step. + +--- + +## 1. General primitive, thin consumer + +The governing rule for anything that could be infrastructure. + +**Layer A β€” the primitive.** Speaks only in bytes, content hashes, DIDs and capabilities. No +product types. No UI framework. No template knowledge. + +**Layer B β€” the consumer.** Produces a generic artifact and calls Layer A. Holds *all* the +product-specific knowledge. + +> **Litmus:** *could a developer who has never heard of your product use this op?* +> If the answer is no, product logic has leaked into the primitive. + +Worked example: a `sites::Deploy` op takes a content-addressed bundle plus config. A "Publish" +button, an admin console and a CLI are three Layer-B consumers of the **same** op. The +orchestrator behind it depends on placement and transport crates β€” never on the product. + +**Enforcement smell:** if writing the primitive tempts you to `use myapp_business::…`, or to +bake in a template assumption, stop. That belongs in Layer B. + +**The payoff is not tidiness.** When the last mile lights up β€” a real session reaching a real +gateway β€” it lights up *every* consumer at once, because they share one op. Three surfaces, one +integration. + +--- + +## 2. API first, UI second + +Build the op, then the surface. Every capability must be callable by **an agent, a CLI, and a +test**. The UI is consumer #1, never the only consumer. + +> If a behaviour only exists inside a component's event handler, it isn't built yet. + +This is also what makes your app usable by AI agents on day one, which on this network is not a +nice-to-have: capabilities are granted to `did:agent:*` identities the same way they are granted +to people. + +### The three-seam control plane + +To add a service you write **one module** and register it in a handful of places. The core never +changes. + +1. **Contract** β€” a struct per op declaring `type Output`, `const SERVICE`, `const PATH`, + `fn required_capability()`. +2. **SDK** β€” `Client::call::()` checks the capability **client-side** (fail fast, no + round-trip), serializes, sends over a **`Transport` seam**, deserializes `O::Output`. Ship + `HttpTransport` for real and `MockTransport` for tests. Add an ergonomic sub-client per + service (`.sites()`, `.storage()`). +3. **Gateway** β€” a `ServiceBackend` re-checks the capability **server-side** and runs the op. + +> **The 404 trap.** Registering the backend is usually not enough β€” there is typically a second, +> hand-maintained list (the router's `parse_service()` arm). Forget it and a fully implemented +> backend returns 404 while every test passes. When you add a service, grep for *every* place +> the service name appears. + +Every service ships an in-memory backend for tests and a real one for production, behind the +same trait. + +### Catalog honesty + +Keep one source-of-truth service registry where each entry carries a `status` +(Available / Preview / Planned / Internal) and a `backing` (the implementing crate). **Make +tests enforce it:** a "live" service must name a non-empty backing crate and ship a code sample; +a "planned" service must be unpriced. This is what stops a console card from claiming GA over a +stub. When you flip a service live, the test forces you to have real backing. + +--- + +## 3. The per-entry CRDT law + +**Every data type is per-entry storage.** Each password, contact, document, preference, paired +device is *individually* encrypted and synced under its own compound key: + +``` +"passwords:{uuid}" -> SyncEntry { blob, updated_at, device_id } +``` + +> **No single-blob formats. Ever.** + +**Why it is a law and not a preference.** Two devices editing *different* entries offline must +**both** survive the merge. A single blob makes every offline edit a whole-vault conflict and +silently destroys one side's work. Only edits to the exact same key may conflict. + +**Corollary for review:** if a diff introduces a struct that serializes a whole collection into +one value before it enters the CRDT, that is the bug. Stop there. + +**Corollary for the model:** model your app's artifact as a *file tree in the database* β€” +`WorkspaceFile { path, kind, content: Vec }` β€” a git-alternative repo. Store **bytes**, so +uploaded images live alongside text, versioned and synced, with no data URIs. + +Adding a synced entity means updating the sync lists in lockstep. Keep the test that asserts the +counts match. + +--- + +## 4. Local-first encrypted storage + +- **Encrypt per record.** A random DEK per row, wrapped under the vault key (AEAD). Store + `{id, user_id, wrapped_dek, ciphertext}`. The list key stays plaintext. +- **Bind the AAD to *position*, not just existence.** + `AAD = namespace β€– record_id β€– field β€– format_version` β€” not `record_id` alone, or ciphertexts + can be relocated and swapped. The format-version byte stops a future format being confused + with an old one under the same key. +- **Per-record DEKs exist so the vault key can rotate.** Rotation becomes "re-wrap N DEKs", not + "re-encrypt and CRDT-merge the entire corpus". **A crypto root with no rotation path is a + permanent liability** β€” re-wrapping under a new passphrase changes the *wrapping*, not the + *bytes*, and does nothing once the bytes leak. +- **96-bit random GCM nonces** are safe under the ~2Β³Β²-per-key birthday bound. Bound the volume + per key before any high-volume single-key use β€” per-record DEKs do this for you. +- **Zeroize everything secret** on **every** path, including error returns. A decrypt-failure + that returns the DEK un-zeroized is a real gap. Redact `Debug`. +- **User-scope every per-account key, and clear all derived state on account switch.** A global + flag or blob keyed without `user_id` leaks across accounts on a shared device. +- **State your at-rest medium's *actual* guarantee, not the docstring's.** Browser session + storage is in-memory and trusted-contexts-only, not "encrypted at rest". That is often fine β€” + claiming otherwise misleads the next reviewer. + +--- + +## 5. Content-addressing = self-verifying delivery + +Objects are identified by their content hash (BLAKE3). **Verify the hash on every hop** β€” cache +fill, origin pull, cross-node retrieve. + +Four properties fall out for free, and they are exactly the properties a mesh needs: + +- Wrong content can never be cached or served. +- A lying node is rejected automatically β€” no reputation system required. +- Retrieval can fail over across replicas, because any replica returning bytes is checked. +- IDs are stable across processes and runs, so a writer and a reader compute the same id + independently. That makes the CID a perfect shared-registry key. + +--- + +## 6. The trust boundary and thin clients + +**The process holding the vault key, the database, or the provider API keys is the trust +boundary.** Everything else is a thin client that relays *intent* β€” text β€” and never keys. + +A browser bubble, a CLI, a mobile shell: each POSTs `{token, message, files[]}` to a +**localhost-only** server inside the boundary; the boundary runs the privileged turn and returns +the result. + +Gate it three ways: **per-session token** baked into the served client, **localhost bind**, and +explicit CORS. Note that `Access-Control-Allow-Origin: *` means *any* page can POST β€” the token +is the real defence, so it must reach the legitimate client and nothing else. + +--- + +## 7. Identity is the account + +There is no separate login. **The DID is the account.** + +A signed grant or action is trusted only because an **enrolled device key** signed it. +Verification is: resolve `DID β†’ DID document β†’ find_entry(device_id) β†’ pubkey`, then check the +signature. Roster rotation (dropping a device) is **free revocation** β€” the resolver returns +`None` and every grant that device signed stops verifying. + +**The client side of the same handshake**, so a request proves who sent it: per request, +`POST /nonce {did, purpose}` β†’ envelope β†’ `sign_assertion(&signer, envelope)` β†’ attach +`Authorization: `. The nonce is **single-use**, so this is a per-request +handshake, not a cached bearer token. The gateway verifies it into a `Caller` before dispatch. + +Design the transport's auth as an enum (`None` / static header / `Signer{did, purpose, signer}`) +so tests and production share one path. + +> **Prove the loop against a real bound server** β€” a `TcpListener` plus a real serve β€” not an +> in-process one-shot. Only the socket path exercises the signing and nonce round-trip. + +--- + +## 8. Cores are sound; bugs live at the seams + +The single highest-value review heuristic here. Crypto and accounting *primitives* are almost +always right. Every serious finding sits at a seam: + +**(a) How an identity is *bound*.** A `did:mata:` is self-certifying β€” the DID +*is* the key β€” but only if the verifier **enforces** it. Recovering the genesis pubkey *from the +DID string* is correct. Resolving `DID β†’ roster` from a mutable central table and trusting +whatever roster you find is identity squatting: whoever can write the table owns the identity. +> **Litmus:** *if I have any valid account, can I claim someone else's un-provisioned DID?* + +**(b) Where keys come to *rest*** β€” and whether they can rotate (Β§4). + +**(c) *Uniformity* of a crypto policy across surfaces.** The most common real bug is the same +check applied on one surface and silently omitted on another. A codebase with four ECDSA +verifiers where only one enforces canonical **low-s** admits malleable `(r, nβˆ’s)` signatures β€” +and now disagrees with itself about what a valid signature *is*. `p256`'s `verify()` accepts +both forms; you must reject high-s yourself. +> **Litmus:** grep every `verify_prehash` / `.verify(` and confirm each has a `normalize_s` +> reject beside it. One missing is a finding. +> **Fix once, in the shared primitive** that every surface imports β€” a single edit clears N +> findings. If you must duplicate a policy, a conformance test is the *floor*, not the guarantee. + +**(d) *Operational* edges** β€” silent decrypt failures, cross-account state, unbounded nonce +stores. An in-memory unbounded nonce store on an unauthenticated endpoint is an OOM *and* a +replay-window-reopens-on-restart bug. + +### The signed-assertion wall β€” verify in this order, fail closed + +1. shape β†’ 2. **audience** match β†’ 3. **purpose** match + allow-list β†’ 4. **issuer** match β†’ +5. resolve DID doc β†’ 6. device in roster β†’ 7. ECDSA verify **with low-s reject** and +point-on-curve β†’ 8. **atomically consume the nonce** (`DELETE … WHERE nonce=$1 RETURNING …`, +never GET-then-DELETE) *after* the signature check β†’ 9. **tamper-check**: the stored envelope +must byte-equal the embedded one. + +Audience, purpose and issuer live **inside the signed canonical bytes** *and* are re-checked, so +a nonce minted for service A cannot be replayed at service B even with a shared nonce store. +Fold all failures into one opaque client-side error; log the granular reason server-side. + +--- + +## 9. The durable seam β€” and the test that lies about it + +An `async trait` with **two impls**: an `InMemory` one for tests and single-process, and a +durable or networked one for production. The mantra is "same code, only the connection differs". + +> **The trap.** An in-process `InMemory` shared via one `Arc` will pass a "two independent +> readers share state" test **without exercising the real cross-process topology at all**. That +> exact test passed while two real gateways could not share an embedded store. + +Always add a test that opens two *independent* connections to the real backing β€” or is honestly +env-gated against a live server β€” before claiming the gap is closed. See `workflow.md` Β§3. diff --git a/skills/building-the-new-internet/deploy.md b/skills/building-the-new-internet/deploy.md new file mode 100644 index 0000000..b982a49 --- /dev/null +++ b/skills/building-the-new-internet/deploy.md @@ -0,0 +1,223 @@ +# deploy.md β€” the road to the MATA distributed cloud + +**disco** (`disco.mata.network`) is the console: sign in with your mID, browse the service +catalog, provision and pay per use. It is what an AWS console would be if the machines belonged +to the people running them. + +This file is what you build against *now* so that launch day is a command, not a migration. + +--- + +## 1. What is actually live β€” read this before you promise anything + +Twelve services. Statuses are honest, not aspirational. + +| Service | Status | Like | What it is | +|---|---|---|---| +| **identity** | **Available** | IAM / Cognito | Sign in with mID; the DID *is* the account | +| **hosting** | Preview | Amplify / Vercel | one `sites deploy` seals β†’ places β†’ replicates β†’ serves off the mesh | +| **edge-functions** | Preview | Lambda / Workers | WASM functions, scale-to-zero, metered | +| **object-storage** | Preview | S3 | durable, content-addressed, self-verifying blobs | +| **cdn** | Preview | CloudFront | edge cache + content-addressed serving | +| **bandwidth** | Preview | data transfer | metered egress, ack-gated, bilateral proof | +| **dns** | Preview | Route 53 | owner-gated zones on your domain, GeoDNS | +| **managed-db** | Preview | RDS / Supabase | hosted SpaceDB (CRDT), metered | +| **containers** | Planned | Fargate | microVM compute | +| **gpu-compute** | Planned | EC2 GPU | GPU passthrough | +| **source-control** | Planned | CodeCommit | git over storage + mID auth | +| **settlement** | Internal | billing | Iron Bank β€” mints and settles `$MATA` for all of the above | + +**"Preview" means two different things, and both are true.** The *primitive* layer is built: +each Preview service maps to real crates and the mesh loop closes **provide β†’ prove β†’ PAID** in +isolation. The *developer self-serve* layer is not fully wired: the console still renders some +mock data and several gateway backends are in-memory stubs. + +So today disco is a real console shell over an honest catalog of real primitives. Build against +the seams; the switch flips under you. + +> **Never ship a claim ahead of the catalog.** If you are writing docs or a landing page, the +> service registry's `status` field is the source of truth, and there are tests enforcing that a +> "live" service names a real backing crate. Copy the status, don't upgrade it. + +--- + +## 2. The five seams β€” build against these and the deploy is free + +The distributed cloud does not ask you to port your app. It asks that you built against seams it +can fill. + +| Seam | You build against | disco fills it with | +|---|---|---| +| **Identity** | `mid-verify` β€” verify locally, no network | the mID roster + KMS resolver | +| **Storage** | `spacedb-sdk` β€” a local replica | `managed-db`, mesh-replicated | +| **Transport** | SpaceDB's `Transport` trait | iroh + relay across the mesh | +| **Placement** | SpaceDB's `ShardStore` | erasure shards, anti-affinity, self-repair | +| **Settlement** | SpaceDB's `Settlement` trait | Iron Bank, `$MATA` per-use metering | + +The dependency arrow is always **operator β†’ your app**, never the reverse. You never depend on a +disco crate. That is the property that keeps your app deployable anywhere, including on your own +hardware, forever. + +--- + +## 3. Identity β€” the piece that is GA today + +Wire this first. It works completely, offline, right now. + +``` +browser/wallet your frontend your backend + | | | + | 1. request sign-in | 0. issue nonce ------>| + |<-----------------------| | + | 2. user consents, | | + | wallet signs JWT | | + |----------------------->| 3. POST {jwt} ------->| + | | | 4. verify_mid_response() + | | | -> did, claims + | | | 5. check_rollback() +``` + +Steps 4 and 5 are **pure functions with no I/O**. There is no JWKS fetch, no DID-resolver call, +no `/token` back-channel, no MAU meter. Your backend can be offline and sign-in still works. + +`verified.did` is your users-table primary key. See `stack.md` Β§1 for the full code and the +three traps (nonce reuse, wildcard audience, skipping rollback). + +--- + +## 4. Storage β€” same API on your laptop and on the mesh + +The migration you do not have to do: + +```rust +// Today β€” a local replica on your own box. No server, no network. +let mut db = Database::open(Identity::generate("did:mata:my-node")?); + +// Launch day β€” the same Database, the same schema, the same ops. +// disco fills the Transport / ShardStore / Settlement seams underneath. +``` + +Nothing above the seam changes. That is the entire point of building on SpaceDB rather than +reaching for Postgres and promising yourself you will migrate later. + +**Two things to get right now, because they are expensive later:** + +1. **Tier every field deliberately** (`stack.md` Β§2). `Strong` returns `Unavailable` under + partition β€” decide *now* which fields are worth that, because changing a tier changes your + app's behaviour under a network split, not just its performance. +2. **Per-entry keys, never a blob** (`architecture.md` Β§3). This one cannot be fixed later + without a data migration and a merge story. + +--- + +## 5. Shipping a site + +### The CLI path + +```sh +# 1. pair this machine with your mID (device-auth, roster-verified grant) +disco party +disco whoami + +# 2. point at a gateway +export DISCO_GATEWAY=https:// + +# 3. deploy a folder +disco sites deploy ./dist --site my-app --domain app.example.com +``` + +Output tells you exactly what it assembled β€” site id, file count, byte count, the op path +(`/sites/deploy`), the URL and the required capability. Without `DISCO_GATEWAY` set it assembles +and stops; **sending to the resource gateway is the go-live step**, at which point the gateway +seals β†’ chunks β†’ places β†’ replicates β†’ registers, and the CDN edge serves it off the mesh. + +`--site` defaults to the parent directory of a build dir, so `./my-app/dist` becomes `my-app`. +Set it explicitly for anything you will deploy twice. + +### The programmatic path β€” same op, from your own code + +```rust +use mata_resource_sdk::{ResourceClient, HttpTransport, PURPOSE_API}; + +let transport = HttpTransport::new(gateway_url) + .with_signer(my_did, PURPOSE_API.to_string(), Box::new(device_signer)); +let client = ResourceClient::new(transport, caller); + +let deployment = client.sites().deploy("my-app", bundle).await?; +let sites = client.sites().list().await?; +let dash = client.sites().dashboard().await?; // health, footprint, delivery, spend +``` + +`with_signer` runs the **live mID handshake per request**: fetch a single-use nonce from the +gateway, sign it into a `Resource-Assertion`, attach it. Not a cached bearer token β€” a fresh +proof every call. Use `with_auth` only for a header signed elsewhere, and `Auth::None` only for +health checks against an open local gateway. + +### The six service sub-clients + +```rust +client.storage().allocate(gb).await?; client.storage().stat(id).await?; +client.functions().deploy(op).await?; client.functions().list().await?; +client.dns().claim("example.com").await?; client.dns().set_record(op).await?; +client.cdn().set_policy(op).await?; client.cdn().deploy_regions(op).await?; +client.database().provision("main", 3).await?; client.database().list().await?; +client.sites().deploy("my-app", bundle).await?; client.sites().dashboard().await?; +``` + +`ResourceClient::call` checks the capability **client-side first** and fails with +`SdkError::Unauthorized` without a round-trip. The gateway re-checks server-side. Both checks +exist on purpose β€” the client-side one is for speed and error quality, never for security. + +**Test against `MockTransport`**, which records calls and replays canned responses, then prove +the real path against a genuinely bound server (`architecture.md` Β§7). + +--- + +## 6. Deploy-readiness checklist + +Tick these while you build, not the week you launch. + +**Identity** +- [ ] Sign-in verifies through `mid-verify` with a server-issued single-use nonce +- [ ] `check_rollback` is called against the last roster version you stored per DID +- [ ] `expected_audience` is your exact origin β€” no wildcard +- [ ] Authorization is capability-based (`mata-cap`), so an agent DID can be granted the same + scoped, expiring, revocable access as a person + +**Storage** +- [ ] All persistence goes through `spacedb-sdk` β€” no connection string anywhere +- [ ] Every field has a deliberate `CrdtType` **and** `Tier` +- [ ] Per-entry compound keys; no struct serializes a whole collection into one CRDT value +- [ ] Per-record DEKs with position-bound AAD, and a written rotation path + +**Build** +- [ ] `cargo check` passes on every target you ship, **including `wasm32-unknown-unknown`** +- [ ] No `*-sys` crate, no `protoc`, no CMake, no system C library in the tree +- [ ] `#[global_allocator]` appears exactly once per deliverable and never in a library +- [ ] Dependencies vaulted and scanned through Deputy + +**Shape** +- [ ] Every capability is an op callable by CLI, test and agent β€” not only by the UI +- [ ] Primitives speak bytes/CIDs/DIDs/capabilities; product types live in the thin consumer +- [ ] Content-addressed artifacts, hash verified on every hop +- [ ] Static assets compressed with `rusty_zstd` before they are sealed + +**Honesty** +- [ ] Every status claim matches the catalog +- [ ] Every performance number in your docs clears the bar in `performance.md` Β§2 +- [ ] No `unwrap()` on a user-reachable path; no "temporary" shim left in + +--- + +## 7. Operating on a mesh β€” what bites after launch + +- **Throttle *mis*-tuning starves legitimate peers** more often than it stops abusers. Tune + against observed traffic and log the rejections; a throttle you cannot see firing is a + throttle you cannot debug. +- **Transport auth is per-stream.** A QUIC-authenticated endpoint id is the identity; gate each + stream, and remember that identity rotation can evade a throttle keyed on it. +- **Stale capability views** are the quiet failure: a node caches a grant that has since been + revoked. Bound the cache lifetime and re-resolve on any authorization failure. +- **Instrument the boundary before you guess.** In a long remote chain, each fix reveals the + next blocker β€” the fastest path is a staged progress signal on the client and a greppable + marker in the daemon log at each hop. See `workflow.md` Β§4. diff --git a/skills/building-the-new-internet/performance.md b/skills/building-the-new-internet/performance.md new file mode 100644 index 0000000..2154aa9 --- /dev/null +++ b/skills/building-the-new-internet/performance.md @@ -0,0 +1,299 @@ +# performance.md β€” fast, without breaking it or lying about it + +Rust is fast. Idiomatic-*looking* Rust routinely leaves large multiples on the table through +unnecessary allocation, repeated work, single-threading, and O(n) container ops. + +Order of operations matters more than any individual technique: + +``` +profile -> the five moves (safe) -> measure -> only then consider unsafe +``` + +Every step below is skippable except the first and the third. + +--- + +## 1. The five moves β€” ~90 % of real codebases + +The worked example: `top_users(log: &str)` over a 20-million-line CSV. Baseline β‰ˆ 1.5 s. + +### Move 1 β€” stop allocating memory you don't need + +Every `.to_string()`, `.clone()`, `.to_owned()`, `format!` or intermediate `.collect()` in a hot +loop is a heap allocation. 20M lines Γ— 2 = 40M allocations doing nothing. + +```rust +// BEFORE β€” owned keys: two allocations per line +let mut counts: HashMap = HashMap::new(); +let user = line.split(',').nth(1).unwrap().to_string(); // alloc + +// AFTER β€” the key borrows from `log`; allocate only the 10 strings you return +let mut counts: HashMap<&str, u64> = HashMap::new(); +let user = line.split(',').nth(1).unwrap(); // a view, no alloc +``` + +β‰ˆ 1.5 s β†’ β‰ˆ 1 s from deleting allocations alone. + +- Prefer `&str` / `&[T]` / `&T` in signatures and loop bodies; push owning to the edges. +- Reuse buffers across iterations (`String::clear` and refill) instead of a fresh allocation. +- Pre-size with `with_capacity` when the count is known. +- `Cow` when a value is *usually* borrowed and occasionally owned. +- Drop throwaway intermediates: `.map(|x| x*x).collect::>().iter().sum()` β†’ `.map(..).sum()`. + +### Move 2 β€” stop repeating work + +A `HashMap` hashes the key on **every** access. `contains_key` β†’ `insert` β†’ `get_mut` hashes the +same key three times. + +```rust +*counts.entry(user).or_insert(0) += 1; // one hash, one slot lookup +``` + +Another ~12 %. Generalize: `entry` / `or_insert_with` / `and_modify` for any read-then-write; +hoist loop-invariant computation (regex compilation, config lookups) out of the loop; watch for +two passes where one would do. + +### Move 3 β€” use every core (Rayon) + +If iterations are independent, a `for` loop is pinning you to one core. + +```rust +use rayon::prelude::*; + +let counts: HashMap<&str, u64> = log + .par_lines() + .fold(HashMap::new, |mut map, line| { // per-thread accumulator + *map.entry(line.split(',').nth(1).unwrap()).or_insert(0) += 1; + map + }) + .reduce(HashMap::new, |mut a, b| { // merge the private maps + for (k, v) in b { *a.entry(k).or_insert(0) += v; } + a + }); +``` + +Down to β‰ˆ 90 ms β€” **>7Γ—** vs baseline. **Private-then-merge, never shared-and-locked**; +contention on a `Mutex` can erase the entire win. + +> **The #1 Rayon compile error:** its `fold`/`reduce` take an identity *closure* (`HashMap::new`), +> not an identity *value* like `std`'s `Iterator::fold`. + +Parallelism has fixed overhead. It loses on small N β€” consider a size threshold with a +sequential fallback, and measure both sides of it. + +### Move 4 β€” pick the right complexity + +`Vec::remove(i)` shifts every later element: O(n) per call, O(nΒ²) across a sweep. Removing one +session near the front of a 1M-element vec shifts ~999,999 elements. + +- `retain(|s| !s.is_idle())` β€” one O(n) compacting pass, order preserved. **Usually what you + want** for bulk filtering. +- `swap_remove(i)` β€” O(1), order destroyed. The sharp tool for removing *one* element by index in + a hot path where order is meaningless (a pool, a free list, an entity soup). Don't advance `i` + afterwards β€” re-check the element swapped in. + +Measured >7,000Γ— on that sweep, from a single method name. **The largest wins are almost always +algorithmic, not constant-factor.** `Vec::contains` is O(n) β€” use a `HashSet`. A linear scan +inside a loop is an accidental O(nΒ²). + +### Move 5 β€” don't hand-roll what a kernel already does, then check it's called + +| you wrote | use instead | why | +|---|---|---| +| `for x in s.iter_mut() { *x = v }` | `s.fill(v)` | broadcast + wide stores β€” a `u16` fill goes 16 elements per 5 instructions | +| `for i in 0..n { d[i] = s[i] }` | `d.copy_from_slice(s)` | lowers to `memcpy`, dispatched to the widest available width | +| `while a[i] == b[i] { i += 1 }` | a common-prefix helper | `pcmpeqb` + movemask + `tzcnt`, ~16Γ— over byte-at-a-time | +| a hand-rolled checksum byte loop | the crate's kernel | CRC32 has a carry-less-multiply form the compiler cannot derive | +| `v.iter().position(|&b| b == x)` on bytes | `memchr` | same | + +**Then verify the fast path is actually reached.** This is the defect that costs most, and every +correctness gate passes it: in rusty_zstd a tested, benchmarked AVX2 checksum kernel carrying a +documented 1.14–1.26Γ— was reachable only from one unit test and one benchmark. The shipping +encoder, decoder and streaming API all took the scalar route β€” the decode side ran the kernel on +**0 %** of its bytes, for months. The two paths agree by design, so no test could see it. + +> **A fast path nothing calls is slow code with extra maintenance.** + +**Settle it with a counter, not by reading the call graph.** Two atomics, tally bytes down each +path, run the real workload, print the percentage. One run, no benchmarking rig, no noise floor. +Anything under 100 % is a bug. + +--- + +## 2. The measurement bar β€” what makes a number admissible + +**No claim without a number that clears this bar.** This is not pedantry; it is how you avoid +shipping a "1.6Γ— win" that was your profiler hashing its own output. + +- **`--release`, always.** Debug builds are 10–100Γ— slower and lie about *where* the time goes. +- **Profile before you touch anything** (`cargo flamegraph`, `samply`, `perf`). Optimizing cold + code is wasted effort β€” and the "obviously hottest" site is regularly not on the shipping path + at all. +- **Baseline β†’ change one thing β†’ re-measure.** Complexity you cannot justify with a number is a + regression in disguise. Revert what doesn't pay. +- **Both arms must do identical work.** The single commonest source of a false result. If one arm + writes 582 MB to disk and the other writes to a null sink, you measured the disk. If your + profiler hashes every output sample, you measured your profiler β€” that was 17 % of its own + runtime in a real case, and it inverted the verdict. +- **Pin the process, interleave the arms (ABBA), N β‰₯ 31, report median *and* minimum.** +- **Run a null arm** β€” measure A against A. Whatever separation that shows is your noise floor, + and any real effect must clear it. +- **Prefer a deterministic count to a duration.** A count, ratio, size or checksum needs no + pinning and no z-score, refutes bad levers before you build them, and is how you catch the + instrument lying. +- **Check the reference's defaults.** A comparison against a tool silently using two cores when + you asked for one is not a comparison. +- **Correctness gate first.** Byte-identical for integer work; tolerance plus SNR for float. A + speed win that changes output is not a win, it is a bug with good timing. + +> Applied to house crates: rusty_alloc's published evidence is **parity**, not superiority. +> rusty_zstd makes **no speed claim** against C zstd yet. Do not repeat a claim that is not in +> the ledger β€” including in your own README. + +--- + +## 3. When `unsafe` is earned β€” and the two checks that usually make it unnecessary + +An `unsafe` backlog is a list of **hypotheses about generated code**. Most of them are wrong. + +### Check one: count the bounds checks the compiler actually emitted + +```sh +cargo rustc --release -p --lib -- --emit asm +# attribute `panic_bounds_check` call sites to symbols by line range +``` + +Audited that way, a codec's five highest-priority "P1" entries came out: + +| entry | emitted checks | verdict | +|---|---:|---| +| "densest bounds-check-per-byte site in the crate" | 14 + 12 β€” genuinely the most | **0 executions** β€” not on the shipping path | +| a masked index, "the check is pure tax" | **0** | premise right, conclusion inverted β€” *because* the masks prove it, LLVM already removed the check | +| scalar quantize | **0** | and the SIMD twin owns the path anyway | +| block extraction | 23 | **the only real one β€” 13 % of encode** | + +Two failure modes, by name: **the code does not run on the path you ship**, and **the compiler +already did it**. + +And when a real one survives, **the fix is often still safe Rust**. That 23-check entry was +per-sample `.min()` clamps, dead on every block (the buffer is padded) but blocking the compiler +from proving the index. Hoisting the edge test to block level turned 256 checks into 2 β€” +**1.14Γ— encode, byte-identical, no `unsafe`.** + +> Reach for `unsafe` when the bound genuinely **cannot** be proven β€” not when it merely has not +> been. + +### Check two: does a safe kernel already exist, and does production reach it? + +See Move 5. Replacing a hand-rolled `unsafe` store loop with safe `fill` has measured 16 elements +per 5 instructions *and* deleted the unsafe block. + +### The discipline, when you do write it + +`unsafe` does not mean "bad code". It means **the compiler cannot prove this is sound, so you are +vouching** β€” and you have to be right, because a wrong `unsafe` is real UB, not a lint. + +The pattern to imitate: **one small, carefully-checked `unsafe` line inside an interface that is +completely safe to use.** + +- **Minimize the surface** β€” the smallest possible block, never a whole `unsafe fn` body. +- **Name the invariant** in a `// SAFETY:` comment. *If you cannot state it, you cannot uphold + it.* +- **Expose a safe API.** Callers never write `unsafe` themselves. +- **Keep the safe twin as the oracle**, and gate the two against each other on every change. +- Set `unsafe_code = "deny"` at the workspace and lift it per-crate, deliberately. + +The patterns worth knowing when the proof is real: `get_unchecked` / `get_unchecked_mut` for +indices already proven in range, uninitialized buffers with `set_len`, `TrustedLen` for exact +size hints, raw-pointer sharing across threads, and zero-copy reinterpretation. Each needs the +invariant written down. + +--- + +## 4. Why the rules are shaped this way β€” the memory model in one page + +Three independent questions collapse onto one rule set: + +1. **Who frees?** β†’ ownership. +2. **How is memory shared?** β†’ **aliasing XOR mutation**. +3. **How is invalid memory prevented?** β†’ no null; `Option` instead. + +The elegance is that rule 2 does double duty: extended across threads it answers *no data races* +for free. **Dangling references and data races are the same bug** β€” aliased mutation β€” seen at +single-thread and multi-thread scope. That is why `Send`/`Sync` fall out of the borrow checker +rather than being bolted on. And every check is at compile time, so safety costs nothing at +runtime. + +### Three principles that generalize beyond Rust + +1. **Push checks from runtime to compile time** (or from later to earlier). A whole bug class + disappears when the check runs before the code can. +2. **Find the one invariant that kills several bug classes at once.** State it explicitly, then + check each change against it. +3. **Make illegal states unrepresentable in the type system**, not guarded at runtime. + +### Two techniques that implement principle 3 + +**Parse-constructor** β€” validate once at construction, then carry a type that is provably valid +everywhere downstream: + +```rust +pub struct Email(String); // private field: outsiders cannot build an invalid Email +impl Email { + pub fn parse(raw: &str) -> Result { /* the ONLY way to make one */ } +} +``` + +**Type-state** β€” model each state of a workflow as its own type. A transition **consumes +`self`**, so the old state is gone; a method valid in one state exists *only* on that type: + +```rust +impl User { pub fn verify(self) -> User { /* ... */ } } +impl User { pub fn send_email(&self, body: &str) { /* ... */ } } +``` + +An unverified user literally has no `send_email` method β€” zero runtime checks, and no +`if user.is_verified` guard for a caller to forget. That is the class of bug a `bool is_verified` +field creates and this pattern deletes. + +**Use this on the stack in this skill:** capability-scoped sessions, sealed records, typed +amounts, `Outcome::Local` vs `Committed` β€” all of it is principle 3 applied. + +--- + +## 5. Async, when you have to look inside it + +Most app code never needs this. When you are reading or hand-rolling a `Future`, one causal chain +explains why four hard features show up at once: + +``` +async fn => Future => poll(Pin<&mut Self>, Context<'_>) + | | + | +-> Context borrows the waker => lifetime + +-> mutating a pinned self needs get_unchecked_mut => unsafe +``` + +- An `async fn` is sugar for a state machine implementing `Future`. Implementing it by hand needs + exactly two things: an `Output` type and a `poll` method. +- `poll` takes `&mut Context<'_>`, carrying a reference to the waker. `'_` is the anonymous + lifetime β€” "infer the scope" β€” and that scope is exactly one `poll` call. +- **`Pin` is a compiler-enforced promise that the value never moves in memory.** It exists + because a future's state machine can hold references *into its own fields*; if it moved, those + would dangle. Actually mutating it needs `get_unchecked_mut`, which the compiler cannot verify + keeps Pin's promise β€” hence the `unsafe` block where you vouch. + +You don't memorize four features; you trace one chain. When you next see `Pin<&mut Self>` or +`Context<'_>`, this is what forced each one β€” and Β§3's discipline is what to apply to the +`unsafe` line it forces. + +--- + +## 6. Specialist territory + +- **Transcendentals in a hot loop** (`exp`, `ln`, `tanh`, `sigmoid`, `erf`, `powf`) can be + replaced with range-reduced polynomials that vectorize. Worth it for exp/ln/tanh; **not** for + `sqrt`/`min`/`max`, which are SSE2 baseline and already vectorized. The round-to-integer step + is where two independent implementations reintroduced the libm call they had just removed. +- **Codec kernels** β€” the `codec-*` skill suite's discipline takes precedence over this file for + encoder/decoder work: profile-first routing, reference-oracle gates, revert-if-not-faster, one + brick per commit. diff --git a/skills/building-the-new-internet/stack.md b/skills/building-the-new-internet/stack.md new file mode 100644 index 0000000..7900cf1 --- /dev/null +++ b/skills/building-the-new-internet/stack.md @@ -0,0 +1,503 @@ +# stack.md β€” every dependency decision, with the traps + +The table in `SKILL.md` Β§1 is the summary. This is the detail: exact crate names, versions, +features, the reason each choice exists, and the mistake each one prevents. + +--- + +## 1. Identity and access β€” mID + +**Repo:** https://github.com/Remade-With-Rust/mid Β· MIT OR Apache-2.0 Β· native + `wasm32` + +A user's identity is **a keypair they hold on their own device** β€” not a row in your database, +not an account on a portal. They hand your service a token signed by their own key; you verify +it **entirely locally**, with zero runtime calls to MATA or anyone. + +| Crate | Use it for | +|---|---| +| `mid-verify` | **Start here.** RP-side verifier: genesis self-sig β†’ roster chain β†’ head version β†’ JWS. A pure function, no I/O. | +| `mid-issuer` | Wallet side β€” mints the self-issued sign-in JWT from an identity snapshot + device key. | +| `mata-sign` | Sign arbitrary content as a `did:mata`, verifiable offline by anyone. | +| `mata-identity` | The user-owned keypair β€” the `did:mata` root. | +| `mata-cap` | One `Capability` / `Caller` / `authorize` model, so every service gates the same way. | +| `kms-client` / `kms-verifier` / `kms-types` | Sovereign-auth envelopes; `kms-client` is `wasm32` + native. | + +Browser/Node side is `@matanetwork/sovereign-id` β€” same protocol, same wire format. Issue on +one, verify on the other. + +```toml +[dependencies] +mid-verify = "0.1" +``` + +```rust +use mid_verify::{verify_mid_response, VerifyConfig}; + +let config = VerifyConfig { + expected_audience: "https://acme.com".into(), // your origin; must equal the token's `aud` + expected_nonce: session_nonce, // the single-use nonce you issued + max_iat_skew_secs: 120, + now_unix_secs: now, // you supply the clock β€” verify stays pure +}; + +let verified = verify_mid_response(jwt, &config)?; +// verified.did -> stable user id (your users-table primary key) +// verified.claims -> only what the user consented to disclose +// verified.genesis_roster_hash -> anchor; one DID always presents the same hash + +// Defeat stolen-device replay. Do not skip this line. +verified.check_rollback(last_seen_version)?; +``` + +**What you do NOT build:** `client_id`, redirect-URI allowlists, a `/token` back-channel, a +JWKS endpoint, a DID-resolver HTTP call, MAU metering, password reset, or a session table. The +token carries its own resolution data and the DID *is* the public key. + +**Traps.** +- Skipping `check_rollback` leaves stolen-device replay open. It is one line. +- `expected_audience` must be your real origin. A wildcard here is an auth bypass. +- The nonce must be single-use and server-issued. Reusing one turns a capture into a login. + +--- + +## 2. Storage β€” SpaceDB, and only SpaceDB + +**Repo:** https://github.com/Remade-With-Rust/spacedb Β· MIT OR Apache-2.0 Β· `spacedb-sdk 0.5.2` + +Local-first, CRDT-native, mesh-replicated. Your data lives encrypted across machines near your +users β€” offline-available, converging automatically, every access gated by a signed, scoped, +revocable capability (for humans **and** AI agents). No connection string, no server, no network +required to start. + +```toml +[dependencies] +spacedb-sdk = "0.5" # composes the whole stack; installs rusty_alloc by default +``` + +```rust +use spacedb_sdk::{ + Database, Schema, CrdtType, Tier, Identity, Capability, SignedCapability, + Scope, Ops, Outcome, StrongResult, +}; + +// 1. Open an offline-first local replica for this device. +let mut db = Database::open(Identity::generate("did:mata:home-1")?); + +// 2. Each field picks its CRDT type AND its consistency tier. +db.define( + Schema::new("profile") + .field("bio", CrdtType::Text, Tier::Convergent) // auto-merges + .field("display_name", CrdtType::Register, Tier::Convergent) + .field("cursor", CrdtType::Register, Tier::Causal) // read-your-writes + .field("visits", CrdtType::Counter, Tier::Convergent) + .field("username", CrdtType::Register, Tier::Strong), // globally unique +); + +// 3. Grant a capability β€” to a person or an AI agent β€” scoped, expiring, revocable, budgeted. +let cap = Capability::grant( + owner.did().clone(), + "did:agent:assistant", + Scope::Collection("profile".into()), + Ops::READ | Ops::WRITE, + )? + .with_expiry(1_702_592_000) + .with_budget(1_000_000); // micro-$MATA it may spend +let mut session = db.session(SignedCapability::sign(cap, &owner)?); + +// 4. Write offline. Every op returns the consistency it ACTUALLY achieved. +let outcome = db.put_register(&mut session, "profile", "display_name", "Ada")?; +assert_eq!(outcome, Outcome::Local); // durable here, converging outward + +// 5. Strong tier when you mean it: globally unique, or it cleanly refuses. +match db.claim_unique(&mut session, "profile", "username", "ada")? { + StrongResult::Committed => { /* yours */ } + StrongResult::Rejected(_) => { /* taken */ } + StrongResult::Unavailable(_) => { /* no quorum right now β€” never a divergent commit */ } +} +``` + +**The model in one line:** open β†’ schema β†’ grant β†’ write/read with honest state β†’ strong when +you mean it. + +**Pick the tier deliberately.** `Convergent` for anything that can merge (text, counters, sets). +`Causal` for anything a user must see their own write of. `Strong` only for global uniqueness β€” +it can return `Unavailable` under partition, and that is the feature. + +**The layers**, each a seam an operator (disco) fills β€” the dependency arrow is always +MATA β†’ SpaceDB, never the reverse: + +| Crate | Layer | Seam | +|---|---|---| +| `spacedb-store` | L0 encrypted KV, typed tables | `KvEngine`, `KeyProvider` | +| `spacedb-crdt` | L1 convergent docs, reactive queries | β€” | +| `spacedb-replica` | L2 anti-entropy sync, honest freshness | `Transport` | +| `spacedb-durability` | L2 erasure shards, placement, self-repair | `ShardStore` | +| `spacedb-access` | L5 mID capabilities, delegation, audit | `KeyDirectory` | +| `spacedb-query` / `-vector` | L4 compute-to-data, on-node RAG | redundant placement | +| `spacedb-consistency` | L3 tiers | strong-tier placement | +| `spacedb-meter` | L6 metering, budgets | `Settlement` | + +**Library authors must opt out of the allocator:** + +```toml +spacedb-sdk = { version = "0.5", default-features = false } +``` + +Cargo features are additive across the whole graph. A *library* that takes `spacedb-sdk` with +default features installs `rusty_alloc` into every application that depends on it β€” and any app +that already chose an allocator then fails to build with an error it cannot fix from its own +manifest. Applications decide the allocator; libraries stay out of it. + +Hardened node profile: `features = ["secure"]` (guard pages, encrypted free lists; ~4–7 % +instructions, measured). + +--- + +## 3. Media β€” remade_ffmpeg_rs + +**Repo:** https://github.com/Remade-With-Rust/remade_ffmpeg_rs Β· Apache-2.0 Β· pre-1.0 + +Decode, encode, transcode, mux and probe audio/video. A ground-up Rust rebuild of FFmpeg with +no FFI, no copyleft, and zero memory-safety CVEs on the core path by construction. + +> **Name-collision trap.** The `rff` crate published on crates.io is an unrelated fuzzy text +> selector. **Always depend by git URL.** + +```toml +[dependencies] +rff = { git = "https://github.com/Remade-With-Rust/remade_ffmpeg_rs", rev = "" } +``` + +```rust +use rff::{Engine, transcode, probe}; + +let engine = Engine::new(); // every built-in codec + container registered +let report = transcode::run(&engine, &spec)?; +``` + +`rff` is the facade β€” it re-exports `rff_core` / `rff_codec` / `rff_format` and builds a wired +`Engine`. The `ffmpeg` / `ffprobe` CLIs are thin wrappers over exactly this API; there is no +logic in them you cannot reach programmatically. Codec crates are `rff-codec-*` (aac, av2, +avif, flac, gif, h264, jpeg, jxl, mp3, opus, pcm, png, vorbis, vp9, webp…), containers +`rff-format-*`. + +**Use it for:** network and edge media conversion β€” transcoding uploads, generating thumbnails +and previews, normalising user media before it enters storage, adaptive delivery. + +**Honest status.** Conformance is bit-exact where claimed (VP9 315/315 vectors). Speed varies +by codec and is reported as measured: AAC ~6Γ— and Vorbis ~5.3Γ— faster than FFmpeg +(frame-parallel), Opus 1.50–1.60Γ— at quality parity, MP3 decode 1.24Γ— on one core, PNG decode +~2.6Γ—; VP9 decode is ~0.16–0.21Γ— and still optimising. Pick the codec knowing the row. + +Sibling single-format crates when you need only one: `rusty_png`, `rusty_jpeg`, `rusty_gif`, +`rusty_flac`, `rusty-opus`, `rusty_h264`, `rusty_av2d`, `rusty_dds`, `rusty-av1-toolkit`. + +--- + +## 4. AI β€” FFAI, and the two-layer rule + +**Repo:** https://github.com/Remade-With-Rust/FFAI Β· published on crates.io + +OCR, ASR/TTS, detection and vision-language in one pure-Rust toolkit. No Python runtime, no +gated weights, no ONNX. + +| Component | Crate | Task | +|---|---|---| +| **Mercury** | `ffai-mercury` | ASR + TTS (Whisper/WhisperX-class, VITS/Piper-class) | +| **Carmenta** | `ffai-carmenta` | OCR β€” documents, screens, change-gated live frames | +| **Diana** | `ffai-diana` | Object detection (YOLO26) + ByteTrack; `ffai-wasm` runs it in a browser | +| **Argus** | `ffai-argus` | VLM captioning / video understanding | +| infra | `ffai-core` (types, engine traits, registry), `ffai-media` (ingest/egress, backed by remade_ffmpeg_rs), `ffai-models` (weight manifests + cache) | | + +```toml +[dependencies] +ffai-core = "0.6" +ffai-diana = "0.7" # add only the engines you actually use +``` + +### The two-layer rule β€” candle and mistral.rs are not alternatives + +mistral.rs is *built on* candle. Picking one does not exclude the other; picking the wrong +**layer** is the actual mistake. + +``` +mistral.rs LLM serving β€” KV-cache paging, GGUF/ISQ, sampling, + | grammar-constrained decoding, tool calling + v +candle tensor spine β€” Tensor/Device, ops, model architectures, + CPU / CUDA / Metal / wasm32 backends +``` + +| Building | Layer | +|---|---| +| Chat, completion, agentic tool calling | **mistral.rs**, behind one `ChatEngine` seam | +| OCR, ASR/TTS, detection, depth, VLM, embeddings, classifiers, re-rankers | **candle** | +| A model in the browser | **candle** β†’ `wasm32` | +| A tensor op inside either | **candle** β€” one `Tensor`/`Device` type across all engines | + +- **Don't** hand-roll an LLM serving loop on raw candle β€” paging, quantization, sampling and + constrained decoding are exactly what mistral.rs already solved. +- **Don't** pull mistral.rs in to run a 20 MB embedding or detection model β€” that is candle's. +- **`candle-core`, `candle-nn`, `candle-transformers` move in LOCKSTEP.** Mixing minor versions + is a type mismatch, not a warning. Pin all three once, in the workspace manifest. +- **Never** route vault-adjacent context to a remote API. Local inference is the default; BYO + cloud keys are an explicit, per-user choice. +- **Never** llama.cpp, Ollama, or any C/C++ inference stack. +- Pair grammar-enforced JSON-schema output with your tool descriptors β€” constrained decoding is + what makes small local models emit valid tool calls. + +**Two C caveats β€” state them, never paper over them.** (a) The `cuda`/`metal` features forward +to candle and pull C/CUDA/Metal tooling: a *knowing* exception to the pure-Rust posture. (b) +`candle-core` takes `tokenizers` with `features = ["onig"]` as a hard dependency, so `onig_sys` +compiles a C regex engine into every native candle build β€” build-time only, no runtime +dependency, target-gated out on `wasm32`, fixable with one feature line upstream. **Never claim +"no C in the tree" for a candle build.** + +--- + +## 5. The allocator β€” rusty_alloc + +**Repo:** https://github.com/Remade-With-Rust/rusty_alloc Β· MIT Β· `rusty_alloc` / `rusty_alloc-api` + +A pure-Rust rebuild of mimalloc's architecture. No C in the tree, permissive licence, runs on +`wasm32-unknown-unknown` with no emscripten β€” the last C allocator out. + +**Version reality (check before you pin):** the repo is at `1.1.0` with a **frozen API**; +crates.io currently serves `1.0.1` for both `rusty_alloc` and `rusty_alloc-api`. Pin exactly +what you verified. Treat **0.3.2 and earlier as unsound on every target** β€” 0.4.0 fixed three +platform-independent use-after-frees. + +### Two ways to adopt it. Both keep it out of your libraries. + +**(a) Your own one-crate seam** β€” preferred for a workspace with several deliverables. The seam +holds the exact pin, the startup configuration and the `secure` feature, so feature code never +names the allocator crate: + +```toml +# crates/myapp-alloc/Cargo.toml +[dependencies] +rusty_alloc-api = { version = "=1.0.1" } +``` + +```rust +// crates/myapp-alloc/src/lib.rs +#![no_std] +pub use rusty_alloc_api::RustyAlloc as Alloc; +``` + +```rust +// crates/myapp-server/src/main.rs β€” the DELIVERABLE, exactly once +#[global_allocator] +static ALLOC: myapp_alloc::Alloc = myapp_alloc::Alloc; +``` + +**(b) `rusty_alloc_default`** β€” the org's ready-made seam (`0.1`), already the default +`rusty-alloc` feature of `rusty_symbols` / `rusty_tokens` / `rusty_a11y` so those can each +default-on without fighting (one link, one allocator): + +```toml +rusty_alloc_default = "0.1" +# hardened: rusty_alloc_default = { version = "0.1", features = ["secure"] } +``` + +If your app already installs an allocator, set `default-features = false` on the UI crates. + +### The law + +**Declare it exactly once, in the deliverable β€” never in a library.** A program may define +exactly one `#[global_allocator]`. A library that declares it forces the choice on every +consumer and makes two such libraries impossible to link into one program. The allocator is a +property of the *deliverable*, not of a component. + +βœ… Every deliverable: desktop/mobile/web entry points, WASM bundles, daemons, every service, +every CLI. When the deliverable is a staticlib/cdylib app, it goes in that crate's `lib.rs`. + +❌ Any shared library crate. In a diff, that is a stop-the-review finding. + +### Adopt it for the safety posture, not for speed + +A double free **aborts** instead of putting a block on a free list twice and handing identical +memory to two owners (~0.4 % overhead). **Treat that abort as a bug to fix, never a check to +disable.** On a mesh node you do not own, a memory bug becoming a visible crash instead of +silent divergence is the whole point β€” a replica that corrupts its own heap is a replica that +lies to the mesh. + +The published evidence is **parity**, not superiority: instruction counts at-or-below mimalloc +on real programs, ~2–16 % under jemalloc, ~18 % under glibc. **Never claim a speed or footprint +win without a measurement that clears the noise floor** (`performance.md` Β§2). + +Enable `secure` on services exposed to untrusted input; `debug_checks` in debug profiles only. +Replaces mimalloc / jemalloc / snmalloc (all C/C++) and `dlmalloc` on `wasm32`. + +--- + +## 6. Compression β€” rusty_zstd + +**Repo:** https://github.com/Remade-With-Rust/rusty_zstd Β· MIT OR Apache-2.0 + +Pure-Rust Zstandard (RFC 8878). Compress (levels βˆ’7…22, all 9 strategies) and decompress, both +directions interop-verified against facebook/zstd v1.5.7. Dictionaries, long-range matching, +seekable frames and multi-threading are dual-gated against the C reference. No C, no FFI on the +core path. + +> **Not on crates.io yet** β€” it is deploying. Depend by git (pin a commit) or path, and switch +> the pin, not the API, when it publishes. + +```toml +[dependencies] +rusty_zstd = { git = "https://github.com/Remade-With-Rust/rusty_zstd", rev = "" } +``` + +```rust +// one-shot +let packed = rusty_zstd::compress(&bytes, 3)?; +let restored = rusty_zstd::decompress(&packed)?; + +// streaming +use rusty_zstd::{Compressor, Decompressor, Flush}; + +// dictionaries β€” the big win on many small similar payloads +let dict = rusty_zstd::train(&samples, &TrainOptions::default())?; +let packed = rusty_zstd::compress_using_dict(&bytes, &dict, 3)?; + +// seekable frames β€” random access without decompressing the whole archive +let packed = rusty_zstd::compress_seekable(&bytes, 3, DEFAULT_FRAME_SIZE)?; +let chunk = rusty_zstd::decompress_frame_at(&packed, i)?; + +// multi-thread for large payloads +let packed = rusty_zstd::compress_mt(&bytes, 3, rusty_zstd::default_nb_workers())?; +``` + +**Where it pays.** Anything crossing a link you pay for or a disk you replicate: SpaceDB values +before encryption, media segments, static site bundles before `disco sites deploy`, log and +telemetry batches, and any internal RPC payload big enough to notice. + +**Order matters: compress, then encrypt.** Ciphertext is incompressible. Compressing *after* +encryption gains nothing and compressing *attacker-influenced* data alongside secrets leaks +length β€” keep secret and attacker-controlled data in separate frames. + +The `no_std` + `alloc` build works for embedded and WASM targets. No speed claim is made against +C zstd yet; do not repeat one that is not in the ledger. + +--- + +## 7. UI chrome β€” thoth (or the split trio) + +**Repos:** `thoth` Β· `rusty_symbols` Β· `rusty_tokens` Β· `rusty_a11y` Β· MIT Β· all `no_std`/wasm-checked + +Everything a UI needs before you write a component. All four default-install `rusty_alloc` +(opt out with `default-features = false`). + +| Crate | Version | What it gives you | +|---|---|---| +| `thoth` | `0.3.0`, git tag `v0.3.0` | all three below, unified: `thoth::symbols` / `::tokens` / `::a11y` | +| `rusty_symbols` | `0.1` crates.io | semantically named Unicode glyph constants + VS15 presentation pinning | +| `rusty_tokens` | `0.2` crates.io | semantic CSS custom-property names, neutral defaults, `:root` sheet emitter | +| `rusty_a11y` | `0.2` crates.io | labelled glyphs, ARIA live regions, status announcements β€” as HTML string builders, no DOM crate, no JS | + +**Why glyph constants and not literals:** application `.rs` files stay ASCII, so a +Windows-1252 round-trip cannot mojibake your icons. One source of truth, `\u{...}` escapes, +presentation pinned so a WebView renders the same glyph on every platform. + +**Why tokens and not hex:** token names *are* the CSS API (`--rt-color-fg`). Defaults are a +small neutral starter; apps override in CSS. `css::root_sheet()` injects the `:root` block into +a WebView or Dioxus app. + +--- + +## 8. The rest of the ladder + +### Crypto β€” never roll it + +| Concern | Use | Never | +|---|---|---| +| Key derivation | **Argon2id** | PBKDF2, bcrypt, raw hashes, "we'll tune it later" | +| Encryption at rest | **AES-256-GCM**, authenticated, per-entry, 12-byte random IV | ECB/CBC, unauthenticated modes, a reused IV | +| Primitives | **RustCrypto** (https://github.com/rustcrypto) | C-backed openssl / libsodium bindings | +| TLS, system tools | **memorysafety.org** β€” rustls, sudo-rs, curl-rust | native-tls / openssl-sys where rustls fits | + +### The Oxi* replacements β€” each deletes a C build + +| Need | Use | Replaces | Hard limit | +|---|---|---|---| +| Text shaping, layout, rasterization | **OxiText** | FreeType, HarfBuzz | the point is WASM + no native build tools | +| Internal wire format (service↔service, RPC) | **oxicode** | JSON on internal hot paths | ❌ **not for public APIs** β€” browsers and third parties get JSON / OpenAPI / gRPC | +| High-precision & arbitrary-size math | **OxiNum** | `rug`, `gmp-mpfr-sys`, `num-bigint`/`num-rational` | ❌ **never for crypto, keys or signatures** β€” not constant-time, leaks via timing on secret data | +| Protobuf codegen | **OxiProto** | `prost-build` + system C++ `protoc` | pure Rust, inline in `cargo build` | + +### Supply chain β€” Deputy + +`cargo install deputy-cli` β€” takes the full transitive closure of your repos, downloads every +crate into a local encrypted vault you own, SHA-256-verifies on acquisition, re-checks on scan, +and gates what reaches production. A re-published `name@version` with different bytes is +flagged, not silently accepted. Stand it up before your first production build, not after your +first incident. + +### The org inventory β€” check it, it moves + +https://github.com/orgs/Remade-With-Rust/repositories + +Infrastructure: `mid`, `sovereign-id` (JS), `spacedb`, `rusty_alloc`, `rusty_alloc_default`, +`deputy`. +Media: `remade_ffmpeg_rs`, `rusty_png`, `rusty_jpeg`, `rusty_gif`, `rusty_flac`, `rusty-opus`, +`rusty_h264`, `rusty_av2d`, `rusty_av2f`, `rusty-av1-toolkit`, `rusty_dds`. +AI: `FFAI`, `mercury`, `carmenta`, `diana`. +UI: `thoth`, `rusty_symbols`, `rusty_tokens`, `rusty_a11y`. +Apps: `starfire`. +Deploying: `rusty_zstd`. + +**Re-check before you build any general-purpose component.** The fastest way: + +```sh +gh repo list Remade-With-Rust --limit 100 --json name,description,updatedAt +``` + +If something on that list does what you were about to write, use it β€” and if it *almost* does, +open an issue there rather than forking the capability into your app. + +--- + +## 9. The scaffold β€” copy this + +```toml +# Cargo.toml (workspace root) β€” every pin lives here, exactly once +[workspace] +resolver = "2" +members = ["crates/*"] + +[workspace.dependencies] +# identity + storage +mid-verify = "0.1" +spacedb-sdk = { version = "0.5", default-features = false } # libs opt out; deliverable opts in +# allocator (through the seam in crates/myapp-alloc) +rusty_alloc-api = "=1.0.1" +# ui chrome +thoth = { git = "https://github.com/Remade-With-Rust/thoth.git", tag = "v0.3.0" } +# media + compression, by git (see Β§3 and Β§6 for why) +rff = { git = "https://github.com/Remade-With-Rust/remade_ffmpeg_rs", rev = "" } +rusty_zstd = { git = "https://github.com/Remade-With-Rust/rusty_zstd", rev = "" } +# ai β€” the three candle crates move in lockstep +candle-core = "0.9" +candle-nn = "0.9" +candle-transformers = "0.9" + +[workspace.lints.rust] +unsafe_code = "deny" # lift it per-crate, with a SAFETY comment, never workspace-wide +``` + +```rust +// crates/myapp-server/src/main.rs β€” a DELIVERABLE +#[global_allocator] +static ALLOC: myapp_alloc::Alloc = myapp_alloc::Alloc; + +#[tokio::main] +async fn main() -> anyhow::Result<()> { + // LongLived for daemons/services/apps; ShortLived for CLIs and one-shots. + myapp_alloc::configure(myapp_alloc::Profile::LongLived); + myapp_server::run().await +} +``` + +`configure` is what turns **purging** on. Purging is opt-in upstream, and opt-in-*off* is the +configuration whose RSS behaviour is least understood β€” with it on, a soak held RSS flat. Every +long-lived deliverable calls it. diff --git a/skills/building-the-new-internet/ui.md b/skills/building-the-new-internet/ui.md new file mode 100644 index 0000000..c6f7016 --- /dev/null +++ b/skills/building-the-new-internet/ui.md @@ -0,0 +1,218 @@ +# ui.md β€” the surface: Dioxus, and the chrome crates that come before it + +One codebase across web, PWA, native desktop (Windows/macOS) and mobile (iOS/Android). +"Cross-platform" means all of them β€” "works on my target" is not working. + +Remember the ordering rule from `architecture.md` Β§2: **the op exists before the button.** This +file is about the button. + +--- + +## 1. Chrome first β€” don't hand-roll glyphs, colours or ARIA + +Before your first component, take the three chrome crates. They are `no_std`, wasm-checked, and +each defaults to installing `rusty_alloc` (opt out with `default-features = false` β€” see +`stack.md` Β§5). + +```toml +# unified +thoth = { git = "https://github.com/Remade-With-Rust/thoth.git", tag = "v0.3.0" } +# or split +rusty_symbols = "0.1" +rusty_tokens = "0.2" +rusty_a11y = "0.2" +``` + +**`rusty_symbols` β€” glyphs as named constants.** Application `.rs` files stay ASCII (`\u{...}` +escapes), so a Windows-1252 round-trip cannot mojibake your icons, and VS15 pinning makes a +WebView render the same glyph on every platform. One source of truth instead of a raw literal +scattered at every site. + +**`rusty_tokens` β€” the theme contract.** Token names *are* the CSS API (`--rt-color-fg`). +Defaults are a small neutral starter; apps override in CSS. `css::root_sheet()` emits the +`:root` block to inject into a WebView or Dioxus app. + +**`rusty_a11y` β€” chrome a screen reader can hear.** Labelled glyphs get `role="img"` names; +sync / saved / offline updates ride polite or assertive live regions. Small HTML string +builders β€” no DOM crate, no JS. + +--- + +## 2. Dioxus 0.7 footguns β€” the ones that cost hours + +### Dynamic `style:` does NOT re-apply on re-render. Dynamic `class:` does. + +The single most expensive UI bug here. A dynamic **`style:`** string attribute is applied **on +mount only** β€” when a signal changes and the component re-renders, the new string is *not* +re-applied to the existing DOM node. + +```rust +// Reverts / doesn't update on re-render: +div { style: if active() { "color: red" } else { "color: gray" } } + +// Updates correctly: +div { class: if active() { "text-accent border-b border-accent" } + else { "text-fg/50 border-b border-transparent" } } +``` + +Symptom: a tab or underline "reverts to the old look" on click even though the signal changed. +Drive visual state with a dynamic `class`. If you genuinely need dynamic inline style, change +the element's `key` to force a remount. + +### A prebuilt `tailwind.css` makes new utility classes silently no-op + +A desktop build ships a **prebuilt** `assets/tailwind.css` containing only the classes present +at build time. A class you add in RSX that is not already in that file β€” `border-b-2`, `-mb-px`, +`gap-0.5` β€” **silently does nothing**. No error, no style. `py-2.5` may be present while +`gap-0.5` is not. + +Before blaming your logic, confirm the class exists. Grep with `-F` because of the escaped dot: +`grep -F "py-2\.5" tailwind.css`. Fix by regenerating Tailwind, or use a class already compiled +in. + +### What `dx serve` actually serves β€” this decides where files go + +| you have | result | +|---|---| +| `asset!("/assets/x.css")` | hashed path (`/assets/x-dxh.css`), **served** β€” the only way `assets/` files get served in dev | +| a raw file in `assets/`, not referenced via `asset!()` | **404 in dev serve** β€” even the template's own fonts and images; they only appear in the built SSG output | +| anything in `public/` | served **verbatim at the web root**: `public/uploads/x.jpg` β†’ `/uploads/x.jpg` | + +**Put runtime, uploaded and static files in `public/`**, so the same clean path works in the +live preview *and* the deployed bundle. Have your build script copy `public/*` into the output +too. Verify serving empirically with marker files and `curl` before you design storage paths. + +### `[web.resource.dev]` is dev-only injection + +In `Dioxus.toml`, `[web.resource.dev] script = ["…"]` injects `