From fe0617dd459cf811fd36729919a64c3bb9ea7fb7 Mon Sep 17 00:00:00 2001 From: Frank McSherry Date: Fri, 7 Aug 2026 11:19:16 -0400 Subject: [PATCH] .agents/skills: add mz-demo-data for live SQL-generated demo data Adds a skill that stands up continuously-updating, realistic synthetic data inside a running Materialize instance using nothing but views over mz_now(). No Kafka, no external load generator, no seed scripts. The technique comes from the "moments" construction in https://github.com/frankmcsherry/blog/blob/master/posts/2024-05-19.md. The audience is people evaluating Materialize rather than people developing it, which is why this reads as a guided path rather than a SQL reference. SKILL.md walks an agent through connecting, picking or designing a domain, proposing the model in plain language before writing any SQL, allocating a byte budget, loading, and proving the result with a heartbeat and an invariant query. Two layers. The scaffold builds a sliding window of timestamps and hashes each one into 16 deterministic bytes. Domains turn those bytes into entities. Primary keys derive from a moment's bytes and child rows re-hash their parent's, so when a moment falls out of the retention window every row derived from it vanishes together. That gives referential integrity without declaring any constraints, and it is what makes the whole approach work. Six domains ship: auctions, ecommerce, banking, iot, clickstream, and zoo. Each bakes in an invariant that holds by construction rather than by check constraint, which is the part that is hard to fake with off-the-shelf data. Banking is the headline: SUM of all balances is exactly zero at every consistent timestamp no matter how many transactions are in flight. Assets live in misc/demo-data/ with .agents/skills/mz-demo-data as a symlink, so the directory can be lifted out on its own if we decide it belongs somewhere other than this repo. Verified against Materialize v26.36.0 in the Docker emulator. All six domains load and coexist, all eight documented invariants hold, and the sliding window advances correctly at a fixed row count. --- .agents/skills/README.md | 11 + .agents/skills/mz-demo-data | 1 + misc/demo-data/README.md | 50 +++ misc/demo-data/SKILL.md | 379 ++++++++++++++++ misc/demo-data/assets/common/people.sql | 40 ++ misc/demo-data/assets/domains/_template.sql | 148 +++++++ misc/demo-data/assets/domains/auctions.sql | 106 +++++ misc/demo-data/assets/domains/banking.sql | 113 +++++ misc/demo-data/assets/domains/clickstream.sql | 131 ++++++ misc/demo-data/assets/domains/ecommerce.sql | 130 ++++++ misc/demo-data/assets/domains/iot.sql | 114 +++++ misc/demo-data/assets/domains/zoo.sql | 411 ++++++++++++++++++ misc/demo-data/assets/scaffold.sql | 113 +++++ misc/demo-data/assets/teardown.sql | 24 + 14 files changed, 1771 insertions(+) create mode 120000 .agents/skills/mz-demo-data create mode 100644 misc/demo-data/README.md create mode 100644 misc/demo-data/SKILL.md create mode 100644 misc/demo-data/assets/common/people.sql create mode 100644 misc/demo-data/assets/domains/_template.sql create mode 100644 misc/demo-data/assets/domains/auctions.sql create mode 100644 misc/demo-data/assets/domains/banking.sql create mode 100644 misc/demo-data/assets/domains/clickstream.sql create mode 100644 misc/demo-data/assets/domains/ecommerce.sql create mode 100644 misc/demo-data/assets/domains/iot.sql create mode 100644 misc/demo-data/assets/domains/zoo.sql create mode 100644 misc/demo-data/assets/scaffold.sql create mode 100644 misc/demo-data/assets/teardown.sql diff --git a/.agents/skills/README.md b/.agents/skills/README.md index 988e7cb1f469e..dd0940cb132a7 100644 --- a/.agents/skills/README.md +++ b/.agents/skills/README.md @@ -45,3 +45,14 @@ choosing which framework to use, start with **mz-test**. | Skill | When to use | What it does | |---|---|---| | **mz-adapter-guide** | Working on or asking about the adapter layer | Correctness invariants and architectural notes for the coordinator, pgwire, peek paths, timestamp oracle, and related crates | + +## Demos + +| Skill | When to use | What it does | +|---|---|---| +| **mz-demo-data** | Standing up live, realistic data to show Materialize off | Builds continuously-updating synthetic data entirely in SQL, no Kafka or external load generator. Ships six domains (auctions, ecommerce, banking, IoT, clickstream, zoo) and a rubric for designing new ones | + +Unlike the skills above, `mz-demo-data` is aimed at people evaluating +Materialize rather than developing it. Its assets live in `misc/demo-data/`, +with `.agents/skills/mz-demo-data` as a symlink, so the directory can be +lifted out on its own. diff --git a/.agents/skills/mz-demo-data b/.agents/skills/mz-demo-data new file mode 120000 index 0000000000000..e2d82f34120fa --- /dev/null +++ b/.agents/skills/mz-demo-data @@ -0,0 +1 @@ +../../misc/demo-data \ No newline at end of file diff --git a/misc/demo-data/README.md b/misc/demo-data/README.md new file mode 100644 index 0000000000000..226310b0ba384 --- /dev/null +++ b/misc/demo-data/README.md @@ -0,0 +1,50 @@ +# mz-demo-data + +Continuously-updating, realistic synthetic data for Materialize demos — +generated entirely in SQL, no external load generator. + +Based on [this blog post](https://github.com/frankmcsherry/blog/blob/master/posts/2024-05-19.md). + +## Quickstart + +```sh +PSQL="psql -p 6875 -h localhost -U materialize" + +$PSQL -f assets/scaffold.sql # moments + random (24h window, 1s tick) +$PSQL -f assets/common/people.sql # shared 256-identity pool +$PSQL -f assets/domains/auctions.sql # or any other domain +``` + +Then in a psql session: + +```sql +COPY (SUBSCRIBE (SELECT COUNT(*) FROM auctions) WITH (progress = true)) TO STDOUT; +``` + +To change the retention window or tick: + +```sql +\set retention '6 hours' +\set tick '1 second' +\i assets/scaffold.sql +``` + +Teardown: `$PSQL -f assets/teardown.sql`. + +## Domains + +* **auctions** — marketplace; auctions with lifecycle and bids +* **ecommerce** — orders, line items, totals (joins shared `people`) +* **banking** — double-entry transactions; `SUM(balances) = 0` invariant +* **iot** — devices, readings, threshold alerts +* **clickstream** — sessions, page views, conversion funnel +* **zoo** — zoo visits, ratings, shipments; four invariants at once + +Load more than one to see Materialize keep multiple data products in sync +over the same shared identity space. + +## Designing a new domain + +See [`SKILL.md`](SKILL.md). The short version: derive PKs from `moment`'s +random bytes, derive FKs by re-hashing the parent's bytes, control +distributions with byte masks, and bake an invariant into the construction. diff --git a/misc/demo-data/SKILL.md b/misc/demo-data/SKILL.md new file mode 100644 index 0000000000000..61910ef084c02 --- /dev/null +++ b/misc/demo-data/SKILL.md @@ -0,0 +1,379 @@ +--- +name: mz-demo-data +description: > + Trigger: "demo data", "synthetic data", "live operational data", "generate + fake data", "create a demo", "load generator alternative", or wants + realistic continuously-updating data in Materialize for a demo. Also "I + need streaming data to show off X" or "build a demo schema for ". + Use this to stand up auctions/bids, ecommerce, banking, IoT, or + clickstream demos — or to design a new domain in the same style. +--- + +# Live operational demo data, generated entirely in SQL + +This skill builds continuously-updating, realistic synthetic data inside a +running Materialize instance — no Kafka, no external load generator, no +seed scripts. Everything is plain views over `mz_now()`. + +The technique comes from [this blog post][blog]. Use the catalog at the +bottom for an existing domain; use the rubric to design new ones. + +[blog]: https://github.com/frankmcsherry/blog/blob/master/posts/2024-05-19.md + +## Quickstart + +```sh +psql -p 6875 -h localhost -U materialize -f assets/scaffold.sql +psql -p 6875 -h localhost -U materialize -f assets/common/people.sql +psql -p 6875 -h localhost -U materialize -f assets/domains/auctions.sql +``` + +Re-running `scaffold.sql` or `people.sql` is safe — both short-circuit if +already loaded. Re-running a domain file errors on the second `CREATE +VIEW`; run `assets/teardown.sql` first to rebuild. + +Then in a psql session: + +```sql +COPY (SUBSCRIBE (SELECT COUNT(*) FROM auctions) WITH (progress = true)) TO STDOUT; +``` + +You should see a heartbeat tick once per second. The count grows for the first +24 hours, then stabilizes at retention/tick (default 86,400). + +To change the window or tick rate, set them before `\i`: + +```sql +\set retention '6 hours' +\set tick '1 second' +\i assets/scaffold.sql +``` + +To tear everything down: `psql -f assets/teardown.sql`. + +## First contact: walking a fresh user from zero to demo + +**Read this section first if you're an agent and a user shows up wanting a +demo.** The skill is meant to feel guided, not like a SQL reference. Drive +the user through these turns: + +### Turn 1 — establish the connection + +Ask once if it isn't already in context: + +> What's the connection? A `psql` command, a `MATERIALIZE_URL`, or +> host/port/user/db? + +Accept any of: a connection string, env var, or the four flags. Don't +proceed until you can run `psql -c 'SELECT mz_version();'` and get a row +back. If they say "localhost defaults", use +`psql -p 6875 -h localhost -U materialize -d materialize`. + +### Turn 2 — pick a domain + +If the user named one ("show me a banking demo"), check the catalog at the +bottom of this file. If it matches, jump to Turn 5 with that file. + +If they have a domain in mind that isn't in the catalog ("I work in +logistics", "we do telecom billing", "give me healthcare"), continue to +Turn 3 — we'll build a new domain together. **Do not** force-fit their +domain onto one in the catalog; that's worse than a fresh one. + +If they don't have a domain in mind, suggest 2–3 from the catalog with a +one-line hook each and let them pick. + +### Turn 3 — propose the model in plain language + +**Not SQL yet.** Show the user the entities and relationships in English, +plus the invariant you'd bake in. Format: + +> Here's what I'd build: +> - **\** (one per moment) — fields: a, b, c +> - **\** (3–5 per top-level) — fields: x, y +> - **\** (static, N rows) — purpose +> +> **Invariant baked in:** \ +> +> **Joins to people?** \ +> +> Sound right, or different entities? + +Wait for their reaction. **Don't write SQL until they sign off.** They will +usually correct one of: an entity name, a missing field, the invariant, or +the cardinality. Cheap to iterate here, expensive later. + +### Turn 4 — show the byte budget + +Once entities are confirmed, show the byte allocation explicitly: + +> Per top-level row, 16 random bytes: +> ``` +> [0..2] id 24-bit +> [3] person_id mod 256, FK to people +> [4] kind mod N +> [5] n_children 1..K +> [6] time offset for due_at +> [7..] free +> ``` +> Children re-hash `(parent.random || child_index)` for their own bytes. + +This is the last cheap-to-change step. If the user wants higher cardinality +in some field, swap bytes here, not later. + +### Turn 5 — write and load + +Copy `assets/domains/_template.sql` to `assets/domains/.sql` and +fill it in following the byte budget. Then: + +```sh +# scaffold is idempotent — safe to run even if already loaded. +psql ... -f assets/scaffold.sql +psql ... -f assets/common/people.sql # only if domain joins people +psql ... -f assets/domains/.sql +``` + +If a previous load of THIS domain is present, scaffold will skip silently +but the domain file will fail on `CREATE VIEW ... already exists`. In that +case, run `assets/teardown.sql` first to wipe and reload. + +### Turn 6 — prove it works, hand over the wheel + +Show the user three queries: + +1. **Heartbeat** — confirms data is flowing: + ```sql + COPY (SUBSCRIBE (SELECT COUNT(*) FROM ) WITH (progress = true)) TO STDOUT; + ``` +2. **Invariant** — confirms correctness; should return the expected fixed + value (usually 0). +3. **One "cool" query** — the demo payoff. For aggregations, + `SELECT ... GROUP BY ...`; for cross-domain, a join through `people`. + +Then offer: "Want to layer another domain on top? You'll see joins through +the shared `people` table stay live too." + +### Things to *not* do during first contact + +- Don't dump the catalog up front. The user came with a domain or wants + guidance — give them one path, not a menu. +- Don't ship SQL without the plain-language proposal first. +- Don't skip the byte budget. It's where misalignments hide (e.g., two + fields sharing byte 5 by accident). +- Don't claim "it's done" until the heartbeat ticks and the invariant + query returns its expected value against THEIR Materialize instance. +- Don't add columns the user didn't ask for. The template's free bytes + exist; leave them free unless the user names a use. + +## The two-layer model + +Every demo built with this skill has exactly two layers: + +1. **Scaffold** (stable). The `moments` view — a sliding window of timestamps + — and the `random` view — MD5(moment) producing 16 deterministic bytes per + moment. **Do not edit per-domain.** Always loaded first; the same scaffold + serves every domain. Lives in `assets/scaffold.sql`. + +2. **Domain** (creative). Views that turn `random` bytes into entities, + relationships, and aggregates. One file per domain in `assets/domains/`. + Domains can compose: load multiple and they cross-join naturally via the + shared `people` table. + +Knowing where the line is matters. If you find yourself wanting to edit the +scaffold from a domain file, you almost certainly want to add another view +on top instead. + +## The four design rules for a domain + +Every domain in the catalog follows these four rules. Apply them when +designing a new one. + +### 1. PKs are derived deterministically from `moment` + +A row's primary key is a function of its moment's random bytes: + +```sql +get_byte(random, 0) + get_byte(random, 1) * 256 + get_byte(random, 2) * 65536 AS id +``` + +**Why:** the same moment always yields the same id, so re-derivation is stable +even though the underlying view is a sliding window. PKs naturally vanish from +the system when their moment falls out of retention. + +### 2. FKs come from re-derivation, not declared constraints + +A child row is generated by re-hashing the parent's random bytes: + +```sql +WITH expanded AS ( + SELECT id AS parent_id, ... + digest(random::text || generate_series(1, n_children)::text, 'md5') AS random + FROM parent_core +) +SELECT ... FROM expanded; +``` + +**Why:** when the parent's moment falls out of retention, every child derived +from it vanishes simultaneously. Referential integrity for free, no +`FOREIGN KEY` declarations needed. This is the key insight that makes the +whole approach work. + +### 3. Distributions are byte-mask choices + +Cardinality is controlled by which bytes you read: + +| Pattern | Cardinality | +|----------------------------------------|------------------| +| `get_byte(random, 0)` | 256 | +| `get_byte(random, 0) + get_byte(random, 1) * 256` | 65,536 | +| `mod(get_byte(random, 0), 5)` | 5 | +| `get_byte(random, 0) < 80` | ~31% boolean | + +**Why:** explicit and predictable. A 256-account banking demo uses one byte +mod 64 for from-account; a 16M-id auction marketplace uses three bytes. Pick +the cardinality you need. + +### 4. Evolution is `moment + interval` + +Time-relative fields are derived from the moment plus a random offset: + +```sql +moment + (get_byte(random, 6)::text || ' minutes')::interval AS end_time +``` + +**Why:** the field is monotone in the moment, so downstream views can filter +on it sensibly. Combined with rule #1, this gives you a temporal lifecycle +(start, end, expiry) tied to the entity's identity. + +## Invariant-by-construction patterns + +The strongest demos rely on invariants that **cannot be violated** because of +how the data is constructed. Two patterns: + +**Sum-to-zero (double entry).** Each event emits two child rows with opposite +signs (`banking.sql`). `SUM(ledger_entries.amount) = 0` holds at every +consistent timestamp, regardless of concurrent transaction volume. This is +the headline Materialize correctness demo and it's almost impossible to fake +with off-the-shelf data. + +**Count matches declared fanout.** A parent row declares `n_items`, and uses +`generate_series(1, n_items)` to emit children. `COUNT(children) per parent = +parent.n_items` is invariant (`ecommerce.sql`). + +When designing a new domain, look for an invariant of this form. A demo +without one is weaker — it shows speed but not correctness. + +## Invariants vs. id collisions + +The blog's construction derives primary keys from 24 bits of MD5 entropy +(`get_byte(random, 0..2)` packed into an int). At default settings — 86,400 +moments in retention — the birthday paradox guarantees roughly 230–500 +id collisions in steady state. **This is intentional and expected.** The +blog accepts it; this skill keeps it for fidelity to the blog and to keep +parent ids in a tractable range. + +Collisions affect which invariant shapes you can honestly claim: + +**Survives collisions (claim freely):** +- Aggregate equalities. `COUNT(children) = SUM(n_children over all parents)`. +- Sum-to-zero. `SUM(ledger_entries.amount) = 0`. +- Subset / FK by re-derivation. `every child.parent_id has matching parent` — + re-derivation guarantees this even when multiple parents share an id (the + child joins to *some* row with that id). +- Distribution-shape claims. `~10% of page_views are /checkout`. + +**Does NOT survive collisions (don't claim, or restate as aggregate):** +- Per-parent fanout equality. `for every parent, COUNT(children with this id) = parent.n_children`. + Two parents sharing an id will sum their children, breaking the equality. +- Per-id uniqueness. `every id appears exactly once`. +- Per-row temporal monotonicity that depends on id grouping. + +When the user asks "what's the invariant," default to the aggregate form +unless the construction makes per-id claims structurally true (e.g. +double-entry, where each transaction emits exactly two entries no matter +what id arithmetic does). + +If a demo *requires* unique ids (e.g. a customer-facing dashboard that +treats id as a key), the fix is to either widen the id space (use 4+ +bytes), use `EXTRACT(EPOCH FROM moment)::bigint` as the id (collision-free +by construction), or accept ~0.5% noise as a feature ("real data is +messy"). All three are reasonable; the catalog uses the blog's 24-bit +form for fidelity. + +## Adapting to a new domain + +When walking a user through this, follow the **First contact** protocol +above — propose-then-confirm, don't write SQL upfront. The mechanical +checklist is below. + +To add `assets/domains/.sql`, work through these steps in order: + +1. **List the entities.** Top-level (one per moment), child rows (per parent + via fanout), shared lookups (static). +2. **Pick the byte budget for each top-level entity.** Random gives you 16 + bytes per moment. Allocate bytes to fields by cardinality: + id (3 bytes), FK to people (1 byte mod 256), category (1 byte mod N), + timing offsets (1 byte), free for amount/value (2–3 bytes). +3. **Decide which existing shared tables you reference.** Today: `people`. If + you reference people, your domain joins the cross-domain "same person + appears in multiple data products" story automatically. +4. **Identify the invariant.** What sum, count, or equality must hold by + construction? Bake it in via fanout or two-legged emission, not as a + `CHECK` constraint. +5. **Copy the template.** `cp assets/domains/_template.sql assets/domains/.sql` + and fill in the TODOs. The template already encodes the standard + `_core` view → public MV → child MV → aggregate VIEW pattern and an + idempotency guard. Keep that structure. +6. **Add two validation queries** at the bottom: a `SUBSCRIBE` heartbeat + and an invariant query that should always return 0 (or a fixed value). + +## Validation + +For every domain, two queries: + +```sql +-- Heartbeat: confirms the domain is live and producing rows. +COPY (SUBSCRIBE (SELECT COUNT(*) FROM ) WITH (progress = true)) TO STDOUT; + +-- Invariant: should return a fixed value (usually 0). +SELECT ... ; +``` + +If the heartbeat ticks but the invariant drifts, the byte budget is +misaligned — likely two entities sharing the same byte for different +purposes. Re-allocate. + +## The catalog + +| File | Domain | Highlights | Joins `people`? | +|---|---|---|---| +| `domains/auctions.sql` | Auctions & bids | Marketplace; lifecycle (end_time); winning-bid demo | No | +| `domains/ecommerce.sql` | Orders, line items, totals | Multi-row child fanout; `order_totals` aggregate-as-view | Yes | +| `domains/banking.sql` | Accounts, double-entry txns | **`SUM(balances) = 0`** invariant. Strongest correctness demo. | Yes | +| `domains/iot.sql` | Devices, readings, alerts | High cardinality; threshold alerts; per-site rollup | No | +| `domains/clickstream.sql`| Sessions, page views, funnel | Funnel analytics; conversion rate by channel | Yes | +| `domains/zoo.sql` | Zoo visits, ratings, shipments | Four invariants at once, richest domain. Front-of-house ratings correlate with back-of-house skim by construction. | Yes | + +Loading more than one ecommerce/banking/clickstream domain at once gives you +the cross-product demo: "show me Person 042's orders, her transactions, and +her browsing session — all live, all consistent." + +## File map + +``` +misc/demo-data/ +├── SKILL.md ← you are here +├── README.md short human intro +└── assets/ + ├── scaffold.sql moments + random; do not edit per-domain + ├── teardown.sql drops everything + ├── common/ + │ └── people.sql 256-identity pool, shared across domains + └── domains/ + ├── _template.sql ← copy this to start a new domain + ├── auctions.sql + ├── ecommerce.sql + ├── banking.sql + ├── iot.sql + ├── clickstream.sql + └── zoo.sql +``` diff --git a/misc/demo-data/assets/common/people.sql b/misc/demo-data/assets/common/people.sql new file mode 100644 index 0000000000000..f43d531b4068c --- /dev/null +++ b/misc/demo-data/assets/common/people.sql @@ -0,0 +1,40 @@ +-- Copyright Materialize, Inc. and contributors. All rights reserved. +-- +-- Use of this software is governed by the Business Source License +-- included in the LICENSE file at the root of this repository. +-- +-- As of the Change Date specified in that file, in accordance with +-- the Business Source License, use of this software will be governed +-- by the Apache License, Version 2.0. + +-- ============================================================================= +-- Shared identity pool: 256 deterministic people, used as customers / holders +-- / users across multiple domains. Letting ecommerce + banking + clickstream +-- all reference the same id space is the "data products staying in sync" demo: +-- "show me Person 042's orders and her recent transactions, live." +-- +-- 256 is chosen so a single `get_byte(random, N)` picks a person uniformly +-- without modulus. +-- +-- Static (not moment-driven). Names/emails are deterministic for repeatability. +-- +-- Load with: \i common/people.sql (after scaffold.sql) +-- ============================================================================= + +SELECT EXISTS (SELECT 1 FROM mz_views WHERE name = 'people') AS already_loaded \gset +\if :already_loaded +\echo people already loaded; skipping. +\else + +CREATE VIEW people AS +SELECT + id::int AS id, + 'Person ' || lpad(id::text, 3, '0') AS name, + 'person' || lpad(id::text, 3, '0') || '@example.com' AS email, + (ARRAY['US-W','US-E','EU','APAC','LATAM'])[1 + mod(id, 5)] AS region, + digest('person:' || id::text, 'md5') AS attrs +FROM generate_series(0, 255) AS id; + +CREATE DEFAULT INDEX ON people; + +\endif diff --git a/misc/demo-data/assets/domains/_template.sql b/misc/demo-data/assets/domains/_template.sql new file mode 100644 index 0000000000000..30d041ab959fa --- /dev/null +++ b/misc/demo-data/assets/domains/_template.sql @@ -0,0 +1,148 @@ +-- Copyright Materialize, Inc. and contributors. All rights reserved. +-- +-- Use of this software is governed by the Business Source License +-- included in the LICENSE file at the root of this repository. +-- +-- As of the Change Date specified in that file, in accordance with +-- the Business Source License, use of this software will be governed +-- by the Apache License, Version 2.0. + +-- ============================================================================= +-- TEMPLATE — copy to .sql, fill in the TODOs, delete this header. +-- +-- This file is a fill-in-the-blank skeleton for a new domain. It encodes the +-- four design rules from SKILL.md: +-- +-- 1. PKs derived deterministically from `random` bytes +-- 2. FKs by re-derivation (re-hash parent random || child index) +-- 3. Distributions by byte mask +-- 4. Time-relative fields via `moment + interval` +-- +-- Prerequisites: scaffold.sql (always). Add `\i common/people.sql` to the +-- prereq list if your domain references people (recommended for cross-domain +-- demos). +-- ============================================================================= + +-- TODO: replace `domain` with your domain's name (e.g. `trip`, `shipment`). +SELECT EXISTS (SELECT 1 FROM mz_views WHERE name = 'TODO_core') AS already_loaded \gset +\if :already_loaded +\echo TODO domain already loaded; skipping. +\else + +-- ----------------------------------------------------------------------------- +-- (Optional) Static lookups. Use for small enumerations: product types, +-- statuses, depots, channels. Skip if not needed. +-- ----------------------------------------------------------------------------- +-- CREATE VIEW TODO_kinds (id, name) AS VALUES +-- (0, 'Kind A'), +-- (1, 'Kind B'), +-- (2, 'Kind C'); + +-- ----------------------------------------------------------------------------- +-- The `_core` view: raw fields extracted directly from random bytes. +-- +-- Byte budget (16 bytes available; allocate by cardinality): +-- [0..2] entity id 24-bit → ~16M space +-- [3] foreign key to people mod 256 +-- [4] lookup index mod N +-- [5] quantity / count 1 + mod(., K) for fanout +-- [6] minor field / jitter +-- [7..] free +-- +-- Edit the SELECT below to match your byte plan. +-- ----------------------------------------------------------------------------- +CREATE VIEW TODO_core AS +SELECT + moment, + random, + get_byte(random, 0) + + get_byte(random, 1) * 256 + + get_byte(random, 2) * 65536 AS id, + get_byte(random, 3) AS person_id, -- FK into people + mod(get_byte(random, 4)::int, 3) AS kind_id, -- FK into TODO_kinds + 1 + mod(get_byte(random, 5)::int, 5) AS n_children, -- fanout for child rows + moment + (get_byte(random, 6)::text || ' minutes')::interval AS due_at -- time-relative field +FROM random; + +-- ----------------------------------------------------------------------------- +-- The public top-level view. Materialize and join lookups for fast reads. +-- This is what demos and validation queries hit. +-- ----------------------------------------------------------------------------- +CREATE MATERIALIZED VIEW TODO AS +SELECT + c.id, + c.person_id, + p.name AS person_name, + -- k.name AS kind, + c.moment AS started_at, + c.due_at +FROM TODO_core c +JOIN people p ON p.id = c.person_id; +-- LEFT JOIN TODO_kinds k ON k.id = c.kind_id; + +-- ----------------------------------------------------------------------------- +-- (Optional) Child rows via fanout. The pattern: +-- 1. generate_series(1, parent.n_children) to expand +-- 2. digest(parent.random || child_index) to re-hash for each child +-- 3. extract child fields from the new random +-- +-- This is rule #2 — FK by re-derivation. When the parent moment falls out +-- of retention, every child generated from it vanishes simultaneously. +-- ----------------------------------------------------------------------------- +CREATE MATERIALIZED VIEW TODO_children AS +WITH expanded AS ( + SELECT + id AS parent_id, + moment AS parent_moment, + generate_series(1, n_children) AS child_no, + digest(random::text || generate_series(1, n_children)::text, 'md5') AS random + FROM TODO_core +) +SELECT + parent_id, + child_no, + get_byte(random, 0) AS field_a, + get_byte(random, 1) + get_byte(random, 2) * 256 AS field_b, + parent_moment + + (child_no::text || ' seconds')::interval AS occurred_at +FROM expanded; + +-- ----------------------------------------------------------------------------- +-- (Optional) Aggregates. The classic Materialize moment: a view that stays +-- correct as data flows in. Group by some FK and sum/count/max. +-- ----------------------------------------------------------------------------- +CREATE VIEW TODO_rollup AS +SELECT + parent_id, + COUNT(*) AS n, + SUM(field_a) AS total_a, + MAX(field_b) AS max_b +FROM TODO_children +GROUP BY parent_id; + +\endif + +-- ----------------------------------------------------------------------------- +-- Validation queries — paste these into a session to confirm the domain works. +-- ----------------------------------------------------------------------------- +-- +-- Heartbeat — should tick continuously: +-- COPY (SUBSCRIBE (SELECT COUNT(*) FROM TODO) WITH (progress = true)) TO STDOUT; +-- +-- Invariant: every child references an existing parent (should be 0). +-- SELECT COUNT(*) FROM TODO_children c +-- LEFT JOIN TODO t ON t.id = c.parent_id WHERE t.id IS NULL; +-- +-- Invariant: total child rows = total declared fanout (aggregate form). +-- A per-parent version of this looks tempting but fails because the 24-bit +-- random `id` space has birthday collisions at 86k+ rows — see SKILL.md +-- "Invariants vs. id collisions". The aggregate form is unaffected. +-- SELECT (SELECT COUNT(*) FROM TODO_children) = +-- (SELECT SUM(n_children) FROM TODO_core) AS fanout_balances; +-- +-- Add at least one *domain-specific* invariant of your own — that's what +-- makes a demo land. Pick a shape that survives id collisions: sum-to-zero +-- over all rows, total count = declared total, monotone timestamps, +-- mutually-exclusive states, subset relationships (X ⊆ Y). Avoid invariants +-- that depend on synthetic ids being unique. +-- ----------------------------------------------------------------------------- diff --git a/misc/demo-data/assets/domains/auctions.sql b/misc/demo-data/assets/domains/auctions.sql new file mode 100644 index 0000000000000..78493d04f7424 --- /dev/null +++ b/misc/demo-data/assets/domains/auctions.sql @@ -0,0 +1,106 @@ +-- Copyright Materialize, Inc. and contributors. All rights reserved. +-- +-- Use of this software is governed by the Business Source License +-- included in the LICENSE file at the root of this repository. +-- +-- As of the Change Date specified in that file, in accordance with +-- the Business Source License, use of this software will be governed +-- by the Apache License, Version 2.0. + +-- ============================================================================= +-- Auctions & Bids. The canonical example from the blog post. +-- +-- Demonstrates: +-- * deterministic PK derivation from moment hash +-- * FK by re-derivation: bids re-hash their auction's random bytes, so when +-- an auction's moment falls out of the retention window, its bids vanish +-- with it. Referential integrity for free. +-- * temporal lifecycle: auctions have an end_time computed from random bytes. +-- +-- Standalone domain: does not reference `people`. (The seller/buyer ids are a +-- larger space than the shared pool, by design — auctions are typically a +-- long-tail marketplace.) +-- +-- Load with: \i scaffold.sql +-- \i domains/auctions.sql +-- ============================================================================= + +-- Item-type lookup. Five categories cycled via `auction.item % 5`. +CREATE VIEW items (id, item) AS VALUES + (0, 'Signed Memorabilia'), + (1, 'City Bar Crawl'), + (2, 'Best Pizza in Town'), + (3, 'Gift Basket'), + (4, 'Custom Art'); + +-- Raw auction stream. One auction per moment. +-- Byte budget for `random` (16 bytes): +-- [0..2] id (24-bit ⇒ ~16M space, sparse → unique with high prob) +-- [3..4] seller (16-bit ⇒ 65k sellers) +-- [5] item type (used both as item lookup and as bid-count fanout below) +-- [6] auction duration in minutes (0..255) +CREATE VIEW auctions_core AS +SELECT + moment, + random, + get_byte(random, 0) + + get_byte(random, 1) * 256 + + get_byte(random, 2) * 65536 AS id, + get_byte(random, 3) + + get_byte(random, 4) * 256 AS seller, + get_byte(random, 5) AS item, + moment + (get_byte(random, 6)::text || ' minutes')::interval AS end_time +FROM random; + +-- The published auctions table — joined to items, materialized for fast joins. +CREATE MATERIALIZED VIEW auctions AS +SELECT auctions_core.id, seller, items.item, end_time +FROM auctions_core, items +WHERE auctions_core.item % 5 = items.id; + +-- Bids. Each auction spawns up to 255 bids via `generate_series`, where the +-- number is itself a random byte (item / bid-count share a byte by design — +-- popular categories get more bids). +-- +-- Each bid re-hashes (auction.random || bid_index) to get its own 16 bytes. +CREATE MATERIALIZED VIEW bids AS +WITH prework AS ( + SELECT + id AS auction_id, + moment AS auction_start, + end_time AS auction_end, + digest(random::text || generate_series(1, get_byte(random, 5))::text, 'md5') AS random + FROM auctions_core +) +SELECT + get_byte(random, 0) + + get_byte(random, 1) * 256 + + get_byte(random, 2) * 65536 AS id, + get_byte(random, 3) + + get_byte(random, 4) * 256 AS buyer, + auction_id, + get_byte(random, 5)::numeric AS amount, + auction_start + (get_byte(random, 6)::text || ' minutes')::interval AS bid_time +FROM prework; + +-- ----------------------------------------------------------------------------- +-- Validation. Paste either of these into a fresh session to confirm liveness: +-- ----------------------------------------------------------------------------- +-- +-- Heartbeat — should tick continuously, count stays near retention/tick: +-- COPY (SUBSCRIBE (SELECT COUNT(*) FROM auctions) WITH (progress = true)) TO STDOUT; +-- +-- Winning bid per auction (the classic Materialize demo): +-- SELECT auction_id, MAX(amount) AS winning_bid +-- FROM bids GROUP BY auction_id; +-- +-- Invariant: every bid references an existing auction (FK by re-derivation). +-- Should always return 0. +-- SELECT COUNT(*) FROM bids b +-- LEFT JOIN auctions a ON b.auction_id = a.id WHERE a.id IS NULL; +-- +-- Note on bid_time: the bid's time offset and the auction's duration come +-- from INDEPENDENT random bytes, so ~50% of bids fire after their auction +-- ends. That's intentional — late-bid attempts are realistic, and showing +-- Materialize handle them as a "filter to live auctions" view is a fine +-- demo on its own. diff --git a/misc/demo-data/assets/domains/banking.sql b/misc/demo-data/assets/domains/banking.sql new file mode 100644 index 0000000000000..29d46eea516b2 --- /dev/null +++ b/misc/demo-data/assets/domains/banking.sql @@ -0,0 +1,113 @@ +-- Copyright Materialize, Inc. and contributors. All rights reserved. +-- +-- Use of this software is governed by the Business Source License +-- included in the LICENSE file at the root of this repository. +-- +-- As of the Change Date specified in that file, in accordance with +-- the Business Source License, use of this software will be governed +-- by the Apache License, Version 2.0. + +-- ============================================================================= +-- Banking: accounts + double-entry transactions +-- +-- Demonstrates: +-- * the "invariant by construction" pattern: every transaction emits TWO +-- ledger entries summing to zero, so SUM(entries.amount) is trivially 0 +-- at every consistent timestamp. This is Materialize's headline correctness +-- property and is hard to demo with synthetic data any other way. +-- * cross-domain join to `people` for account holders +-- +-- Prerequisites: scaffold.sql, common/people.sql +-- ============================================================================= + +-- Static account directory: 64 accounts. Each held by someone from `people`. +-- account_id is 0..63 so a single byte mod 64 picks one uniformly. +CREATE VIEW accounts AS +SELECT + id::int AS id, + 'ACCT-' || lpad(id::text, 4, '0') AS number, + -- Hash the account id with a salt to pick a holder, so the assignment + -- isn't a trivial id == holder_id mapping. + get_byte(digest('acct:' || id::text, 'md5'), 0) AS holder_id, + (ARRAY['Checking','Savings','Credit'])[1 + mod(id, 3)] AS account_type +FROM generate_series(0, 63) AS id; + +CREATE DEFAULT INDEX ON accounts; + +-- One transaction per moment. +-- Byte budget: +-- [0..2] transaction_id +-- [3] from_account, mod 64 +-- [4] to_account: derived from `from` + offset in [1..63] (mod 64) so the +-- two accounts are guaranteed distinct. +-- [5..6] amount in cents (16-bit ⇒ up to $655.35 per txn) +CREATE VIEW transactions_core AS +SELECT + moment, + random, + get_byte(random, 0) + + get_byte(random, 1) * 256 + + get_byte(random, 2) * 65536 AS id, + mod(get_byte(random, 3)::int, 64) AS from_account, + mod( + mod(get_byte(random, 3)::int, 64) + + 1 + mod(get_byte(random, 4)::int, 63), + 64 + ) AS to_account, + ((get_byte(random, 5) + get_byte(random, 6) * 256))::numeric + / 100.0 AS amount +FROM random; + +CREATE MATERIALIZED VIEW transactions AS +SELECT + t.id, + t.moment AS posted_at, + t.from_account, + t.to_account, + t.amount +FROM transactions_core t; + +-- Double-entry: each transaction emits two ledger rows that sum to zero. +-- generate_series(1,2) fans out each txn; we pick from vs. to by row index. +CREATE MATERIALIZED VIEW ledger_entries AS +WITH expanded AS ( + SELECT id, moment, from_account, to_account, amount, + generate_series(1, 2) AS leg + FROM transactions_core +) +SELECT + id AS transaction_id, + moment AS posted_at, + CASE WHEN leg = 1 THEN from_account ELSE to_account END AS account_id, + CASE WHEN leg = 1 THEN -amount ELSE amount END AS amount +FROM expanded; + +-- Running balance per account. The headline view: stays correct under +-- arbitrary concurrent reads + writes because of strict serializability. +CREATE VIEW account_balances AS +SELECT + a.id, + a.number, + a.account_type, + p.name AS holder_name, + p.region AS holder_region, + COALESCE(SUM(le.amount), 0) AS balance +FROM accounts a +LEFT JOIN ledger_entries le ON le.account_id = a.id +LEFT JOIN people p ON p.id = a.holder_id +GROUP BY a.id, a.number, a.account_type, p.name, p.region; + +-- ----------------------------------------------------------------------------- +-- Validation: +-- +-- Heartbeat (count of in-flight transactions): +-- COPY (SUBSCRIBE (SELECT COUNT(*) FROM transactions) WITH (progress = true)) TO STDOUT; +-- +-- THE invariant: total of all balances is exactly zero. This is the demo: +-- you can hit this query repeatedly while millions of transactions land, +-- and it will return 0 every single time. +-- SELECT SUM(balance) FROM account_balances; +-- +-- Per-holder net position (joins to people, useful for cross-domain demos): +-- SELECT holder_name, SUM(balance) FROM account_balances GROUP BY holder_name; +-- ----------------------------------------------------------------------------- diff --git a/misc/demo-data/assets/domains/clickstream.sql b/misc/demo-data/assets/domains/clickstream.sql new file mode 100644 index 0000000000000..e5859d7102909 --- /dev/null +++ b/misc/demo-data/assets/domains/clickstream.sql @@ -0,0 +1,131 @@ +-- Copyright Materialize, Inc. and contributors. All rights reserved. +-- +-- Use of this software is governed by the Business Source License +-- included in the LICENSE file at the root of this repository. +-- +-- As of the Change Date specified in that file, in accordance with +-- the Business Source License, use of this software will be governed +-- by the Apache License, Version 2.0. + +-- ============================================================================= +-- Clickstream: sessions → page_views → conversions +-- +-- Demonstrates: +-- * funnel analytics as a maintained view +-- * window-like behavior via per-session offsets within the moment +-- * cross-domain join to `people` for user identity +-- +-- Prerequisites: scaffold.sql, common/people.sql +-- ============================================================================= + +-- One session per moment. +-- Byte budget: +-- [0..2] session_id +-- [3] user_id (mod 256 ⇒ people pool) +-- [4] n_views (clamped 3..15) +-- [5] source channel index +-- [6] device-type index +CREATE VIEW sessions_core AS +SELECT + moment, + random, + get_byte(random, 0) + + get_byte(random, 1) * 256 + + get_byte(random, 2) * 65536 AS id, + get_byte(random, 3) AS user_id, + 3 + mod(get_byte(random, 4)::int, 13) AS n_views, + (ARRAY['organic','paid','referral','direct','email']) + [1 + mod(get_byte(random, 5)::int, 5)] AS source, + (ARRAY['desktop','mobile','tablet']) + [1 + mod(get_byte(random, 6)::int, 3)] AS device +FROM random; + +CREATE MATERIALIZED VIEW sessions AS +SELECT + s.id, + s.user_id, + p.name AS user_name, + p.region AS user_region, + s.moment AS started_at, + s.source, + s.device, + s.n_views +FROM sessions_core s +JOIN people p ON p.id = s.user_id; + +-- Page views: each session emits n_views events. Pages are chosen by hash and +-- skewed so /checkout is rarer than /landing (the funnel narrows). +-- Page distribution: low byte values → /landing, /product, /search (common), +-- high byte values → /cart, /checkout (rare). This produces a realistic funnel. +CREATE MATERIALIZED VIEW page_views AS +WITH expanded AS ( + SELECT + id AS session_id, + moment AS session_start, + generate_series(1, n_views) AS view_no, + digest(random::text || generate_series(1, n_views)::text, 'md5') AS random + FROM sessions_core +) +SELECT + session_id, + view_no, + -- Each view offset within the session, in seconds. View N happens at + -- session_start + (N + small jitter) seconds. + session_start + + (view_no::text || ' seconds')::interval + + (mod(get_byte(random, 1)::int, 30)::text || ' seconds')::interval + AS viewed_at, + -- Skewed page picker: bias towards top-of-funnel. + CASE + WHEN get_byte(random, 0) < 80 THEN '/landing' + WHEN get_byte(random, 0) < 150 THEN '/product' + WHEN get_byte(random, 0) < 200 THEN '/search' + WHEN get_byte(random, 0) < 230 THEN '/cart' + ELSE '/checkout' + END AS path +FROM expanded; + +-- Conversions: sessions that reached /checkout. +CREATE VIEW conversions AS +SELECT DISTINCT + pv.session_id, + s.user_id, + s.user_name, + s.source, + s.started_at +FROM page_views pv +JOIN sessions s ON s.id = pv.session_id +WHERE pv.path = '/checkout'; + +-- Funnel: pageview counts per stage, the classic clickstream dashboard. +CREATE VIEW funnel AS +SELECT path, COUNT(*) AS views +FROM page_views +GROUP BY path; + +-- Conversion rate by source channel. A useful "stays correct under load" demo. +CREATE VIEW conversion_by_source AS +SELECT + s.source, + COUNT(DISTINCT s.id) AS sessions, + COUNT(DISTINCT c.session_id) AS conversions, + COUNT(DISTINCT c.session_id)::float + / NULLIF(COUNT(DISTINCT s.id), 0) AS conversion_rate +FROM sessions s +LEFT JOIN conversions c ON c.session_id = s.id +GROUP BY s.source; + +-- ----------------------------------------------------------------------------- +-- Validation: +-- +-- Heartbeat: +-- COPY (SUBSCRIBE (SELECT COUNT(*) FROM sessions) WITH (progress = true)) TO STDOUT; +-- +-- Current funnel: +-- SELECT * FROM funnel ORDER BY views DESC; +-- +-- Cross-domain demo (requires ecommerce.sql also loaded): +-- Which users converted AND placed an order in the window? +-- SELECT c.user_name FROM conversions c +-- WHERE EXISTS (SELECT 1 FROM orders o WHERE o.customer_id = c.user_id); +-- ----------------------------------------------------------------------------- diff --git a/misc/demo-data/assets/domains/ecommerce.sql b/misc/demo-data/assets/domains/ecommerce.sql new file mode 100644 index 0000000000000..354d8947ea3ce --- /dev/null +++ b/misc/demo-data/assets/domains/ecommerce.sql @@ -0,0 +1,130 @@ +-- Copyright Materialize, Inc. and contributors. All rights reserved. +-- +-- Use of this software is governed by the Business Source License +-- included in the LICENSE file at the root of this repository. +-- +-- As of the Change Date specified in that file, in accordance with +-- the Business Source License, use of this software will be governed +-- by the Apache License, Version 2.0. + +-- ============================================================================= +-- E-commerce: orders → line_items → order_totals +-- +-- Demonstrates: +-- * multi-row child generation via generate_series fanout +-- * cross-domain join to shared `people` (customer FK) +-- * aggregate-as-view (`order_totals`) — the classic MV use case +-- +-- Prerequisites: scaffold.sql, common/people.sql +-- ============================================================================= + +-- Static product catalog. Sixteen products keeps line_items tractable. +CREATE VIEW products (id, name, category) AS VALUES + ( 0, 'Coffee Beans 1lb', 'Grocery'), + ( 1, 'Olive Oil 500ml', 'Grocery'), + ( 2, 'Dish Soap', 'Household'), + ( 3, 'Paper Towels 6pk', 'Household'), + ( 4, 'Notebook A5', 'Stationery'), + ( 5, 'Ballpoint Pen 12pk', 'Stationery'), + ( 6, 'USB-C Cable 1m', 'Electronics'), + ( 7, 'Wireless Mouse', 'Electronics'), + ( 8, 'T-Shirt Plain', 'Apparel'), + ( 9, 'Wool Socks', 'Apparel'), + (10, 'Yoga Mat', 'Fitness'), + (11, 'Resistance Band Set', 'Fitness'), + (12, 'Cast Iron Skillet', 'Kitchen'), + (13, 'Chef Knife 8in', 'Kitchen'), + (14, 'Houseplant Pothos', 'Home'), + (15, 'Candle Lavender', 'Home'); + +-- One order per moment. +-- Byte budget for `random`: +-- [0..2] order id (24-bit space) +-- [3] customer_id (mod 256 ⇒ aligns with people pool) +-- [6] n_items, clamped to 1..8 +CREATE VIEW orders_core AS +SELECT + moment, + random, + get_byte(random, 0) + + get_byte(random, 1) * 256 + + get_byte(random, 2) * 65536 AS id, + get_byte(random, 3) AS customer_id, + 1 + mod(get_byte(random, 6)::int, 8) AS n_items +FROM random; + +CREATE MATERIALIZED VIEW orders AS +SELECT + o.id, + o.customer_id, + p.name AS customer_name, + p.region AS customer_region, + o.moment AS placed_at, + o.n_items +FROM orders_core o +JOIN people p ON p.id = o.customer_id; + +-- Line items: each order spawns n_items via generate_series, each line +-- re-hashes (order.random || line_no) for its own bytes. +-- Line-item byte budget: +-- [0..1] product_id (mod 16) +-- [2] qty (1..8) +-- [3..5] unit_price_cents (24-bit ⇒ up to ~$167k; realistic skew via mask) +CREATE MATERIALIZED VIEW line_items AS +WITH expanded AS ( + SELECT + id AS order_id, + generate_series(1, n_items) AS line_no, + digest(random::text || generate_series(1, n_items)::text, 'md5') AS random + FROM orders_core +) +SELECT + order_id, + line_no, + mod(get_byte(random, 0) + get_byte(random, 1) * 256, 16) AS product_id, + 1 + mod(get_byte(random, 2)::int, 8) AS qty, + -- Prices skewed small: 24-bit cents, but we keep low byte dominant + -- so most items are cheap and a few are pricey. + (get_byte(random, 3) + + get_byte(random, 4) * 256 + + get_byte(random, 5) * 4)::numeric / 100.0 AS unit_price +FROM expanded; + +-- Aggregate-as-view. THIS is the demo: the running sum stays consistent with +-- line_items in real time, no batch job, no consistency window. +CREATE VIEW order_totals AS +SELECT + li.order_id, + SUM(li.qty * li.unit_price) AS total +FROM line_items li +GROUP BY li.order_id; + +-- Customer running spend across the retention window. Shows people-side joins. +CREATE VIEW customer_spend AS +SELECT + o.customer_id, + o.customer_name, + COUNT(DISTINCT o.id) AS orders_placed, + SUM(li.qty * li.unit_price) AS total_spent +FROM orders o +JOIN line_items li ON li.order_id = o.id +GROUP BY o.customer_id, o.customer_name; + +-- ----------------------------------------------------------------------------- +-- Validation: +-- +-- Heartbeat: +-- COPY (SUBSCRIBE (SELECT COUNT(*) FROM orders) WITH (progress = true)) TO STDOUT; +-- +-- Invariant: every line_item references an existing order (should be 0). +-- SELECT COUNT(*) FROM line_items li +-- LEFT JOIN orders o ON o.id = li.order_id WHERE o.id IS NULL; +-- +-- Invariant: total line-items = total declared fanout, in aggregate. +-- (A per-order version of this fails because the 24-bit `id` space has +-- birthday-paradox collisions at 86k+ rows — see SKILL.md "Invariants vs. +-- id collisions". The aggregate form is unaffected since both sides count +-- rows regardless of id.) +-- SELECT (SELECT COUNT(*) FROM line_items) = +-- (SELECT SUM(n_items) FROM orders_core) AS fanout_balances; +-- ----------------------------------------------------------------------------- diff --git a/misc/demo-data/assets/domains/iot.sql b/misc/demo-data/assets/domains/iot.sql new file mode 100644 index 0000000000000..401e045a668db --- /dev/null +++ b/misc/demo-data/assets/domains/iot.sql @@ -0,0 +1,114 @@ +-- Copyright Materialize, Inc. and contributors. All rights reserved. +-- +-- Use of this software is governed by the Business Source License +-- included in the LICENSE file at the root of this repository. +-- +-- As of the Change Date specified in that file, in accordance with +-- the Business Source License, use of this software will be governed +-- by the Apache License, Version 2.0. + +-- ============================================================================= +-- IoT: devices + readings + alerts +-- +-- Demonstrates: +-- * high-cardinality fan-out (multiple readings per moment) +-- * per-device aggregates over a sliding window (running avg, max) +-- * threshold-based alerting as a continuously-maintained view +-- +-- Standalone domain: no `people` join. Devices are their own identity space. +-- +-- Prerequisites: scaffold.sql +-- ============================================================================= + +-- Static device fleet of 128 devices. Each has a sensor type and a per-device +-- alert threshold. Threshold derived from the device id so it's stable. +CREATE VIEW devices AS +SELECT + id::int AS id, + 'dev-' || lpad(id::text, 4, '0') AS device_name, + (ARRAY['temperature','pressure','humidity','vibration']) + [1 + mod(id, 4)] AS sensor_type, + -- Site picked deterministically from the id; 16 sites of ~8 devices each. + 'site-' || lpad((mod(id, 16))::text, 2, '0') AS site, + -- Threshold: most devices ~200, a few aggressive ones ~150. + 150 + mod(id * 7, 100) AS threshold +FROM generate_series(0, 127) AS id; + +CREATE DEFAULT INDEX ON devices; + +-- Readings. Each moment emits 8 readings, each one re-hashing to pick a +-- device and a value. Total cardinality ≈ retention_seconds × 8. +-- Reading byte budget (per re-hash): +-- [0] device_id (mod 128) +-- [1..2] reading value (16-bit, scaled into roughly the threshold range) +CREATE MATERIALIZED VIEW readings AS +WITH expanded AS ( + SELECT + moment, + digest(random::text || generate_series(1, 8)::text, 'md5') AS random, + generate_series(1, 8) AS slot + FROM random +) +SELECT + moment AS observed_at, + slot, + mod(get_byte(random, 0)::int, 128) AS device_id, + (get_byte(random, 1) + get_byte(random, 2) * 256) + ::numeric / 100.0 AS value +FROM expanded; + +-- Per-device running aggregates over the live retention window. This is the +-- IoT demo: a streaming dashboard that stays current. +CREATE VIEW device_stats AS +SELECT + d.id AS device_id, + d.device_name, + d.sensor_type, + d.site, + COUNT(*) AS readings_in_window, + AVG(r.value) AS avg_value, + MAX(r.value) AS max_value, + MIN(r.value) AS min_value +FROM devices d +JOIN readings r ON r.device_id = d.id +GROUP BY d.id, d.device_name, d.sensor_type, d.site; + +-- Alerts: readings exceeding their device's threshold. A continuously +-- maintained, joinable, streaming alert table. +CREATE MATERIALIZED VIEW alerts AS +SELECT + r.observed_at, + r.device_id, + d.device_name, + d.sensor_type, + d.site, + r.value, + d.threshold, + r.value - d.threshold AS overshoot +FROM readings r +JOIN devices d ON d.id = r.device_id +WHERE r.value > d.threshold; + +-- Per-site alert load. Useful for "which site is hot right now" dashboards. +CREATE VIEW site_alert_load AS +SELECT + site, + COUNT(*) AS active_alerts, + MAX(overshoot) AS worst_overshoot +FROM alerts +GROUP BY site; + +-- ----------------------------------------------------------------------------- +-- Validation: +-- +-- Heartbeat: +-- COPY (SUBSCRIBE (SELECT COUNT(*) FROM readings) WITH (progress = true)) TO STDOUT; +-- +-- Top noisy sites right now: +-- SELECT * FROM site_alert_load ORDER BY active_alerts DESC LIMIT 5; +-- +-- Invariant: every alert is reachable from a reading (FK integrity). 0 expected: +-- SELECT COUNT(*) FROM alerts a +-- LEFT JOIN readings r ON r.observed_at = a.observed_at AND r.device_id = a.device_id +-- WHERE r.device_id IS NULL; +-- ----------------------------------------------------------------------------- diff --git a/misc/demo-data/assets/domains/zoo.sql b/misc/demo-data/assets/domains/zoo.sql new file mode 100644 index 0000000000000..a3a27535fae10 --- /dev/null +++ b/misc/demo-data/assets/domains/zoo.sql @@ -0,0 +1,411 @@ +-- Copyright Materialize, Inc. and contributors. All rights reserved. +-- +-- Use of this software is governed by the Business Source License +-- included in the LICENSE file at the root of this repository. +-- +-- As of the Change Date specified in that file, in accordance with +-- the Business Source License, use of this software will be governed +-- by the Apache License, Version 2.0. + +-- ============================================================================= +-- Zoo of Disappointing Animals (a front for mostly crime) +-- +-- Two layers, one schema: +-- Front of house: visits, ratings of 64 disappointing animals. +-- Back of house: shipments with consistent skim, "consultations" that only +-- happen on skeptical-mood visits. +-- +-- The correlation between disappointment and crime is baked in: skeptical +-- guests rate harsher AND trigger consultations, so hours with high average +-- disappointment also have higher envelope totals. The cover IS the tell. +-- +-- Invariants: +-- 1. COUNT(ratings) per visit = visits.n_animals_seen (fanout) +-- 2. SUM(shipments.declared - actual) is monotone non-positive (skim) +-- 3. consultations.visit_id ⊆ visits where mood = 'skeptical' (subset) +-- 4. consultations.client_id is always a real visitor (re-derivation) +-- +-- Prerequisites: scaffold.sql, common/people.sql +-- ============================================================================= + +SELECT EXISTS (SELECT 1 FROM mz_views WHERE name = 'visits_core') AS already_loaded \gset +\if :already_loaded +\echo zoo already loaded; skipping. +\else + +-- ----------------------------------------------------------------------------- +-- Static lookups +-- ----------------------------------------------------------------------------- + +CREATE VIEW moods (id, name) AS VALUES + (0, 'hopeful'), + (1, 'skeptical'), + (2, 'dragged_along'), + (3, 'field_trip'); + +CREATE VIEW crate_labels (id, name) AS VALUES + (0, 'fresh hay'), + (1, 'veterinary supplies'), + (2, 'enrichment toys'), + (3, 'do not open'); + +CREATE VIEW service_codes (id, name) AS VALUES + (0, 'tax advisory'), + (1, 'logistics'), + (2, 'asset relocation'), + (3, 'reputation management'), + (4, 'witness preparation'), + (5, 'inventory adjustment'), + (6, 'jurisdictional planning'), + (7, 'unspecified consulting'); + +-- Species roster (the brochure version). +CREATE VIEW species (id, name, expected_disappointment) AS VALUES + (0, 'Capybara', 2), + (1, 'Sleeping Lion', 5), + (2, 'Pigeon (Allegedly Eagle)', 5), + (3, 'Sloth', 3), + (4, 'Empty Enclosure', 5), + (5, 'Goat', 2), + (6, 'Tiger (Painted Mule)', 5), + (7, 'Lizard in a Jar', 4), + (8, 'Bear (Costume)', 4), + (9, 'Penguin (Plastic)', 5), + (10, 'Rabbit', 1), + (11, 'Snake (Garden Hose)', 5); + +-- 64 animals, each with a name, species, and a specific letdown reason. +-- animal_id is 0..63 so one byte mod 64 picks uniformly. +CREATE VIEW animals AS +SELECT + id::int AS id, + 'ANI-' || lpad(id::text, 3, '0') AS tag, + (ARRAY[ + 'Bartholomew','Mister Whiskers','Doctor Paws','Sir Naps-a-Lot', + 'Greg','The Disappointment','Beatrice','Captain Snore', + 'Marquis de Lounge','Pebbles','Twiggy','Mayor McMolt', + 'Lord Featherbottom','Janet','Cinder','Pudding', + 'Stumpy','Princess Vacant','Reginald','The Smell', + 'Ottoman','Doorstop','Ms. Hibernation','Mister Allegedly', + 'The Glistener','Frank','Fenestra','Dim Bulb', + 'Sergeant Nope','Roy','Crouton','The Refunder', + 'Hazelnut','Hugo','Mavis','Crinkle', + 'The Audit','Snickers','Phyllis','Buttercup', + 'Vince','Mister Eyes','Lump','The Suggestion', + 'Pretzel','Boris','Calamity','Tuffet', + 'Dr. Disappointment','The Husk','Marbles','The Receipt', + 'Twitch','The Bystander','Norbert','Cricket', + 'The Witness','Edna','Lasagna','The Defendant', + 'Pinto','Whimsy','The Inheritance','The Final Straw' + ])[1 + id] AS name, + mod(get_byte(digest('species:' || id::text, 'md5'), 0)::int, 12) AS species_id, + (ARRAY[ + 'sleeps 22 hours a day', + 'is actually a rock with eyes glued on', + 'smaller than the brochure photo', + 'technically a pigeon', + 'enclosure is just a mirror', + 'has not moved since 2019', + 'turns out to be a costume', + 'enclosure card reads "TBD"', + 'visible only on tuesdays', + 'painted to look like another animal', + 'is a taxidermy with a fan blowing on it', + 'smells worse than expected', + 'enclosure contains only a sign reading SOON', + 'absent due to "training"', + 'considerably damper than promised' + ])[1 + mod(get_byte(digest('reason:' || id::text, 'md5'), 0)::int, 15)] AS letdown_reason +FROM generate_series(0, 63) AS id; + +CREATE DEFAULT INDEX ON animals; + +-- ----------------------------------------------------------------------------- +-- Visits: one per moment. +-- +-- Byte budget on `random`: +-- [0..2] visit_id +-- [3] visitor_id mod 256 (FK to people) +-- [4] mood mod 4 +-- [5] n_animals_seen = (byte mod 5) + 1 +-- [6] arrival offset in seconds +-- [7] consultation count seed (mod 3, gated by mood = skeptical) +-- ----------------------------------------------------------------------------- +CREATE VIEW visits_core AS +SELECT + moment, + random, + get_byte(random, 0) + + get_byte(random, 1) * 256 + + get_byte(random, 2) * 65536 AS id, + get_byte(random, 3) AS visitor_id, + mod(get_byte(random, 4)::int, 4) AS mood_id, + 1 + mod(get_byte(random, 5)::int, 5) AS n_animals_seen, + moment + (get_byte(random, 6)::text || ' seconds')::interval AS arrived_at, + mod(get_byte(random, 7)::int, 3) AS consultation_seed +FROM random; + +CREATE MATERIALIZED VIEW visits AS +SELECT + v.id, + v.visitor_id, + p.name AS visitor_name, + p.region AS visitor_region, + m.name AS mood, + v.n_animals_seen, + v.arrived_at, + -- Consultations only happen when the guest is "skeptical". + CASE WHEN m.name = 'skeptical' THEN v.consultation_seed ELSE 0 END + AS n_consultations +FROM visits_core v +JOIN people p ON p.id = v.visitor_id +JOIN moods m ON m.id = v.mood_id; + +-- ----------------------------------------------------------------------------- +-- Ratings: one row per (visit, animal seen). Re-hash random with the rating +-- index. Skeptical guests rate +2 stars harsher (clamped to 5). That single +-- nudge produces the correlation between front-of-house disappointment and +-- back-of-house activity. +-- +-- Child random bytes: +-- [0] animal_id mod 64 +-- [1] base stars = (byte mod 5) + 1; bumped to +2 if parent mood skeptical +-- [2] viewed_at offset (seconds after arrival) +-- ----------------------------------------------------------------------------- +CREATE MATERIALIZED VIEW ratings AS +WITH expanded AS ( + SELECT + v.id AS visit_id, + v.arrived_at, + v.mood_id, + generate_series(1, v.n_animals_seen) AS rating_no, + digest(v.random::text || 'rating' || + generate_series(1, v.n_animals_seen)::text, 'md5') AS random + FROM visits_core v +) +SELECT + visit_id, + rating_no, + mod(get_byte(random, 0)::int, 64) AS animal_id, + LEAST( + 5, + 1 + mod(get_byte(random, 1)::int, 5) + + CASE WHEN mood_id = 1 THEN 2 ELSE 0 END + ) AS stars, + arrived_at + (get_byte(random, 2)::text || ' seconds')::interval + AS viewed_at +FROM expanded; + +-- ----------------------------------------------------------------------------- +-- Shipments: one per moment, re-hashing `random || 'shipment'`. Independent +-- entropy from visits so the two streams are not trivially correlated. +-- +-- Byte budget: +-- [0..2] shipment_id +-- [3] handler_id mod 256 (FK to people) +-- [4] crate_label mod 4 +-- [5] declared_weight = byte + 50 (50..305 kg) +-- [6] skim = byte mod 8 (subtracted, always) +-- ----------------------------------------------------------------------------- +CREATE VIEW shipments_core AS +SELECT + moment, + digest(random::text || 'shipment', 'md5') AS random +FROM random; + +CREATE MATERIALIZED VIEW shipments AS +SELECT + moment AS arrived_at, + get_byte(random, 0) + + get_byte(random, 1) * 256 + + get_byte(random, 2) * 65536 AS id, + get_byte(random, 3) AS handler_id, + p.name AS handler_name, + cl.name AS crate_label, + (get_byte(random, 5) + 50)::numeric AS declared_weight_kg, + ((get_byte(random, 5) + 50) - mod(get_byte(random, 6)::int, 8))::numeric + AS actual_weight_kg, + mod(get_byte(random, 6)::int, 8)::numeric AS skim_kg +FROM shipments_core +JOIN people p ON p.id = get_byte(random, 3) +JOIN crate_labels cl ON cl.id = mod(get_byte(random, 4)::int, 4); + +-- ----------------------------------------------------------------------------- +-- Consultations: 0..2 child rows per visit, gated by mood = skeptical. +-- client_id is CARRIED from the parent visit (not re-rolled) — that's how +-- "the guest meeting" maps to "the visitor in the cover story". +-- +-- Child random bytes: +-- [0..2] consultation_id +-- [4] service_code mod 8 +-- [5] envelope_mm = byte (0..255) +-- ----------------------------------------------------------------------------- +CREATE MATERIALIZED VIEW consultations AS +WITH gated AS ( + SELECT + v.id AS visit_id, + v.visitor_id AS client_id, + v.arrived_at, + v.consultation_seed, + v.random + FROM visits_core v + JOIN moods m ON m.id = v.mood_id + WHERE m.name = 'skeptical' + AND v.consultation_seed > 0 +), +expanded AS ( + SELECT + visit_id, + client_id, + arrived_at, + generate_series(1, consultation_seed) AS meeting_no, + digest(random::text || 'consult' || + generate_series(1, consultation_seed)::text, 'md5') AS random + FROM gated +) +SELECT + e.visit_id, + e.meeting_no, + get_byte(e.random, 0) + + get_byte(e.random, 1) * 256 + + get_byte(e.random, 2) * 65536 AS id, + e.client_id, + p.name AS client_name, + sc.name AS service_code, + get_byte(e.random, 5)::int AS envelope_mm, + e.arrived_at + (e.meeting_no * 7 || ' minutes')::interval AS met_at +FROM expanded e +JOIN people p ON p.id = e.client_id +JOIN service_codes sc ON sc.id = mod(get_byte(e.random, 4)::int, 8); + +-- ----------------------------------------------------------------------------- +-- Aggregates: the demos. +-- ----------------------------------------------------------------------------- + +-- The headline join. Per-hour: how disappointing was the cover, and how busy +-- was the back of house? High avg_disappointment should track high envelope +-- totals, because both ride the skeptical-mood bytes. +CREATE VIEW cover_quality AS +WITH visit_disappointment AS ( + SELECT v.id AS visit_id, v.arrived_at, AVG(r.stars)::numeric(10,2) AS avg_stars + FROM visits v + JOIN ratings r ON r.visit_id = v.id + GROUP BY v.id, v.arrived_at +), +hourly_visits AS ( + SELECT + date_trunc('hour', arrived_at) AS hour, + COUNT(*) AS visits, + AVG(avg_stars)::numeric(10,2) AS avg_disappointment + FROM visit_disappointment + GROUP BY date_trunc('hour', arrived_at) +), +hourly_shipments AS ( + SELECT + date_trunc('hour', arrived_at) AS hour, + SUM(skim_kg) AS skim_kg + FROM shipments + GROUP BY date_trunc('hour', arrived_at) +), +hourly_consults AS ( + SELECT + date_trunc('hour', met_at) AS hour, + COUNT(*) AS consultations, + SUM(envelope_mm) AS envelope_mm_total + FROM consultations + GROUP BY date_trunc('hour', met_at) +) +SELECT + v.hour, + v.visits, + v.avg_disappointment, + COALESCE(s.skim_kg, 0) AS skim_kg, + COALESCE(c.consultations, 0) AS consultations, + COALESCE(c.envelope_mm_total, 0) AS envelope_mm_total +FROM hourly_visits v +LEFT JOIN hourly_shipments s ON s.hour = v.hour +LEFT JOIN hourly_consults c ON c.hour = v.hour; + +-- Which animals attract the worst reviews? (The leaderboard of letdowns.) +CREATE VIEW animal_disappointment AS +SELECT + a.tag, + a.name, + sp.name AS species, + a.letdown_reason, + COUNT(*) AS sightings, + AVG(r.stars)::numeric(10,2) AS avg_stars +FROM ratings r +JOIN animals a ON a.id = r.animal_id +JOIN species sp ON sp.id = a.species_id +GROUP BY a.tag, a.name, sp.name, a.letdown_reason; + +-- Which services run hottest, and under how much cover? +CREATE VIEW business_efficiency AS +WITH visit_cover AS ( + SELECT v.id AS visit_id, AVG(r.stars)::numeric(10,2) AS cover_score + FROM visits v JOIN ratings r ON r.visit_id = v.id + GROUP BY v.id +) +SELECT + c.service_code, + COUNT(*) AS meetings, + AVG(c.envelope_mm)::numeric(10,2) AS avg_envelope_mm, + SUM(c.envelope_mm) AS total_envelope_mm, + AVG(vc.cover_score)::numeric(10,2) AS avg_cover_disappointment +FROM consultations c +JOIN visit_cover vc ON vc.visit_id = c.visit_id +GROUP BY c.service_code; + +-- Cross-domain hook: per-person rap sheet. Joins client_id back through +-- people for use with banking / ecommerce demos. +CREATE VIEW client_activity AS +SELECT + p.id AS person_id, + p.name AS person_name, + p.region, + COUNT(*) AS consultations, + SUM(c.envelope_mm) AS envelope_mm_total +FROM consultations c +JOIN people p ON p.id = c.client_id +GROUP BY p.id, p.name, p.region; + +\endif + +-- ----------------------------------------------------------------------------- +-- Validation queries: +-- +-- Heartbeat: +-- COPY (SUBSCRIBE (SELECT COUNT(*) FROM visits) WITH (progress = true)) TO STDOUT; +-- +-- Invariant 1: total ratings = total declared animals seen (aggregate fanout). +-- SELECT (SELECT COUNT(*) FROM ratings) = +-- (SELECT SUM(n_animals_seen) FROM visits) AS fanout_balances; +-- +-- Invariant 2: skim is monotone non-positive in aggregate. +-- SELECT SUM(declared_weight_kg - actual_weight_kg) AS total_skim_kg FROM shipments; +-- -- should always be >= 0 (we're skimming, not adding) +-- +-- Invariant 3: count of consultations equals total declared meetings from +-- skeptical visits. Aggregate form — survives 24-bit id collisions. +-- (The per-row JOIN version reports false positives when two visits share an +-- id and one is skeptical and the other isn't; see SKILL.md "Invariants vs. +-- id collisions".) +-- SELECT (SELECT COUNT(*) FROM consultations) = +-- (SELECT SUM(CASE WHEN m.name = 'skeptical' +-- THEN mod(get_byte(v.random, 7)::int, 3) ELSE 0 END) +-- FROM visits_core v JOIN moods m ON m.id = v.mood_id) AS consultations_match; +-- +-- Demo: hours where the cover ran best AND the back of house was busy. +-- SELECT * FROM cover_quality ORDER BY hour DESC LIMIT 20; +-- +-- Demo: the leaderboard of letdowns. +-- SELECT * FROM animal_disappointment ORDER BY avg_stars DESC, sightings DESC LIMIT 10; +-- +-- Demo (with banking loaded): people who are both clients AND have large +-- round-number transactions. +-- SELECT ca.person_name, ca.envelope_mm_total, ab.balance +-- FROM client_activity ca +-- JOIN account_balances ab ON ab.holder_name = ca.person_name +-- ORDER BY ca.envelope_mm_total DESC LIMIT 20; +-- ----------------------------------------------------------------------------- diff --git a/misc/demo-data/assets/scaffold.sql b/misc/demo-data/assets/scaffold.sql new file mode 100644 index 0000000000000..dd3d75b0f1ecb --- /dev/null +++ b/misc/demo-data/assets/scaffold.sql @@ -0,0 +1,113 @@ +-- Copyright Materialize, Inc. and contributors. All rights reserved. +-- +-- Use of this software is governed by the Business Source License +-- included in the LICENSE file at the root of this repository. +-- +-- As of the Change Date specified in that file, in accordance with +-- the Business Source License, use of this software will be governed +-- by the Apache License, Version 2.0. + +-- ============================================================================= +-- Moments scaffold: a sliding window of timestamps you can hash for entropy. +-- +-- This file is stable boilerplate. Domains build on top of `moments` and +-- `random`; they should never need to edit anything here. +-- +-- The trick: build a 130-year `generate_series` of years, but filter so only +-- the years near `mz_now()` survive. Cascade through days/hours/minutes/seconds. +-- The `UNION ALL SELECT * FROM empty` at each level blocks the optimizer from +-- inlining and pre-computing the whole timeline. +-- +-- Load with: \i scaffold.sql +-- ============================================================================= + +-- Knobs. Override before \i, e.g.: +-- \set retention '6 hours' +-- \i scaffold.sql +\if :{?retention} \else \set retention '1 day' \endif +\if :{?tick} \else \set tick '1 second' \endif + +\echo Scaffold: retention=:retention, tick=:tick + +-- Idempotency: if `empty` already exists in the current database, assume +-- the scaffold is loaded and skip. Run teardown.sql first to rebuild. +SELECT EXISTS (SELECT 1 FROM mz_tables WHERE name = 'empty') AS already_loaded \gset +\if :already_loaded +\echo Scaffold already loaded; skipping. (Run teardown.sql first to rebuild.) +\else + +CREATE TABLE empty (e TIMESTAMP); + +-- Each year-long interval of interest. +CREATE VIEW years AS +SELECT * +FROM generate_series( + '1970-01-01 00:00:00+00', + '2099-01-01 00:00:00+00', + '1 year') year +WHERE mz_now() BETWEEN year AND year + '1 year' + '1 day'; + +-- Each day-long interval of interest. +CREATE VIEW days AS +SELECT * FROM ( + SELECT generate_series(year, year + '1 year' - '1 day'::interval, '1 day') AS day + FROM years + UNION ALL SELECT * FROM empty +) +WHERE mz_now() BETWEEN day AND day + '1 day' + '1 day'; + +-- Each hour-long interval of interest. +CREATE VIEW hours AS +SELECT * FROM ( + SELECT generate_series(day, day + '1 day' - '1 hour'::interval, '1 hour') AS hour + FROM days + UNION ALL SELECT * FROM empty +) +WHERE mz_now() BETWEEN hour AND hour + '1 hour' + '1 day'; + +-- Each minute-long interval of interest. +CREATE VIEW minutes AS +SELECT * FROM ( + SELECT generate_series(hour, hour + '1 hour' - '1 minute'::interval, '1 minute') AS minute + FROM hours + UNION ALL SELECT * FROM empty +) +WHERE mz_now() BETWEEN minute AND minute + '1 minute' + '1 day'; + +-- Each second-long interval of interest. +CREATE VIEW seconds AS +SELECT * FROM ( + SELECT generate_series(minute, minute + '1 minute' - '1 second'::interval, '1 second') AS second + FROM minutes + UNION ALL SELECT * FROM empty +) +WHERE mz_now() BETWEEN second AND second + '1 second' + '1 day'; + +-- Indexes in order. Each level depends on the prior being indexed so the +-- expansion fires incrementally rather than re-scanning the cascade. +CREATE DEFAULT INDEX ON years; +CREATE DEFAULT INDEX ON days; +CREATE DEFAULT INDEX ON hours; +CREATE DEFAULT INDEX ON minutes; +CREATE DEFAULT INDEX ON seconds; + +-- Public surface #1: a sliding-window stream of timestamps. +-- Cardinality = retention / tick. Defaults give 86,400 rows (24h of seconds). +-- The `mod(...)` clause thins to the requested tick rate. Tick must be a +-- whole number of seconds. +CREATE VIEW moments AS +SELECT second AS moment FROM seconds +WHERE mz_now() >= second + AND mz_now() < second + :'retention'::interval + AND mod(EXTRACT(EPOCH FROM second)::bigint, + EXTRACT(EPOCH FROM :'tick'::interval)::bigint) = 0; + +-- Public surface #2: deterministic pseudorandom bytes per moment. +-- Use `get_byte(random, N)` in domains to pull 0..255 values for fields, +-- foreign keys, distributions. Re-hashing `random || something` lets a row +-- spawn many child rows that stay stable across re-derivation. +CREATE VIEW random AS +SELECT moment, digest(moment::text, 'md5') AS random +FROM moments; + +\endif diff --git a/misc/demo-data/assets/teardown.sql b/misc/demo-data/assets/teardown.sql new file mode 100644 index 0000000000000..4da2f23f2281d --- /dev/null +++ b/misc/demo-data/assets/teardown.sql @@ -0,0 +1,24 @@ +-- Copyright Materialize, Inc. and contributors. All rights reserved. +-- +-- Use of this software is governed by the Business Source License +-- included in the LICENSE file at the root of this repository. +-- +-- As of the Change Date specified in that file, in accordance with +-- the Business Source License, use of this software will be governed +-- by the Apache License, Version 2.0. + +-- Drop everything created by the scaffold and any loaded domains. +-- Idempotent: safe to run repeatedly, safe to run when only some domains +-- were loaded. Order matters: drop `empty CASCADE` first to kill the +-- moments chain and everything derived from it, then drop the static +-- lookup tables that don't transitively depend on `empty`. + +-- Scaffold + everything reachable from a moment. +DROP TABLE IF EXISTS empty CASCADE; + +-- Static lookups (have no dep on `empty`, must be dropped explicitly). +DROP VIEW IF EXISTS people CASCADE; -- common +DROP VIEW IF EXISTS items CASCADE; -- auctions +DROP VIEW IF EXISTS products CASCADE; -- ecommerce +DROP VIEW IF EXISTS accounts CASCADE; -- banking +DROP VIEW IF EXISTS devices CASCADE; -- iot