diff --git a/.claude/rules/archetypes.md b/.claude/rules/archetypes.md new file mode 100644 index 00000000..0a38bf46 --- /dev/null +++ b/.claude/rules/archetypes.md @@ -0,0 +1,60 @@ +--- +description: Authoring rules for ECS systems that iterate archetype rows. +globs: "**/*.ts" +--- + +# Archetypes — iteration rules + +## Express selection in the query, not the loop + +`queryArchetypes(include, { exclude })` accepts both required and excluded +component lists. Use them. Don't query a wider set and then skip rows or +archetypes with an `if`. + +```ts +// ❌ post-filter +for (const arch of db.store.queryArchetypes(["position", "rotation"])) { + if (arch.components.has("_worldMatrix")) continue; + ... +} + +// ✅ declarative +for (const arch of db.store.queryArchetypes( + ["position", "rotation"], + { exclude: ["_worldMatrix"] }, +)) { + ... +} +``` + +## When every row migrates out, iterate tail → head + +Archetypes are densely packed. Removing or migrating a row that isn't the +last one triggers a hole-fill: the tail row is moved into the gap. +Iterating `0 → rowCount-1` while migrating every row pays this cost on +every iteration. Iterating `rowCount-1 → 0` means each removal is from +the tail — no shift, indices ahead of the cursor stay valid. + +```ts +// ❌ shifts the tail into every hole, and the snapshot allocation is +// only there to survive the shifts. +const ids = [...]; +for (let i = 0; i < arch.rowCount; i++) ids[i] = arch.columns.id.get(i); +for (const id of ids) db.store.update(id, { _worldMatrix: Mat4x4.identity }); + +// ✅ no shifts, no allocation +for (let i = arch.rowCount - 1; i >= 0; i--) { + db.store.update(arch.columns.id.get(i), { _worldMatrix: Mat4x4.identity }); +} +``` + +If only *some* rows migrate (filter inside the loop), snapshot the ids +you'll touch — forward iteration is fine because rows you don't touch +stay put. + +## Don't snapshot what the query already filters + +A snapshot of "all entity ids in this archetype right now" is only +needed when forward iteration would invalidate. Reverse iteration +removes the need; an `exclude` clause removes the need to look at rows +that don't qualify in the first place. diff --git a/.claude/rules/namespace.md b/.claude/rules/namespace.md index 46b116a9..a00ad63c 100644 --- a/.claude/rules/namespace.md +++ b/.claude/rules/namespace.md @@ -48,3 +48,60 @@ as both type and namespace (`LogLevel.is(x)`, `LogLevel.values`). avoid cycling through `public.ts`. - Add `is` / `values` / per-member descriptors only when an external consumer actually needs them — not preemptively. + +## Multi-declaration files (when lumping is OK) + +Default stays one type / one helper per file. Two named exceptions: + +- **`-functions.ts`** — sibling functions operating on one owned + type. Lives in that type's folder. Use when functions are small enough + that adjacency in one editor pane beats one-file-per-function. +- **`-schema.ts`** — TypeScript projection of a borrowed data + format (file format, wire protocol, third-party API). Plain `export + interface` only — no namespace, no helpers. Lives next to the parser/ + emitter, not in a type folder. + +Never `*-types.ts`. "Types" is a meta-word the `.ts` extension already +implies; the name has to predict the contents. If you can't beat +`-types`, the file isn't a real lump — split per type, or inline at use +site. + +## Domain namespaces (for types you don't own) + +When utility functions operate on a platform or third-party type (e.g. +`GPUDevice`, `GPUTexture`), use a **domain namespace** instead: a folder +named after the concept, with a namespace export but *no type re-export*. + +``` +src/gpu/ + gpu.ts # `export * as GPU from "./public.js"` — no type alias + public.ts # re-exports every public helper + .ts # one declaration per file +``` + +```ts +// gpu/gpu.ts +export * as GPU from "./public.js"; + +// gpu/public.ts +export { createCubemap } from "./create-cubemap.js"; +export { cubeFaceView } from "./cube-face-view.js"; +``` + +Consumers import the namespace and get discoverability without shadowing +the platform type: + +```ts +import { GPU } from "@adobe/data-graphics"; +GPU.createCubemap(device, 256, "rgba16float"); +``` + +**Name by purpose, not by the external type.** `GPU`, not `GPUDevice`. + +**Tree-shaking still works**: bundlers (Rollup/Rolldown) trace static +property accesses (`GPU.createCubemap`) and exclude unused helpers. +Dynamic access (`GPU[name]`) defeats this — avoid it. + +**When NOT to use a domain namespace**: if a utility is only useful as +an internal implementation detail of a specific plugin (not callable by +consumers), keep it private to that plugin's folder instead. diff --git a/.claude/rules/plugin-modelling.md b/.claude/rules/plugin-modelling.md new file mode 100644 index 00000000..65e50970 --- /dev/null +++ b/.claude/rules/plugin-modelling.md @@ -0,0 +1,170 @@ +--- +paths: + - '**/*plugin*' + - '**/*plugin*/**' +--- + +# Plugin modelling — model vs. system, authored vs. derived + +Each plugin has two orthogonal axes: + +1. **Authored** state — the user inserts / sets it. Designed by the human + architect. +2. **Derived** state — a system produces it from other state. Implementation; + owned by the AI. + +The human's mental model of a project is the union of every plugin's +**authored surface**. Systems and derived state are how the world updates +and renders — read their interface contract, not their innards. + +## Two tiers of model plugins + +- **Core** — the primary record types every consumer reads. Small, fixed, + shared. +- **Authoring abstractions** — higher-level intents (orbit, animation, + procedural shape, particle emitter, constraint, …) that systems expand + into core state. Open-ended; each project picks what it needs. + +A plugin almost never holds both authored and derived data the user cares +about. If it appears to, the derived data is the *implementation* of the +authoring abstraction. + +## File naming + +Each file whose primary export is a `Database.Plugin` (created or +combined) ends in `-plugin.ts` — e.g. `node-plugin.ts`, `model-plugin.ts`, +`pbr-core-plugin.ts`. The suffix is for discovery: cursor rules, +codebase grep, and "find all plugins" tooling match `**/*-plugin.ts` +without inspecting contents. + +The exported constant's name depends on whether the folder also hosts a +type namespace: + +- **Pure-plugin folder** (no owned type) → export the concept name + itself, lowercase. `node-plugin.ts` exports `node`. Import sites read + `import { node } from "..."; Database.Plugin.combine(node, ...)`. +- **Type + plugin folder** (folder hosts both an owned type and its + plugin) → the plugin file exports `plugin`, re-exported through the + type's namespace. `camera-plugin.ts` exports `plugin`; `camera/public.ts` + re-exports it; consumers reach it as `Camera.plugin`. One import gives + both the type and the plugin. + +Files with helper exports, value types, schemas, or shaders do **not** +take the suffix — only plugin files. + +## Resources and archetypes as types + +Prefer **one resource per plugin**, of an owned type that the folder also +hosts. `light/` has one `light` resource of type `Light`; `orbit/` has +one `orbit` resource of type `Orbit`. Splitting a single coherent +concept into siblings (`lightDirection`, `lightColor`, `ambientStrength`) +duplicates the concept's identity at every read site. + +For each **archetype** the plugin declares, add a TypeScript type with +the same name describing one row's authored shape. `Node` is the type +for the `Node` archetype's row. The components stay separate per +typed-buffer-column convention; the type names the bundle so consumers +can declare `const node: Node` after a read. + +Both cases follow the same folder shape: + +``` +/ + .ts # type + namespace + public.ts # re-exports plugin (and any helpers) + -plugin.ts # exports `plugin` +``` + +Consumers reach the plugin as `Concept.plugin` and the type as +`Concept`. Pure-plugin folders (no owned type) still export their +plugin under a lowercase concept name — see "File naming" below. + +**Exceptions:** +- Plugins whose resources have genuinely independent lifecycles + (`graphics` — `device`, `canvas`, `commandEncoder`, each set at + different times) keep them as separate resources. Consolidating + would force any one write to replace the whole struct. +- Ephemeral implementation archetypes (e.g. `_PbrPrimitive`, + `_VisibleMaterial`) don't need a value type — consumers iterate + them by archetype, never construct one. + +## Shared conventions + +- **Uppercase identifier** = archetype. **Lowercase** = component or + resource. Disambiguated by where declared (archetype vs. component vs. + resource section of the data plugin). +- **Entity references** are typed `EntityId`; field names drop the `Ref` + suffix — the type carries the signal. +- **Implementation prefixes** (`pbr`, `ibl`, …) belong in code where they + disambiguate cross-plugin reuse. Drop them in the conceptual view. +- **`_` prefix** = ephemeral / derived / not part of the data model. + Applies to **components, archetypes, and resources** in code — the + prefix makes "this is system-owned, not user-authored" visible at every + read site. The human ignores it when modelling; system-graph views + still display it. Does **not** apply to plugin names, system names, or + transactions — those are grouped by feature folder, and the folder + already communicates whether a plugin is authored surface or + implementation. +- **Inline `: Type`** only on `_`-prefixed names. Persistent items get + their type from the data plugin where they're declared. +- **Components use JSON schemas**, resources use `{ default: X as T }`. + Schemas (e.g. `Entity.schema`, `Mat4x4.schema`, `Boolean.schema`, + `{ type: "string" }`) enable typed-buffer column storage — the right + choice when a value is stored once per entity. Resources are a single + slot per database, so the runtime cost of typed storage isn't worth + it; the default-pattern is the convention there. Use the default + pattern on components *only* for runtime-only objects (GPU buffers, + bind groups, closures, complex JS objects with no schema). + +## Model plugin format + +Authored surface only — no derived state, no systems. + +``` +pluginName + components + fieldName: Type + ... + resources + fieldName: Type + ... + archetypes + ArchetypeName: [field, field, ...OtherArchetype] +``` + +- `...Other` composes another archetype's component list. +- Shape comments (`// { … }`) stay on the same line; do not wrap. + +## System plugin format + +``` +systemName // high-level summary + query: ArchetypeExpr [, ArchetypeExpr ...] // omit if resource-only + read: + name // persistent — type from data plugin + _name: Type // ephemeral — type inline + write: + name // writes a component / resource + _Archetype // creates new entities of this archetype + // free-text side effect // draw calls, network sends, … +``` + +- `read:` and `write:` are outlined one-per-line. +- A system can have multiple independent queries; comma-separate them on + the `query:` line. + +## Query DSL + +- `Archetype` — entities matching the archetype. +- `+component` — require this additional component. +- `-component` — exclude entities that have it. +- `Archetype+a-b+c` — left-to-right chain of `+` / `-`. +- `Q1, Q2, ...` — multiple independent queries the system reads from. + +## Designing or discussing a plugin + +1. Write the authored surface (model format) **first**. +2. Then each system as its own interface card with `query` / `read` / + `write`. +3. The graph of `write:` → `read:` edges is the implementation; the union + of authored surfaces is the human's mental model. diff --git a/.claude/rules/type-casts.md b/.claude/rules/type-casts.md index 427eea2a..168ae90f 100644 --- a/.claude/rules/type-casts.md +++ b/.claude/rules/type-casts.md @@ -4,28 +4,81 @@ paths: - 'packages/**/*.tsx' --- -# Type casts — never to silence a problem +# Type casts — only two valid uses -`as` exists to assert a runtime invariant the type system cannot infer -but you have proven (post-validation narrowing, branded types, -third-party declarations that are imprecisely typed upstream). It is -not a way to make a type error go away. +`as` is dangerous. It silently turns off the type checker for the cast +expression. Every cast is a place a future refactor can break in +silence. There are exactly two valid uses: -If a cast is the only thing making code compile, the type system is -trying to tell you something. The fix lives at the source of the lying -type — almost always a declaration *you* control — not at the -consumer. +1. **Proving a type the compiler cannot see.** A runtime invariant you + have established but TypeScript has no way to verify — e.g. + "validated JSON matches `User.schema`", "this `unknown` from a loose + API is `number` because the schema declared it", "this DOM node is + `HTMLInputElement` because the selector matched `input`". -## Example +2. **Widening a literal or default to a useful container type.** A + resource or component default is `0` (the literal) without help. + `0 as F32` widens it to `F32` so the resource can hold any number, + not just zero. Same for `[] as User[]`, `false as boolean`, + `{} as State`. Without the cast, the inferred type is so narrow it + blocks every later assignment. + +Anything else is the cast hiding a real defect. Most often: an identity +cast that neither narrows nor widens, used to make a type error go +away without fixing the underlying type. + +## The acceptance test + +Before every `as`, articulate in one sentence which of the two it is: + +- **Case 1:** "I know `` is `` because ." +- **Case 2:** "I'm widening `` so the container can hold any ``." + +If your sentence reduces to "TypeScript is wrong" or "I just want this +to compile", the cast is hiding a real defect — fix the type at its +source, do not cast at the consumer. + +## Common violations + +### Identity casts (neither widening nor narrowing) ```ts -// packages/event-bus/src/emit.ts (your code) +// ❌ `cameraAngle + delta` is already F32 (= number). Cast does nothing. +t.resources.cameraAngle = (t.resources.cameraAngle + delta) as F32; + +// ✅ +t.resources.cameraAngle = t.resources.cameraAngle + delta; +``` + +### Casting `any` to a specific shape just to call a property + +```ts +// ❌ `prev` is already any; the cast adds no information. +const n = (prev as ArrayLike).length; + +// ✅ +const n = prev.length; +``` + +### Casting wider than necessary + +```ts +// ❌ Loses the value-side type for no reason. +const map = t.componentSchemas as Record; + +// ✅ Same key widening, but values stay typed as Schema. +const map = t.componentSchemas as Record; +``` + +### Casting to silence an error you should fix at the source + +```ts +// declaration (your code) export const emit = (name: string, payload: unknown): void => { return inner.send(name, payload); // actually returns Promise }; -// at a consumer -// ❌ cast hides the lie; every caller will eventually duplicate it +// ❌ cast hides the lie; every caller will duplicate it const ack = await (emit("save", data) as Promise); // ✅ fix the declaration once; every consumer benefits @@ -34,17 +87,6 @@ export const emit = (name: string, payload: unknown): Promise => { }; ``` -## Acceptance test for `as` - -You may cast when you can articulate, in one sentence, **the runtime -invariant the type system cannot see** — e.g. "the JSON was just -validated against `User.schema`", "this brand was just enforced by -`brandUser()`", "this DOM node is `HTMLInputElement` because the -selector matched `input`". - -If your sentence reduces to "TypeScript is wrong" or "I just want this -to compile", the cast is hiding a real defect. - ## When the lying type is in code you control Fix the declaration. A consumer-side cast is a leaky patch every other diff --git a/.claude/skills/think/SKILL.md b/.claude/skills/think/SKILL.md new file mode 100644 index 00000000..29feb79f --- /dev/null +++ b/.claude/skills/think/SKILL.md @@ -0,0 +1,47 @@ +--- +name: think +description: Reflective Thought Composition. Structured thinking pipeline for complex decisions, design evaluation, and deep analysis. Use when quality of reasoning matters more than speed of response. +--- + +# think + +Reflective Thought Composition (RTC) — a structured thinking pipeline that expands the thinking process of any model. Adapted from paralleldrive/aidd `aidd-rtc`. + +``` +fn think(input, options) { + show work: + 🎯 restate |> 💡 ideate |> 🪞 reflectSelfCritically |> + 🔭 expandOrthogonally |> ⚖️ scoreRankEvaluate |> 💬 respond +} +``` + +Commands { + /think [--compact] [--depth N] [prompt] Reflective Thought Composition — think deeply and critically over multiple reasoning paths prior to responding. +} + +Options { + --compact 🗜️🐘🤔💭 Compress thinking: SPR🧠 associative. Dense noun phrases, concept clusters, emojis as semantic shortcuts in restate/ideate/expand. Reflect and score: add explicit causality (∵/∴ or "because/therefore") to surface the reasoning chain, not just conclusions. Every internal stage: load-bearing tokens only, no filler. 💬Respond = full natural language, standalone, structured. + --depth -d [1..10] (default: 10) Response density. 1 = a few words per step, 10 = several bullet points per step. +} + +## The stages + +- 🎯 **restate** — restate the problem in your own words; surface the real goal and constraints. +- 💡 **ideate** — generate candidate approaches/answers, breadth over polish. +- 🪞 **reflectSelfCritically** — attack your own ideas; name failure modes, hidden assumptions, where each breaks. +- 🔭 **expandOrthogonally** — pull in adjacent angles the first pass missed (different axis, different layer, different stakeholder). +- ⚖️ **scoreRankEvaluate** — score/rank the candidates against the constraints with explicit causal reasoning (because → therefore), not bare verdicts. +- 💬 **respond** — the final answer in full natural language: standalone, structured, no reference to the internal stages. + +## When to use each option + +``` +(thinking itself is the goal — improving reasoning quality) => /think --compact +(communicating depth to the user is the goal) => /think --depth N +(both) => /think --compact --depth N +``` + +`--compact` is for internal reasoning passes where the output feeds another step, not directly to the user. Think deeply but compactly — every token earns its place. Switch to natural language at 💬 respond. + +**Pass:** remove any word → lose meaning. Reflect/score show explicit causal chain, not just conclusions. +**Fail:** consultant prose. Hedging. Filler. Conclusions without reasoning. Polish before the respond stage. diff --git a/.github/workflows/deploy-docs.yml b/.github/workflows/deploy-docs.yml index 127b52b4..a42f276a 100644 --- a/.github/workflows/deploy-docs.yml +++ b/.github/workflows/deploy-docs.yml @@ -40,12 +40,20 @@ jobs: - run: pnpm --filter @adobe/data-lit build + - run: pnpm --filter @adobe/data-gpu build + - run: pnpm --filter data-p2p-tictactoe build env: CI: true - run: cp -r packages/data-p2p-tictactoe/dist packages/data/docs/p2p-tictactoe + - run: pnpm --filter data-gpu-samples build + env: + CI: true + + - run: cp -r packages/data-gpu-samples/dist packages/data/docs/gpu-samples + - uses: actions/configure-pages@v5 with: enablement: true diff --git a/.gitignore b/.gitignore index db2de35a..bf63f497 100644 --- a/.gitignore +++ b/.gitignore @@ -48,3 +48,6 @@ source.txt docs/api packages/data/docs/api packages/data/references + +# Playwright MCP capture artifacts +.playwright-mcp/ diff --git a/.vscode/settings.json b/.vscode/settings.json index 8252fca4..4c70df1c 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -1,3 +1,4 @@ { - "typescript.tsdk": "${workspaceFolder}/node_modules/.pnpm/node_modules/typescript" + "typescript.tsdk": "node_modules/.pnpm/node_modules/typescript/lib", + "typescript.enablePromptUseWorkspaceTsdk": true } diff --git a/NEXT_STEPS.md b/NEXT_STEPS.md new file mode 100644 index 00000000..42f7f83c --- /dev/null +++ b/NEXT_STEPS.md @@ -0,0 +1,94 @@ +# Suggested next steps + +Items we've parked while pursuing other work. Not in priority order; +priorities shift with what we just shipped. + +## Tests + +We've built a lot without test coverage: animation sampling, schema- +driven interpolator dispatch, skinning matrix math, glTF parsing +(skin, animations, vertex packing), the orbit-camera plugin's +auto-fit. Useful targets: + +- `componentwiseLerp` — round-trips for Vec3, scalar, edge cases. +- `interpolate` — schema-declared interpolator dispatch (`Quat.slerp` + picked over default lerp). +- `sampleTrack` — keyframe bracket binary search at boundaries and + inside spans; step / linear modes. +- `parseGltfSkin` — joint-parent map for a glTF where joint order is + not topological. +- `parseGltfAnimations` — channels targeting non-joint nodes are + filtered out; jointIndex resolution. +- `pbrSkinningMatrixSystem` — `inverse(modelWorld) × jointWorld × IBM` + produces identity at bind pose. + +## Shadow mapping + +Biggest visual upgrade still missing from the IBL renderer. Standard +shape: + +1. Depth-only render pass from the light's POV → depth texture. +2. Fragment shader samples the shadow map with PCF. +3. New plugin `pbrShadow` (or extends `pbrIbl`) owns the depth pass, + the shadow map texture/sampler resources, and the extra bind group. + +Cascaded shadow maps for the directional light, omnidirectional cube +shadow maps for point lights — pick one first. + +## Multi-clip animation blending + +The animation player drives one clip. Real-world rigs blend walk→run +on a velocity parameter, or layer an upper-body action over a +locomotion base. Two reasonable extensions: + +- **List of weighted clips on the player.** Tracks from each clip are + sampled, then linearly combined per-target-component using the + player's per-clip weight. Works for crossfades and additive layers. +- **Animation state machine.** Higher level: nodes are clips, + transitions have durations and conditions. State machine is itself + an entity component; a system advances it. + +The first is a smaller step and unlocks crossfades. + +## CubicSpline interpolation + +The `interpolate` function throws on `cubicSpline`. glTF rarely uses +it but it's a real spec mode. Each keyframe stores +`[inTangent, value, outTangent]`; Hermite blend between adjacent +keyframes. Quaternion variant normalizes after the Hermite. Add an +`InterpolatorFn` signature that accepts the tangent data, register +it on `Quat.schema` for slerp-equivalent cubic. + +## Asset URL deduplication + +Two `insertGeometry({ pbrModelUrl: "fox.glb" })` calls fetch and +upload Fox twice. Add a `Map` in the loader +closure: on `insertGeometry`, if the URL is already known and that +geometry is still alive, reuse the entity id instead of inserting a +new one. Watch for the case where the original was deleted. + +## Skinned-mesh instancing + +Today each skinned Model gets its own draw call with its own skeleton +bind group. For crowds this is the bottleneck. Pack all instances' +joint matrices into one storage buffer keyed by +`instance_index × jointCount + jointIdx`; the skinned vertex shader +indexes into it using its instance_index. One draw call for N +identical-rigged characters. + +## Camera improvements + +- Orthographic/perspective switching at the camera resource level + (the type already has an `orthographic` lerp factor). +- `Camera.screenToWorldRay(x, y)` for picking. +- Pick-test plugin: cast a ray at click, return the first hit entity. + +## Smaller items + +- HDR tone mapping options (we hardcode ACES; Reinhard, Khronos + Neutral are alternatives). +- Per-material `alphaMode: "BLEND"` proper handling — currently + treated like OPAQUE. +- `setLight` accepts intensity scale separate from color. +- Move `useOrbitDragCamera` and `useOrbitCameraControl` out of samples + into a shared place if a non-sample consumer wants them. diff --git a/instanced-post-refactor.png b/instanced-post-refactor.png new file mode 100644 index 00000000..80e7b499 Binary files /dev/null and b/instanced-post-refactor.png differ diff --git a/package.json b/package.json index 577205ae..b1e7d500 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "data-monorepo", - "version": "0.9.68", + "version": "0.9.69", "private": true, "scripts": { "build": "pnpm -r run build", @@ -10,8 +10,9 @@ "typecheck": "pnpm -r run typecheck", "dev": "pnpm -r --parallel run dev", "dev:data": "pnpm --filter @adobe/data run dev", + "dev-gpu": "pnpm --parallel --filter @adobe/data --filter @adobe/data-gpu --filter data-gpu-samples run dev", "link": "pnpm -r --filter @adobe/data* run link", - "publish": "sh -c 'for x in \"$@\"; do OTP=\"$x\"; done; export NPM_CONFIG_OTP=\"$OTP\"; pnpm -r --filter @adobe/data --filter @adobe/data-react --filter @adobe/data-lit --filter @adobe/data-solid run publish-public' sh", + "publish": "sh -c 'for x in \"$@\"; do OTP=\"$x\"; done; export NPM_CONFIG_OTP=\"$OTP\"; pnpm -r --filter @adobe/data --filter @adobe/data-react --filter @adobe/data-lit --filter @adobe/data-solid --filter @adobe/data-gpu run publish-public' sh", "bump": "pnpm version patch --no-git-tag-version && V=$(node -p \"require('$PWD/package.json').version\") && pnpm -r exec pnpm version $V --no-git-tag-version --allow-same-version", "release": "pnpm bump && pnpm publish", "bp": "pnpm bump && pnpm run publish" diff --git a/packages/data-gpu-samples/.gitignore b/packages/data-gpu-samples/.gitignore new file mode 100644 index 00000000..7af0e76a --- /dev/null +++ b/packages/data-gpu-samples/.gitignore @@ -0,0 +1,2 @@ +public/models/ +public/env/ diff --git a/packages/data-gpu-samples/CLAUDE.md b/packages/data-gpu-samples/CLAUDE.md new file mode 100644 index 00000000..3c56b483 --- /dev/null +++ b/packages/data-gpu-samples/CLAUDE.md @@ -0,0 +1,49 @@ +# CLAUDE.md — data-gpu-samples + +## Never commit binary assets to this repo + +Model files (`.glb`, `.gltf`, `.bin`), HDR environment maps, textures, and +audio belong on public CDNs, not in git. The repo's `.gitignore` excludes +`public/models/` and `public/env/`. Do not work around the ignore by +checking files in elsewhere or by adding them to a `dist/` folder. + +The repo's git history has been rewritten once already to remove +accidentally-committed binaries; doing so again is expensive and error- +prone. Catch it before the commit. + +## Where to source assets + +When a sample needs a model or environment map, link to a public source +that hosts the file under a permissive license. Two reliable options: + +- **glTF models** — Khronos Sample Assets repo, raw GitHub: + ``` + https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Assets/main/Models//glTF-Binary/.glb + ``` + Examples already in use: `DamagedHelmet`, `MetalRoughSpheres`, + `AntiqueCamera`, `Fox`. License: see the model's own `LICENSE.md` in + the Khronos repo (most are CC-BY 4.0). + +- **HDR environment maps** — Poly Haven: + ``` + https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/.hdr + ``` + Examples in use: `studio_small_09_1k`, `venice_sunset_1k`, + `kloppenheim_02_1k`. License: CC0. + +Use the `1k` HDR variant unless a sample is specifically showcasing +high-res IBL — `2k` and `4k` HDRs are megabytes of needless bandwidth +for a sample. + +## Attribution + +When a sample uses a third-party asset, add a one-line comment near +the asset URL crediting the source and license — e.g. +```ts +// Fox glTF © Khronos Group, CC-BY 4.0 +const MODEL_URL = "https://raw.githubusercontent.com/.../Fox.glb"; +``` + +## When a sample genuinely needs a custom asset + +Get explicit permission from the engineer with acceptance of the size, file type and location. diff --git a/packages/data-gpu-samples/index.html b/packages/data-gpu-samples/index.html new file mode 100644 index 00000000..58e0d485 --- /dev/null +++ b/packages/data-gpu-samples/index.html @@ -0,0 +1,25 @@ + + + + + + Data Graphics Samples + + + +
+ + + diff --git a/packages/data-gpu-samples/package.json b/packages/data-gpu-samples/package.json new file mode 100644 index 00000000..05a1019e --- /dev/null +++ b/packages/data-gpu-samples/package.json @@ -0,0 +1,24 @@ +{ + "name": "data-gpu-samples", + "version": "0.9.69", + "description": "WebGPU samples built on @adobe/data-gpu", + "type": "module", + "private": true, + "scripts": { + "dev": "vite", + "dev:all": "pnpm --parallel --filter @adobe/data --filter @adobe/data-gpu --filter data-gpu-samples run dev", + "build": "vite build" + }, + "dependencies": { + "@adobe/data": "workspace:*", + "@adobe/data-gpu": "workspace:*", + "@adobe/data-lit": "workspace:*", + "lit": "^3.3.1" + }, + "devDependencies": { + "@webgpu/types": "^0.1.61", + "typescript": "^5.8.3", + "vite": "^5.1.1", + "vite-plugin-checker": "^0.12.0" + } +} diff --git a/packages/data-gpu-samples/src/hooks/use-orbit-camera-control.ts b/packages/data-gpu-samples/src/hooks/use-orbit-camera-control.ts new file mode 100644 index 00000000..b21ae7b6 --- /dev/null +++ b/packages/data-gpu-samples/src/hooks/use-orbit-camera-control.ts @@ -0,0 +1,17 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { useElement, useEffect } from "@adobe/data-lit"; +import { attachOrbitDrag, type OrbitDragService } from "@adobe/data-gpu"; + +/** + * Wires the host element's drag gesture to the `orbit` plugin via + * `attachOrbitDrag`. Drag rotates the orbit (pausing auto-spin); + * drag end un-pauses. + */ +export function useOrbitCameraControl( + service: OrbitDragService, + options: { sensitivity?: number } = {}, +): void { + const element = useElement(); + useEffect(() => attachOrbitDrag(element as HTMLElement, service, options), [element, service]); +} diff --git a/packages/data-gpu-samples/src/hooks/use-orbit-drag-camera.ts b/packages/data-gpu-samples/src/hooks/use-orbit-drag-camera.ts new file mode 100644 index 00000000..8b13f1d7 --- /dev/null +++ b/packages/data-gpu-samples/src/hooks/use-orbit-drag-camera.ts @@ -0,0 +1,34 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { useDraggable, useElement, useRef } from "@adobe/data-lit"; + +/** + * Drag-to-orbit camera hook. Attaches a pointer drag listener to the host + * element and emits the per-event horizontal pixel delta. Callers wire the + * delta into whatever transaction rotates their camera. + * + * Used across PBR / IBL / solar-system samples that share this gesture. + */ +export function useOrbitDragCamera( + onDelta: (deltaX: number) => void, + onEnd?: () => void, +): void { + const element = useElement(); + // Latest-callbacks ref so useDraggable's effect deps stay stable. + const callbacks = useRef({ onDelta, onEnd }); + callbacks.current.onDelta = onDelta; + callbacks.current.onEnd = onEnd; + + const state = useRef({ lastDx: 0 }); + useDraggable(element, { + minDragDistance: 1, + onDragStart: () => { state.current.lastDx = 0; }, + onDrag: (_e, _pos, delta) => { + const dx = delta[0] - state.current.lastDx; + state.current.lastDx = delta[0]; + callbacks.current.onDelta(dx); + }, + onDragEnd: () => { callbacks.current.onEnd?.(); }, + onDragCancel: () => { callbacks.current.onEnd?.(); }, + }, []); +} diff --git a/packages/data-gpu-samples/src/main.ts b/packages/data-gpu-samples/src/main.ts new file mode 100644 index 00000000..36913db0 --- /dev/null +++ b/packages/data-gpu-samples/src/main.ts @@ -0,0 +1,9 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { render } from "lit"; +import { SampleContainer } from "./sample-container/sample-container.js"; + +const app = document.getElementById("app"); +if (app) { + render(SampleContainer(), app); +} diff --git a/packages/data-gpu-samples/src/sample-container/sample-container-element.ts b/packages/data-gpu-samples/src/sample-container/sample-container-element.ts new file mode 100644 index 00000000..26b8d6d7 --- /dev/null +++ b/packages/data-gpu-samples/src/sample-container/sample-container-element.ts @@ -0,0 +1,58 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { LitElement, html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { HelloTriangle } from "../samples/hello-triangle/hello-triangle.js"; +import { PbrModelIbl } from "../samples/pbr-model-ibl/pbr-model-ibl.js"; +import { MetalRoughSpheres } from "../samples/metal-rough-spheres/metal-rough-spheres.js"; +import { PbrIblInstanced } from "../samples/pbr-ibl-instanced/pbr-ibl-instanced.js"; +import { SolarSystem } from "../samples/solar-system/solar-system.js"; +import { AntiqueCamera } from "../samples/antique-camera/antique-camera.js"; +import { SkinnedFox } from "../samples/skinned-fox/skinned-fox.js"; +import { Boids } from "../samples/boids/boids.js"; +import { RigidStack } from "../samples/rigid-stack/rigid-stack.js"; +import { Ragdoll } from "../samples/ragdoll/ragdoll.js"; + +const tagName = "sample-container"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: SampleContainerElement; + } +} + +@customElement(tagName) +export class SampleContainerElement extends LitElement { + static styles = css` + :host { display: block; width: 100%; } + nav { display: flex; gap: 0.5rem; padding: 1rem; flex-wrap: wrap; } + a { color: #7af; text-decoration: none; padding: 0.25rem 0.75rem; border: 1px solid #7af; border-radius: 4px; } + a:hover { background: #7af2; } + .content { padding: 1rem; } + `; + + private get sample(): string { + return new URLSearchParams(window.location.search).get("sample") ?? "hello-triangle"; + } + + override render() { + const samples = ["hello-triangle", "pbr-model-ibl", "metal-rough-spheres", "pbr-ibl-instanced", "solar-system", "antique-camera", "skinned-fox", "boids", "rigid-stack", "ragdoll"]; + return html` + +
+ ${this.sample === "hello-triangle" ? HelloTriangle() : ""} + ${this.sample === "pbr-model-ibl" ? PbrModelIbl() : ""} + ${this.sample === "metal-rough-spheres" ? MetalRoughSpheres() : ""} + ${this.sample === "pbr-ibl-instanced" ? PbrIblInstanced() : ""} + ${this.sample === "solar-system" ? SolarSystem() : ""} + ${this.sample === "antique-camera" ? AntiqueCamera() : ""} + ${this.sample === "skinned-fox" ? SkinnedFox() : ""} + ${this.sample === "boids" ? Boids() : ""} + ${this.sample === "rigid-stack" ? RigidStack() : ""} + ${this.sample === "ragdoll" ? Ragdoll() : ""} +
+ `; + } +} diff --git a/packages/data-gpu-samples/src/sample-container/sample-container.ts b/packages/data-gpu-samples/src/sample-container/sample-container.ts new file mode 100644 index 00000000..f5502b34 --- /dev/null +++ b/packages/data-gpu-samples/src/sample-container/sample-container.ts @@ -0,0 +1,9 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html } from "lit"; +import type { TemplateResult } from "lit"; + +export const SampleContainer = (): TemplateResult => { + void import("./sample-container-element.js"); + return html``; +}; diff --git a/packages/data-gpu-samples/src/samples/antique-camera/antique-camera-element.ts b/packages/data-gpu-samples/src/samples/antique-camera/antique-camera-element.ts new file mode 100644 index 00000000..2424020e --- /dev/null +++ b/packages/data-gpu-samples/src/samples/antique-camera/antique-camera-element.ts @@ -0,0 +1,57 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +// AntiqueCamera glTF model © Khronos Group, CC-BY 4.0 + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { pbrModelIblPlugin } from "../pbr-model-ibl/pbr-model-ibl-service.js"; +import { useOrbitCameraControl } from "../../hooks/use-orbit-camera-control.js"; + +const tagName = "antique-camera"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: AntiqueCameraElement; + } +} + +const MODEL_URL = "https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Assets/main/Models/AntiqueCamera/glTF-Binary/AntiqueCamera.glb"; +const ENV_URL = "https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/studio_small_09_1k.hdr"; + +@customElement(tagName) +export class AntiqueCameraElement extends DatabaseElement { + static styles = css` + :host { display: block; } + .stage { position: relative; } + canvas { display: block; border: 1px solid #333; background: #111; cursor: grab; } + canvas.dragging { cursor: grabbing; } + .hint { position: absolute; top: 0.5rem; right: 0.5rem; padding: 0.25rem 0.5rem; background: rgba(0,0,0,0.5); color: #aaa; font: 11px/1 ui-monospace, monospace; border-radius: 4px; } + `; + + get plugin() { return pbrModelIblPlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + service.transactions.initializeScene({ + modelUrl: MODEL_URL, + envUrl: ENV_URL, + lightColor: [0.2, 0.2, 0.2], + orbitFit: { radiusFactor: 1.4, heightFactor: 0.3 }, + }); + }, [canvas, service]); + + useOrbitCameraControl(service); + + return html` +
+ +
drag to orbit
+
+ `; + } +} diff --git a/packages/data-gpu-samples/src/samples/antique-camera/antique-camera.ts b/packages/data-gpu-samples/src/samples/antique-camera/antique-camera.ts new file mode 100644 index 00000000..e16f7e36 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/antique-camera/antique-camera.ts @@ -0,0 +1,9 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html } from "lit"; +import type { TemplateResult } from "lit"; + +export const AntiqueCamera = (): TemplateResult => { + void import("./antique-camera-element.js"); + return html``; +}; diff --git a/packages/data-gpu-samples/src/samples/boids/boids-element.ts b/packages/data-gpu-samples/src/samples/boids/boids-element.ts new file mode 100644 index 00000000..c7e85f68 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/boids/boids-element.ts @@ -0,0 +1,71 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { boidsPlugin } from "./boids-service.js"; +import { useOrbitCameraControl } from "../../hooks/use-orbit-camera-control.js"; + +const tagName = "boids-sample"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: BoidsElement; + } +} + +@customElement(tagName) +export class BoidsElement extends DatabaseElement { + static styles = css` + :host { display: block; } + .stage { position: relative; } + canvas { display: block; border: 1px solid #333; background: #000; cursor: grab; } + canvas.dragging { cursor: grabbing; } + .hint { position: absolute; top: 0.5rem; right: 0.5rem; padding: 0.25rem 0.5rem; background: rgba(0,0,0,0.5); color: #aaa; font: 11px/1 ui-monospace, monospace; border-radius: 4px; } + `; + + get plugin() { return boidsPlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + service.transactions.setLight({ color: [0.9, 0.9, 0.95] }); + service.transactions.initializeScene(); + }, [canvas, service]); + + // Cursor scare: project the pointer onto the orbit-focal plane and + // feed it to the compute shader as a fleeing target. + useEffect(() => { + if (!canvas) return; + const onMove = (e: PointerEvent) => { + const rect = canvas.getBoundingClientRect(); + service.transactions.setScareFromScreen({ + x: e.clientX - rect.left, + y: e.clientY - rect.top, + width: rect.width, + height: rect.height, + }); + }; + const onLeave = () => service.transactions.disableScare(); + canvas.addEventListener("pointermove", onMove); + canvas.addEventListener("pointerleave", onLeave); + return () => { + canvas.removeEventListener("pointermove", onMove); + canvas.removeEventListener("pointerleave", onLeave); + }; + }, [canvas, service]); + + useOrbitCameraControl(service); + + return html` +
+ +
move mouse to scare · drag to orbit
+
+ `; + } +} diff --git a/packages/data-gpu-samples/src/samples/boids/boids-service.ts b/packages/data-gpu-samples/src/samples/boids/boids-service.ts new file mode 100644 index 00000000..f8c1b1fc --- /dev/null +++ b/packages/data-gpu-samples/src/samples/boids/boids-service.ts @@ -0,0 +1,467 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Vec3, type F32 } from "@adobe/data/math"; +import { SceneUniforms, graphics, Orbit } from "@adobe/data-gpu"; +import { computeShader, renderShader } from "./boids-shaders.js"; + +// --- Tunables -------------------------------------------------------------- + +// Cell size must be ≥ the boid view radius so the 3×3×3 cell scan in the +// update pass covers every possible neighbour. We currently use viewR ≈ 2× +// separationDist; at separationDist = 1.0 that's 2.0, hence cellSize = 2.0. +const WORLD_EXTENT = 10; // half-extent of cubic toroidal world +const GRID_DIM = 10; // 1000 cells, cellSize = 2.0 +const CELL_COUNT = GRID_DIM ** 3; +const CELL_SIZE = (WORLD_EXTENT * 2) / GRID_DIM; +// Tuned for visible flocking. With viewR ≈ 1.6 and the 20-unit cube the +// average neighbour count is ~8 — Reynolds' classic boid count band. +const DEFAULT_BOIDS = 4000; + +// Per-vertex stride for the boid arrowhead (position + normal, packed). +const MESH_STRIDE = 24; + +// 5 × u32 indirect-draw args. +const DRAW_ARGS_SIZE = 20; +// 2 × vec4f per boid. +const BOID_STATE_STRIDE = 32; +// 12 scalar + 2 × vec4f params. +const PARAMS_SIZE = 96; + +const SCARE_RADIUS = 3.0; +const SCARE_GAIN = 18.0; + +// --- Initial state: clumped positions + curl-noise-like velocity field ---- + +// Sum-of-sines stand-in for 3D Perlin. Features at ~6-unit scale concentrate +// boids into a handful of blobs rather than a uniform fog, so flocks exist +// from frame 1. +function densityNoise(x: number, y: number, z: number): number { + const a = Math.sin(x * 0.45 + 0.3) * Math.cos(y * 0.45 + 1.1) * Math.sin(z * 0.45 + 2.4); + const b = Math.sin(x * 0.95 + 2.7) * Math.cos(y * 0.95 + 0.6) * Math.sin(z * 0.95 + 1.8); + return a + 0.5 * b; +} + +// Smooth, swirly velocity field. Boids in the same blob start moving in the +// same direction so alignment locks them into a coherent flock immediately. +function flowField(x: number, y: number, z: number): [number, number, number] { + const f = 0.35; + return [ + Math.sin(y * f + 0.2) * Math.cos(z * f - 0.7) * 3, + Math.sin(z * f + 1.4) * Math.cos(x * f + 0.3) * 3, + Math.sin(x * f + 2.1) * Math.cos(y * f + 1.6) * 3, + ]; +} + +function buildInitialState(count: number): Float32Array { + const out = new Float32Array(count * 8); + const halfRange = WORLD_EXTENT * 0.85; + for (let i = 0; i < count; i++) { + // Rejection-sample positions where densityNoise is high. + let x = 0, y = 0, z = 0; + let placed = false; + for (let tries = 0; tries < 32; tries++) { + x = (Math.random() * 2 - 1) * halfRange; + y = (Math.random() * 2 - 1) * halfRange; + z = (Math.random() * 2 - 1) * halfRange; + if (densityNoise(x, y, z) > -0.1) { placed = true; break; } + } + if (!placed) { + x = (Math.random() * 2 - 1) * halfRange; + y = (Math.random() * 2 - 1) * halfRange; + z = (Math.random() * 2 - 1) * halfRange; + } + const [vx, vy, vz] = flowField(x, y, z); + out[i * 8 + 0] = x; + out[i * 8 + 1] = y; + out[i * 8 + 2] = z; + out[i * 8 + 3] = 0; + out[i * 8 + 4] = vx; + out[i * 8 + 5] = vy; + out[i * 8 + 6] = vz; + out[i * 8 + 7] = 0; + } + return out; +} + +// --- Mesh: 4-vertex tetrahedral arrowhead, +Z forward ---------------------- + +function arrowheadMesh(): { vertices: Float32Array; indices: Uint16Array; indexCount: number } { + // position (3f) + outward normal (3f). Vertex normals taken as normalize(position - centroid). + const v: number[] = []; + const push = (x: number, y: number, z: number) => { + const len = Math.hypot(x, y, z) || 1; + v.push(x, y, z, x / len, y / len, z / len); + }; + // 3× the previous scale so each boid is visible against the IBL skybox. + push(0.00, 0.000, 0.45); // tip + push(0.12, -0.075, -0.15); // back-bot-right + push(-0.12, -0.075, -0.15); // back-bot-left + push(0.00, 0.150, -0.15); // back-top + const indices = new Uint16Array([ + 0, 1, 3, // tip-right-top + 0, 3, 2, // tip-top-left + 0, 2, 1, // tip-bottom + 1, 2, 3, // back + ]); + return { vertices: new Float32Array(v), indices, indexCount: indices.length }; +} + +// --- Plugin ---------------------------------------------------------------- + +interface ComputePipelines { + clearCells: GPUComputePipeline; + populateGrid: GPUComputePipeline; + prefixSum: GPUComputePipeline; + binBoids: GPUComputePipeline; + updateBoids: GPUComputePipeline; +} + +interface BoidGpu { + params: GPUBuffer; + stateA: GPUBuffer; + stateB: GPUBuffer; + cellCounts: GPUBuffer; + cellOffsets: GPUBuffer; + cellWriteCursors: GPUBuffer; + sortedIndices: GPUBuffer; + drawArgs: GPUBuffer; + meshVB: GPUBuffer; + meshIB: GPUBuffer; + indexCount: number; + /** computeBG[0]: read=A,write=B; computeBG[1]: read=B,write=A. */ + computeBG: [GPUBindGroup, GPUBindGroup]; + /** renderBG[0]: read=B (matches computeBG[0]'s writeState); [1]: read=A. */ + renderBG: [GPUBindGroup, GPUBindGroup]; + pipelines: ComputePipelines; + renderPipeline: GPURenderPipeline; +} + +export const boidsPlugin = Database.Plugin.create({ + extends: Database.Plugin.combine(graphics, SceneUniforms.plugin, Orbit.plugin), + resources: { + boidsCount: { default: DEFAULT_BOIDS as number, transient: true }, + boidsGpu: { default: null as BoidGpu | null, transient: true }, + boidsPingFrame: { default: 0 as number, transient: true }, + boidsSceneBG: { default: null as GPUBindGroup | null, transient: true }, + boidsSceneBuffer: { default: null as GPUBuffer | null, transient: true }, + // Cursor scare ray. Origin is the camera eye; dir is the unit vector + // from the eye through the cursor toward the far plane. Boids near + // this line are pushed perpendicular to it. + boidsScareOrigin: { default: [0, 0, 0] as Vec3, transient: true }, + boidsScareDir: { default: [0, 0, 1] as Vec3, transient: true }, + boidsScareActive: { default: 0 as F32, transient: true }, + }, + transactions: { + setBoidsCount(t, count: number) { + t.resources.boidsCount = count; + // Force re-init by clearing the GPU bundle; the init system will rebuild. + t.resources.boidsGpu = null; + }, + /** Cast a ray from the camera eye through the cursor toward the far plane. + * Any boid near that line — at any depth — will be scared. + * `x`/`y` are pixel coords (e.g. from a PointerEvent relative to the canvas). + * Internally converts to NDC (x/y ∈ [-1, 1], origin at canvas center, y-up). */ + setScareFromScreen(t, args: { x: number; y: number; width: number; height: number }) { + const cam = t.resources.camera; + if (!cam) return; + const fwd = Vec3.normalize(Vec3.subtract(cam.target, cam.position)); + const right = Vec3.normalize(Vec3.cross(fwd, cam.up)); + const upOrtho = Vec3.cross(right, fwd); + const tanHalfFov = Math.tan(cam.fieldOfView / 2); + // NDC conversion: x ∈ [-1,1] left→right, y ∈ [-1,1] bottom→top. + const ndcX = (args.x / args.width) * 2 - 1; + const ndcY = 1 - (args.y / args.height) * 2; + const rx = ndcX * tanHalfFov * cam.aspect; + const ry = ndcY * tanHalfFov; + const dir = Vec3.normalize([ + fwd[0] + right[0] * rx + upOrtho[0] * ry, + fwd[1] + right[1] * rx + upOrtho[1] * ry, + fwd[2] + right[2] * rx + upOrtho[2] * ry, + ]); + t.resources.boidsScareOrigin = [cam.position[0], cam.position[1], cam.position[2]]; + t.resources.boidsScareDir = dir; + t.resources.boidsScareActive = 1; + }, + disableScare(t) { + t.resources.boidsScareActive = 0; + }, + initializeScene(t) { + t.resources.orbit = { + ...t.resources.orbit, + center: [0, 0, 0], + radius: WORLD_EXTENT * 1.6, + height: WORLD_EXTENT * 0.3, + autoSpinSpeed: 0.08, + nearFactor: 0.005, + farFactor: 4, + }; + }, + }, + systems: { + boidsInit: { + create: db => () => { + const { device, boidsGpu, boidsCount } = db.store.resources; + if (!device || boidsGpu || boidsCount === 0) return; + + const mesh = arrowheadMesh(); + + const meshVB = device.createBuffer({ + size: mesh.vertices.byteLength, + usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(meshVB, 0, mesh.vertices); + + const meshIB = device.createBuffer({ + size: mesh.indices.byteLength, + usage: GPUBufferUsage.INDEX | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(meshIB, 0, mesh.indices); + + const params = device.createBuffer({ + size: PARAMS_SIZE, + usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST, + }); + + const stateBytes = boidsCount * BOID_STATE_STRIDE; + const stateA = device.createBuffer({ + size: stateBytes, + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + const stateB = device.createBuffer({ + size: stateBytes, + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + + device.queue.writeBuffer(stateA, 0, buildInitialState(boidsCount)); + + const cellCounts = device.createBuffer({ + size: CELL_COUNT * 4, + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + const cellOffsets = device.createBuffer({ + size: CELL_COUNT * 4, + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + const cellWriteCursors = device.createBuffer({ + size: CELL_COUNT * 4, + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + const sortedIndices = device.createBuffer({ + size: boidsCount * 4, + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + const drawArgs = device.createBuffer({ + size: DRAW_ARGS_SIZE, + usage: GPUBufferUsage.INDIRECT | GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(drawArgs, 0, new Uint32Array([mesh.indexCount, boidsCount, 0, 0, 0])); + + // Compute layout + pipelines + const computeLayout = device.createBindGroupLayout({ + entries: [ + { binding: 0, visibility: GPUShaderStage.COMPUTE, buffer: { type: "uniform" } }, + { binding: 1, visibility: GPUShaderStage.COMPUTE, buffer: { type: "read-only-storage" } }, + { binding: 2, visibility: GPUShaderStage.COMPUTE, buffer: { type: "storage" } }, + { binding: 3, visibility: GPUShaderStage.COMPUTE, buffer: { type: "storage" } }, + { binding: 4, visibility: GPUShaderStage.COMPUTE, buffer: { type: "storage" } }, + { binding: 5, visibility: GPUShaderStage.COMPUTE, buffer: { type: "storage" } }, + { binding: 6, visibility: GPUShaderStage.COMPUTE, buffer: { type: "storage" } }, + { binding: 7, visibility: GPUShaderStage.COMPUTE, buffer: { type: "storage" } }, + ], + }); + const computePipelineLayout = device.createPipelineLayout({ + bindGroupLayouts: [computeLayout], + }); + const computeModule = device.createShaderModule({ code: computeShader }); + const makeCp = (entry: string): GPUComputePipeline => device.createComputePipeline({ + layout: computePipelineLayout, + compute: { module: computeModule, entryPoint: entry }, + }); + const pipelines: ComputePipelines = { + clearCells: makeCp("clear_cells"), + populateGrid: makeCp("populate_grid"), + prefixSum: makeCp("prefix_sum"), + binBoids: makeCp("bin_boids"), + updateBoids: makeCp("update_boids"), + }; + + // Two ping-pong compute bind groups. + const makeComputeBG = (read: GPUBuffer, write: GPUBuffer): GPUBindGroup => + device.createBindGroup({ + layout: computeLayout, + entries: [ + { binding: 0, resource: { buffer: params } }, + { binding: 1, resource: { buffer: read } }, + { binding: 2, resource: { buffer: write } }, + { binding: 3, resource: { buffer: cellCounts } }, + { binding: 4, resource: { buffer: cellOffsets } }, + { binding: 5, resource: { buffer: cellWriteCursors } }, + { binding: 6, resource: { buffer: sortedIndices } }, + { binding: 7, resource: { buffer: drawArgs } }, + ], + }); + const computeBG: [GPUBindGroup, GPUBindGroup] = [ + makeComputeBG(stateA, stateB), + makeComputeBG(stateB, stateA), + ]; + + // Render layout + pipeline + const sceneLayout = device.createBindGroupLayout({ + entries: [{ + binding: 0, + visibility: GPUShaderStage.VERTEX | GPUShaderStage.FRAGMENT, + buffer: { type: "uniform" }, + }], + }); + const boidStorageLayout = device.createBindGroupLayout({ + entries: [{ + binding: 0, + visibility: GPUShaderStage.VERTEX, + buffer: { type: "read-only-storage" }, + }], + }); + const renderModule = device.createShaderModule({ code: renderShader }); + const renderPipeline = device.createRenderPipeline({ + layout: device.createPipelineLayout({ bindGroupLayouts: [sceneLayout, boidStorageLayout] }), + vertex: { + module: renderModule, + entryPoint: "vs_main", + buffers: [{ + arrayStride: MESH_STRIDE, + stepMode: "vertex", + attributes: [ + { format: "float32x3", offset: 0, shaderLocation: 0 }, + { format: "float32x3", offset: 12, shaderLocation: 1 }, + ], + }], + }, + fragment: { + module: renderModule, + entryPoint: "fs_main", + targets: [{ format: db.store.resources.canvasFormat }], + }, + primitive: { topology: "triangle-list", cullMode: "none" }, + depthStencil: { + format: db.store.resources.depthFormat, + depthWriteEnabled: true, + depthCompare: "less", + }, + }); + const renderBG: [GPUBindGroup, GPUBindGroup] = [ + device.createBindGroup({ + layout: boidStorageLayout, + entries: [{ binding: 0, resource: { buffer: stateB } }], + }), + device.createBindGroup({ + layout: boidStorageLayout, + entries: [{ binding: 0, resource: { buffer: stateA } }], + }), + ]; + + db.store.resources.boidsGpu = { + params, stateA, stateB, + cellCounts, cellOffsets, cellWriteCursors, + sortedIndices, drawArgs, + meshVB, meshIB, indexCount: mesh.indexCount, + computeBG, renderBG, pipelines, renderPipeline, + }; + }, + schedule: { during: ["postUpdate"] }, + }, + boidsCompute: { + create: db => () => { + const gpu = db.store.resources.boidsGpu; + const { device, commandEncoder, boidsCount } = db.store.resources; + if (!gpu || !device || !commandEncoder) return; + + const dt = db.store.resources.frameTime.dt; + + // Refresh params each frame. + const params = new ArrayBuffer(PARAMS_SIZE); + const f32 = new Float32Array(params); + const u32 = new Uint32Array(params); + f32[0] = dt; // dt + f32[1] = CELL_SIZE; // cellSize + f32[2] = WORLD_EXTENT; // worldExtent + u32[3] = boidsCount; // boidsCount + u32[4] = GRID_DIM; // gridDim + u32[5] = CELL_COUNT; // cellCount + u32[6] = gpu.indexCount; // indexCount + f32[7] = 0.8; // separationDist → viewR ≈ 1.6 + f32[8] = 3.0; // separationGain (strong push-apart) + f32[9] = 2.0; // alignmentGain (strong velocity matching) + f32[10] = 0.6; // cohesionGain (looser pull-in) + f32[11] = 9.0; // maxSpeed + // scareOrigin (vec4 at offset 48): xyz = eye pos, w = active flag. + const scareOrigin = db.store.resources.boidsScareOrigin; + f32[12] = scareOrigin[0]; + f32[13] = scareOrigin[1]; + f32[14] = scareOrigin[2]; + f32[15] = db.store.resources.boidsScareActive; + // scareDir (vec4 at offset 64): xyz = unit eye→cursor direction. + const scareDir = db.store.resources.boidsScareDir; + f32[16] = scareDir[0]; + f32[17] = scareDir[1]; + f32[18] = scareDir[2]; + // scareTuning (vec4 at offset 80): x = radius, y = gain. + f32[20] = SCARE_RADIUS; + f32[21] = SCARE_GAIN; + device.queue.writeBuffer(gpu.params, 0, params); + + const ping = db.store.resources.boidsPingFrame & 1; + const bg = gpu.computeBG[ping]; + + const pass = commandEncoder.beginComputePass(); + pass.setBindGroup(0, bg); + + const cellWG = Math.ceil(CELL_COUNT / 64); + const boidWG = Math.ceil(boidsCount / 64); + + pass.setPipeline(gpu.pipelines.clearCells); + pass.dispatchWorkgroups(cellWG); + pass.setPipeline(gpu.pipelines.populateGrid); + pass.dispatchWorkgroups(boidWG); + pass.setPipeline(gpu.pipelines.prefixSum); + pass.dispatchWorkgroups(1); + pass.setPipeline(gpu.pipelines.binBoids); + pass.dispatchWorkgroups(boidWG); + pass.setPipeline(gpu.pipelines.updateBoids); + pass.dispatchWorkgroups(boidWG); + pass.end(); + + db.store.resources.boidsPingFrame = ping + 1; + }, + schedule: { during: ["physics"], after: ["boidsInit"] }, + }, + boidsRender: { + create: db => () => { + const gpu = db.store.resources.boidsGpu; + const { renderPassEncoder, device, _sceneUniformsBuffer, boidsSceneBuffer, boidsSceneBG } = db.store.resources; + if (!gpu || !renderPassEncoder || !device || !_sceneUniformsBuffer) return; + + let sceneBG = boidsSceneBG; + if (boidsSceneBuffer !== _sceneUniformsBuffer || !sceneBG) { + sceneBG = device.createBindGroup({ + layout: gpu.renderPipeline.getBindGroupLayout(0), + entries: [{ binding: 0, resource: { buffer: _sceneUniformsBuffer } }], + }); + db.store.resources.boidsSceneBG = sceneBG; + db.store.resources.boidsSceneBuffer = _sceneUniformsBuffer; + } + + // The render bind group matches whichever buffer compute just wrote. + const written = (db.store.resources.boidsPingFrame - 1) & 1; + renderPassEncoder.setPipeline(gpu.renderPipeline); + renderPassEncoder.setBindGroup(0, sceneBG); + renderPassEncoder.setBindGroup(1, gpu.renderBG[written]); + renderPassEncoder.setVertexBuffer(0, gpu.meshVB); + renderPassEncoder.setIndexBuffer(gpu.meshIB, "uint16"); + renderPassEncoder.drawIndexedIndirect(gpu.drawArgs, 0); + }, + schedule: { during: ["render"] }, + }, + }, +}); + +export type BoidsService = Database.Plugin.ToDatabase; diff --git a/packages/data-gpu-samples/src/samples/boids/boids-shaders.ts b/packages/data-gpu-samples/src/samples/boids/boids-shaders.ts new file mode 100644 index 00000000..b3c7ffcc --- /dev/null +++ b/packages/data-gpu-samples/src/samples/boids/boids-shaders.ts @@ -0,0 +1,260 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +// All-in-one compute module for the boids sample. Five entry points share one +// bind group layout so we can swap ping-pong groups without rebuilding +// pipelines. State is exchanged through bindings 1 (read) and 2 (write); on +// alternating frames the bound buffers are swapped. +export const computeShader = /* wgsl */ ` +struct Params { + dt: f32, + cellSize: f32, + worldExtent: f32, + boidsCount: u32, + gridDim: u32, + cellCount: u32, + indexCount: u32, + separationDist: f32, + separationGain: f32, + alignmentGain: f32, + cohesionGain: f32, + maxSpeed: f32, + // Scare ray origin (eye): xyz = world position, w = 1 when active else 0. + scareOrigin: vec4f, + // Scare ray direction (unit, eye → cursor through far plane). xyz used; w padding. + scareDir: vec4f, + // x = radius (perpendicular distance cap), y = gain. zw padding. + scareTuning: vec4f, +} + +struct BoidState { + pos: vec4f, + vel: vec4f, +} + +@group(0) @binding(0) var P: Params; +@group(0) @binding(1) var readState: array; +@group(0) @binding(2) var writeState: array; +@group(0) @binding(3) var cellCounts: array>; +@group(0) @binding(4) var cellOffsets: array; +@group(0) @binding(5) var cellWriteCursors: array>; +@group(0) @binding(6) var sortedIndices: array; +@group(0) @binding(7) var drawArgs: array; + +fn cellIndexOf(pos: vec3f) -> u32 { + let half = P.worldExtent; + let gd = i32(P.gridDim); + let p = (pos + vec3f(half)) / P.cellSize; + let c = clamp(vec3i(p), vec3i(0), vec3i(gd - 1)); + return u32(c.z * gd * gd + c.y * gd + c.x); +} + +@compute @workgroup_size(64) +fn clear_cells(@builtin(global_invocation_id) gid: vec3u) { + let i = gid.x; + if (i >= P.cellCount) { return; } + atomicStore(&cellCounts[i], 0u); +} + +@compute @workgroup_size(64) +fn populate_grid(@builtin(global_invocation_id) gid: vec3u) { + let i = gid.x; + if (i >= P.boidsCount) { return; } + let cell = cellIndexOf(readState[i].pos.xyz); + atomicAdd(&cellCounts[cell], 1u); +} + +// Single-thread exclusive prefix sum over cellCounts → cellOffsets, also +// initialises cellWriteCursors so binBoids can atomic-scatter. ~512 iters at +// gridDim=8 — sub-microsecond on any GPU. +@compute @workgroup_size(1) +fn prefix_sum() { + var sum: u32 = 0u; + for (var i: u32 = 0u; i < P.cellCount; i = i + 1u) { + let c = atomicLoad(&cellCounts[i]); + cellOffsets[i] = sum; + atomicStore(&cellWriteCursors[i], sum); + sum = sum + c; + } +} + +@compute @workgroup_size(64) +fn bin_boids(@builtin(global_invocation_id) gid: vec3u) { + let i = gid.x; + if (i >= P.boidsCount) { return; } + let cell = cellIndexOf(readState[i].pos.xyz); + let slot = atomicAdd(&cellWriteCursors[cell], 1u); + sortedIndices[slot] = i; +} + +@compute @workgroup_size(64) +fn update_boids(@builtin(global_invocation_id) gid: vec3u) { + let i = gid.x; + if (i >= P.boidsCount) { return; } + + let me = readState[i]; + let pos = me.pos.xyz; + let vel = me.vel.xyz; + + let half = P.worldExtent; + let gd = i32(P.gridDim); + let myCell = clamp(vec3i((pos + vec3f(half)) / P.cellSize), vec3i(0), vec3i(gd - 1)); + + var cohesion = vec3f(0.0); + var alignment = vec3f(0.0); + var separation = vec3f(0.0); + var neighbors: u32 = 0u; + let sepR2 = P.separationDist * P.separationDist; + let viewR2 = sepR2 * 4.0; + + for (var dz: i32 = -1; dz <= 1; dz = dz + 1) { + for (var dy: i32 = -1; dy <= 1; dy = dy + 1) { + for (var dx: i32 = -1; dx <= 1; dx = dx + 1) { + let nc = myCell + vec3i(dx, dy, dz); + if (any(nc < vec3i(0)) || any(nc >= vec3i(gd))) { continue; } + let cell = u32(nc.z * gd * gd + nc.y * gd + nc.x); + let start = cellOffsets[cell]; + let count = atomicLoad(&cellCounts[cell]); + for (var k: u32 = 0u; k < count; k = k + 1u) { + let other = sortedIndices[start + k]; + if (other == i) { continue; } + let os = readState[other]; + let d = os.pos.xyz - pos; + let dist2 = dot(d, d); + if (dist2 < 0.0001 || dist2 > viewR2) { continue; } + cohesion = cohesion + os.pos.xyz; + alignment = alignment + os.vel.xyz; + if (dist2 < sepR2) { + separation = separation - d / sqrt(dist2); + } + neighbors = neighbors + 1u; + } + } + } + } + + var newVel = vel; + if (neighbors > 0u) { + let nf = f32(neighbors); + let cohForce = (cohesion / nf - pos) * P.cohesionGain; + let alnForce = (alignment / nf - vel) * P.alignmentGain; + let sepForce = separation * P.separationGain; + newVel = newVel + (cohForce + alnForce + sepForce) * P.dt; + } + + // Cursor scare: outward force perpendicular to the eye→cursor ray. + // Boids near the line — at any depth — feel it, proportional to how + // close they are to the line. Boids behind the camera are ignored. + if (P.scareOrigin.w > 0.5) { + let origin = P.scareOrigin.xyz; + let dir = P.scareDir.xyz; + let toBoid = pos - origin; + let tAlong = dot(toBoid, dir); + if (tAlong > 0.0) { + let perp = toBoid - dir * tAlong; + let d2 = dot(perp, perp); + let r = P.scareTuning.x; + if (d2 < r * r && d2 > 0.0001) { + let d = sqrt(d2); + let strength = (1.0 - d / r) * P.scareTuning.y; + newVel = newVel + (perp / d) * strength * P.dt; + } + } + } + + let speed = length(newVel); + if (speed > P.maxSpeed) { + newVel = newVel * (P.maxSpeed / speed); + } else if (speed < 0.5) { + // Give isolated boids a small forward kick so they don't stall. + newVel = newVel + vec3f(0.0, 0.0, 0.5) * P.dt; + } + + // Toroidal wrap-around: flocks never hit a wall, they flow through. + var newPos = pos + newVel * P.dt; + let extent = P.worldExtent; + let size = 2.0 * extent; + newPos = ((newPos + vec3f(extent)) - floor((newPos + vec3f(extent)) / size) * size) - vec3f(extent); + + writeState[i] = BoidState(vec4f(newPos, 0.0), vec4f(newVel, 0.0)); + + // Boid 0 writes the indirect draw args for the render pass. + if (i == 0u) { + drawArgs[0] = P.indexCount; + drawArgs[1] = P.boidsCount; + drawArgs[2] = 0u; + drawArgs[3] = 0u; + drawArgs[4] = 0u; + } +} +`; + +export const renderShader = /* wgsl */ ` +struct SceneUniforms { + viewProjectionMatrix: mat4x4, + lightDirection: vec3f, + ambientStrength: f32, + lightColor: vec3f, + cameraPosition: vec3f, +} + +struct BoidState { + pos: vec4f, + vel: vec4f, +} + +@group(0) @binding(0) var scene: SceneUniforms; +@group(1) @binding(0) var boids: array; + +struct VIn { + @location(0) position: vec3f, + @location(1) normal: vec3f, +} + +struct VOut { + @builtin(position) clip: vec4f, + @location(0) worldNormal: vec3f, + @location(1) color: vec3f, +} + +@vertex +fn vs_main(@builtin(instance_index) ii: u32, in: VIn) -> VOut { + let state = boids[ii]; + let pos = state.pos.xyz; + let vel = state.vel.xyz; + + // Build a TBN with forward = velocity direction. + let forward = normalize(vel + vec3f(0.0001, 0.0, 0.0)); + let upHint = vec3f(0.0, 1.0, 0.0); + let right = normalize(cross(forward, upHint)); + let up = cross(right, forward); + + let worldPos = pos + + in.position.x * right + + in.position.y * up + + in.position.z * forward; + let worldNormal = in.normal.x * right + + in.normal.y * up + + in.normal.z * forward; + + var out: VOut; + out.clip = scene.viewProjectionMatrix * vec4f(worldPos, 1.0); + out.worldNormal = worldNormal; + // Hue from facing direction: ±x, ±y, ±z each map to a distinct RGB + // wedge by remapping the unit vector from [-1,1] to [0,1] per channel. + // Intensity from speed: slow boids dim, fast ones bright. + let hue = (forward + vec3f(1.0)) * 0.5; + let speedT = saturate(length(vel) / 5.0); + let intensity = mix(0.35, 1.0, speedT); + out.color = hue * intensity; + return out; +} + +@fragment +fn fs_main(in: VOut) -> @location(0) vec4f { + let n = normalize(in.worldNormal); + let L = normalize(-scene.lightDirection); + let diff = max(dot(n, L), 0.0); + let lit = in.color * (scene.ambientStrength + diff * scene.lightColor); + return vec4f(lit, 1.0); +} +`; diff --git a/packages/data-gpu-samples/src/samples/boids/boids.ts b/packages/data-gpu-samples/src/samples/boids/boids.ts new file mode 100644 index 00000000..a1e52794 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/boids/boids.ts @@ -0,0 +1,9 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html } from "lit"; +import type { TemplateResult } from "lit"; + +export const Boids = (): TemplateResult => { + void import("./boids-element.js"); + return html``; +}; diff --git a/packages/data-gpu-samples/src/samples/hello-triangle/hello-triangle-element.ts b/packages/data-gpu-samples/src/samples/hello-triangle/hello-triangle-element.ts new file mode 100644 index 00000000..e59c9f07 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/hello-triangle/hello-triangle-element.ts @@ -0,0 +1,31 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { helloTrianglePlugin } from "./hello-triangle-service.js"; + +const tagName = "hello-triangle"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: HelloTriangleElement; + } +} + +@customElement(tagName) +export class HelloTriangleElement extends DatabaseElement { + static styles = css`:host { display: block; } canvas { display: block; border: 1px solid #333; }`; + + get plugin() { return helloTrianglePlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + }, [canvas, service]); + return html``; + } +} diff --git a/packages/data-gpu-samples/src/samples/hello-triangle/hello-triangle-service.ts b/packages/data-gpu-samples/src/samples/hello-triangle/hello-triangle-service.ts new file mode 100644 index 00000000..efa3c223 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/hello-triangle/hello-triangle-service.ts @@ -0,0 +1,70 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { graphics } from "@adobe/data-gpu"; + +const triangleShader = ` +struct VertexOutput { + @builtin(position) position: vec4f, + @location(0) color: vec4f, +} + +@vertex +fn vs_main(@builtin(vertex_index) i: u32) -> VertexOutput { + var pos = array( + vec2f( 0.0, 0.5), + vec2f(-0.5, -0.5), + vec2f( 0.5, -0.5), + ); + var colors = array( + vec4f(1.0, 0.0, 0.0, 1.0), + vec4f(0.0, 1.0, 0.0, 1.0), + vec4f(0.0, 0.0, 1.0, 1.0), + ); + var out: VertexOutput; + out.position = vec4f(pos[i], 0.0, 1.0); + out.color = colors[i]; + return out; +} + +@fragment +fn fs_main(in: VertexOutput) -> @location(0) vec4f { + return in.color; +} +`; + +export const helloTrianglePlugin = Database.Plugin.create({ + extends: graphics, + systems: { + hello_triangle_render: { + create: db => { + let pipeline: GPURenderPipeline | null = null; + return () => { + const { device, renderPassEncoder, canvasFormat, depthFormat } = db.store.resources; + if (!device || !renderPassEncoder) return; + + if (!pipeline) { + const module = device.createShaderModule({ code: triangleShader }); + pipeline = device.createRenderPipeline({ + layout: "auto", + vertex: { module, entryPoint: "vs_main" }, + fragment: { module, entryPoint: "fs_main", targets: [{ format: canvasFormat }] }, + primitive: { topology: "triangle-list" }, + depthStencil: { + format: depthFormat, + depthWriteEnabled: true, + depthCompare: "less", + }, + }); + } + + renderPassEncoder.setPipeline(pipeline); + renderPassEncoder.draw(3); + }; + }, + schedule: { during: ["render"] } + }, + }, +}); + +export type HelloTriangleService = Database.Plugin.ToDatabase; diff --git a/packages/data-gpu-samples/src/samples/hello-triangle/hello-triangle.ts b/packages/data-gpu-samples/src/samples/hello-triangle/hello-triangle.ts new file mode 100644 index 00000000..9e190e50 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/hello-triangle/hello-triangle.ts @@ -0,0 +1,9 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html } from "lit"; +import type { TemplateResult } from "lit"; + +export const HelloTriangle = (): TemplateResult => { + void import("./hello-triangle-element.js"); + return html``; +}; diff --git a/packages/data-gpu-samples/src/samples/metal-rough-spheres/metal-rough-spheres-element.ts b/packages/data-gpu-samples/src/samples/metal-rough-spheres/metal-rough-spheres-element.ts new file mode 100644 index 00000000..d55d0ffd --- /dev/null +++ b/packages/data-gpu-samples/src/samples/metal-rough-spheres/metal-rough-spheres-element.ts @@ -0,0 +1,56 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { pbrModelIblPlugin } from "../pbr-model-ibl/pbr-model-ibl-service.js"; +import { useOrbitCameraControl } from "../../hooks/use-orbit-camera-control.js"; + +const tagName = "metal-rough-spheres"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: MetalRoughSpheresElement; + } +} + +const MODEL_URL = "https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Assets/main/Models/MetalRoughSpheres/glTF-Binary/MetalRoughSpheres.glb"; +const ENV_URL = "https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/venice_sunset_1k.hdr"; + +@customElement(tagName) +export class MetalRoughSpheresElement extends DatabaseElement { + static styles = css` + :host { display: block; } + .stage { position: relative; } + canvas { display: block; border: 1px solid #333; background: #111; cursor: grab; } + canvas.dragging { cursor: grabbing; } + .hint { position: absolute; top: 0.5rem; right: 0.5rem; padding: 0.25rem 0.5rem; background: rgba(0,0,0,0.5); color: #aaa; font: 11px/1 ui-monospace, monospace; border-radius: 4px; } + `; + + get plugin() { return pbrModelIblPlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + service.transactions.initializeScene({ + modelUrl: MODEL_URL, + envUrl: ENV_URL, + lightColor: [0, 0, 0], + orbitFit: { radiusFactor: 1.2, heightFactor: 0 }, + }); + }, [canvas, service]); + + useOrbitCameraControl(service); + + return html` +
+ +
drag to orbit
+
+ `; + } +} diff --git a/packages/data-gpu-samples/src/samples/metal-rough-spheres/metal-rough-spheres.ts b/packages/data-gpu-samples/src/samples/metal-rough-spheres/metal-rough-spheres.ts new file mode 100644 index 00000000..6761faa7 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/metal-rough-spheres/metal-rough-spheres.ts @@ -0,0 +1,9 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html } from "lit"; +import type { TemplateResult } from "lit"; + +export const MetalRoughSpheres = (): TemplateResult => { + void import("./metal-rough-spheres-element.js"); + return html``; +}; diff --git a/packages/data-gpu-samples/src/samples/pbr-ibl-instanced/pbr-ibl-instanced-element.ts b/packages/data-gpu-samples/src/samples/pbr-ibl-instanced/pbr-ibl-instanced-element.ts new file mode 100644 index 00000000..ea815916 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/pbr-ibl-instanced/pbr-ibl-instanced-element.ts @@ -0,0 +1,57 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { pbrIblInstancedPlugin } from "./pbr-ibl-instanced-service.js"; +import { useOrbitCameraControl } from "../../hooks/use-orbit-camera-control.js"; + +const tagName = "pbr-ibl-instanced"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: PbrIblInstancedElement; + } +} + +const MODEL_URL = "https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Assets/main/Models/DamagedHelmet/glTF-Binary/DamagedHelmet.glb"; +const ENV_URL = "https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/studio_small_09_1k.hdr"; + +@customElement(tagName) +export class PbrIblInstancedElement extends DatabaseElement { + static styles = css` + :host { display: block; } + .stage { position: relative; } + canvas { display: block; border: 1px solid #333; background: #111; cursor: grab; } + canvas.dragging { cursor: grabbing; } + .hint { position: absolute; top: 0.5rem; right: 0.5rem; padding: 0.25rem 0.5rem; background: rgba(0,0,0,0.5); color: #aaa; font: 11px/1 ui-monospace, monospace; border-radius: 4px; } + `; + + get plugin() { return pbrIblInstancedPlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + service.transactions.initializeScene({ + modelUrl: MODEL_URL, + envUrl: ENV_URL, + lightColor: [0.3, 0.3, 0.3], + grid: 4, + spacing: 2.5, + }); + }, [canvas, service]); + + useOrbitCameraControl(service); + + return html` +
+ +
drag to orbit
+
+ `; + } +} diff --git a/packages/data-gpu-samples/src/samples/pbr-ibl-instanced/pbr-ibl-instanced-service.ts b/packages/data-gpu-samples/src/samples/pbr-ibl-instanced/pbr-ibl-instanced-service.ts new file mode 100644 index 00000000..efae7d14 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/pbr-ibl-instanced/pbr-ibl-instanced-service.ts @@ -0,0 +1,45 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Vec3 } from "@adobe/data/math"; +import { pbrRender, Model, Orbit } from "@adobe/data-gpu"; + +export const pbrIblInstancedPlugin = Database.Plugin.create({ + extends: Database.Plugin.combine(pbrRender, Orbit.plugin), + transactions: { + initializeScene(t, args: { + modelUrl: string; + envUrl?: string; + lightColor?: Vec3; + grid: number; + spacing: number; + }): number { + t.resources.light = { + ...t.resources.light, + environmentUrl: args.envUrl ?? t.resources.light.environmentUrl, + color: args.lightColor ?? t.resources.light.color, + }; + const geoId = Model.plugin.transactions.insertGeometry(t, { modelUrl: args.modelUrl }); + const offset = (args.grid - 1) / 2; + for (let x = 0; x < args.grid; x++) { + for (let z = 0; z < args.grid; z++) { + Model.plugin.transactions.insertModel(t, { + geometry: geoId, + position: [(x - offset) * args.spacing, 0, (z - offset) * args.spacing], + }); + } + } + t.resources.orbit = { + ...t.resources.orbit, + fitGeometry: geoId, + fitRadiusOffset: offset * args.spacing, + fitRadiusFactor: 2, + fitHeightFactor: 0.5, + fitCenter: [0, 0, 0], + }; + return geoId; + }, + }, +}); + +export type PbrIblInstancedService = Database.Plugin.ToDatabase; diff --git a/packages/data-gpu-samples/src/samples/pbr-ibl-instanced/pbr-ibl-instanced.ts b/packages/data-gpu-samples/src/samples/pbr-ibl-instanced/pbr-ibl-instanced.ts new file mode 100644 index 00000000..31fdf831 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/pbr-ibl-instanced/pbr-ibl-instanced.ts @@ -0,0 +1,9 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html } from "lit"; +import type { TemplateResult } from "lit"; + +export const PbrIblInstanced = (): TemplateResult => { + void import("./pbr-ibl-instanced-element.js"); + return html``; +}; diff --git a/packages/data-gpu-samples/src/samples/pbr-model-ibl/pbr-model-ibl-element.ts b/packages/data-gpu-samples/src/samples/pbr-model-ibl/pbr-model-ibl-element.ts new file mode 100644 index 00000000..fcd55875 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/pbr-model-ibl/pbr-model-ibl-element.ts @@ -0,0 +1,56 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { pbrModelIblPlugin } from "./pbr-model-ibl-service.js"; +import { useOrbitCameraControl } from "../../hooks/use-orbit-camera-control.js"; + +const tagName = "pbr-model-ibl"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: PbrModelIblElement; + } +} + +const MODEL_URL = "https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Assets/main/Models/DamagedHelmet/glTF-Binary/DamagedHelmet.glb"; +const ENV_URL = "https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/studio_small_09_1k.hdr"; + +@customElement(tagName) +export class PbrModelIblElement extends DatabaseElement { + static styles = css` + :host { display: block; } + .stage { position: relative; } + canvas { display: block; border: 1px solid #333; background: #111; cursor: grab; } + canvas.dragging { cursor: grabbing; } + .hint { position: absolute; top: 0.5rem; right: 0.5rem; padding: 0.25rem 0.5rem; background: rgba(0,0,0,0.5); color: #aaa; font: 11px/1 ui-monospace, monospace; border-radius: 4px; } + `; + + get plugin() { return pbrModelIblPlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + service.transactions.initializeScene({ + modelUrl: MODEL_URL, + envUrl: ENV_URL, + lightColor: [0.4, 0.4, 0.4], + orbitFit: { radiusFactor: 1.6, heightFactor: 0.25 }, + }); + }, [canvas, service]); + + useOrbitCameraControl(service); + + return html` +
+ +
drag to orbit
+
+ `; + } +} diff --git a/packages/data-gpu-samples/src/samples/pbr-model-ibl/pbr-model-ibl-service.ts b/packages/data-gpu-samples/src/samples/pbr-model-ibl/pbr-model-ibl-service.ts new file mode 100644 index 00000000..1d9c5259 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/pbr-model-ibl/pbr-model-ibl-service.ts @@ -0,0 +1,34 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Vec3 } from "@adobe/data/math"; +import { pbrRender, Model, Orbit } from "@adobe/data-gpu"; + +export const pbrModelIblPlugin = Database.Plugin.create({ + extends: Database.Plugin.combine(pbrRender, Orbit.plugin), + transactions: { + initializeScene(t, args: { + modelUrl: string; + envUrl?: string; + lightColor?: Vec3; + orbitFit?: { radiusFactor: number; heightFactor: number }; + }): number { + t.resources.light = { + ...t.resources.light, + environmentUrl: args.envUrl ?? t.resources.light.environmentUrl, + color: args.lightColor ?? t.resources.light.color, + }; + const geoId = Model.plugin.transactions.insertGeometry(t, { modelUrl: args.modelUrl }); + Model.plugin.transactions.insertModel(t, { geometry: geoId }); + t.resources.orbit = { + ...t.resources.orbit, + fitGeometry: geoId, + fitRadiusFactor: args.orbitFit?.radiusFactor ?? t.resources.orbit.fitRadiusFactor, + fitHeightFactor: args.orbitFit?.heightFactor ?? t.resources.orbit.fitHeightFactor, + }; + return geoId; + }, + }, +}); + +export type PbrModelIblService = Database.Plugin.ToDatabase; diff --git a/packages/data-gpu-samples/src/samples/pbr-model-ibl/pbr-model-ibl.ts b/packages/data-gpu-samples/src/samples/pbr-model-ibl/pbr-model-ibl.ts new file mode 100644 index 00000000..92b3a216 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/pbr-model-ibl/pbr-model-ibl.ts @@ -0,0 +1,9 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html } from "lit"; +import type { TemplateResult } from "lit"; + +export const PbrModelIbl = (): TemplateResult => { + void import("./pbr-model-ibl-element.js"); + return html``; +}; diff --git a/packages/data-gpu-samples/src/samples/ragdoll/ragdoll-jolt-element.ts b/packages/data-gpu-samples/src/samples/ragdoll/ragdoll-jolt-element.ts new file mode 100644 index 00000000..abcdddbc --- /dev/null +++ b/packages/data-gpu-samples/src/samples/ragdoll/ragdoll-jolt-element.ts @@ -0,0 +1,50 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +// CesiumMan glTF © Cesium / Khronos Group, CC-BY 4.0 + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { ragdollJoltPlugin } from "./ragdoll-service.js"; +import { useOrbitCameraControl } from "../../hooks/use-orbit-camera-control.js"; + +const tagName = "ragdoll-jolt"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: RagdollJoltElement; + } +} + +const MODEL_URL = "https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Assets/main/Models/CesiumMan/glTF-Binary/CesiumMan.glb"; +const ENV_URL = "https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/studio_small_09_1k.hdr"; + +@customElement(tagName) +export class RagdollJoltElement extends DatabaseElement { + static styles = css` + :host { display: block; } + .stage { position: relative; } + canvas { display: block; border: 1px solid #333; background: #111; cursor: grab; } + canvas.dragging { cursor: grabbing; } + .hint { position: absolute; top: 0.5rem; left: 0.5rem; padding: 0.25rem 0.5rem; background: rgba(0,0,0,0.5); color: #aaa; font: 11px/1 ui-monospace, monospace; border-radius: 4px; } + `; + + get plugin() { return ragdollJoltPlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + service.transactions.seedStandardMaterials(); + service.transactions.initializeScene({ modelUrl: MODEL_URL, envUrl: ENV_URL }); + }, [canvas, service]); + useOrbitCameraControl(service); + return html` +
+ +
Jolt · native Ragdoll · drag to orbit
+
+ `; + } +} diff --git a/packages/data-gpu-samples/src/samples/ragdoll/ragdoll-rapier-element.ts b/packages/data-gpu-samples/src/samples/ragdoll/ragdoll-rapier-element.ts new file mode 100644 index 00000000..38122200 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/ragdoll/ragdoll-rapier-element.ts @@ -0,0 +1,50 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +// CesiumMan glTF © Cesium / Khronos Group, CC-BY 4.0 + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { ragdollRapierPlugin } from "./ragdoll-service.js"; +import { useOrbitCameraControl } from "../../hooks/use-orbit-camera-control.js"; + +const tagName = "ragdoll-rapier"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: RagdollRapierElement; + } +} + +const MODEL_URL = "https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Assets/main/Models/CesiumMan/glTF-Binary/CesiumMan.glb"; +const ENV_URL = "https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/studio_small_09_1k.hdr"; + +@customElement(tagName) +export class RagdollRapierElement extends DatabaseElement { + static styles = css` + :host { display: block; } + .stage { position: relative; } + canvas { display: block; border: 1px solid #333; background: #111; cursor: grab; } + canvas.dragging { cursor: grabbing; } + .hint { position: absolute; top: 0.5rem; left: 0.5rem; padding: 0.25rem 0.5rem; background: rgba(0,0,0,0.5); color: #aaa; font: 11px/1 ui-monospace, monospace; border-radius: 4px; } + `; + + get plugin() { return ragdollRapierPlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + service.transactions.seedStandardMaterials(); + service.transactions.initializeScene({ modelUrl: MODEL_URL, envUrl: ENV_URL }); + }, [canvas, service]); + useOrbitCameraControl(service); + return html` +
+ +
Rapier · free-ball ragdoll · drag to orbit
+
+ `; + } +} diff --git a/packages/data-gpu-samples/src/samples/ragdoll/ragdoll-service.ts b/packages/data-gpu-samples/src/samples/ragdoll/ragdoll-service.ts new file mode 100644 index 00000000..a5cebf54 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/ragdoll/ragdoll-service.ts @@ -0,0 +1,72 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { pbrRender, pbrSkinning, boneColliders, joltRagdoll, physicsRenderBridge, shapeGeometry, ragdollTrigger, rapierSolver, Model, Orbit } from "@adobe/data-gpu"; + +/** + * ragdoll — a rigged humanoid walks, then goes limp and collapses onto the floor. + * The same scene runs **side by side on both solvers** through a shared base + * (`ragdollScene`): per-bone capsules are auto-fitted from the skin and track the + * walk, then `triggerRagdoll` flips them to dynamic so the skinned mesh flops. + * + * The two panels use **different ragdoll backends** through the same scene + + * `ragdollTrigger`: Jolt runs its **native `Ragdoll`** (`joltRagdoll` — swing-twist + * limits, parent-child collision filtering, pose-driven), while Rapier runs **our + * generic `boneColliders`** (free-ball, since Rapier's binding has no cone joint). + */ +const ragdollScene = Database.Plugin.create({ + extends: Database.Plugin.combine(pbrRender, pbrSkinning, shapeGeometry, physicsRenderBridge, ragdollTrigger, Orbit.plugin), + transactions: { + initializeScene(t, args: { modelUrl: string; envUrl?: string }): number { + t.resources.light = { ...t.resources.light, environmentUrl: args.envUrl ?? t.resources.light.environmentUrl, color: [0.55, 0.55, 0.55] }; + const geoId = Model.plugin.transactions.insertGeometry(t, { modelUrl: args.modelUrl }); + Model.plugin.transactions.insertModel(t, { geometry: geoId, position: [0, 0.9, 0] }); // lifted, so the ragdoll drops onto the floor + t.archetypes.StaticCollider.insert({ + colliderShape: "box", halfExtents: [4, 0.25, 4], material: t.resources.materials.stone, + position: [0, -0.25, 0], rotation: [0, 0, 0, 1], + }); + t.resources.orbit = { ...t.resources.orbit, center: [0, 0.4, 0], radius: 3.2, height: 1.4, autoSpinSpeed: 0.15 }; + return geoId; + }, + }, + systems: { + // Start the model's first clip looping once skeleton + clips have loaded. + autoplayAnimation: { + schedule: { during: ["update"] }, + create: db => { + const started = new Set(); + return () => { + for (const arch of db.store.queryArchetypes(["_skeletonJoints", "_skeletonGeometry"])) { + const ids = arch.columns.id, jc = arch.columns._skeletonJoints, gc = arch.columns._skeletonGeometry; + for (let i = 0; i < arch.rowCount; i++) { + const skeleton = ids.get(i); + if (started.has(skeleton)) continue; + const clips = (db.store.read(gc.get(i)) as { _animationClipRefs?: number[] } | null)?._animationClipRefs ?? []; + if (clips.length === 0) continue; + db.store.archetypes.Animation.insert({ + animationClipRef: clips[0], animationTargets: [...jc.get(i)], + animationTime: 0, animationSpeed: 1, animationLoop: true, animationPlaying: true, + }); + started.add(skeleton); + } + } + }; + }, + }, + // Walk for a few seconds, then go limp (the active ragdoll backend handles it). + autoRagdoll: { + schedule: { during: ["update"] }, + create: db => { + let fired = false; + return () => { + if (fired || db.store.resources.frameTime.elapsed < 4) return; + db.transactions.triggerRagdoll(); + fired = true; + }; + }, + }, + }, +}); + +export const ragdollRapierPlugin = Database.Plugin.combine(ragdollScene, boneColliders, rapierSolver); +export const ragdollJoltPlugin = Database.Plugin.combine(ragdollScene, joltRagdoll); // joltRagdoll brings joltSolver diff --git a/packages/data-gpu-samples/src/samples/ragdoll/ragdoll.ts b/packages/data-gpu-samples/src/samples/ragdoll/ragdoll.ts new file mode 100644 index 00000000..c1dd4c4f --- /dev/null +++ b/packages/data-gpu-samples/src/samples/ragdoll/ragdoll.ts @@ -0,0 +1,20 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html } from "lit"; +import type { TemplateResult } from "lit"; + +/** + * A rigged humanoid walks then ragdolls, on both solvers side by side: Jolt + * (cone/swing-twist anatomical limits) next to Rapier (free-ball — its binding + * has no cone constraint). Same scene + skeleton through the shared base. + */ +export const Ragdoll = (): TemplateResult => { + void import("./ragdoll-jolt-element.js"); + void import("./ragdoll-rapier-element.js"); + return html` +
+ + +
+ `; +}; diff --git a/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-debug-render.ts b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-debug-render.ts new file mode 100644 index 00000000..5405e92f --- /dev/null +++ b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-debug-render.ts @@ -0,0 +1,202 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { graphics, physicsData, ColliderShape, SceneUniforms } from "@adobe/data-gpu"; +import { rigidStackShader } from "./rigid-stack-render.wgsl.js"; + +/** + * The original lightweight debug renderer for rigid-stack — flat Lambertian, + * bodies tinted by their material's baseColorFactor and brightened by speed, + * geometry vertex-pulled from packed instance buffers. Kept as a selectable + * alternative to `pbrRender`: combine `rigidStackDebugRender` instead of + * `pbrRender` (and drop shapeGeometry/physicsRenderBridge) to use it. + */ +const POSITION_STRIDE = 12; +const CUBE_STRIDE = 24; +const POSE_STRIDE = 32; +const PROPS_STRIDE = 64; +const MAX_INSTANCES = 600; +const BIN = 7; + +function unitSphere(rings: number, segments: number): { vertices: Float32Array; indices: Uint16Array } { + const verts: number[] = []; + for (let ring = 0; ring <= rings; ring++) { + const theta = (ring / rings) * Math.PI, y = Math.cos(theta), rs = Math.sin(theta); + for (let seg = 0; seg <= segments; seg++) { + const phi = (seg / segments) * Math.PI * 2; + verts.push(rs * Math.cos(phi), y, rs * Math.sin(phi)); + } + } + const indices: number[] = []; + const stride = segments + 1; + for (let ring = 0; ring < rings; ring++) for (let seg = 0; seg < segments; seg++) { + const a = ring * stride + seg, b = a + stride; + indices.push(a, b, a + 1, a + 1, b, b + 1); + } + return { vertices: new Float32Array(verts), indices: new Uint16Array(indices) }; +} + +function unitCube(): Float32Array { + const faces: { n: number[]; v: number[][] }[] = [ + { n: [1, 0, 0], v: [[1, -1, -1], [1, 1, -1], [1, 1, 1], [1, -1, 1]] }, + { n: [-1, 0, 0], v: [[-1, -1, 1], [-1, 1, 1], [-1, 1, -1], [-1, -1, -1]] }, + { n: [0, 1, 0], v: [[-1, 1, -1], [-1, 1, 1], [1, 1, 1], [1, 1, -1]] }, + { n: [0, -1, 0], v: [[-1, -1, 1], [-1, -1, -1], [1, -1, -1], [1, -1, 1]] }, + { n: [0, 0, 1], v: [[1, -1, 1], [1, 1, 1], [-1, 1, 1], [-1, -1, 1]] }, + { n: [0, 0, -1], v: [[-1, -1, -1], [-1, 1, -1], [1, 1, -1], [1, -1, -1]] }, + ]; + const out: number[] = []; + for (const f of faces) { + const [a, b, c, d] = f.v; + for (const v of [a, b, c, a, c, d]) out.push(v[0], v[1], v[2], f.n[0], f.n[1], f.n[2]); + } + return new Float32Array(out); +} + +function groundQuad(half: number, y: number): Float32Array { + const e = half + 1; + return new Float32Array([-e, y, -e, e, y, -e, e, y, e, -e, y, -e, e, y, e, -e, y, e]); +} + +interface RigidGpu { + spherePipeline: GPURenderPipeline; + boxPipeline: GPURenderPipeline; + groundPipeline: GPURenderPipeline; + sceneLayout: GPUBindGroupLayout; + bodyLayout: GPUBindGroupLayout; + sphereVB: GPUBuffer; sphereIB: GPUBuffer; sphereIndexCount: number; + cubeVB: GPUBuffer; cubeVertexCount: number; + groundVB: GPUBuffer; groundVertexCount: number; + poseBuffer: GPUBuffer; + propsBuffer: GPUBuffer; + sceneBG: Map; + bodyBG: GPUBindGroup; +} + +const RENDER_COMPONENTS = ["position", "rotation", "halfExtents", "colliderShape", "linearVelocity", "material"] as const; + +export const rigidStackDebugRender = Database.Plugin.create({ + extends: Database.Plugin.combine(graphics, physicsData, SceneUniforms.plugin), + resources: { + rigidGpu: { default: null as RigidGpu | null, transient: true }, + }, + systems: { + rigidRenderInit: { + schedule: { during: ["postUpdate"] }, + create: db => () => { + const { device, rigidGpu, canvasFormat, depthFormat } = db.store.resources; + if (!device || rigidGpu) return; + const module = device.createShaderModule({ code: rigidStackShader }); + const sceneLayout = device.createBindGroupLayout({ entries: [{ binding: 0, visibility: GPUShaderStage.VERTEX | GPUShaderStage.FRAGMENT, buffer: { type: "uniform" } }] }); + const bodyLayout = device.createBindGroupLayout({ entries: [ + { binding: 0, visibility: GPUShaderStage.VERTEX, buffer: { type: "read-only-storage" } }, + { binding: 1, visibility: GPUShaderStage.VERTEX, buffer: { type: "read-only-storage" } }, + ] }); + const vbLayout: GPUVertexBufferLayout = { arrayStride: POSITION_STRIDE, attributes: [{ shaderLocation: 0, offset: 0, format: "float32x3" }] }; + const cubeVbLayout: GPUVertexBufferLayout = { arrayStride: CUBE_STRIDE, attributes: [{ shaderLocation: 0, offset: 0, format: "float32x3" }, { shaderLocation: 1, offset: 12, format: "float32x3" }] }; + const depthStencil: GPUDepthStencilState = { format: depthFormat, depthWriteEnabled: true, depthCompare: "less" }; + const spherePipeline = device.createRenderPipeline({ layout: device.createPipelineLayout({ bindGroupLayouts: [sceneLayout, bodyLayout] }), vertex: { module, entryPoint: "vs_sphere", buffers: [vbLayout] }, fragment: { module, entryPoint: "fs_lit", targets: [{ format: canvasFormat }] }, primitive: { topology: "triangle-list", cullMode: "back" }, depthStencil }); + const boxPipeline = device.createRenderPipeline({ layout: device.createPipelineLayout({ bindGroupLayouts: [sceneLayout, bodyLayout] }), vertex: { module, entryPoint: "vs_box", buffers: [cubeVbLayout] }, fragment: { module, entryPoint: "fs_lit", targets: [{ format: canvasFormat }] }, primitive: { topology: "triangle-list", cullMode: "none" }, depthStencil }); + const groundPipeline = device.createRenderPipeline({ layout: device.createPipelineLayout({ bindGroupLayouts: [sceneLayout] }), vertex: { module, entryPoint: "vs_ground", buffers: [vbLayout] }, fragment: { module, entryPoint: "fs_ground", targets: [{ format: canvasFormat }] }, primitive: { topology: "triangle-list", cullMode: "none" }, depthStencil }); + + const sphere = unitSphere(12, 18); + const sphereVB = device.createBuffer({ size: sphere.vertices.byteLength, usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST }); + device.queue.writeBuffer(sphereVB, 0, sphere.vertices); + const sphereIB = device.createBuffer({ size: sphere.indices.byteLength, usage: GPUBufferUsage.INDEX | GPUBufferUsage.COPY_DST }); + device.queue.writeBuffer(sphereIB, 0, sphere.indices); + const cube = unitCube(); + const cubeVB = device.createBuffer({ size: cube.byteLength, usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST }); + device.queue.writeBuffer(cubeVB, 0, cube); + const ground = groundQuad(BIN, 0); + const groundVB = device.createBuffer({ size: ground.byteLength, usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST }); + device.queue.writeBuffer(groundVB, 0, ground); + + const poseBuffer = device.createBuffer({ size: MAX_INSTANCES * POSE_STRIDE, usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST }); + const propsBuffer = device.createBuffer({ size: MAX_INSTANCES * PROPS_STRIDE, usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST }); + const bodyBG = device.createBindGroup({ layout: bodyLayout, entries: [{ binding: 0, resource: { buffer: poseBuffer } }, { binding: 1, resource: { buffer: propsBuffer } }] }); + + db.store.resources.rigidGpu = { + spherePipeline, boxPipeline, groundPipeline, sceneLayout, bodyLayout, + sphereVB, sphereIB, sphereIndexCount: sphere.indices.length, + cubeVB, cubeVertexCount: cube.length / 6, + groundVB, groundVertexCount: ground.length / 3, + poseBuffer, propsBuffer, + sceneBG: new Map(), + bodyBG, + }; + }, + }, + rigidRender: { + schedule: { during: ["render"] }, + create: db => { + const pose = new Float32Array(MAX_INSTANCES * 8); + const props = new Float32Array(MAX_INSTANCES * 16); + let matColor = new Map(); + let matCount = -1; + return () => { + const gpu = db.store.resources.rigidGpu; + const { device, renderPassEncoder, _sceneUniformsBuffer } = db.store.resources; + if (!gpu || !device || !renderPassEncoder || !_sceneUniformsBuffer) return; + + let mc = 0; + for (const arch of db.store.queryArchetypes(["name", "baseColorFactor"])) mc += arch.rowCount; + if (mc !== matCount) { + matColor = new Map(); + for (const arch of db.store.queryArchetypes(["name", "baseColorFactor"])) { + const id = arch.columns.id, bc = arch.columns.baseColorFactor; + for (let r = 0; r < arch.rowCount; r++) { + const c = bc.get(r); + matColor.set(id.get(r), [c[0], c[1], c[2]]); + } + } + matCount = mc; + } + + let n = 0; + for (const arch of db.store.queryArchetypes(RENDER_COMPONENTS)) { + const pos = arch.columns.position, ori = arch.columns.rotation, he = arch.columns.halfExtents; + const cs = arch.columns.colliderShape, lv = arch.columns.linearVelocity, mt = arch.columns.material; + for (let r = 0; r < arch.rowCount && n < MAX_INSTANCES; r++, n++) { + const p = pos.get(r), q = ori.get(r), e = he.get(r), v = lv.get(r); + const shapeIdx = ColliderShape.toIndex(cs.get(r)); + const col = matColor.get(mt.get(r)) ?? [0.7, 0.7, 0.7]; + const bound = shapeIdx === 1 ? Math.hypot(e[0], e[1], e[2]) : e[0]; + pose[n * 8] = p[0]; pose[n * 8 + 1] = p[1]; pose[n * 8 + 2] = p[2]; pose[n * 8 + 3] = bound; + pose[n * 8 + 4] = q[0]; pose[n * 8 + 5] = q[1]; pose[n * 8 + 6] = q[2]; pose[n * 8 + 7] = q[3]; + props[n * 16] = v[0]; props[n * 16 + 1] = v[1]; props[n * 16 + 2] = v[2]; + props[n * 16 + 4] = col[0]; props[n * 16 + 5] = col[1]; props[n * 16 + 6] = col[2]; + props[n * 16 + 12] = e[0]; props[n * 16 + 13] = e[1]; props[n * 16 + 14] = e[2]; props[n * 16 + 15] = shapeIdx; + } + } + if (n === 0) return; + device.queue.writeBuffer(gpu.poseBuffer, 0, pose, 0, n * 8); + device.queue.writeBuffer(gpu.propsBuffer, 0, props, 0, n * 16); + + let sceneBG = gpu.sceneBG.get(_sceneUniformsBuffer); + if (!sceneBG) { + sceneBG = device.createBindGroup({ layout: gpu.sceneLayout, entries: [{ binding: 0, resource: { buffer: _sceneUniformsBuffer } }] }); + gpu.sceneBG.set(_sceneUniformsBuffer, sceneBG); + } + + renderPassEncoder.setPipeline(gpu.groundPipeline); + renderPassEncoder.setBindGroup(0, sceneBG); + renderPassEncoder.setVertexBuffer(0, gpu.groundVB); + renderPassEncoder.draw(gpu.groundVertexCount); + + renderPassEncoder.setPipeline(gpu.spherePipeline); + renderPassEncoder.setBindGroup(0, sceneBG); + renderPassEncoder.setBindGroup(1, gpu.bodyBG); + renderPassEncoder.setVertexBuffer(0, gpu.sphereVB); + renderPassEncoder.setIndexBuffer(gpu.sphereIB, "uint16"); + renderPassEncoder.drawIndexed(gpu.sphereIndexCount, n); + + renderPassEncoder.setPipeline(gpu.boxPipeline); + renderPassEncoder.setBindGroup(0, sceneBG); + renderPassEncoder.setBindGroup(1, gpu.bodyBG); + renderPassEncoder.setVertexBuffer(0, gpu.cubeVB); + renderPassEncoder.draw(gpu.cubeVertexCount, n); + }; + }, + }, + }, +}); diff --git a/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-jolt-element.ts b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-jolt-element.ts new file mode 100644 index 00000000..f841d93c --- /dev/null +++ b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-jolt-element.ts @@ -0,0 +1,50 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { rigidStackJoltPlugin } from "./rigid-stack-service.js"; +import { useOrbitCameraControl } from "../../hooks/use-orbit-camera-control.js"; + +const tagName = "rigid-stack-jolt"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: RigidStackJoltElement; + } +} + +/** Same scene + render path as the CPU element, driven by the Jolt solver plugin. */ +@customElement(tagName) +export class RigidStackJoltElement extends DatabaseElement { + static styles = css` + :host { display: block; } + .stage { position: relative; } + canvas { display: block; border: 1px solid #333; background: #000; cursor: grab; } + canvas.dragging { cursor: grabbing; } + .hint { position: absolute; top: 0.5rem; left: 0.5rem; padding: 0.25rem 0.5rem; background: rgba(0,0,0,0.5); color: #aaa; font: 11px/1 ui-monospace, monospace; border-radius: 4px; } + `; + + get plugin() { return rigidStackJoltPlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + service.transactions.seedStandardMaterials(); + service.transactions.initializeScene(); + }, [canvas, service]); + + useOrbitCameraControl(service); + + return html` +
+ +
Jolt (default) · drag to orbit
+
+ `; + } +} diff --git a/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-rapier-element.ts b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-rapier-element.ts new file mode 100644 index 00000000..9d0375f8 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-rapier-element.ts @@ -0,0 +1,51 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { rigidStackRapierPlugin } from "./rigid-stack-service.js"; +import { useOrbitCameraControl } from "../../hooks/use-orbit-camera-control.js"; + +const tagName = "rigid-stack-rapier"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: RigidStackRapierElement; + } +} + +/** Same scene + render path as the CPU element, driven by the Rapier solver + * plugin instead — the side-by-side reference. */ +@customElement(tagName) +export class RigidStackRapierElement extends DatabaseElement { + static styles = css` + :host { display: block; } + .stage { position: relative; } + canvas { display: block; border: 1px solid #333; background: #000; cursor: grab; } + canvas.dragging { cursor: grabbing; } + .hint { position: absolute; top: 0.5rem; left: 0.5rem; padding: 0.25rem 0.5rem; background: rgba(0,0,0,0.5); color: #aaa; font: 11px/1 ui-monospace, monospace; border-radius: 4px; } + `; + + get plugin() { return rigidStackRapierPlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + service.transactions.seedStandardMaterials(); + service.transactions.initializeScene(); + }, [canvas, service]); + + useOrbitCameraControl(service); + + return html` +
+ +
Rapier (alt: many dynamics) · drag to orbit
+
+ `; + } +} diff --git a/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-render.wgsl.ts b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-render.wgsl.ts new file mode 100644 index 00000000..03e897ff --- /dev/null +++ b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-render.wgsl.ts @@ -0,0 +1,99 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +/** + * Debug render for the rigid-stack sample. Like physics-drop, bodies are + * instanced and vertex-pulled from two storage buffers, but the CPU solver + * packs different `props`: + * - `pose` (2 vec4f/body): pos + boundingRadius, orientation quat. + * - `props` (4 vec4f/body): vel + _, materialColor + _, _, halfExtent + shape. + * + * Bodies are tinted by their material's base albedo (so wood/steel/ice read at + * a glance) and brightened — hue preserved — by speed, so moving bodies pop. + */ +export const rigidStackShader = /* wgsl */ ` +struct SceneUniforms { + viewProjectionMatrix: mat4x4, + lightDirection: vec3f, + ambientStrength: f32, + lightColor: vec3f, + cameraPosition: vec3f, +} + +@group(0) @binding(0) var scene: SceneUniforms; +@group(1) @binding(0) var pose: array; // 2 / body +@group(1) @binding(1) var props: array; // 4 / body + +fn qRot(q: vec4f, v: vec3f) -> vec3f { + let t = 2.0 * cross(q.xyz, v); + return v + q.w * t + cross(q.xyz, t); +} + +fn bodyColor(ii: u32) -> vec3f { + let base = props[ii * 4u + 1u].xyz; // material albedo + let speed = length(props[ii * 4u].xyz); + // brighten while moving (hue retained); resting bodies show true material. + return base * (1.0 + clamp(speed / 6.0, 0.0, 1.0) * 1.4); +} + +struct VOut { + @builtin(position) clip: vec4f, + @location(0) normal: vec3f, + @location(1) color: vec3f, +} + +@vertex +fn vs_sphere(@builtin(instance_index) ii: u32, @location(0) meshPos: vec3f) -> VOut { + let p = pose[ii * 2u]; + let isSphere = props[ii * 4u + 3u].w < 0.5; + let r = select(0.0, p.w, isSphere); + var out: VOut; + out.clip = scene.viewProjectionMatrix * vec4f(p.xyz + meshPos * r, 1.0); + out.normal = normalize(meshPos); + out.color = bodyColor(ii); + return out; +} + +@vertex +fn vs_box(@builtin(instance_index) ii: u32, @location(0) cubePos: vec3f, @location(1) cubeNormal: vec3f) -> VOut { + let center = pose[ii * 2u].xyz; + let q = pose[ii * 2u + 1u]; + let he = props[ii * 4u + 3u].xyz; + let isBox = props[ii * 4u + 3u].w > 0.5; + let scale = select(vec3f(0.0), he, isBox); + var out: VOut; + out.clip = scene.viewProjectionMatrix * vec4f(center + qRot(q, cubePos * scale), 1.0); + out.normal = qRot(q, cubeNormal); + out.color = bodyColor(ii); + return out; +} + +@fragment +fn fs_lit(in: VOut) -> @location(0) vec4f { + let n = normalize(in.normal); + let L = normalize(-scene.lightDirection); + let diff = max(dot(n, L), 0.0); + let lit = in.color * (scene.ambientStrength + diff * scene.lightColor); + return vec4f(lit, 1.0); +} + +struct GOut { + @builtin(position) clip: vec4f, + @location(0) world: vec3f, +} + +@vertex +fn vs_ground(@location(0) pos: vec3f) -> GOut { + var out: GOut; + out.clip = scene.viewProjectionMatrix * vec4f(pos, 1.0); + out.world = pos; + return out; +} + +@fragment +fn fs_ground(in: GOut) -> @location(0) vec4f { + let c = floor(in.world.x) + floor(in.world.z); + let odd = (c - 2.0 * floor(c * 0.5)) < 0.5; + let shade = select(0.10, 0.16, odd); + return vec4f(vec3f(shade), 1.0); +} +`; diff --git a/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-service.ts b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-service.ts new file mode 100644 index 00000000..01fb2711 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack-service.ts @@ -0,0 +1,260 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, type Entity } from "@adobe/data/ecs"; +import { Quat } from "@adobe/data/math"; +import { pbrRender, rapierSolver, joltSolver, shapeGeometry, physicsRenderBridge, modelCollider, jointData, ColliderShape, Orbit } from "@adobe/data-gpu"; + +// Studio HDR for IBL © Poly Haven, CC0. +const ENV_URL = "https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/studio_small_09_1k.hdr"; +// DamagedHelmet glTF © Khronos Group, CC-BY 4.0 — dropped as a dynamic body whose +// convex-hull collider is auto-generated from its mesh (modelCollider). +const HELMET_URL = "https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Assets/main/Models/DamagedHelmet/glTF-Binary/DamagedHelmet.glb"; + +// Scene config. The bin is split into two zones so the moving bar doesn't +// demolish the stack: the **right** half holds the stable stack; the **left** +// half is the dynamic playground (sweeping bar + ramp). Drops rain across the +// whole bin, so some land on the stack while the bar — confined to the left — +// never reaches it. +const BIN = 12; // half-extent of the floor / containing walls +const STACK_W = 4, STACK_D = 4, STACK_H = 4; // dynamic block stack (unit cubes) +const STACK_CX = 6.5; // stack sits in the right zone, clear of the bar +const SPAWN_INTERVAL = 0.18; // seconds between dynamic drops +const SPAWN_DELAY = 2.5; // let the bare stack settle first, to verify it holds +const DYNAMIC_CAP = 120; // stop spawning at this many dropped bodies +const SPAWN_CX = 1.0; // drops rain across the whole bin — some top the stack +const SPAWN_SPREAD = 8.0; // (right), the rest feed the bar + ramp (left) +const SPAWN_HEIGHT = 14; +// Kinematic bar: sweeps only the left zone (centre −4.5 ± 4.5 ⇒ x ∈ [−9, 0]), so +// it churns the dropped bodies and ramp without ever reaching the stack at +6.5. +const SWEEP_CX = -4.5, SWEEP_AMP = 4.5, SWEEP_SPEED = 0.7, SWEEP_Y = 1.0; +const IDENTITY: [number, number, number, number] = [...Quat.identity]; + +/** + * One deterministic drop. The sequence is precomputed from a fixed seed so that + * two services running different solvers (Rapier vs Jolt) get the *identical* + * scene — a fair side-by-side comparison. + */ +interface Drop { shape: ColliderShape; he: [number, number, number]; pos: [number, number, number]; quat: [number, number, number, number]; points?: Float32Array; } + +function seededRng(seed: number): () => number { + let a = seed >>> 0; + return () => { a |= 0; a = (a + 0x6d2b79f5) | 0; let t = Math.imul(a ^ (a >>> 15), 1 | a); t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; return ((t ^ (t >>> 14)) >>> 0) / 4294967296; }; +} +function randomQuat(r: () => number): [number, number, number, number] { + const u1 = r(), u2 = r(), u3 = r(); + const a = Math.sqrt(1 - u1), b = Math.sqrt(u1); + return [a * Math.sin(2 * Math.PI * u2), a * Math.cos(2 * Math.PI * u2), b * Math.sin(2 * Math.PI * u3), b * Math.cos(2 * Math.PI * u3)]; +} +/** A small cloud of points scattered near a sphere's surface — the solver hulls + * it into a random convex polyhedron (gem-like). */ +function randomHullPoints(r: () => number, count: number, radius: number): Float32Array { + const out = new Float32Array(count * 3); + for (let i = 0; i < count; i++) { + const u = r() * 2 - 1, theta = r() * 2 * Math.PI, s = Math.sqrt(1 - u * u), rad = radius * (0.7 + 0.3 * r()); + out[i * 3] = rad * s * Math.cos(theta); out[i * 3 + 1] = rad * u; out[i * 3 + 2] = rad * s * Math.sin(theta); + } + return out; +} +const DROPS: Drop[] = (() => { + const r = seededRng(0x1234abcd), out: Drop[] = []; + for (let i = 0; i < DYNAMIC_CAP; i++) { + const k = r(); + const shape: ColliderShape = k < 0.28 ? "box" : k < 0.5 ? "capsule" : k < 0.72 ? "hull" : "sphere"; + // box: 3 half-extents; capsule: [radius, cyl half-height]; hull: halfExtents unused; sphere: [radius] + const he: [number, number, number] = shape === "box" ? [0.3 + r() * 0.3, 0.3 + r() * 0.3, 0.3 + r() * 0.3] + : shape === "capsule" ? [0.28 + r() * 0.15, 0.35 + r() * 0.35, 0] + : [0.3 + r() * 0.4, 0, 0]; + out.push({ + shape, he, + pos: [SPAWN_CX + (r() * 2 - 1) * SPAWN_SPREAD, SPAWN_HEIGHT, (r() * 2 - 1) * SPAWN_SPREAD], + quat: shape === "sphere" ? [0, 0, 0, 1] : randomQuat(r), + points: shape === "hull" ? randomHullPoints(r, 9, 0.45 + r() * 0.2) : undefined, + }); + } + return out; +})(); + +/** + * rigid-stack scene — a 4×4×4 dynamic wood stack rests on a stone floor inside a + * walled bin; mixed-material bodies drop in and knock it around, rendered + * through the unified PBR + IBL path. Floor and walls are immovable + * `StaticCollider` boxes; every body becomes renderable via `physicsRenderBridge`. + * + * This **base** plugin is solver-agnostic — scene + spawning only. A solver is + * combined in below (`rapierSolver` or `joltSolver`); the same scene runs on + * either through the shared `physicsData` seam, so the two are compared side by + * side. The drop sequence is deterministic (seeded), so both see it identically. + */ +const rigidStackScene = Database.Plugin.create({ + extends: Database.Plugin.combine(pbrRender, shapeGeometry, physicsRenderBridge, modelCollider, jointData, Orbit.plugin), + resources: { + _spawnAccum: { default: 0 as number, transient: true }, + _spawnElapsed: { default: 0 as number, transient: true }, + _spawnedDynamic: { default: 0 as number, transient: true }, + _sweeper: { default: 0 as Entity, transient: true }, // the kinematic bar + }, + transactions: { + initializeScene(t) { + t.resources.orbit = { + ...t.resources.orbit, + center: [0, 1, 0], radius: 30, height: 22, autoSpinSpeed: 0.12, + }; + t.resources.light = { + ...t.resources.light, + environmentUrl: ENV_URL, + direction: [-2, -5, -3], color: [1.0, 0.98, 0.92], ambientStrength: 0.4, + }; + // Stone bin: a floor slab (top face at y = 0) and four walls, all + // immovable StaticCollider boxes. The render bridge gives them + // geometry once the shape meshes load — no separate render-only prop. + const stone = t.resources.materials.stone; + const wall = (position: [number, number, number], halfExtents: [number, number, number]) => + t.archetypes.StaticCollider.insert({ colliderShape: "box", halfExtents, material: stone, position, rotation: IDENTITY }); + wall([0, -0.5, 0], [BIN + 1, 0.5, BIN + 1]); // floor slab (top at y = 0) + const WH = 3; // wall half-height (walls 0 → 2·WH) + wall([ BIN, WH, 0], [0.5, WH, BIN + 1]); // +x + wall([-BIN, WH, 0], [0.5, WH, BIN + 1]); // −x + wall([0, WH, BIN], [BIN + 1, WH, 0.5]); // +z + wall([0, WH, -BIN], [BIN + 1, WH, 0.5]); // −z + // A static triangle-mesh ramp (level geometry) leaning into one corner — + // dropped bodies land on it and slide down. World-space verts; up-facing + // winding so it renders (front faces) and collides (trimesh is two-sided). + t.archetypes.MeshCollider.insert({ + colliderShape: "mesh", halfExtents: [0, 0, 0], material: t.resources.materials.steel, + position: [0, 0, 0], rotation: IDENTITY, + colliderMesh: { + // sloped quad in the left zone: high at the −x wall, low toward centre + positions: new Float32Array([-BIN + 1.5, 4.5, 6, -BIN + 1.5, 4.5, -6, -1, 0.6, -6, -1, 0.6, 6]), + indices: new Uint32Array([0, 2, 1, 0, 3, 2]), + }, + }); + // Dynamic block stack: a grid of unit cubes resting on the floor. + // A small gap on every axis avoids initial face-coincidence + // (degenerate SAT normals); they settle into contact. + const wood = t.resources.materials.wood; + const GAP = 1.04; + const x0 = -(STACK_W - 1) / 2 * GAP, z0 = -(STACK_D - 1) / 2 * GAP; + for (let y = 0; y < STACK_H; y++) { + for (let x = 0; x < STACK_W; x++) { + for (let z = 0; z < STACK_D; z++) { + t.archetypes.RigidBody.insert({ + bodyType: "dynamic", colliderShape: "box", halfExtents: [0.5, 0.5, 0.5], material: wood, + position: [STACK_CX + x0 + x * GAP, 0.55 + y * 1.04, z0 + z * GAP], + rotation: IDENTITY, linearVelocity: [0, 0, 0], angularVelocity: [0, 0, 0], + }); + } + } + } + // A kinematic steel bar that sweeps across the bin, plowing the stack — + // its pose is authored each frame by the `sweep` system; the solver moves + // it as a kinematic body that pushes the dynamics but is never pushed back. + t.resources._sweeper = t.archetypes.RigidBody.insert({ + bodyType: "kinematic", colliderShape: "box", halfExtents: [0.4, 1.0, BIN - 1], material: t.resources.materials.steel, + position: [SWEEP_CX - SWEEP_AMP, SWEEP_Y, 0], rotation: IDENTITY, linearVelocity: [0, 0, 0], angularVelocity: [0, 0, 0], + }); + // A downloaded glTF model dropped as dynamic bodies: it renders in full + // detail but collides as a convex hull auto-generated from its mesh + // (colliderShape "hull" + no collision data ⇒ modelCollider fills it). One + // shared Geometry, three staggered instances. (To hand-author the collider + // instead, pass `convexPoints`/`colliderMesh` here and generation is skipped.) + const helmet = t.archetypes.Geometry.insert({ modelUrl: HELMET_URL }); + for (let i = 0; i < 3; i++) { + t.archetypes.ModelBody.insert({ + geometry: helmet, scale: [1.5, 1.5, 1.5], visible: true, parent: 0, + bodyType: "dynamic", colliderShape: "hull", halfExtents: [0, 0, 0], material: t.resources.materials.steel, + position: [SPAWN_CX - 2 + i * 2, 15 + i * 5, -1 + i], + rotation: randomQuat(seededRng(0x5eed + i)), linearVelocity: [0, 0, 0], angularVelocity: [0, 0, 0], + }); + } + // A hanging chain: a static anchor + capsule links joined by `point` (ball) + // joints, so it swings freely and the sweeping bar knocks it around. Each + // link's ends are at ±(halfHeight + radius) in its local Y; consecutive + // anchors coincide in world space, so the chain forms taut. + const LINK_HY = 0.4, LINK_R = 0.22, LINK_END = LINK_HY + LINK_R, LINK_LEN = 2 * LINK_END; + const CX = SWEEP_CX, CZ = BIN - 3, ANCHOR_Y = 9, ANCHOR_HH = 0.25; + const steel = t.resources.materials.steel; + const anchor = t.archetypes.StaticCollider.insert({ + colliderShape: "box", halfExtents: [0.3, ANCHOR_HH, 0.3], material: steel, + position: [CX, ANCHOR_Y, CZ], rotation: IDENTITY, + }); + let prev = anchor, prevBottom: [number, number, number] = [0, -ANCHOR_HH, 0]; + for (let i = 0; i < 6; i++) { + const link = t.archetypes.RigidBody.insert({ + bodyType: "dynamic", colliderShape: "capsule", halfExtents: [LINK_R, LINK_HY, 0], material: steel, + position: [CX, ANCHOR_Y - ANCHOR_HH - LINK_END - i * LINK_LEN, CZ], + rotation: IDENTITY, linearVelocity: [0, 0, 0], angularVelocity: [0, 0, 0], + }); + t.archetypes.Joint.insert({ + jointType: "point", jointBodyA: prev, jointBodyB: link, + jointAnchorA: prevBottom, jointAnchorB: [0, LINK_END, 0], + jointAxis: [0, 0, 1], jointMinLimit: 0, jointMaxLimit: 0, jointSwingLimit: 0, + }); + prev = link; prevBottom = [0, -LINK_END, 0]; + } + // A cone (swing-twist) joint, off in a clear corner: a horizontal capsule + // arm pinned at its inner end to a floating post, free to swing only within + // a ~36° cone of the +x reference. Gravity droops it; on joltSolver the cone + // stops it at the limit (it sticks out, leaning), while rapierSolver (no cone + // limit in its binding) lets it hang straight down — the documented difference. + const ARM_END = 1.1, ARM_ROT: [number, number, number, number] = [0, 0, -Math.SQRT1_2, Math.SQRT1_2]; // local +Y → world +x + const post = t.archetypes.StaticCollider.insert({ + colliderShape: "box", halfExtents: [0.3, 0.3, 0.3], material: steel, + position: [-3, 5, -BIN + 1.5], rotation: IDENTITY, + }); + const arm = t.archetypes.RigidBody.insert({ + bodyType: "dynamic", colliderShape: "capsule", halfExtents: [0.2, 0.9, 0], material: steel, + position: [-3 + ARM_END, 5, -BIN + 1.5], rotation: ARM_ROT, linearVelocity: [0, 0, 0], angularVelocity: [0, 0, 0], + }); + t.archetypes.Joint.insert({ + jointType: "cone", jointBodyA: post, jointBodyB: arm, + jointAnchorA: [0, 0, 0], jointAnchorB: [0, -ARM_END, 0], + jointAxis: [1, 0, 0], jointMinLimit: 0, jointMaxLimit: 0, jointSwingLimit: Math.PI / 5, + }); + }, + spawnBody(t, args: { index: number }) { + const d = DROPS[args.index]; + const material = Object.values(t.resources.materials)[args.index % Object.keys(t.resources.materials).length]; + const common = { + bodyType: "dynamic" as const, colliderShape: d.shape, halfExtents: d.he, material, + position: d.pos, rotation: d.quat, linearVelocity: [0, 0, 0] as [number, number, number], angularVelocity: [0, 0, 0] as [number, number, number], + }; + // hull bodies carry their authored point cloud (the ConvexBody archetype) + if (d.shape === "hull" && d.points) t.archetypes.ConvexBody.insert({ ...common, convexPoints: d.points }); + else t.archetypes.RigidBody.insert(common); + }, + }, + systems: { + // Drop a dynamic body every SPAWN_INTERVAL until the cap. + spawner: { + schedule: { during: ["update"] }, + create: db => () => { + if (db.store.resources._spawnedDynamic >= DYNAMIC_CAP) return; + const dt = db.store.resources.frameTime.dt; + const elapsed = db.store.resources._spawnElapsed + dt; + db.store.resources._spawnElapsed = elapsed; + if (elapsed < SPAWN_DELAY) return; + let accum = db.store.resources._spawnAccum + dt; + while (accum >= SPAWN_INTERVAL && db.store.resources._spawnedDynamic < DYNAMIC_CAP) { + accum -= SPAWN_INTERVAL; + db.transactions.spawnBody({ index: db.store.resources._spawnedDynamic }); + db.store.resources._spawnedDynamic = db.store.resources._spawnedDynamic + 1; + } + db.store.resources._spawnAccum = accum; + }, + }, + // Author the kinematic bar's pose: a horizontal sweep along x. The solver + // reads this pose and drives the kinematic body to it each step. + sweep: { + schedule: { during: ["update"] }, + create: db => () => { + const id = db.store.resources._sweeper; + if (!id) return; + const x = SWEEP_CX + Math.sin(db.store.resources.frameTime.elapsed * SWEEP_SPEED - Math.PI / 2) * SWEEP_AMP; + db.store.update(id, { position: [x, SWEEP_Y, 0] }); + }, + }, + }, +}); + +export const rigidStackRapierPlugin = Database.Plugin.combine(rigidStackScene, rapierSolver); +export const rigidStackJoltPlugin = Database.Plugin.combine(rigidStackScene, joltSolver); diff --git a/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack.ts b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack.ts new file mode 100644 index 00000000..60d4e111 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/rigid-stack/rigid-stack.ts @@ -0,0 +1,21 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html } from "lit"; +import type { TemplateResult } from "lit"; + +/** + * Same scene, two reference solvers side by side: Jolt (the default) next to + * Rapier. Both run the identical (seeded) drop sequence through the shared + * `physicsData` seam, so the comparison is apples-to-apples. See the solver + * guidance in @adobe/data-gpu's physics/solvers/README.md. + */ +export const RigidStack = (): TemplateResult => { + void import("./rigid-stack-jolt-element.js"); + void import("./rigid-stack-rapier-element.js"); + return html` +
+ + +
+ `; +}; diff --git a/packages/data-gpu-samples/src/samples/skinned-fox/skinned-fox-element.ts b/packages/data-gpu-samples/src/samples/skinned-fox/skinned-fox-element.ts new file mode 100644 index 00000000..d8a37a43 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/skinned-fox/skinned-fox-element.ts @@ -0,0 +1,57 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +// Fox glTF © Khronos Group, CC-BY 4.0 + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { skinnedFoxPlugin } from "./skinned-fox-service.js"; +import { useOrbitCameraControl } from "../../hooks/use-orbit-camera-control.js"; + +const tagName = "skinned-fox"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: SkinnedFoxElement; + } +} + +const MODEL_URL = "https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Assets/main/Models/Fox/glTF-Binary/Fox.glb"; +const ENV_URL = "https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/studio_small_09_1k.hdr"; + +@customElement(tagName) +export class SkinnedFoxElement extends DatabaseElement { + static styles = css` + :host { display: block; } + .stage { position: relative; } + canvas { display: block; border: 1px solid #333; background: #111; cursor: grab; } + canvas.dragging { cursor: grabbing; } + .hint { position: absolute; top: 0.5rem; right: 0.5rem; padding: 0.25rem 0.5rem; background: rgba(0,0,0,0.5); color: #aaa; font: 11px/1 ui-monospace, monospace; border-radius: 4px; } + `; + + get plugin() { return skinnedFoxPlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + service.transactions.initializeScene({ + modelUrl: MODEL_URL, + envUrl: ENV_URL, + lightColor: [0.3, 0.3, 0.3], + orbitFit: { radiusFactor: 1.6, heightFactor: 0.4 }, + }); + }, [canvas, service]); + + useOrbitCameraControl(service); + + return html` +
+ +
drag to orbit
+
+ `; + } +} diff --git a/packages/data-gpu-samples/src/samples/skinned-fox/skinned-fox-service.ts b/packages/data-gpu-samples/src/samples/skinned-fox/skinned-fox-service.ts new file mode 100644 index 00000000..2ffd6503 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/skinned-fox/skinned-fox-service.ts @@ -0,0 +1,34 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Vec3 } from "@adobe/data/math"; +import { pbrRender, pbrSkinning, Model, Orbit } from "@adobe/data-gpu"; + +export const skinnedFoxPlugin = Database.Plugin.create({ + extends: Database.Plugin.combine(pbrRender, pbrSkinning, Orbit.plugin), + transactions: { + initializeScene(t, args: { + modelUrl: string; + envUrl?: string; + lightColor?: Vec3; + orbitFit?: { radiusFactor: number; heightFactor: number }; + }): number { + t.resources.light = { + ...t.resources.light, + environmentUrl: args.envUrl ?? t.resources.light.environmentUrl, + color: args.lightColor ?? t.resources.light.color, + }; + const geoId = Model.plugin.transactions.insertGeometry(t, { modelUrl: args.modelUrl }); + Model.plugin.transactions.insertModel(t, { geometry: geoId }); + t.resources.orbit = { + ...t.resources.orbit, + fitGeometry: geoId, + fitRadiusFactor: args.orbitFit?.radiusFactor ?? t.resources.orbit.fitRadiusFactor, + fitHeightFactor: args.orbitFit?.heightFactor ?? t.resources.orbit.fitHeightFactor, + }; + return geoId; + }, + }, +}); + +export type SkinnedFoxService = Database.Plugin.ToDatabase; diff --git a/packages/data-gpu-samples/src/samples/skinned-fox/skinned-fox.ts b/packages/data-gpu-samples/src/samples/skinned-fox/skinned-fox.ts new file mode 100644 index 00000000..f44bbdfa --- /dev/null +++ b/packages/data-gpu-samples/src/samples/skinned-fox/skinned-fox.ts @@ -0,0 +1,9 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html } from "lit"; +import type { TemplateResult } from "lit"; + +export const SkinnedFox = (): TemplateResult => { + void import("./skinned-fox-element.js"); + return html``; +}; diff --git a/packages/data-gpu-samples/src/samples/solar-system/create-sphere.ts b/packages/data-gpu-samples/src/samples/solar-system/create-sphere.ts new file mode 100644 index 00000000..8f061663 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/solar-system/create-sphere.ts @@ -0,0 +1,60 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +/** + * Produces unit-radius UV-sphere GPU buffers laid out for StandardVertex + * (position + normal + tangent + uv). Only used by the solar-system sample + * to procedurally generate planet geometry. + */ +export function createSphereBuffers( + device: GPUDevice, + rings: number, + segments: number, +): { vertexBuffer: GPUBuffer; indexBuffer: GPUBuffer; indexCount: number; indexFormat: GPUIndexFormat } { + const verts: number[] = []; + const idxs: number[] = []; + + for (let r = 0; r <= rings; r++) { + const phi = (r / rings) * Math.PI; + const y = Math.cos(phi); + const sinPhi = Math.sin(phi); + for (let s = 0; s <= segments; s++) { + const theta = (s / segments) * 2 * Math.PI; + const cosT = Math.cos(theta); + const sinT = Math.sin(theta); + const x = sinPhi * cosT; + const z = sinPhi * sinT; + verts.push( + x, y, z, + x, y, z, + -sinT, 0, cosT, 1, + s / segments, r / rings, + ); + } + } + + for (let r = 0; r < rings; r++) { + for (let s = 0; s < segments; s++) { + const a = r * (segments + 1) + s; + const b = a + 1; + const c = a + segments + 1; + const d = c + 1; + idxs.push(a, c, b, b, c, d); + } + } + + const vertexData = new Float32Array(verts); + const vertexBuffer = device.createBuffer({ + size: vertexData.byteLength, + usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(vertexBuffer, 0, vertexData); + + const indexData = new Uint16Array(idxs); + const indexBuffer = device.createBuffer({ + size: indexData.byteLength, + usage: GPUBufferUsage.INDEX | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(indexBuffer, 0, indexData); + + return { vertexBuffer, indexBuffer, indexCount: idxs.length, indexFormat: "uint16" }; +} diff --git a/packages/data-gpu-samples/src/samples/solar-system/solar-system-element.ts b/packages/data-gpu-samples/src/samples/solar-system/solar-system-element.ts new file mode 100644 index 00000000..52094387 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/solar-system/solar-system-element.ts @@ -0,0 +1,55 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html, css } from "lit"; +import { customElement } from "lit/decorators.js"; +import { DatabaseElement, useEffect, useElement } from "@adobe/data-lit"; +import { solarSystemPlugin } from "./solar-system-service.js"; +import { useOrbitCameraControl } from "../../hooks/use-orbit-camera-control.js"; + +const tagName = "solar-system"; + +declare global { + interface HTMLElementTagNameMap { + [tagName]: SolarSystemElement; + } +} + +const ENV_URL = "https://dl.polyhaven.org/file/ph-assets/HDRIs/hdr/1k/studio_small_09_1k.hdr"; + +@customElement(tagName) +export class SolarSystemElement extends DatabaseElement { + static styles = css` + :host { display: block; } + .stage { position: relative; } + canvas { display: block; border: 1px solid #333; background: #000; cursor: grab; } + canvas.dragging { cursor: grabbing; } + .hint { position: absolute; top: 0.5rem; right: 0.5rem; padding: 0.25rem 0.5rem; background: rgba(0,0,0,0.5); color: #aaa; font: 11px/1 ui-monospace, monospace; border-radius: 4px; } + `; + + get plugin() { return solarSystemPlugin; } + + override render() { + const canvas = useElement("canvas"); + const service = this.service; + + useEffect(() => { + if (!canvas) return; + service.transactions.setCanvas(canvas); + service.transactions.setLight({ environmentUrl: ENV_URL, color: [0.1, 0.1, 0.1] }); + service.transactions.initializeScene(); + }, [canvas, service]); + + useOrbitCameraControl(service); + + const onClick = (e: MouseEvent) => { + service.actions.pickAndFit({ x: e.offsetX, y: e.offsetY }); + }; + + return html` +
+ +
drag to orbit · click to focus
+
+ `; + } +} diff --git a/packages/data-gpu-samples/src/samples/solar-system/solar-system-service.ts b/packages/data-gpu-samples/src/samples/solar-system/solar-system-service.ts new file mode 100644 index 00000000..2c6b8795 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/solar-system/solar-system-service.ts @@ -0,0 +1,129 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Vec3 } from "@adobe/data/math"; +import { + animation, + Model, + Orbit, + pbrRender, + picking, + type AnimationTrack, +} from "@adobe/data-gpu"; +import { sphere } from "./sphere-plugin.js"; + +const TWO_PI = Math.PI * 2; +const ORBIT_SEGMENTS = 64; + +function buildOrbitTrack(radius: number): AnimationTrack { + const times = new Float32Array(ORBIT_SEGMENTS + 1); + const values = new Float32Array((ORBIT_SEGMENTS + 1) * 3); + for (let i = 0; i <= ORBIT_SEGMENTS; i++) { + const t = (i / ORBIT_SEGMENTS) * TWO_PI; + times[i] = t; + values[i * 3] = Math.cos(t) * radius; + values[i * 3 + 2] = Math.sin(t) * radius; + } + return { targetIndex: 0, component: "position", times, values, interpolation: "linear" }; +} + +interface PlanetSpec { + color: [number, number, number, number]; + metallic: number; + roughness: number; + emissive?: Vec3; + position: Vec3; + scale: Vec3; + parent?: number; + orbitRadius?: number; + orbitSpeed?: number; +} + +export const solarSystemPlugin = Database.Plugin.create({ + extends: Database.Plugin.combine(pbrRender, sphere, animation, Orbit.plugin, picking), + transactions: { + initializeScene(t) { + t.resources.orbit = { + ...t.resources.orbit, + center: [0, 0, 0], + radius: 28, + height: 10, + autoSpinSpeed: 0.12, + }; + + const insertPlanet = (spec: PlanetSpec): number => { + const geo = sphere.transactions.insertSphere(t, { + color: spec.color, + emissive: spec.emissive, + metallic: spec.metallic, + roughness: spec.roughness, + rings: 32, + segments: 64, + }); + const planetId = Model.plugin.transactions.insertModel(t, { + geometry: geo, + position: spec.position, + scale: spec.scale, + parent: spec.parent ?? 0, + }); + if (spec.orbitRadius !== undefined && spec.orbitSpeed !== undefined) { + const clip = t.archetypes.AnimationClip.insert({ + animationClipTracks: [buildOrbitTrack(spec.orbitRadius)], + animationClipDuration: TWO_PI, + }); + t.archetypes.Animation.insert({ + animationClipRef: clip, + animationTargets: [planetId], + animationTime: 0, + animationSpeed: spec.orbitSpeed, + animationLoop: true, + animationPlaying: true, + }); + } + return planetId; + }; + + const sun = insertPlanet({ + color: [1.0, 0.92, 0.6, 1.0], emissive: [3.0, 2.5, 0.8], + metallic: 0, roughness: 1.0, + position: [0, 0, 0], scale: [2.5, 2.5, 2.5], + }); + insertPlanet({ + color: [0.55, 0.5, 0.45, 1.0], metallic: 0.1, roughness: 0.9, + position: [4.5, 0, 0], scale: [0.25, 0.25, 0.25], parent: sun, + orbitRadius: 4.5, orbitSpeed: 4.0, + }); + insertPlanet({ + color: [0.9, 0.75, 0.4, 1.0], metallic: 0.0, roughness: 0.95, + position: [7, 0, 0], scale: [0.5, 0.5, 0.5], parent: sun, + orbitRadius: 7.0, orbitSpeed: 1.6, + }); + const earth = insertPlanet({ + color: [0.15, 0.45, 0.85, 1.0], metallic: 0.05, roughness: 0.85, + position: [10, 0, 0], scale: [0.55, 0.55, 0.55], parent: sun, + orbitRadius: 10.0, orbitSpeed: 1.0, + }); + insertPlanet({ + color: [0.65, 0.65, 0.65, 1.0], metallic: 0.0, roughness: 0.95, + position: [1.2, 0, 0], scale: [0.15, 0.15, 0.15], parent: earth, + orbitRadius: 1.2, orbitSpeed: 13.4, + }); + insertPlanet({ + color: [0.82, 0.32, 0.18, 1.0], metallic: 0.1, roughness: 0.9, + position: [14, 0, 0], scale: [0.4, 0.4, 0.4], parent: sun, + orbitRadius: 14.0, orbitSpeed: 0.53, + }); + }, + }, + actions: { + /** Pick the Model under the cursor; if any, reframe the orbit on it. */ + pickAndFit(db, args: { x: number; y: number }) { + const hit = db.actions.pickFromScreen(args); + if (!hit) return; + const geo = db.read(hit.entity)?.geometry; + if (geo) db.transactions.setOrbit({ fitGeometry: geo }); + }, + }, +}); + +export type SolarSystemService = Database.Plugin.ToDatabase; diff --git a/packages/data-gpu-samples/src/samples/solar-system/solar-system.ts b/packages/data-gpu-samples/src/samples/solar-system/solar-system.ts new file mode 100644 index 00000000..64e8edaa --- /dev/null +++ b/packages/data-gpu-samples/src/samples/solar-system/solar-system.ts @@ -0,0 +1,9 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { html } from "lit"; +import type { TemplateResult } from "lit"; + +export const SolarSystem = (): TemplateResult => { + void import("./solar-system-element.js"); + return html``; +}; diff --git a/packages/data-gpu-samples/src/samples/solar-system/sphere-plugin.ts b/packages/data-gpu-samples/src/samples/solar-system/sphere-plugin.ts new file mode 100644 index 00000000..0a593e61 --- /dev/null +++ b/packages/data-gpu-samples/src/samples/solar-system/sphere-plugin.ts @@ -0,0 +1,98 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Mat4x4 } from "@adobe/data/math"; +import { pbrCore, core, VisibleMaterial } from "@adobe/data-gpu"; +import { createSphereBuffers } from "./create-sphere.js"; + +interface SphereSpec extends VisibleMaterial.ColorMaterialOptions { + rings: number; + segments: number; +} + +/** + * Sample-local plugin: lets the solar-system author `Sphere` entities with a + * color/material spec. A system materializes each into the GPU primitives the + * PBR renderer consumes, so the resulting entity id can be referenced as a + * Model's `geometry`. + */ +export const sphere = Database.Plugin.create({ + extends: Database.Plugin.combine(pbrCore, core), + components: { + sphereSpec: { default: null as unknown as SphereSpec }, + }, + archetypes: { + Sphere: ["sphereSpec"], + }, + transactions: { + insertSphere(t, options: VisibleMaterial.ColorMaterialOptions & { rings?: number; segments?: number }): number { + const spec: SphereSpec = { + color: options.color, + emissive: options.emissive, + metallic: options.metallic, + roughness: options.roughness, + rings: options.rings ?? 32, + segments: options.segments ?? 64, + }; + return t.archetypes.Sphere.insert({ sphereSpec: spec }); + }, + _insertSphereGeometry(t, args: { + geometry: number; + materialBindGroup: GPUBindGroup; + vertexBuffer: GPUBuffer; + indexBuffer: GPUBuffer; + indexCount: number; + indexFormat: GPUIndexFormat; + }) { + const materialId = t.archetypes._VisibleMaterial.insert({ + ephemeral: true, + _materialBindGroup: args.materialBindGroup, + _geometry: args.geometry, + }); + t.archetypes._PbrPrimitive.insert({ + ephemeral: true, + _geometry: args.geometry, + _material: materialId, + _vertexBuffer: args.vertexBuffer, + _skinVertexBuffer: null, + _indexBuffer: args.indexBuffer, + _indexCount: args.indexCount, + _indexFormat: args.indexFormat, + _nodeLocalMatrix: Mat4x4.identity, + }); + }, + }, + systems: { + _sphereGenerateSystem: { + create: db => { + const built = new Set(); + return () => { + const { device } = db.store.resources; + if (!device) return; + for (const arch of db.store.queryArchetypes(["sphereSpec"])) { + const ids = arch.columns.id; + const specs = arch.columns.sphereSpec; + for (let i = 0; i < arch.rowCount; i++) { + const id = ids.get(i) as number; + if (built.has(id)) continue; + built.add(id); + + const spec = specs.get(i) as SphereSpec; + const geo = createSphereBuffers(device, spec.rings, spec.segments); + const materialBindGroup = VisibleMaterial.createColorBindGroup(device, spec); + db.transactions._insertSphereGeometry({ + geometry: id, + materialBindGroup, + vertexBuffer: geo.vertexBuffer, + indexBuffer: geo.indexBuffer, + indexCount: geo.indexCount, + indexFormat: geo.indexFormat, + }); + } + } + }; + }, + schedule: { during: ["preUpdate"] }, + }, + }, +}); diff --git a/packages/data-gpu-samples/tsconfig.json b/packages/data-gpu-samples/tsconfig.json new file mode 100644 index 00000000..3d7c9571 --- /dev/null +++ b/packages/data-gpu-samples/tsconfig.json @@ -0,0 +1,17 @@ +{ + "compilerOptions": { + "target": "ES2020", + "useDefineForClassFields": false, + "lib": ["ES2020", "DOM", "DOM.Iterable"], + "module": "ESNext", + "skipLibCheck": true, + "types": ["@webgpu/types", "vite/client"], + "moduleResolution": "bundler", + "resolveJsonModule": true, + "isolatedModules": true, + "noEmit": true, + "experimentalDecorators": true, + "strict": true + }, + "include": ["src"] +} diff --git a/packages/data-gpu-samples/vite.config.ts b/packages/data-gpu-samples/vite.config.ts new file mode 100644 index 00000000..cff54b38 --- /dev/null +++ b/packages/data-gpu-samples/vite.config.ts @@ -0,0 +1,19 @@ +import { defineConfig } from "vite"; +import checker from "vite-plugin-checker"; + +export default defineConfig({ + plugins: [checker({ typescript: true })], + optimizeDeps: { + esbuildOptions: { + tsconfigRaw: { + compilerOptions: { experimentalDecorators: true, useDefineForClassFields: false }, + }, + }, + }, + root: ".", + // In production (GitHub Pages) the app lives at /data/graphics-samples/. + // The dev server ignores this — it always serves from /. + base: process.env.CI ? "/data/graphics-samples/" : "/", + build: { outDir: "dist" }, + server: { port: 3008, open: false }, +}); diff --git a/packages/data-gpu/CLAUDE.md b/packages/data-gpu/CLAUDE.md new file mode 100644 index 00000000..10575c68 --- /dev/null +++ b/packages/data-gpu/CLAUDE.md @@ -0,0 +1,98 @@ +# CLAUDE.md — data-gpu + +## Hot-loop performance: direct typed-array column access in systems + +Per-frame systems (the physics solver, the renderers, anything in `update`/ +`physics`/`render`) run over every body/instance every frame. In those loops, +**never use `column.get(i)` / `column.set(i, v)` for `Vec3`/`Quat`/struct +columns** — `get` allocates a fresh array per call, so a few-hundred-element +loop churns thousands of short-lived arrays/frame and the GC stalls show up as +frame hitches. + +Instead: + +- **Read/write the backing memory directly.** Struct columns (`Vec3`, `Quat`, + any fixed-array/object schema) are flat `Float32Array` buffers. Hoist the + array once per archetype and index it: `Vec3` stride 3, `Quat` stride 4. + + ```ts + const posArr = arch.columns.position.getTypedArray(); // not column.get(i) + for (let i = 0; i < arch.rowCount; i++) { + const o = i * 3; + state.pos[o] = posArr[o]; state.pos[o + 1] = posArr[o + 1]; state.pos[o + 2] = posArr[o + 2]; + } + ``` + +- **Systems may write component columns directly — no transaction.** + Transactions are for authored edits / undo / sync; a per-frame solver writing + its own derived state writes the typed array directly (zero transaction + overhead, and `column.set` does no change-notification anyway). + +- **Keep per-element math allocation-free.** Helpers called per row take an + out-array + offset and write into it (return scalars), rather than returning + a fresh `{...}` / `[...]`. See `ColliderShape.massProperties` (writes inverse + inertia into the solver array) and `composeTrs` in `pbr-render-plugin` (writes + a TRS matrix into a pooled arena). Reuse scratch buffers across frames; grow, + never reallocate per frame. + +Worked examples: the CPU solver gather/scatter (`cpu-xpbd-plugin.ts`) and the +primitive instance packing (`pbr-render-plugin.ts`) — both are zero-allocation +in steady state. + +(For the orthogonal "closure → class for V8 hidden classes" optimization on the +`@adobe/data` buffer interfaces, see the `performance` skill.) + +## ECS traversal: select in the query, never iterate rows you'll skip + +When writing **any** system, before the loop ask: *which rows do I actually need +this frame, and does the query already exclude the rest?* Iterating a wide set +and `if`-skipping inside the loop is the recurring perf bug. The cost is +invisible at tens of rows and dominant at thousands (see the worked numbers +below), so make the query do the filtering. (`.claude/rules/archetypes.md` is +the canonical rule; this is the data-gpu-specific checklist.) + +- **Push selection into `queryArchetypes(include, { exclude })`.** Require the + components you read; exclude the ones that disqualify a row. Don't query a + superset and filter. Distinguish by *archetype shape*, not by a per-row value + test, wherever you can: e.g. `StaticCollider` has no velocity columns, so the + dynamic gather/writeback simply never matches statics — no `if (isStatic)`. + +- **Process-once work → tag + exclude.** If a system does one-time setup per + entity (mirroring a body into an external engine, deriving `_worldBounds`, + uploading a buffer), add a private `_`-prefixed **`True` tag** when done and + `exclude` it from the query. Steady state then matches **zero archetypes** and + iterates nothing, instead of re-scanning every entity every frame to re-skip + it. Worked example: the Rapier/Jolt solver sync (`_rapierBody` / `_joltBody`). + Measured at 20k static + 64 dynamic: Jolt **0.53 → 0.07 ms/frame** (the scan + *was* the whole per-frame cost), Rapier **1.51 → 1.13**. + +- **Iterate tail→head when every visited row migrates out.** Adding/removing a + component (`db.store.update(id, { _tag: … })`, delete) migrates the row to + another archetype; doing it mid-iteration triggers a hole-fill (tail row moved + into the gap). Forward iteration pays that on every row and invalidates + indices ahead; `for (let r = rowCount - 1; r >= 0; r--)` removes only the tail, + so unvisited rows stay put. (See `world-bounds-plugin`, `transform-plugin`.) + +- **`getTypedArray()` for the hot path; `column.get()` for cold migrating loops.** + The every-frame path (e.g. solver writeback over all dynamics) reads/writes the + backing `Float32Array` directly — no allocation (see the hot-loop section + above). But a loop that **migrates rows as it goes** (the tag-on-completion + pattern) must NOT hold a hoisted `getTypedArray()` across the migration — the + buffer can move. Such loops run only over *new* rows (cold), so use + `column.get()` there: robust to the migration, and the small per-row + allocation is amortized to once-per-entity-lifetime. + +## Mirroring ECS state to an external engine (WASM physics, GPU, …) + +Each frame: (1) **sync** new entities into the engine — tag+exclude as above so +this is O(new), not O(all); (2) step the engine; (3) **write back** only the +subset that changed (e.g. dynamics — never statics) straight into the columns +via `getTypedArray()`. Copy *into* the engine and *back* through typed-array +indexing, not per-row object literals. + +Residual to watch: the **read-back from the engine** is bounded by the engine's +JS binding. Some (e.g. `rapier3d-compat`) allocate a fresh `{x,y,z}` per getter +call — a few allocations per moving body per frame. It scales with the *moving* +count only; if that grows large, drop to the engine's raw/flat-buffer bindings +to read straight into the typed arrays. Flagged at the call site in +`rapier-solver-plugin.ts`. diff --git a/packages/data-gpu/LICENSE b/packages/data-gpu/LICENSE new file mode 100644 index 00000000..4290d535 --- /dev/null +++ b/packages/data-gpu/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2025-2026 Adobe, Inc. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/packages/data-gpu/package.json b/packages/data-gpu/package.json new file mode 100644 index 00000000..ff157484 --- /dev/null +++ b/packages/data-gpu/package.json @@ -0,0 +1,34 @@ +{ + "name": "@adobe/data-gpu", + "version": "0.9.69", + "description": "Adobe data WebGPU plugins and types for graphics and compute", + "type": "module", + "private": false, + "publishConfig": { + "registry": "https://registry.npmjs.org/", + "access": "public" + }, + "scripts": { + "build": "cp ../../LICENSE . && tsc -b", + "dev": "tsc -b -w", + "test": "vitest --run --passWithNoTests", + "publish-public": "pnpm build && pnpm publish --no-git-checks --access public" + }, + "sideEffects": false, + "dependencies": { + "@adobe/data": "workspace:*", + "@dimforge/rapier3d-compat": "^0.19.3", + "jolt-physics": "^1.0.0" + }, + "devDependencies": { + "@webgpu/types": "^0.1.61", + "typescript": "^5.8.3", + "vitest": "^1.6.0" + }, + "exports": { + ".": { + "import": "./dist/index.js", + "types": "./dist/index.d.ts" + } + } +} diff --git a/packages/data-gpu/src/core/core-plugin.ts b/packages/data-gpu/src/core/core-plugin.ts new file mode 100644 index 00000000..a5644b8d --- /dev/null +++ b/packages/data-gpu/src/core/core-plugin.ts @@ -0,0 +1,92 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, scheduler } from "@adobe/data/ecs"; +import { FrameTime } from "./frame-time/frame-time.js"; + +async function getWebGPUDevice() { + // Guard for headless hosts (Node) where `navigator` doesn't exist: the + // simulation runs without a GPU device; only rendering needs one. + if (typeof navigator === "undefined" || !navigator.gpu) { + return null; + } + const adapter = await navigator.gpu.requestAdapter(); + if (!adapter) { + return null; + } + return adapter.requestDevice(); +} + +/** + * GPU runtime + frame scheduler — the shared base for both `graphics` + * (presentation) and `physics` (compute). Owns the WebGPU device, the + * per-frame command encoder, and the ordered frame phases. Contains nothing + * about canvases or render passes — those live in `graphics`. + * + * Phase order: input → preUpdate → update → postUpdate → physics → preRender + * → render → postRender. The command encoder is created at `preUpdate` and + * submitted at `postRender`; consumers record render passes (graphics) or + * compute passes (physics) into it during the phases between. + * + * Composes `FrameTime` over the scheduler so every downstream system has the + * `frameTime` resource — per-frame `dt` / `elapsed` — without re-deriving it. + */ +export const core = Database.Plugin.create({ + extends: Database.Plugin.combine(scheduler, FrameTime.plugin), + resources: { + device: { default: null as GPUDevice | null, transient: true }, + commandEncoder: { default: null as GPUCommandEncoder | null, transient: true }, + }, + systems: { + input: { + create: _db => () => {}, + }, + preUpdate: { + create: db => () => { + const { device } = db.store.resources; + if (device) { + db.store.resources.commandEncoder = device.createCommandEncoder(); + } + }, + schedule: { after: ["input"] }, + }, + update: { + create: _db => () => {}, + schedule: { after: ["preUpdate"] }, + }, + postUpdate: { + create: _db => () => {}, + schedule: { after: ["update"] }, + }, + physics: { + create: _db => () => {}, + schedule: { after: ["postUpdate"] }, + }, + preRender: { + create: _db => () => {}, + schedule: { after: ["physics"] }, + }, + render: { + create: db => { + getWebGPUDevice().then(device => { + db.store.resources.device = device; + }); + return () => {}; + }, + schedule: { after: ["preRender"] }, + }, + // Submits the frame's command encoder. Graphics ends its render pass + // `before: ["postRender"]` (so the pass is closed first); physics + // finishes its compute passes during the `physics` phase — both record + // into this encoder beforehand. + postRender: { + create: db => () => { + const { commandEncoder, device } = db.store.resources; + if (commandEncoder && device) { + device.queue.submit([commandEncoder.finish()]); + db.store.resources.commandEncoder = null; + } + }, + schedule: { after: ["render"] }, + }, + }, +}); diff --git a/packages/data-gpu/src/core/frame-time/frame-time-plugin.ts b/packages/data-gpu/src/core/frame-time/frame-time-plugin.ts new file mode 100644 index 00000000..78af176f --- /dev/null +++ b/packages/data-gpu/src/core/frame-time/frame-time-plugin.ts @@ -0,0 +1,44 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import type { FrameTime } from "./frame-time.js"; + +/** + * Largest per-frame delta (seconds) the clock reports. A backgrounded tab or + * the very first frame would otherwise produce a multi-second `dt` that throws + * every time-stepped system off; clamping here means no consumer has to. + */ +const MAX_DELTA = 0.1; + +/** + * Publishes per-frame wall-clock timing as the `frameTime` resource. Declares + * its system but does not extend the scheduler — so pure-data consumers (and + * their tests) can pull in the `frameTime` resource without dragging in + * `requestAnimationFrame`. Composed with a scheduler (via `core`), the system + * runs in the first tier (no dependencies), so every later phase — update, + * physics, render — reads a `dt`/`elapsed` already advanced for this frame. + */ +export const plugin = Database.Plugin.create({ + resources: { + frameTime: { + default: { now: 0, dt: 0, elapsed: 0 } satisfies FrameTime as FrameTime, + transient: true, + }, + }, + systems: { + _frameTime: { + create: db => { + let last = 0; + return () => { + const now = performance.now(); + // first frame (last === 0) reports dt 0 — no work has elapsed yet + const raw = last === 0 ? 0 : (now - last) / 1000; + last = now; + const dt = raw < MAX_DELTA ? raw : MAX_DELTA; + const elapsed = db.store.resources.frameTime.elapsed + dt; + db.store.resources.frameTime = { now, dt, elapsed }; + }; + }, + }, + }, +}); diff --git a/packages/data-gpu/src/core/frame-time/frame-time.ts b/packages/data-gpu/src/core/frame-time/frame-time.ts new file mode 100644 index 00000000..c2b2ebb7 --- /dev/null +++ b/packages/data-gpu/src/core/frame-time/frame-time.ts @@ -0,0 +1,21 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export * as FrameTime from "./public.js"; + +/** + * Wall-clock timing for the current frame, published once per frame by + * `FrameTime.plugin`. Any system that advances state over time reads this + * instead of calling `performance.now()` and tracking its own `lastTime`. + */ +export interface FrameTime { + /** `performance.now()` (ms) sampled at the start of this frame. */ + readonly now: number; + /** + * Seconds since the previous frame, clamped to a sane maximum so a + * stalled tab or the first frame can't inject a huge time jump. A solver + * that needs a tighter stability bound clamps `dt` further at its own seam. + */ + readonly dt: number; + /** Seconds since the plugin started — the running sum of clamped `dt`. */ + readonly elapsed: number; +} diff --git a/packages/data-gpu/src/core/frame-time/public.ts b/packages/data-gpu/src/core/frame-time/public.ts new file mode 100644 index 00000000..5ef45c50 --- /dev/null +++ b/packages/data-gpu/src/core/frame-time/public.ts @@ -0,0 +1,3 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export { plugin } from "./frame-time-plugin.js"; diff --git a/packages/data-gpu/src/graphics/animation/animation-data-plugin.ts b/packages/data-gpu/src/graphics/animation/animation-data-plugin.ts new file mode 100644 index 00000000..030d83c8 --- /dev/null +++ b/packages/data-gpu/src/graphics/animation/animation-data-plugin.ts @@ -0,0 +1,119 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, Entity } from "@adobe/data/ecs"; +import { Boolean, F32, Time, True, type Schema } from "@adobe/data/schema"; +import { AnimationTrack } from "../animation/animation-track/animation-track.js"; + +const animationBase = Database.Plugin.create({ + components: { + animationClipTracks: { default: [] as AnimationTrack[] }, + animationClipDuration: Time.schema, + animationClipRef: Entity.schema, + animationTargets: { default: [] as Entity[] }, + animationTime: Time.schema, + animationSpeed: { ...F32.schema, default: 1 }, + animationLoop: Boolean.schema, + animationPlaying: Boolean.schema, + animationObservable: True.schema, + }, + archetypes: { + AnimationClip: ["animationClipTracks", "animationClipDuration"], + Animation: [ + "animationClipRef", + "animationTargets", + "animationTime", + "animationSpeed", + "animationLoop", + "animationPlaying", + ], + AnimationObservable: [ + "animationClipRef", + "animationTargets", + "animationTime", + "animationSpeed", + "animationLoop", + "animationPlaying", + "animationObservable", + ], + }, +}); + +type AnimationStore = Database.Plugin.ToStore; + +export const animationData = Database.Plugin.create({ + extends: animationBase, + transactions: { + insertAnimationClip(t, args: { tracks: AnimationTrack[]; duration: number }): number { + return t.archetypes.AnimationClip.insert({ + animationClipTracks: args.tracks, + animationClipDuration: args.duration, + }); + }, + insertAnimation( + t, + args: { + animationClipRef: number; + animationTargets: number[]; + animationTime?: number; + animationSpeed?: number; + animationLoop?: boolean; + animationPlaying?: boolean; + observable?: boolean; + }, + ): number { + const values = { + animationClipRef: args.animationClipRef, + animationTargets: args.animationTargets, + animationTime: args.animationTime ?? 0, + animationSpeed: args.animationSpeed ?? 1, + animationLoop: args.animationLoop ?? false, + animationPlaying: args.animationPlaying ?? true, + }; + return args.observable + ? t.archetypes.AnimationObservable.insert({ ...values, animationObservable: true }) + : t.archetypes.Animation.insert(values); + }, + advanceAnimations(t: AnimationStore, args: { dt: number; observable: boolean }) { + const componentSchemas = t.componentSchemas as Record; + const arch = args.observable ? t.archetypes.AnimationObservable : t.archetypes.Animation; + const ids = arch.columns.id; + const clipRefs = arch.columns.animationClipRef; + const targetsCol = arch.columns.animationTargets; + const times = arch.columns.animationTime; + const speeds = arch.columns.animationSpeed; + const loops = arch.columns.animationLoop; + const playings = arch.columns.animationPlaying; + const { dt } = args; + const rowCount = arch.rowCount; + for (let i = 0; i < rowCount; i++) { + if (!playings.get(i)) continue; + const playerId = ids.get(i); + const clipId = clipRefs.get(i); + const clip = t.read(clipId); + if (!clip?.animationClipTracks || !clip.animationClipDuration || clip.animationClipDuration <= 0) continue; + const speed = speeds.get(i); + const loop = loops.get(i); + const duration = clip.animationClipDuration; + let newTime = times.get(i) + dt * speed; + if (loop) { + newTime = ((newTime % duration) + duration) % duration; + } else if (newTime >= duration) { + newTime = duration; + t.update(playerId, { animationPlaying: false }); + } else if (newTime < 0) { + newTime = 0; + } + t.update(playerId, { animationTime: newTime }); + const targets = targetsCol.get(i); + for (const track of clip.animationClipTracks) { + const target = targets[track.targetIndex]; + if (target === undefined || target === 0) continue; + const schema = componentSchemas[track.component]; + if (!schema) continue; + const value = AnimationTrack.sample(track, schema, newTime); + t.update(target, { [track.component]: value }); + } + } + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/animation/animation-plugin.test.ts b/packages/data-gpu/src/graphics/animation/animation-plugin.test.ts new file mode 100644 index 00000000..469422fb --- /dev/null +++ b/packages/data-gpu/src/graphics/animation/animation-plugin.test.ts @@ -0,0 +1,161 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { describe, it, expect } from "vitest"; +import { Database, type Store } from "@adobe/data/ecs"; +import { F32 } from "@adobe/data/math"; +import { animation } from "./animation-plugin.js"; +import type { AnimationTrack } from "./animation-track/animation-track.js"; + +function scalarTrack(component: string, points: readonly [time: number, value: number][]): AnimationTrack { + const times = new Float32Array(points.length); + const values = new Float32Array(points.length); + for (let i = 0; i < points.length; i++) { + times[i] = points[i][0]; + values[i] = points[i][1]; + } + return { targetIndex: 0, component, times, values, interpolation: "linear" }; +} + +function createTestDb() { + // The `placeholder` resource exists only to consume entity id 0, which the + // animation code treats as a sentinel "no target". In real scenes the + // first non-resource entity is never 0 either. + return Database.create(Database.Plugin.create({ + extends: animation, + components: { + value: F32.schema, + }, + resources: { + placeholder: { default: 0 as number }, + }, + archetypes: { + Target: ["value"], + }, + transactions: { + insertTarget(t): number { + return t.archetypes.Target.insert({ value: 0 }); + }, + }, + })); +} + +const linearClip = (db: ReturnType) => + db.transactions.insertAnimationClip({ + tracks: [scalarTrack("value", [[0, 0], [1, 10]])], + duration: 1, + }); + +describe("animation.advanceAnimations", () => { + it("advances time and writes the sampled value to the target", () => { + const db = createTestDb(); + const targetId = db.transactions.insertTarget() as number; + const clipId = linearClip(db); + db.transactions.insertAnimation({ + animationClipRef: clipId, + animationTargets: [targetId], + }); + + db.transactions.advanceAnimations({ dt: 0.25, observable: false }); + + // 0.25s into a 1s clip linearly interpolating 0→10 → value = 2.5. + expect(db.read(targetId)?.value).toBeCloseTo(2.5, 5); + }); + + it("loops time within the clip duration", () => { + const db = createTestDb(); + const targetId = db.transactions.insertTarget() as number; + const clipId = linearClip(db); + db.transactions.insertAnimation({ + animationClipRef: clipId, + animationTargets: [targetId], + animationLoop: true, + }); + + db.transactions.advanceAnimations({ dt: 2.25, observable: false }); + + // 2.25 % 1 = 0.25 → same sampled value as the first test. + expect(db.read(targetId)?.value).toBeCloseTo(2.5, 5); + }); + + it("stops a non-looping player when it reaches duration", () => { + const db = createTestDb(); + const targetId = db.transactions.insertTarget() as number; + const clipId = linearClip(db); + const playerId = db.transactions.insertAnimation({ + animationClipRef: clipId, + animationTargets: [targetId], + animationLoop: false, + }) as number; + + db.transactions.advanceAnimations({ dt: 2, observable: false }); + + const player = db.read(playerId); + expect(player?.animationTime).toBe(1); + expect(player?.animationPlaying).toBe(false); + }); + + it("only touches players whose animationObservable flag matches the args", () => { + const db = createTestDb(); + const observableTarget = db.transactions.insertTarget() as number; + const directTarget = db.transactions.insertTarget() as number; + const clipId = linearClip(db); + db.transactions.insertAnimation({ + animationClipRef: clipId, + animationTargets: [observableTarget], + observable: true, + }); + db.transactions.insertAnimation({ + animationClipRef: clipId, + animationTargets: [directTarget], + }); + + // observable: true touches only the flagged player. + db.transactions.advanceAnimations({ dt: 0.5, observable: true }); + expect(db.read(observableTarget)?.value).toBeCloseTo(5, 5); + expect(db.read(directTarget)?.value).toBe(0); + + // observable: false touches only the non-flagged player. + db.transactions.advanceAnimations({ dt: 0.5, observable: false }); + expect(db.read(directTarget)?.value).toBeCloseTo(5, 5); + }); + + it("fires entity observers for the observable path and skips them for the direct path", () => { + const db = createTestDb(); + const observableTarget = db.transactions.insertTarget() as number; + const directTarget = db.transactions.insertTarget() as number; + const clipId = linearClip(db); + db.transactions.insertAnimation({ + animationClipRef: clipId, + animationTargets: [observableTarget], + observable: true, + }); + db.transactions.insertAnimation({ + animationClipRef: clipId, + animationTargets: [directTarget], + }); + + const observableNotifications: number[] = []; + const directNotifications: number[] = []; + db.observe.entity(observableTarget)(v => { if (v) observableNotifications.push(v.value as number); }); + db.observe.entity(directTarget)(v => { if (v) directNotifications.push(v.value as number); }); + // Drop the initial-value notifications. + observableNotifications.length = 0; + directNotifications.length = 0; + + // Observable path: writes flow through the transaction → observers fire. + db.transactions.advanceAnimations({ dt: 0.5, observable: true }); + expect(observableNotifications.length).toBeGreaterThan(0); + expect(directNotifications.length).toBe(0); + + // Direct path: same `advanceAnimations` body, but with the raw store + // as `t`. Underlying value changes; observers must stay silent. + // (`db.store` is exposed at runtime but not on the public Database + // type — systems get a `Database & { store }` view; tests reach in.) + const directStore = (db as unknown as { store: Store }).store; + const before = db.read(directTarget)?.value; + animation.transactions.advanceAnimations(directStore, { dt: 0.5, observable: false }); + const after = db.read(directTarget)?.value; + expect(after).not.toBe(before); + expect(directNotifications.length).toBe(0); + }); +}); diff --git a/packages/data-gpu/src/graphics/animation/animation-plugin.ts b/packages/data-gpu/src/graphics/animation/animation-plugin.ts new file mode 100644 index 00000000..4d33073b --- /dev/null +++ b/packages/data-gpu/src/graphics/animation/animation-plugin.ts @@ -0,0 +1,26 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { FrameTime } from "../../core/frame-time/frame-time.js"; +import { animationData } from "./animation-data-plugin.js"; + +/** + * Full animation plugin — declarative data (from `scene/animation-plugin.ts`) + * plus the per-frame `animationSampleSystem` that advances all playing entities. + * + * Exported as `animation` from index.ts. The data-only part lives in + * `scene/animation-plugin.ts` and is included in the `scene` aggregator. + */ +export const animation = Database.Plugin.create({ + extends: Database.Plugin.combine(animationData, FrameTime.plugin), + systems: { + animationSampleSystem: { + create: db => () => { + const dt = db.store.resources.frameTime.dt; + if (dt <= 0) return; + db.transactions.advanceAnimations({ dt, observable: true }); + animationData.transactions.advanceAnimations(db.store, { dt, observable: false }); + }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/animation/animation-track/animation-track.ts b/packages/data-gpu/src/graphics/animation/animation-track/animation-track.ts new file mode 100644 index 00000000..0b5c6f6d --- /dev/null +++ b/packages/data-gpu/src/graphics/animation/animation-track/animation-track.ts @@ -0,0 +1,17 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { InterpolationMode } from "../interpolation-mode/interpolation-mode.js"; + +export interface AnimationTrack { + /** Index into the AnimationPlayer's animationTargets array. */ + readonly targetIndex: number; + /** Component name to write on the resolved target entity. */ + readonly component: string; + /** Sorted keyframe timestamps. */ + readonly times: Float32Array; + /** Packed keyframe values (componentsPerKey * times.length entries). */ + readonly values: Float32Array; + readonly interpolation: InterpolationMode; +} + +export * as AnimationTrack from "./public.js"; diff --git a/packages/data-gpu/src/graphics/animation/animation-track/componentwise-lerp.ts b/packages/data-gpu/src/graphics/animation/animation-track/componentwise-lerp.ts new file mode 100644 index 00000000..e8a6ad21 --- /dev/null +++ b/packages/data-gpu/src/graphics/animation/animation-track/componentwise-lerp.ts @@ -0,0 +1,24 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Schema } from "@adobe/data/schema"; + +/** + * Default linear interpolation when a schema does not declare its own + * `interpolators.linear`. Walks the schema once and lerps numeric leaves. + * Supports `type: "number"` scalars and `type: "array"` of numbers. + */ +export function componentwiseLerp(schema: Schema, prev: any, next: any, t: number): any { + if (schema.type === "number") { + return prev + (next - prev) * t; + } + if (schema.type === "array") { + const n = prev.length; + const out = new Array(n); + const itemSchema = schema.items ?? { type: "number" }; + for (let i = 0; i < n; i++) { + out[i] = componentwiseLerp(itemSchema, prev[i], next[i], t); + } + return out; + } + throw new Error(`componentwiseLerp: no default interpolator for schema type "${schema.type}"`); +} diff --git a/packages/data-gpu/src/graphics/animation/animation-track/interpolate.ts b/packages/data-gpu/src/graphics/animation/animation-track/interpolate.ts new file mode 100644 index 00000000..8fdbcbc1 --- /dev/null +++ b/packages/data-gpu/src/graphics/animation/animation-track/interpolate.ts @@ -0,0 +1,24 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Schema } from "@adobe/data/schema"; +import type { InterpolationMode } from "../interpolation-mode/interpolation-mode.js"; +import { componentwiseLerp } from "./componentwise-lerp.js"; + +/** + * Dispatches to a schema-declared interpolator if present, otherwise falls + * back to a sensible default: `step` returns `next`, `linear` walks the schema + * and lerps numeric leaves. `cubicSpline` requires a schema override. + */ +export function interpolate( + schema: Schema, + mode: InterpolationMode, + prev: any, + next: any, + t: number, +): any { + const custom = schema.interpolators?.[mode]; + if (custom) return custom(prev, next, t); + if (mode === "step") return prev; + if (mode === "linear") return componentwiseLerp(schema, prev, next, t); + throw new Error(`interpolate: schema type "${schema.type}" has no "${mode}" interpolator`); +} diff --git a/packages/data-gpu/src/graphics/animation/animation-track/public.ts b/packages/data-gpu/src/graphics/animation/animation-track/public.ts new file mode 100644 index 00000000..cf4e76c6 --- /dev/null +++ b/packages/data-gpu/src/graphics/animation/animation-track/public.ts @@ -0,0 +1,3 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export { sample } from "./sample.js"; diff --git a/packages/data-gpu/src/graphics/animation/animation-track/sample.ts b/packages/data-gpu/src/graphics/animation/animation-track/sample.ts new file mode 100644 index 00000000..d7b4f96c --- /dev/null +++ b/packages/data-gpu/src/graphics/animation/animation-track/sample.ts @@ -0,0 +1,48 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Schema } from "@adobe/data/schema"; +import type { AnimationTrack } from "./animation-track.js"; +import { interpolate } from "./interpolate.js"; + +/** + * Samples a track at `time` and returns the interpolated component value. + * `time` must already be wrapped into the clip's [0, duration] range by the caller. + */ +export function sample(track: AnimationTrack, schema: Schema, time: number): number | number[] { + const { times, values } = track; + const keyCount = times.length; + const stride = values.length / keyCount; + + if (keyCount === 1 || time <= times[0]) { + return readKey(values, 0, stride); + } + const last = keyCount - 1; + if (time >= times[last]) { + return readKey(values, last, stride); + } + + let lo = 0; + let hi = last; + while (hi - lo > 1) { + const mid = (lo + hi) >> 1; + if (times[mid] <= time) lo = mid; + else hi = mid; + } + + const tPrev = times[lo]; + const tNext = times[hi]; + const span = tNext - tPrev; + const t = span > 0 ? (time - tPrev) / span : 0; + + const prev = readKey(values, lo, stride); + const next = readKey(values, hi, stride); + return interpolate(schema, track.interpolation, prev, next, t); +} + +function readKey(values: Float32Array, index: number, stride: number): number | number[] { + if (stride === 1) return values[index]; + const out = new Array(stride); + const base = index * stride; + for (let i = 0; i < stride; i++) out[i] = values[base + i]; + return out; +} diff --git a/packages/data-gpu/src/graphics/animation/interpolation-mode/interpolation-mode.ts b/packages/data-gpu/src/graphics/animation/interpolation-mode/interpolation-mode.ts new file mode 100644 index 00000000..1bcc6534 --- /dev/null +++ b/packages/data-gpu/src/graphics/animation/interpolation-mode/interpolation-mode.ts @@ -0,0 +1,8 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; +import { schema } from "./schema.js"; + +export type InterpolationMode = Schema.ToType; + +export * as InterpolationMode from "./public.js"; diff --git a/packages/data-gpu/src/graphics/animation/interpolation-mode/public.ts b/packages/data-gpu/src/graphics/animation/interpolation-mode/public.ts new file mode 100644 index 00000000..fa4419e7 --- /dev/null +++ b/packages/data-gpu/src/graphics/animation/interpolation-mode/public.ts @@ -0,0 +1,3 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export { schema } from "./schema.js"; diff --git a/packages/data-gpu/src/graphics/animation/interpolation-mode/schema.ts b/packages/data-gpu/src/graphics/animation/interpolation-mode/schema.ts new file mode 100644 index 00000000..61101553 --- /dev/null +++ b/packages/data-gpu/src/graphics/animation/interpolation-mode/schema.ts @@ -0,0 +1,5 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; + +export const schema = { type: "string", enum: ["linear", "step", "cubicSpline"] } as const satisfies Schema; diff --git a/packages/data-gpu/src/graphics/camera/camera-plugin.ts b/packages/data-gpu/src/graphics/camera/camera-plugin.ts new file mode 100644 index 00000000..a0ca139a --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/camera-plugin.ts @@ -0,0 +1,37 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { graphics } from "../graphics-plugin.js"; +import { Camera } from "./camera.js"; + +export const plugin = Database.Plugin.create({ + extends: graphics, + resources: { + camera: { + default: { + aspect: 16 / 9, + fieldOfView: Math.PI / 4, + nearPlane: 0.1, + farPlane: 100.0, + position: [0, 0, 10], + target: [0, 0, 0], + up: [0, 1, 0], + orthographic: 0, + } satisfies Camera as Camera, + }, + }, + systems: { + updateCameraAspect: { + create: db => () => { + const { canvas } = db.store.resources; + let { camera: cam } = db.store.resources; + if (!canvas || !cam) return; + const aspect = canvas.width / canvas.height; + if (cam.aspect !== aspect) { + db.store.resources.camera = { ...cam, aspect }; + } + }, + schedule: { during: ["preRender"] } + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/camera/camera.ts b/packages/data-gpu/src/graphics/camera/camera.ts new file mode 100644 index 00000000..cd623328 --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/camera.ts @@ -0,0 +1,8 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; +import { schema } from "./schema.js"; + +export type Camera = Schema.ToType; + +export * as Camera from "./public.js"; diff --git a/packages/data-gpu/src/graphics/camera/orbit/attach-orbit-drag.ts b/packages/data-gpu/src/graphics/camera/orbit/attach-orbit-drag.ts new file mode 100644 index 00000000..6e166172 --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/orbit/attach-orbit-drag.ts @@ -0,0 +1,62 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +/** + * Minimal service interface required by attachOrbitDrag — just the two + * orbit transactions it calls. Any service that combines Orbit.plugin + * satisfies this. + */ +export interface OrbitDragService { + transactions: { + addOrbitAngle(delta: number): void; + resumeAutoSpin(): void; + }; +} + +/** + * Attaches pointer-drag listeners to `element` that drive orbit rotation. + * Returns a dispose function to remove the listeners. + * + * Framework-agnostic — pass the returned dispose to whatever cleanup + * mechanism your framework provides (useEffect return, onUnmount, etc.). + * + * @example + * // Vanilla JS + * const dispose = attachOrbitDrag(canvas, service); + * // later: + * dispose(); + * + * @example + * // Lit (inside render()) + * useEffect(() => attachOrbitDrag(canvas, service), [canvas, service]); + */ +export function attachOrbitDrag( + element: HTMLElement, + service: OrbitDragService, + options?: { sensitivity?: number }, +): () => void { + const sensitivity = options?.sensitivity ?? 0.01; + let lastX = 0; + + const onDown = (e: PointerEvent) => { + lastX = e.clientX; + element.setPointerCapture(e.pointerId); + }; + const onMove = (e: PointerEvent) => { + if (!e.buttons) return; + service.transactions.addOrbitAngle((lastX - e.clientX) * sensitivity); + lastX = e.clientX; + }; + const onUp = () => service.transactions.resumeAutoSpin(); + + element.addEventListener("pointerdown", onDown); + element.addEventListener("pointermove", onMove); + element.addEventListener("pointerup", onUp); + element.addEventListener("pointercancel", onUp); + + return () => { + element.removeEventListener("pointerdown", onDown); + element.removeEventListener("pointermove", onMove); + element.removeEventListener("pointerup", onUp); + element.removeEventListener("pointercancel", onUp); + }; +} diff --git a/packages/data-gpu/src/graphics/camera/orbit/orbit-data-plugin.ts b/packages/data-gpu/src/graphics/camera/orbit/orbit-data-plugin.ts new file mode 100644 index 00000000..6bd4d126 --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/orbit/orbit-data-plugin.ts @@ -0,0 +1,44 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Camera } from "../camera.js"; +import type { Orbit } from "./orbit.js"; + +/** + * Declarative orbit camera state — the authored surface. + * Systems in `orbit-system-plugin.ts` drive the `camera` resource from this. + */ +export const orbitData = Database.Plugin.create({ + extends: Camera.plugin, + resources: { + orbit: { + default: { + center: [0, 0, 0], + radius: 3, + height: 0, + angle: 0, + autoSpin: true, + autoSpinSpeed: 0.5, + nearFactor: 0.01, + farFactor: 4, + fitGeometry: 0, + fitRadiusFactor: 1.5, + fitHeightFactor: 0.25, + fitRadiusOffset: 0, + fitCenter: null, + } satisfies Orbit as Orbit, + }, + }, + transactions: { + setOrbit(t, args: Partial) { + t.resources.orbit = { ...t.resources.orbit, ...args }; + }, + addOrbitAngle(t, delta: number) { + const cur = t.resources.orbit; + t.resources.orbit = { ...cur, angle: cur.angle + delta, autoSpin: false }; + }, + resumeAutoSpin(t) { + t.resources.orbit = { ...t.resources.orbit, autoSpin: true }; + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/camera/orbit/orbit-plugin.ts b/packages/data-gpu/src/graphics/camera/orbit/orbit-plugin.ts new file mode 100644 index 00000000..142a6cc0 --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/orbit/orbit-plugin.ts @@ -0,0 +1,13 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { orbitData } from "./orbit-data-plugin.js"; +import { orbitSystem } from "./orbit-system-plugin.js"; + +/** + * Orbit camera — data + systems combined. Add to any service that needs + * a draggable, auto-spinning, model-fitting camera. + * + * Exported as `Orbit.plugin` via orbit.ts → public.ts. + */ +export const plugin = Database.Plugin.combine(orbitData, orbitSystem); diff --git a/packages/data-gpu/src/graphics/camera/orbit/orbit-system-plugin.ts b/packages/data-gpu/src/graphics/camera/orbit/orbit-system-plugin.ts new file mode 100644 index 00000000..06121c45 --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/orbit/orbit-system-plugin.ts @@ -0,0 +1,74 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import type { Aabb, Vec3 } from "@adobe/data/math"; +import { orbitData } from "./orbit-data-plugin.js"; + +/** + * Systems that drive `camera` from the `orbit` resource each frame. + * + * - **_orbitCameraSystem**: updates camera position/target/near/far from the + * orbit polar coords; auto-spins when `orbit.autoSpin` is true. + * - **_orbitAutoFitSystem**: when `orbit.fitGeometry` is non-zero, reads that + * Geometry's `_bounds`, sizes the orbit to fit, and zeroes the field. + */ +export const orbitSystem = Database.Plugin.create({ + extends: orbitData, + systems: { + _orbitCameraSystem: { + create: db => { + return () => { + const { orbit, camera: cam } = db.store.resources; + if (!cam) return; + if (orbit.autoSpin) { + const dt = db.store.resources.frameTime.dt; + db.store.resources.orbit = { ...orbit, angle: orbit.angle + dt * orbit.autoSpinSpeed }; + } + const angle = db.store.resources.orbit.angle; + db.store.resources.camera = { + ...cam, + position: [ + orbit.center[0] + Math.sin(angle) * orbit.radius, + orbit.center[1] + orbit.height, + orbit.center[2] + Math.cos(angle) * orbit.radius, + ], + target: orbit.center, + nearPlane: Math.max(orbit.radius * orbit.nearFactor, 0.1), + farPlane: Math.max(orbit.radius * orbit.farFactor, 100), + }; + }; + }, + schedule: { during: ["update"] }, + }, + _orbitAutoFitSystem: { + create: db => () => { + const orbit = db.store.resources.orbit; + if (!orbit.fitGeometry) return; + // orbit.fitGeometry holds a Geometry entity id. `_bounds` lives + // on modelLoader's schema, which orbit doesn't extend — read as + // unknown and narrow manually. + const entity = db.store.read(orbit.fitGeometry) as { _bounds?: Aabb } | null; + const bounds = entity?._bounds; + if (!bounds) return; + const size = Math.max( + bounds.max[0] - bounds.min[0], + bounds.max[1] - bounds.min[1], + bounds.max[2] - bounds.min[2], + ); + const center = orbit.fitCenter ?? [ + (bounds.min[0] + bounds.max[0]) / 2, + (bounds.min[1] + bounds.max[1]) / 2, + (bounds.min[2] + bounds.max[2]) / 2, + ] as Vec3; + db.store.resources.orbit = { + ...orbit, + center, + radius: size * orbit.fitRadiusFactor + orbit.fitRadiusOffset, + height: size * orbit.fitHeightFactor, + fitGeometry: 0, + }; + }, + schedule: { during: ["preUpdate"] }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/camera/orbit/orbit.ts b/packages/data-gpu/src/graphics/camera/orbit/orbit.ts new file mode 100644 index 00000000..78748ca8 --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/orbit/orbit.ts @@ -0,0 +1,36 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Vec3 } from "@adobe/data/math"; + +/** + * Orbit camera state — a polar arrangement around a center point with an + * auto-spin and a one-shot auto-fit to a Geometry's bounds. + * + * - `center` / `radius` / `height` / `angle` define the camera placement. + * - `nearFactor` / `farFactor` scale the near and far planes so they + * track the orbit radius (models authored in any unit render without + * clipping). + * - `autoSpin` / `autoSpinSpeed` rotate the camera each frame when the + * user isn't dragging. + * - `fitGeometry` is an entity id; when non-zero, the auto-fit system + * reads that Geometry's bounds on the first available frame, sizes the + * orbit, and zeros the field. `fitRadiusFactor` / `fitHeightFactor` / + * `fitRadiusOffset` / `fitCenter` shape the fit. + */ +export interface Orbit { + center: Vec3; + radius: number; + height: number; + angle: number; + autoSpin: boolean; + autoSpinSpeed: number; + nearFactor: number; + farFactor: number; + fitGeometry: number; + fitRadiusFactor: number; + fitHeightFactor: number; + fitRadiusOffset: number; + fitCenter: Vec3 | null; +} + +export * as Orbit from "./public.js"; diff --git a/packages/data-gpu/src/graphics/camera/orbit/public.ts b/packages/data-gpu/src/graphics/camera/orbit/public.ts new file mode 100644 index 00000000..56740880 --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/orbit/public.ts @@ -0,0 +1,7 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export { plugin } from "./orbit-plugin.js"; +export { orbitData } from "./orbit-data-plugin.js"; +export { orbitSystem } from "./orbit-system-plugin.js"; +export { attachOrbitDrag } from "./attach-orbit-drag.js"; +export type { OrbitDragService } from "./attach-orbit-drag.js"; diff --git a/packages/data-gpu/src/graphics/camera/public.ts b/packages/data-gpu/src/graphics/camera/public.ts new file mode 100644 index 00000000..431aba43 --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/public.ts @@ -0,0 +1,6 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export * from "./schema.js"; +export * from "./to-view-projection.js"; +export * from "./screen-to-world-ray.js"; +export { plugin } from "./camera-plugin.js"; diff --git a/packages/data-gpu/src/graphics/camera/schema.ts b/packages/data-gpu/src/graphics/camera/schema.ts new file mode 100644 index 00000000..a8ba0f12 --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/schema.ts @@ -0,0 +1,21 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { F32, Vec3 } from "@adobe/data/math"; +import { Schema } from "@adobe/data/schema"; + +export const schema = { + type: "object", + properties: { + aspect: F32.schema, + fieldOfView: F32.schema, + nearPlane: F32.schema, + farPlane: F32.schema, + position: Vec3.schema, + target: Vec3.schema, + up: Vec3.schema, + // 0 = perspective, 1 = orthographic, fractional = hybrid blend + orthographic: F32.schema, + }, + required: ["aspect", "fieldOfView", "nearPlane", "farPlane", "position", "target", "up", "orthographic"], + additionalProperties: false, +} as const satisfies Schema; diff --git a/packages/data-gpu/src/graphics/camera/screen-to-world-ray.ts b/packages/data-gpu/src/graphics/camera/screen-to-world-ray.ts new file mode 100644 index 00000000..2c4d4a43 --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/screen-to-world-ray.ts @@ -0,0 +1,47 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Mat4x4, Vec3, Vec4, Line3 } from "@adobe/data/math"; +import type { Camera } from "./camera.js"; +import { toViewProjection } from "./to-view-projection.js"; + +/** + * Builds a world-space ray from a canvas pixel coordinate. + * Pixel origin is top-left (matching `offsetX`/`offsetY` on pointer events). + * Internally converts to NDC (Normalized Device Coordinates: x/y ∈ [-1, 1], + * origin at canvas center, y-up) before unprojecting via the inverse + * view-projection matrix. + */ +export const screenToWorldRay = ( + cam: Camera, + screenX: number, + screenY: number, + canvasWidth: number, + canvasHeight: number, + rayLength = 1000, +): Line3 => { + // NDC: x ∈ [-1, 1] left→right, y ∈ [-1, 1] bottom→top. + const ndcX = (screenX / canvasWidth) * 2 - 1; + const ndcY = 1 - (screenY / canvasHeight) * 2; + + // WebGPU NDC z ∈ [0, 1]: near plane at 0, far plane at 1. + const nearPoint: Vec4 = [ndcX, ndcY, 0, 1]; + const farPoint: Vec4 = [ndcX, ndcY, 1, 1]; + + const invVP = Mat4x4.inverse(toViewProjection(cam)); + + const nearW = Mat4x4.multiplyVec4(invVP, nearPoint); + const farW = Mat4x4.multiplyVec4(invVP, farPoint); + + const near: Vec3 = [nearW[0] / nearW[3], nearW[1] / nearW[3], nearW[2] / nearW[3]]; + const far: Vec3 = [farW[0] / farW[3], farW[1] / farW[3], farW[2] / farW[3]]; + + const dir: Vec3 = [near[0] - far[0], near[1] - far[1], near[2] - far[2]]; + const len = Math.sqrt(dir[0] ** 2 + dir[1] ** 2 + dir[2] ** 2); + + if (len < 0.0001) return { a: near, b: far }; + + const norm: Vec3 = [dir[0] / len, dir[1] / len, dir[2] / len]; + const end: Vec3 = [near[0] + norm[0] * rayLength, near[1] + norm[1] * rayLength, near[2] + norm[2] * rayLength]; + + return { a: near, b: end }; +}; diff --git a/packages/data-gpu/src/graphics/camera/to-view-projection.ts b/packages/data-gpu/src/graphics/camera/to-view-projection.ts new file mode 100644 index 00000000..ddb19f0e --- /dev/null +++ b/packages/data-gpu/src/graphics/camera/to-view-projection.ts @@ -0,0 +1,29 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Mat4x4, Vec3 } from "@adobe/data/math"; +import type { Camera } from "./camera.js"; + +export const toViewProjection = (cam: Camera): Mat4x4 => { + const lookAt = Mat4x4.lookAt(cam.position, cam.target, cam.up); + + const f = 1.0 / Math.tan(cam.fieldOfView / 2); + const d = Vec3.distance(cam.position, cam.target); + + // WebGPU clip-space convention: z_ndc ∈ [0, 1]. + // Perspective wants m[2][2] = far/(near-far); orthographic wants d/(near-far) + // so that z_clip(z_view = -far) = (perspective: far / w_clip=far → 1) + // (orthographic: d / w_clip=d → 1). + // Blend the two cases by the orthographic factor. + const m22 = ((1 - cam.orthographic) * cam.farPlane + cam.orthographic * d) / (cam.nearPlane - cam.farPlane); + const m23 = cam.nearPlane * m22; + + // Fourth row blends: w' = (1 - orthographic) * (-z) + orthographic * d + const perspective: Mat4x4 = [ + f / cam.aspect, 0, 0, 0, + 0, f, 0, 0, + 0, 0, m22, -(1 - cam.orthographic), + 0, 0, m23, cam.orthographic * d, + ]; + + return Mat4x4.multiply(perspective, lookAt); +}; diff --git a/packages/data-gpu/src/graphics/graphics-plugin.ts b/packages/data-gpu/src/graphics/graphics-plugin.ts new file mode 100644 index 00000000..17a62d3a --- /dev/null +++ b/packages/data-gpu/src/graphics/graphics-plugin.ts @@ -0,0 +1,108 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Vec4 } from "@adobe/data/math"; +import { core } from "../core/core-plugin.js"; + +/** + * Graphics presentation — extends the `core` GPU runtime with everything + * about drawing to a canvas: the swap-chain context, color/depth formats, + * the depth texture, and the per-frame render pass. + * + * Render-pass lifecycle, sandwiched between core's phase anchors with hard + * ordering edges (`during` is only a soft hint, so we use after/before): + * `beginRenderPass` runs after `preRender` and before `render` (pass open + * before draws); draw systems run `during: ["render"]`; `endRenderPass` runs + * after `render` and before `postRender` (pass closed after draws, before + * core submits the encoder). Compute-only consumers (e.g. `physics`) depend + * on `core` directly and never touch any of this. + */ +export const graphics = Database.Plugin.create({ + extends: core, + resources: { + renderPassEncoder: { default: null as GPURenderPassEncoder | null, transient: true }, + depthTexture: { default: null as GPUTexture | null, transient: true }, + clearColor: { default: [0, 0, 0, 1] as Vec4, transient: true }, + canvas: { default: null as HTMLCanvasElement | null, transient: true }, + canvasContext: { default: null as GPUCanvasContext | null, transient: true }, + // Guarded for headless hosts (Node): this default is evaluated at module + // load, and `navigator` is browser-only. Rendering re-derives the real + // preferred format when a canvas is configured; the placeholder is unused. + canvasFormat: { + default: (typeof navigator !== "undefined" && navigator.gpu + ? navigator.gpu.getPreferredCanvasFormat() + : "bgra8unorm") as GPUTextureFormat, + transient: true, + }, + depthFormat: { default: "depth24plus" as GPUTextureFormat, transient: true }, + }, + transactions: { + setCanvas(t, canvas: HTMLCanvasElement | null) { + t.resources.canvas = canvas; + }, + }, + systems: { + configureCanvas: { + create: db => () => { + const { device } = db.store.resources; + if (!device) return; + let { canvasContext } = db.store.resources; + const { canvas, canvasFormat } = db.store.resources; + if (canvas && !canvasContext) { + canvasContext = db.store.resources.canvasContext = canvas.getContext("webgpu"); + if (!canvasContext) { + throw new Error("No WebGPU context"); + } + canvasContext.configure({ + device, + format: canvasFormat, + alphaMode: "premultiplied", + }); + } + }, + schedule: { after: ["preUpdate"], before: ["update"] }, + }, + beginRenderPass: { + create: db => () => { + const { canvas, commandEncoder, clearColor, canvasContext, device, depthFormat } = db.store.resources; + let { depthTexture } = db.store.resources; + if (!commandEncoder || !canvasContext || !device || !canvas) return; + + const canvasSize: [number, number] = [canvas.width, canvas.height]; + if (!depthTexture || depthTexture.width !== canvasSize[0] || depthTexture.height !== canvasSize[1]) { + depthTexture = db.store.resources.depthTexture = device.createTexture({ + size: canvasSize, + format: depthFormat, + usage: GPUTextureUsage.RENDER_ATTACHMENT, + }); + } + + db.store.resources.renderPassEncoder = commandEncoder.beginRenderPass({ + colorAttachments: [{ + clearValue: clearColor, + loadOp: "clear", + storeOp: "store", + view: canvasContext.getCurrentTexture().createView(), + }], + depthStencilAttachment: { + view: depthTexture.createView(), + depthClearValue: 1.0, + depthLoadOp: "clear", + depthStoreOp: "store", + }, + }); + }, + schedule: { after: ["preRender"], before: ["render"] }, + }, + endRenderPass: { + create: db => () => { + const { renderPassEncoder } = db.store.resources; + if (renderPassEncoder) { + renderPassEncoder.end(); + db.store.resources.renderPassEncoder = null; + } + }, + schedule: { after: ["render"], before: ["postRender"] }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/picking/pick-hit.ts b/packages/data-gpu/src/graphics/picking/pick-hit.ts new file mode 100644 index 00000000..7880f027 --- /dev/null +++ b/packages/data-gpu/src/graphics/picking/pick-hit.ts @@ -0,0 +1,13 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Entity } from "@adobe/data/ecs"; + +/** + * One ray-pick result — the Model the ray hit and the parametric distance + * along the ray segment (`alpha ∈ [0,1]` where 0 is the ray's near point + * `a` and 1 is the far point `b` from `screenToWorldRay`). + */ +export interface PickHit { + readonly entity: Entity; + readonly distance: number; +} diff --git a/packages/data-gpu/src/graphics/picking/picking-plugin.ts b/packages/data-gpu/src/graphics/picking/picking-plugin.ts new file mode 100644 index 00000000..ff6a6b04 --- /dev/null +++ b/packages/data-gpu/src/graphics/picking/picking-plugin.ts @@ -0,0 +1,101 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Aabb, type Line3 } from "@adobe/data/math"; +import { True } from "@adobe/data/schema"; +import { Camera } from "../camera/camera.js"; +import { screenToWorldRay } from "../camera/screen-to-world-ray.js"; +import { worldBounds } from "../scene/model/world-bounds-plugin.js"; +import type { PickHit } from "./pick-hit.js"; + +/** + * Picking — ray-against-PickableModel hit testing. + * + * `PickableModel` extends `Model` with a `pickable` flag. The archetype + * gives a clean opt-in: insert via `insertPickableModel` and the entity + * participates in picking; plain `Model` entities are invisible to the + * ray-caster. + * + * `pickRay(ray)` scans every visible PickableModel with a `_worldBounds` + * column. `pickFromScreen({ x, y })` takes pixel coords (e.g. from + * `offsetX` / `offsetY` on a pointer event), converts them to NDC + * (Normalized Device Coordinates, x/y ∈ [-1, 1], origin at center), + * then reconstructs the eye→cursor ray and delegates to `pickRay`. + * + * Exposed as **actions** (not transactions) because picks return a + * `PickHit | null` value — transactions are restricted to `Entity | void`. + * + * Future: replace the linear scan with a `broadphase` resource of type + * `Broadphase` exposing `queryRay(ray, cb)` and `queryAabb(box, cb)`. + */ + +const pickingBase = Database.Plugin.create({ + extends: Database.Plugin.combine(worldBounds, Camera.plugin), + components: { + pickable: True.schema, + }, + archetypes: { + PickableModel: ["geometry", "position", "rotation", "scale", "visible", "parent", "pickable"], + }, + transactions: { + insertPickableModel(t, args: { + geometry: number; + position?: readonly [number, number, number]; + rotation?: readonly [number, number, number, number]; + scale?: readonly [number, number, number]; + parent?: number; + }): number { + return t.archetypes.PickableModel.insert({ + geometry: args.geometry, + position: args.position ?? [0, 0, 0], + rotation: args.rotation ?? [0, 0, 0, 1], + scale: args.scale ?? [1, 1, 1], + visible: true, + parent: args.parent ?? 0, + pickable: true, + }); + }, + }, +}); + +// `picking` adds only actions (no new C/R/A), so the `db` passed to every +// action is structurally assignable to this type — full component typing +// with no `any`. +type PickingDB = Database.Plugin.ToDatabase; + +const pickRayImpl = (db: PickingDB, ray: Line3): PickHit | null => { + let best: PickHit | null = null; + for (const arch of db.queryArchetypes([ + "geometry", "visible", "pickable", "_worldBounds", + ])) { + const ids = arch.columns.id; + const vis = arch.columns.visible; + const bounds = arch.columns._worldBounds; + for (let i = 0; i < arch.rowCount; i++) { + if (!vis.get(i)) continue; + const alpha = Aabb.lineIntersection(bounds.get(i), ray); + if (alpha < 0) continue; + if (best === null || alpha < best.distance) { + best = { entity: ids.get(i), distance: alpha }; + } + } + } + return best; +}; + +export const picking = Database.Plugin.create({ + extends: pickingBase, + actions: { + pickRay(db, ray: Line3): PickHit | null { + return pickRayImpl(db, ray); + }, + pickFromScreen(db, args: { x: number; y: number }): PickHit | null { + const cam = db.resources.camera; + const canvas = db.resources.canvas; + if (!cam || !canvas) return null; + // Convert pixel coords → NDC (x/y ∈ [-1, 1], origin at canvas center). + const ray = screenToWorldRay(cam, args.x, args.y, canvas.width, canvas.height); + return pickRayImpl(db, ray); + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/plugins.md b/packages/data-gpu/src/graphics/plugins.md new file mode 100644 index 00000000..5d41af20 --- /dev/null +++ b/packages/data-gpu/src/graphics/plugins.md @@ -0,0 +1,120 @@ +```mermaid +graph LR + +%% ── Infrastructure ─────────────────────────────────────────────────────────── + graphics["**graphics** + res: device, canvas, commandEncoder + renderPassEncoder, clearColor, depthTexture + phases: input › preUpdate › update + physics › preRender › render"] + +%% ── Core model plugins ─────────────────────────────────────────────────────── + Node["**Node.plugin** + comp: position, rotation, scale + parent, visible, pickable + arch: Node"] + + Camera["**Camera.plugin** + res: camera: Camera"] + + Light["**Light.plugin** + res: light: Light + {direction, color, ambientStrength, environmentUrl}"] + + Model["**Model.plugin** + comp: geometry, modelUrl + arch: Geometry, Model"] + + scene["**scene** + (combines Node, Camera, Light, Model)"] + +%% ── Authoring abstractions ─────────────────────────────────────────────────── + animation["**animation** + comp: animationClipTracks, animationClipDuration + animationClipRef, animationTargets, animationTime + animationSpeed, animationLoop, animationPlaying + arch: AnimationClip, Animation, AnimationObservable"] + + Orbit["**Orbit.plugin** + res: orbit: Orbit + {center, radius, height, angle, autoSpin, fitGeometry…}"] + +%% ── System plugins ─────────────────────────────────────────────────────────── + sceneUniforms["**SceneUniforms.plugin** + (packs camera + light into GPU uniform buffer)"] + + transform["**transform** + (derives _worldMatrix per-frame from TRS)"] + + pbrCore["**pbrCore** + arch: _VisibleMaterial, _PbrPrimitive"] + + modelLoader["**modelLoader** + (async glTF → GPU primitives, + writes _bounds to Geometry)"] + + pbrSkinning["**pbrSkinning** + (joint matrices per-frame, arch: _Skeleton)"] + + picking["**picking** + actions: pickRay, pickFromNdc → PickHit"] + +%% ── Render aggregators ─────────────────────────────────────────────────────── + pbrIblRender["**pbrIblRender** + res: _iblEnvironment, _iblIrradiance + _iblPrefiltered, _iblBrdfLut"] + + pbrDirectRender["**pbrDirectRender** + (direct lighting, no IBL)"] + +%% ── Extends edges ──────────────────────────────────────────────────────────── + Camera --> graphics + Light --> graphics + modelLoader --> graphics + Model --> Node + scene --> Node + scene --> Camera + scene --> Light + scene --> Model + + Orbit --> Camera + sceneUniforms --> Camera + sceneUniforms --> Light + + transform --> Node + modelLoader --> pbrCore + modelLoader --> Model + modelLoader --> animation + + pbrSkinning --> modelLoader + pbrSkinning --> pbrCore + pbrSkinning --> transform + pbrSkinning --> animation + + picking --> modelLoader + picking --> transform + picking --> Camera + + pbrIblRender --> pbrCore + pbrIblRender --> modelLoader + pbrIblRender --> sceneUniforms + pbrIblRender --> transform + + pbrDirectRender --> pbrCore + pbrDirectRender --> modelLoader + pbrDirectRender --> sceneUniforms + pbrDirectRender --> transform + +%% ── Styling ────────────────────────────────────────────────────────────────── + classDef core fill:#1a3a5c,stroke:#4a8fc4,color:#fff + classDef authoring fill:#2a4a2a,stroke:#6ab06a,color:#fff + classDef system fill:#3a2a1a,stroke:#c48a4a,color:#fff + classDef renderer fill:#3a1a3a,stroke:#c46ab0,color:#fff + classDef infra fill:#1a1a3a,stroke:#6a6ab0,color:#fff + + class Node,Camera,Light,Model,scene core + class animation,Orbit authoring + class transform,pbrCore,modelLoader,pbrSkinning,sceneUniforms,picking system + class pbrIblRender,pbrDirectRender renderer + class graphics infra +``` diff --git a/packages/data-gpu/src/graphics/rendering/bone-collider-plugin.ts b/packages/data-gpu/src/graphics/rendering/bone-collider-plugin.ts new file mode 100644 index 00000000..d6003402 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/bone-collider-plugin.ts @@ -0,0 +1,182 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, Entity } from "@adobe/data/ecs"; +import { True } from "@adobe/data/schema"; +import { Vec3, Quat, Mat4x4 } from "@adobe/data/math"; +import { physicsData } from "../../physics/physics-data-plugin.js"; +import { jointData } from "../../physics/joint/joint-plugin.js"; +import { fitBoneCapsules } from "../../physics/ragdoll/fit-bone-capsules.js"; +import { ragdollTrigger } from "./ragdoll-trigger-plugin.js"; +import { pbrSkinning } from "./skinning/skinning-plugin.js"; +import { modelLoader } from "../scene/model/model-loader-plugin.js"; +import { transform } from "../scene/node/transform-plugin.js"; + +/** + * boneColliders — fits a capsule to each bone of a skinned model and ragdolls it. + * + * **Alive:** once the skin loads, `fitBoneCapsules` gives a per-bone capsule + * (bone-local offset + dims); we spawn a **kinematic** capsule per bone and each + * frame drive it to `jointWorldMatrix · offset`, so the capsules track the + * animation and collide with the world. + * + * **Ragdoll** (`triggerRagdoll`): connect each capsule to its nearest + * capsule-bearing ancestor with a point joint (anchored at the shared joint), flip + * every capsule kinematic→dynamic, and stop the animation. Now physics drives the + * capsules; `reconcileRagdoll` writes each bone's pose *back* onto its skeleton + * joint (world → bone-local), so the skinned mesh goes limp and flops. (Anatomical + * cone limits + Jolt support are follow-ups; see physics/README.md.) + */ + +/** Column-major rigid transform from a position + unit quaternion. */ +function compose(p: ArrayLike, q: ArrayLike): Mat4x4 { + return Mat4x4.multiply(Mat4x4.translation(p[0], p[1], p[2]), Quat.toMat4([q[0], q[1], q[2], q[3]])); +} + +/** Orthonormalised rotation of a column-major matrix, as a quaternion. */ +function rotationOf(m: Mat4x4): Quat { + let m00 = m[0], m10 = m[1], m20 = m[2], m01 = m[4], m11 = m[5], m21 = m[6], m02 = m[8], m12 = m[9], m22 = m[10]; + const sx = Math.hypot(m00, m10, m20) || 1, sy = Math.hypot(m01, m11, m21) || 1, sz = Math.hypot(m02, m12, m22) || 1; + m00 /= sx; m10 /= sx; m20 /= sx; m01 /= sy; m11 /= sy; m21 /= sy; m02 /= sz; m12 /= sz; m22 /= sz; + const tr = m00 + m11 + m22; + if (tr > 0) { const s = Math.sqrt(tr + 1) * 2; return [(m21 - m12) / s, (m02 - m20) / s, (m10 - m01) / s, 0.25 * s]; } + if (m00 > m11 && m00 > m22) { const s = Math.sqrt(1 + m00 - m11 - m22) * 2; return [0.25 * s, (m01 + m10) / s, (m02 + m20) / s, (m21 - m12) / s]; } + if (m11 > m22) { const s = Math.sqrt(1 + m11 - m00 - m22) * 2; return [(m01 + m10) / s, 0.25 * s, (m12 + m21) / s, (m02 - m20) / s]; } + const s = Math.sqrt(1 + m22 - m00 - m11) * 2; return [(m02 + m20) / s, (m12 + m21) / s, 0.25 * s, (m10 - m01) / s]; +} + +interface SkinGeometry { _cpuSkin?: { positions: Float32Array; joints: Uint32Array; weights: Float32Array } | null; _skinInverseBindMatrices?: Float32Array | null } + +export const boneColliders = Database.Plugin.create({ + extends: Database.Plugin.combine(physicsData, jointData, ragdollTrigger, pbrSkinning, modelLoader, transform), + components: { + _boneJoint: Entity.schema, // the skeleton joint this capsule tracks + _boneOffsetPos: Vec3.schema, // capsule offset in the bone's bind-local frame + _boneOffsetRot: Quat.schema, + _ragdollBuilt: True.schema, // tag: this skeleton's bone capsules have been generated + }, + archetypes: { + // a kinematic capsule body bound to a skeleton joint (collisionGroup 1 ⇒ the + // ragdoll's bones never collide with each other, only the world) + BoneCapsule: ["bodyType", "colliderShape", "halfExtents", "material", "collisionGroup", "position", "rotation", "linearVelocity", "angularVelocity", "_boneJoint", "_boneOffsetPos", "_boneOffsetRot"], + }, + systems: { + // Once a skeleton's skin has loaded, fit + spawn its bone capsules (one-time; + // tag + exclude). Tail→head since tagging the skeleton migrates its row. + generateBoneColliders: { + schedule: { during: ["postUpdate"] }, + create: db => () => { + const material = (Object.values(db.store.resources.materials)[0] as Entity | undefined) ?? (0 as Entity); + for (const arch of db.store.queryArchetypes(["_skeletonJoints", "_skeletonGeometry"], { exclude: ["_ragdollBuilt"] })) { + const ids = arch.columns.id, jc = arch.columns._skeletonJoints, gc = arch.columns._skeletonGeometry; + for (let i = arch.rowCount - 1; i >= 0; i--) { + const skeleton = ids.get(i), joints = jc.get(i), geo = gc.get(i); + const g = db.store.read(geo) as SkinGeometry | null; + if (!g?._cpuSkin || !g._skinInverseBindMatrices) continue; // skin not loaded yet + for (const c of fitBoneCapsules({ jointCount: joints.length, inverseBindMatrices: g._skinInverseBindMatrices, skin: g._cpuSkin })) { + db.store.archetypes.BoneCapsule.insert({ + bodyType: "kinematic", colliderShape: "capsule", halfExtents: [c.radius, c.halfHeight, 0], material, collisionGroup: 1, + position: [0, 0, 0], rotation: [0, 0, 0, 1], linearVelocity: [0, 0, 0], angularVelocity: [0, 0, 0], + _boneJoint: joints[c.jointIndex], _boneOffsetPos: c.offsetPosition, _boneOffsetRot: c.offsetRotation, + }); + } + db.store.update(skeleton, { _ragdollBuilt: true }); + } + } + }, + }, + // Alive: place every *kinematic* bone capsule at jointWorldMatrix · offset + // (the solver then kinematic-drives the engine body there). Dynamic capsules + // are ragdolling — physics owns them, so skip. + trackBoneColliders: { + schedule: { during: ["postUpdate"], after: ["generateBoneColliders", "transformSystem"] }, + create: db => () => { + for (const arch of db.store.queryArchetypes(["_boneJoint", "_boneOffsetPos", "_boneOffsetRot", "bodyType", "position", "rotation"])) { + const bj = arch.columns._boneJoint, op = arch.columns._boneOffsetPos, orot = arch.columns._boneOffsetRot, bt = arch.columns.bodyType; + const posCol = arch.columns.position, rotCol = arch.columns.rotation; + for (let i = 0; i < arch.rowCount; i++) { + if (bt.get(i) === "dynamic") continue; // ragdolling — physics drives it + const jw = db.store.get(bj.get(i), "_worldMatrix") as Mat4x4 | undefined; + if (!jw) continue; + posCol.set(i, Mat4x4.multiplyVec3(jw, op.get(i))); + rotCol.set(i, Quat.multiply(rotationOf(jw), orot.get(i))); + } + } + }, + }, + // On trigger: joint each capsule to its nearest capsule-bearing ancestor (at + // the shared joint point, anchors from the current pose), flip every capsule + // to dynamic, and stop the animation. The solver flips the engine bodies and + // mirrors the joints; the root capsule (no ancestor) falls free, dragging the rest. + ragdollOnTrigger: { + schedule: { during: ["postUpdate"], after: ["trackBoneColliders"] }, + create: db => () => { + if (!db.store.resources._ragdollTrigger) return; + db.store.resources._ragdollTrigger = false; + // joint entity → its capsule + current world pose + rotation + const byJoint = new Map(); + for (const arch of db.store.queryArchetypes(["_boneJoint", "position", "rotation"])) { + const ids = arch.columns.id, bj = arch.columns._boneJoint, pc = arch.columns.position, rc = arch.columns.rotation; + for (let i = 0; i < arch.rowCount; i++) { const r = rc.get(i); byJoint.set(bj.get(i), { capsule: ids.get(i), world: compose(pc.get(i), r), rot: r }); } + } + for (const arch of db.store.queryArchetypes(["_boneJoint", "position", "rotation"])) { + const ids = arch.columns.id, bj = arch.columns._boneJoint, pc = arch.columns.position, rc = arch.columns.rotation; + for (let i = 0; i < arch.rowCount; i++) { + const joint = bj.get(i); + // nearest ancestor joint that has a capsule + let pj = (db.store.get(joint, "parent") as Entity | undefined) ?? (0 as Entity); + while (pj && !byJoint.has(pj)) pj = (db.store.get(pj, "parent") as Entity | undefined) ?? (0 as Entity); + const parent = pj ? byJoint.get(pj) : undefined; + if (!parent) continue; // root capsule: unconstrained + const jw = db.store.get(joint, "_worldMatrix") as Mat4x4 | undefined; + const origin: Vec3 = jw ? [jw[12], jw[13], jw[14]] : [pc.get(i)[0], pc.get(i)[1], pc.get(i)[2]]; + const childRot = rc.get(i); + const childWorld = compose(pc.get(i), childRot); + // cone reference = the child bone's current axis (its capsule +Y), in the + // parent's frame; the swing-twist joint then limits how far the bone can + // deviate from this rest pose — anatomical limits, not a free ball. + const boneDirWorld = Quat.rotateVec3(childRot, [0, 1, 0]); + const axis = Quat.rotateVec3(Quat.inverse(parent.rot), boneDirWorld); + db.store.archetypes.Joint.insert({ + jointType: "cone", jointBodyA: parent.capsule, jointBodyB: ids.get(i), + jointAnchorA: Mat4x4.multiplyVec3(Mat4x4.inverse(parent.world), origin), + jointAnchorB: Mat4x4.multiplyVec3(Mat4x4.inverse(childWorld), origin), + jointAxis: axis, jointMinLimit: -0.5, jointMaxLimit: 0.5, jointSwingLimit: 0.9, // ~±29° twist, ~52° swing cone + }); + } + } + for (const arch of db.store.queryArchetypes(["_boneJoint", "bodyType"])) { + const ids = arch.columns.id; + for (let i = 0; i < arch.rowCount; i++) db.store.update(ids.get(i), { bodyType: "dynamic" }); + } + for (const arch of db.store.queryArchetypes(["animationPlaying"])) { + const ids = arch.columns.id; + for (let i = 0; i < arch.rowCount; i++) db.store.update(ids.get(i), { animationPlaying: false }); + } + }, + }, + // Ragdolling: write each dynamic bone capsule's pose back onto its skeleton + // joint as a local TRS (jointWorld = capsuleWorld · offset⁻¹, then relative to + // the parent's world), so the skinned mesh follows the physics. Two passes so + // a child uses its parent's *capsule-derived* world, keeping the chain consistent. + reconcileRagdoll: { + schedule: { during: ["preRender"], after: ["transformSystem"] }, + create: db => () => { + const jointWorld = new Map(); + for (const arch of db.store.queryArchetypes(["_boneJoint", "_boneOffsetPos", "_boneOffsetRot", "bodyType", "position", "rotation"])) { + const bj = arch.columns._boneJoint, op = arch.columns._boneOffsetPos, orot = arch.columns._boneOffsetRot, bt = arch.columns.bodyType, pc = arch.columns.position, rc = arch.columns.rotation; + for (let i = 0; i < arch.rowCount; i++) { + if (bt.get(i) !== "dynamic") continue; + jointWorld.set(bj.get(i), Mat4x4.multiply(compose(pc.get(i), rc.get(i)), Mat4x4.inverse(compose(op.get(i), orot.get(i))))); + } + } + if (jointWorld.size === 0) return; + for (const [joint, jw] of jointWorld) { + const parent = (db.store.get(joint, "parent") as Entity | undefined) ?? (0 as Entity); + const parentW = jointWorld.get(parent) ?? (parent ? db.store.get(parent, "_worldMatrix") as Mat4x4 | undefined : undefined) ?? Mat4x4.identity; + const local = Mat4x4.multiply(Mat4x4.inverse(parentW), jw); + db.store.update(joint, { position: [local[12], local[13], local[14]], rotation: rotationOf(local) }); + } + }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/display-transform-plugin.ts b/packages/data-gpu/src/graphics/rendering/display-transform-plugin.ts new file mode 100644 index 00000000..86fd1a32 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/display-transform-plugin.ts @@ -0,0 +1,26 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Vec3, Quat } from "@adobe/data/math"; + +/** + * The **display transform** seam — the pose a renderer should actually draw at, + * which may differ from the canonical `position`/`rotation` (the authoritative + * gameplay/sim state). When physics runs on a fixed clock faster or slower than + * the render rate, an interpolation pass writes the blended prev→current pose + * here each render frame so motion stays smooth; gameplay still reads the + * canonical pose. + * + * Both `_`-prefixed (derived, not authored). A renderer reads `_renderPosition`/ + * `_renderRotation` *when present* and otherwise falls back to the canonical + * `position`/`rotation` — so this plugin is a pure, physics-free dependency: + * graphics-only scenes register the components (harmless, unused) and bodies that + * nothing interpolates simply never gain them. The producer is + * `interpolation-plugin` (physics → display); the consumer is any renderer. + */ +export const displayTransform = Database.Plugin.create({ + components: { + _renderPosition: Vec3.schema, // derived: interpolated pose to render at (else use `position`) + _renderRotation: Quat.schema, + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/brdf.wgsl.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/brdf.wgsl.ts new file mode 100644 index 00000000..b201f50c --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/brdf.wgsl.ts @@ -0,0 +1,43 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +// Shared WGSL fragment: Cook-Torrance metallic-roughness BRDF building blocks. +// Concatenated into the direct and (future) IBL shader modules so both paths +// agree on the math. + +export default /* wgsl */ ` +const PI: f32 = 3.14159265359; +const MIN_ROUGHNESS: f32 = 0.04; + +// Trowbridge-Reitz / GGX normal distribution. +fn d_ggx(nDotH: f32, alpha: f32) -> f32 { + let a2 = alpha * alpha; + let denom = nDotH * nDotH * (a2 - 1.0) + 1.0; + return a2 / (PI * denom * denom); +} + +// Smith G with Schlick-GGX. +fn g_smith(nDotV: f32, nDotL: f32, alpha: f32) -> f32 { + let k = (alpha + 1.0) * (alpha + 1.0) / 8.0; + let gv = nDotV / (nDotV * (1.0 - k) + k); + let gl = nDotL / (nDotL * (1.0 - k) + k); + return gv * gl; +} + +// Schlick Fresnel. +fn f_schlick(vDotH: f32, f0: vec3f) -> vec3f { + let f = pow(clamp(1.0 - vDotH, 0.0, 1.0), 5.0); + return f0 + (vec3f(1.0) - f0) * f; +} + +// Roughness-aware Schlick for IBL: avoids over-strong rim at high roughness. +fn f_schlick_roughness(nDotV: f32, f0: vec3f, roughness: f32) -> vec3f { + let r = vec3f(1.0 - roughness); + return f0 + (max(r, f0) - f0) * pow(max(1.0 - nDotV, 0.0), 5.0); +} + +// Narkowicz ACES filmic tonemap fit. Cheap, looks clearly filmic. +fn tone_map_aces(x: vec3f) -> vec3f { + let a = 2.51; let b = 0.03; let c = 2.43; let d = 0.59; let e = 0.14; + return clamp((x * (a * x + b)) / (x * (c * x + d) + e), vec3f(0.0), vec3f(1.0)); +} +`; diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl-render-plugin.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl-render-plugin.ts new file mode 100644 index 00000000..6d0f2a1f --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl-render-plugin.ts @@ -0,0 +1,415 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Mat4x4, Vec3 } from "@adobe/data/math"; +import { pbrCore } from "../pbr-core-plugin.js"; +import { modelLoader } from "../../scene/model/model-loader-plugin.js"; +import { transform } from "../../scene/node/transform-plugin.js"; +import { buildIblResources } from "./ibl/build-ibl-resources.js"; +import { parseHdr } from "./ibl/parse-hdr.js"; +import { VisibleMaterial } from "../visible-material/visible-material.js"; +import { SceneUniforms } from "../../scene/scene-uniforms/scene-uniforms.js"; +import { StandardVertex } from "../standard-vertex/standard-vertex.js"; +import { SkinningAttributes } from "../skinning/skinning-attributes/skinning-attributes.js"; +import { buildIblShader } from "./ibl-shader.wgsl.js"; +import skyboxShader from "./skybox-shader.wgsl.js"; + +const PREFILTERED_MIP_COUNT = 7; + +function createIblBindGroupLayout(device: GPUDevice): GPUBindGroupLayout { + return device.createBindGroupLayout({ + entries: [ + { binding: 0, visibility: GPUShaderStage.FRAGMENT, texture: { sampleType: "float", viewDimension: "cube" } }, + { binding: 1, visibility: GPUShaderStage.FRAGMENT, texture: { sampleType: "float", viewDimension: "cube" } }, + { binding: 2, visibility: GPUShaderStage.FRAGMENT, texture: { sampleType: "float", viewDimension: "2d" } }, + { binding: 3, visibility: GPUShaderStage.FRAGMENT, sampler: { type: "filtering" } }, + ], + }); +} + +function createSkyboxBindGroupLayout(device: GPUDevice): GPUBindGroupLayout { + return device.createBindGroupLayout({ + entries: [ + { binding: 0, visibility: GPUShaderStage.VERTEX, buffer: { type: "uniform" } }, + { binding: 1, visibility: GPUShaderStage.FRAGMENT, texture: { sampleType: "float", viewDimension: "cube" } }, + { binding: 2, visibility: GPUShaderStage.FRAGMENT, sampler: { type: "filtering" } }, + ], + }); +} + +/** + * pbrIblRender — IBL PBR rendering aggregator. Combines the implementation + * plugins needed to draw `Model` entities with image-based lighting: + * + * - pbrCore (ephemeral primitive/material shape declarations) + * - modelLoader (glTF → primitives) + * - SceneUniforms.plugin (camera + light packed into a GPU buffer) + * - transform (Node TRS → _worldMatrix) + * + * The render system is skin-aware — if a primitive has a `_skinVertexBuffer`, + * it uses the skinned pipeline and looks up the matching joint bind group. + * Add `pbrSkinning` separately to populate those joint matrices for skinned + * meshes. + * + * Adds its own systems: + * - iblInitSystem : light.environmentUrl → IBL cube/2D textures + * - pbrIblRenderSystem: visible Models → drawIndexed calls + * + * `_PbrPrimitive` archetype. + */ +export const pbrIblRender = Database.Plugin.create({ + extends: Database.Plugin.combine(pbrCore, modelLoader, SceneUniforms.plugin, transform), + resources: { + _iblEnvironment: { default: null as GPUTexture | null, transient: true }, + _iblIrradiance: { default: null as GPUTexture | null, transient: true }, + _iblPrefiltered: { default: null as GPUTexture | null, transient: true }, + _iblBrdfLut: { default: null as GPUTexture | null, transient: true }, + }, + systems: { + iblInitSystem: { + create: db => { + let started = false; + return () => { + if (started) return; + const { device, light } = db.store.resources; + if (!device) return; + const environmentUrl = light.environmentUrl; + started = true; + + const buildAndAssign = (hdr?: Awaited>) => { + const r = buildIblResources(device, { + prefilteredMipCount: PREFILTERED_MIP_COUNT, + hdrSource: hdr, + }); + db.store.resources._iblEnvironment = r.environment; + db.store.resources._iblIrradiance = r.irradiance; + db.store.resources._iblPrefiltered = r.prefiltered; + db.store.resources._iblBrdfLut = r.brdfLut; + }; + + if (environmentUrl) { + fetch(environmentUrl) + .then(r => { + if (!r.ok) throw new Error(`HDR fetch failed: ${r.status}`); + return r.arrayBuffer(); + }) + .then(buf => buildAndAssign(parseHdr(buf))) + .catch(err => { + console.warn("[_pbrIblRender] HDR load failed; using procedural fallback", err); + buildAndAssign(); + }); + } else { + buildAndAssign(); + } + }; + }, + schedule: { during: ["preRender"] }, + }, + pbrIblRenderSystem: { + create: db => { + let pipeline: GPURenderPipeline | null = null; + let skinnedPipeline: GPURenderPipeline | null = null; + let skyboxPipeline: GPURenderPipeline | null = null; + let sceneLayout: GPUBindGroupLayout | null = null; + let materialLayout: GPUBindGroupLayout | null = null; + let iblLayout: GPUBindGroupLayout | null = null; + let instanceLayout: GPUBindGroupLayout | null = null; + let skinnedInstanceLayout: GPUBindGroupLayout | null = null; + let skyboxLayout: GPUBindGroupLayout | null = null; + let iblSampler: GPUSampler | null = null; + let skyboxUniformBuffer: GPUBuffer | null = null; + let sceneBindGroup: GPUBindGroup | null = null; + let iblBindGroup: GPUBindGroup | null = null; + let skyboxBindGroup: GPUBindGroup | null = null; + let cachedSceneBuffer: GPUBuffer | null = null; + let cachedIblIrradiance: GPUTexture | null = null; + let cachedSkyEnvironment: GPUTexture | null = null; + const skyboxScratch = new Float32Array(12); + type InstanceEntry = { buffer: GPUBuffer; bindGroup: GPUBindGroup; capacity: number }; + const instanceCache = new Map(); + let instanceScratch = new Float32Array(16); + + return () => { + const { + device, renderPassEncoder, canvasFormat, depthFormat, _sceneUniformsBuffer, + _iblEnvironment, _iblIrradiance, _iblPrefiltered, _iblBrdfLut, camera, + } = db.store.resources; + if (!device || !renderPassEncoder || !_sceneUniformsBuffer || !camera) return; + if (!_iblEnvironment || !_iblIrradiance || !_iblPrefiltered || !_iblBrdfLut) return; + + if (!sceneLayout) sceneLayout = SceneUniforms.createBindGroupLayout(device); + if (!materialLayout) materialLayout = VisibleMaterial.createBindGroupLayout(device); + if (!iblLayout) iblLayout = createIblBindGroupLayout(device); + if (!instanceLayout) instanceLayout = device.createBindGroupLayout({ + entries: [{ binding: 0, visibility: GPUShaderStage.VERTEX, buffer: { type: "read-only-storage" } }], + }); + if (!skinnedInstanceLayout) skinnedInstanceLayout = device.createBindGroupLayout({ + entries: [ + { binding: 0, visibility: GPUShaderStage.VERTEX, buffer: { type: "read-only-storage" } }, + { binding: 1, visibility: GPUShaderStage.VERTEX, buffer: { type: "read-only-storage" } }, + ], + }); + if (!skyboxLayout) skyboxLayout = createSkyboxBindGroupLayout(device); + if (!iblSampler) { + iblSampler = device.createSampler({ + magFilter: "linear", + minFilter: "linear", + mipmapFilter: "linear", + addressModeU: "clamp-to-edge", + addressModeV: "clamp-to-edge", + addressModeW: "clamp-to-edge", + }); + } + if (!skyboxUniformBuffer) { + skyboxUniformBuffer = device.createBuffer({ + size: 48, + usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST, + }); + } + + if (!pipeline) { + const module = device.createShaderModule({ + code: buildIblShader({ prefilteredMipCount: PREFILTERED_MIP_COUNT, skinned: false }), + }); + pipeline = device.createRenderPipeline({ + layout: device.createPipelineLayout({ + bindGroupLayouts: [sceneLayout, materialLayout, iblLayout, instanceLayout], + }), + vertex: { module, entryPoint: "vs_main", buffers: [StandardVertex.layout] }, + fragment: { module, entryPoint: "fs_main", targets: [{ format: canvasFormat }] }, + primitive: { topology: "triangle-list", cullMode: "back" }, + depthStencil: { format: depthFormat, depthWriteEnabled: true, depthCompare: "less" }, + }); + } + if (!skinnedPipeline) { + const module = device.createShaderModule({ + code: buildIblShader({ prefilteredMipCount: PREFILTERED_MIP_COUNT, skinned: true }), + }); + skinnedPipeline = device.createRenderPipeline({ + layout: device.createPipelineLayout({ + bindGroupLayouts: [sceneLayout, materialLayout, iblLayout, skinnedInstanceLayout], + }), + vertex: { module, entryPoint: "vs_main", buffers: [StandardVertex.layout, SkinningAttributes.layout] }, + fragment: { module, entryPoint: "fs_main", targets: [{ format: canvasFormat }] }, + primitive: { topology: "triangle-list", cullMode: "back" }, + depthStencil: { format: depthFormat, depthWriteEnabled: true, depthCompare: "less" }, + }); + } + if (!skyboxPipeline) { + const sm = device.createShaderModule({ code: skyboxShader }); + skyboxPipeline = device.createRenderPipeline({ + layout: device.createPipelineLayout({ bindGroupLayouts: [skyboxLayout] }), + vertex: { module: sm, entryPoint: "vs_skybox" }, + fragment: { module: sm, entryPoint: "fs_skybox", targets: [{ format: canvasFormat }] }, + primitive: { topology: "triangle-list" }, + depthStencil: { format: depthFormat, depthWriteEnabled: false, depthCompare: "always" }, + }); + } + + if (_sceneUniformsBuffer !== cachedSceneBuffer || !sceneBindGroup) { + sceneBindGroup = device.createBindGroup({ + layout: sceneLayout, + entries: [{ binding: 0, resource: { buffer: _sceneUniformsBuffer } }], + }); + cachedSceneBuffer = _sceneUniformsBuffer; + } + if (_iblIrradiance !== cachedIblIrradiance || !iblBindGroup) { + iblBindGroup = device.createBindGroup({ + layout: iblLayout, + entries: [ + { binding: 0, resource: _iblIrradiance.createView({ dimension: "cube" }) }, + { binding: 1, resource: _iblPrefiltered.createView({ dimension: "cube" }) }, + { binding: 2, resource: _iblBrdfLut.createView() }, + { binding: 3, resource: iblSampler }, + ], + }); + cachedIblIrradiance = _iblIrradiance; + } + if (_iblEnvironment !== cachedSkyEnvironment || !skyboxBindGroup) { + skyboxBindGroup = device.createBindGroup({ + layout: skyboxLayout, + entries: [ + { binding: 0, resource: { buffer: skyboxUniformBuffer } }, + { binding: 1, resource: _iblEnvironment.createView({ dimension: "cube" }) }, + { binding: 2, resource: iblSampler }, + ], + }); + cachedSkyEnvironment = _iblEnvironment; + } + + // Camera basis for the skybox view rays. + const forward = Vec3.normalize(Vec3.subtract(camera.target, camera.position)); + const right = Vec3.normalize(Vec3.cross(forward, camera.up)); + const upOrtho = Vec3.cross(right, forward); + const tanHalfFov = Math.tan(camera.fieldOfView / 2); + skyboxScratch[0] = right[0]; + skyboxScratch[1] = right[1]; + skyboxScratch[2] = right[2]; + skyboxScratch[3] = camera.aspect; + skyboxScratch[4] = upOrtho[0]; + skyboxScratch[5] = upOrtho[1]; + skyboxScratch[6] = upOrtho[2]; + skyboxScratch[7] = tanHalfFov; + skyboxScratch[8] = forward[0]; + skyboxScratch[9] = forward[1]; + skyboxScratch[10] = forward[2]; + skyboxScratch[11] = 0; + device.queue.writeBuffer(skyboxUniformBuffer, 0, skyboxScratch); + + renderPassEncoder.setPipeline(skyboxPipeline); + renderPassEncoder.setBindGroup(0, skyboxBindGroup); + renderPassEncoder.draw(3); + + // Collect visible Models grouped by geometry. World matrix comes + // from each Model's `_worldMatrix` component (written by _transform). + interface ModelEntry { id: number; matrix: Mat4x4 } + const modelsByGeo = new Map(); + for (const arch of db.store.queryArchetypes(["geometry", "visible", "_worldMatrix"])) { + const ids = arch.columns.id; + const geoRefs = arch.columns.geometry; + const vis = arch.columns.visible; + const worldMats = arch.columns._worldMatrix; + for (let i = 0; i < arch.rowCount; i++) { + if (!vis.get(i)) continue; + const id = ids.get(i); + const m = worldMats.get(i); + const geoId = geoRefs.get(i); + let arr = modelsByGeo.get(geoId); + if (!arr) { arr = []; modelsByGeo.set(geoId, arr); } + arr.push({ id, matrix: m }); + } + } + + const materialMap = new Map(); + for (const arch of db.store.queryArchetypes(["_materialBindGroup"])) { + const ids = arch.columns.id; + const bgs = arch.columns._materialBindGroup; + for (let i = 0; i < arch.rowCount; i++) { + const bg = bgs.get(i); if (bg) materialMap.set(ids.get(i), bg); + } + } + + const jointBindGroupByModel = new Map(); + for (const arch of db.store.queryArchetypes([ + "_skeletonModelRef", + "_skeletonJointMatrixBindGroup", + ])) { + const modelRefs = arch.columns._skeletonModelRef; + const bgs = arch.columns._skeletonJointMatrixBindGroup; + for (let i = 0; i < arch.rowCount; i++) { + const bg = bgs.get(i); + if (bg) jointBindGroupByModel.set(modelRefs.get(i), bg); + } + } + + renderPassEncoder.setBindGroup(0, sceneBindGroup); + renderPassEncoder.setBindGroup(2, iblBindGroup); + + let lastPipeline: GPURenderPipeline | null = null; + let lastMat: GPUBindGroup | null = null; + let lastInstBG: GPUBindGroup | null = null; + for (const archetype of db.store.queryArchetypes([ + "_vertexBuffer", + "_skinVertexBuffer", + "_indexBuffer", + "_indexCount", + "_indexFormat", + "_material", + "_geometry", + "_nodeLocalMatrix", + ])) { + const vbs = archetype.columns._vertexBuffer; + const skinVbs = archetype.columns._skinVertexBuffer; + const ibs = archetype.columns._indexBuffer; + const counts = archetype.columns._indexCount; + const formats = archetype.columns._indexFormat; + const matRefs = archetype.columns._material; + const geoRefs = archetype.columns._geometry; + const nodeMats = archetype.columns._nodeLocalMatrix; + const primIds = archetype.columns.id; + for (let i = 0; i < archetype.rowCount; i++) { + const geoId = geoRefs.get(i); + const models = modelsByGeo.get(geoId); + if (!models) continue; + + const skinBuf = skinVbs.get(i); + const nodeMatrix = nodeMats.get(i); + const primId = primIds.get(i); + const mat = materialMap.get(matRefs.get(i)); + if (!mat) continue; + + if (skinBuf) { + // Skinned: one draw per Model; the skeleton owns the + // 2-binding bind group with instance + joint matrices. + if (lastPipeline !== skinnedPipeline) { + renderPassEncoder.setPipeline(skinnedPipeline); + lastPipeline = skinnedPipeline; + lastInstBG = null; + } + if (mat !== lastMat) { + renderPassEncoder.setBindGroup(1, mat); + lastMat = mat; + } + renderPassEncoder.setVertexBuffer(0, vbs.get(i)!); // _PbrPrimitive archetype guarantees non-null + renderPassEncoder.setVertexBuffer(1, skinBuf); + renderPassEncoder.setIndexBuffer(ibs.get(i)!, formats.get(i)); // _PbrPrimitive archetype guarantees non-null + for (const model of models) { + const skeletonBG = jointBindGroupByModel.get(model.id); + if (!skeletonBG) continue; + if (skeletonBG !== lastInstBG) { + renderPassEncoder.setBindGroup(3, skeletonBG); + lastInstBG = skeletonBG; + } + renderPassEncoder.drawIndexed(counts.get(i), 1); + } + continue; + } + + // Static: instanced draw. + if (lastPipeline !== pipeline) { + renderPassEncoder.setPipeline(pipeline); + lastPipeline = pipeline; + lastInstBG = null; + } + const instCount = models.length; + let entry = instanceCache.get(primId); + if (!entry || entry.capacity < instCount) { + entry?.buffer.destroy(); + const buffer = device.createBuffer({ + size: Math.max(instCount * 64, 64), + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + const bindGroup = device.createBindGroup({ + layout: instanceLayout, + entries: [{ binding: 0, resource: { buffer } }], + }); + entry = { buffer, bindGroup, capacity: instCount }; + instanceCache.set(primId, entry); + } + if (instanceScratch.length < instCount * 16) { + instanceScratch = new Float32Array(instCount * 16); + } + for (let j = 0; j < instCount; j++) { + instanceScratch.set(Mat4x4.multiply(models[j].matrix, nodeMatrix), j * 16); + } + device.queue.writeBuffer(entry.buffer, 0, instanceScratch, 0, instCount * 16); + + if (mat !== lastMat) { + renderPassEncoder.setBindGroup(1, mat); + lastMat = mat; + } + if (entry.bindGroup !== lastInstBG) { + renderPassEncoder.setBindGroup(3, entry.bindGroup); + lastInstBG = entry.bindGroup; + } + renderPassEncoder.setVertexBuffer(0, vbs.get(i)!); // _PbrPrimitive archetype guarantees non-null + renderPassEncoder.setIndexBuffer(ibs.get(i)!, formats.get(i)); // _PbrPrimitive archetype guarantees non-null + renderPassEncoder.drawIndexed(counts.get(i), instCount); + } + } + }; + }, + schedule: { during: ["render"], after: ["beginRenderPass", "iblInitSystem", "transformSystem"], before: ["endRenderPass"] }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl-shader.wgsl.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl-shader.wgsl.ts new file mode 100644 index 00000000..c85f42a3 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl-shader.wgsl.ts @@ -0,0 +1,185 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { wgslStructFields } from "@adobe/data/typed-buffer"; +import { schema as sceneUniformsSchema } from "../../scene/scene-uniforms/schema.js"; +import { schema as visibleMaterialSchema } from "../visible-material/schema.js"; +import brdf from "./brdf.wgsl.js"; + +export interface IblShaderOptions { + prefilteredMipCount: number; + /** When true, the vertex shader accepts JOINTS_0 / WEIGHTS_0 attributes and + * a joint-matrix storage buffer in bind group 4, and blends each vertex by + * the four weighted joint matrices before applying the instance transform. */ + skinned: boolean; +} + +export function buildIblShader(options: IblShaderOptions): string { + // Skinned variant adds jointMatrices as binding 1 of group 3 (sharing the + // slot with instance matrices) so we stay within WebGPU's default + // maxBindGroups = 4 limit. + const skinningBindings = options.skinned ? /* wgsl */ ` +@group(3) @binding(1) var jointMatrices: array>; +` : ""; + + const vertexInputs = options.skinned ? /* wgsl */ ` +struct VertexInput { + @location(0) position: vec3f, + @location(1) normal: vec3f, + @location(2) tangent: vec4f, + @location(3) uv: vec2f, + @location(4) joints: vec4u, + @location(5) weights: vec4f, +}` : /* wgsl */ ` +struct VertexInput { + @location(0) position: vec3f, + @location(1) normal: vec3f, + @location(2) tangent: vec4f, + @location(3) uv: vec2f, +}`; + + const vertexMain = options.skinned ? /* wgsl */ ` +@vertex +fn vs_main(@builtin(instance_index) instanceIndex: u32, in: VertexInput) -> VertexOutput { + let skinMat = + jointMatrices[in.joints.x] * in.weights.x + + jointMatrices[in.joints.y] * in.weights.y + + jointMatrices[in.joints.z] * in.weights.z + + jointMatrices[in.joints.w] * in.weights.w; + let m = instances[instanceIndex] * skinMat; + let m3 = mat3x3(m[0].xyz, m[1].xyz, m[2].xyz); + let normalMat = mat3x3( + cross(m3[1], m3[2]), + cross(m3[2], m3[0]), + cross(m3[0], m3[1]), + ); + let worldPos = m * vec4f(in.position, 1.0); + var out: VertexOutput; + out.clipPosition = scene.viewProjectionMatrix * worldPos; + out.worldPosition = worldPos.xyz; + out.normal = normalize(normalMat * in.normal); + out.tangent = normalize(m3 * in.tangent.xyz); + out.bitangent = normalize(cross(out.normal, out.tangent) * in.tangent.w); + out.uv = in.uv; + return out; +}` : /* wgsl */ ` +@vertex +fn vs_main(@builtin(instance_index) instanceIndex: u32, in: VertexInput) -> VertexOutput { + let m = instances[instanceIndex]; + let m3 = mat3x3(m[0].xyz, m[1].xyz, m[2].xyz); + let normalMat = mat3x3( + cross(m3[1], m3[2]), + cross(m3[2], m3[0]), + cross(m3[0], m3[1]), + ); + let worldPos = m * vec4f(in.position, 1.0); + var out: VertexOutput; + out.clipPosition = scene.viewProjectionMatrix * worldPos; + out.worldPosition = worldPos.xyz; + out.normal = normalize(normalMat * in.normal); + out.tangent = normalize(m3 * in.tangent.xyz); + out.bitangent = normalize(cross(out.normal, out.tangent) * in.tangent.w); + out.uv = in.uv; + return out; +}`; + + return /* wgsl */ ` +struct SceneUniforms { +${wgslStructFields(sceneUniformsSchema)} +} + +struct VisibleMaterial { +${wgslStructFields(visibleMaterialSchema)} +} + +@group(0) @binding(0) var scene: SceneUniforms; + +@group(1) @binding(0) var material: VisibleMaterial; +@group(1) @binding(1) var baseColorTexture: texture_2d; +@group(1) @binding(2) var metallicRoughnessTexture: texture_2d; +@group(1) @binding(3) var normalTexture: texture_2d; +@group(1) @binding(4) var occlusionTexture: texture_2d; +@group(1) @binding(5) var emissiveTexture: texture_2d; +@group(1) @binding(6) var pbrSampler: sampler; + +@group(2) @binding(0) var iblIrradiance: texture_cube; +@group(2) @binding(1) var iblPrefiltered: texture_cube; +@group(2) @binding(2) var iblBrdfLut: texture_2d; +@group(2) @binding(3) var iblSampler: sampler; + +@group(3) @binding(0) var instances: array>; +${skinningBindings} +const PREFILTERED_MIP_COUNT: f32 = ${options.prefilteredMipCount.toFixed(1)}; + +${vertexInputs} + +struct VertexOutput { + @builtin(position) clipPosition: vec4f, + @location(0) worldPosition: vec3f, + @location(1) normal: vec3f, + @location(2) tangent: vec3f, + @location(3) bitangent: vec3f, + @location(4) uv: vec2f, +} + +${vertexMain} + +${brdf} + +@fragment +fn fs_main(in: VertexOutput) -> @location(0) vec4f { + let baseColor = textureSample(baseColorTexture, pbrSampler, in.uv) * material.baseColorFactor; + + let mr = textureSample(metallicRoughnessTexture, pbrSampler, in.uv); + let metallic = mr.b * material.metallicFactor; + var roughness = mr.g * material.roughnessFactor; + roughness = max(roughness, MIN_ROUGHNESS); + let alpha = roughness * roughness; + + let occlusion = textureSample(occlusionTexture, pbrSampler, in.uv).r; + let emissive = textureSample(emissiveTexture, pbrSampler, in.uv).rgb * material.emissiveFactor; + + let nSampled = textureSample(normalTexture, pbrSampler, in.uv).rgb * 2.0 - vec3f(1.0); + let nScaled = vec3f(nSampled.xy * material.normalScale, nSampled.z); + let tbn = mat3x3(in.tangent, in.bitangent, in.normal); + let N = normalize(tbn * nScaled); + + let V = normalize(scene.cameraPosition - in.worldPosition); + let nDotV = max(dot(N, V), 0.001); + let R = reflect(-V, N); + + let f0 = mix(vec3f(0.04), baseColor.rgb, metallic); + + // --- IBL contribution (split-sum) --- + let F_ibl = f_schlick_roughness(nDotV, f0, roughness); + let kD_ibl = (vec3f(1.0) - F_ibl) * (1.0 - metallic); + + let irradiance = textureSampleLevel(iblIrradiance, iblSampler, N, 0.0).rgb; + let diffuseIbl = kD_ibl * irradiance * baseColor.rgb; + + let mipLevel = roughness * (PREFILTERED_MIP_COUNT - 1.0); + let prefiltered = textureSampleLevel(iblPrefiltered, iblSampler, R, mipLevel).rgb; + let envBrdf = textureSampleLevel(iblBrdfLut, iblSampler, vec2f(nDotV, roughness), 0.0).rg; + let specularIbl = prefiltered * (F_ibl * envBrdf.x + envBrdf.y); + + let ambient = (diffuseIbl + specularIbl) * mix(1.0, occlusion, material.occlusionStrength); + + // --- Direct light (keeps parity with the direct renderer's single light) --- + let L = normalize(-scene.lightDirection); + let H = normalize(V + L); + let nDotL = max(dot(N, L), 0.0); + let nDotH = max(dot(N, H), 0.0); + let vDotH = max(dot(V, H), 0.0); + let D_d = d_ggx(nDotH, alpha); + let G_d = g_smith(nDotV, nDotL, alpha); + let F_d = f_schlick(vDotH, f0); + let spec_d = (D_d * G_d * F_d) / (4.0 * nDotV * nDotL + 0.0001); + let kD_d = (vec3f(1.0) - F_d) * (1.0 - metallic); + let direct = (kD_d * baseColor.rgb / PI + spec_d) * scene.lightColor * nDotL; + + let color = direct + ambient + emissive; + let mapped = tone_map_aces(color); + let gamma = pow(mapped, vec3f(1.0 / 2.2)); + return vec4f(gamma, baseColor.a); +} +`; +} diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/brdf-lut.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/brdf-lut.ts new file mode 100644 index 00000000..0392a2ce --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/brdf-lut.ts @@ -0,0 +1,101 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import iblMath from "./ibl-math.wgsl.js"; +import { FULLSCREEN_VS } from "./render-helpers.js"; + +// Karis split-sum BRDF integration. Encodes the F0-independent part of the +// specular IBL integral. Lookup is (NdotV, roughness) → (scale, bias) such +// that specular_IBL = prefiltered * (F0 * scale + bias). +const BRDF_LUT_FS = /* wgsl */ ` +struct Params { size: u32, _pad: u32, _pad2: u32, _pad3: u32 }; +@group(0) @binding(0) var params: Params; + +const SAMPLES: u32 = 1024u; + +fn integrate(nDotV: f32, roughness: f32) -> vec2f { + let V = vec3f(sqrt(1.0 - nDotV * nDotV), 0.0, nDotV); + // N = (0,0,1): work directly in tangent space. importance_sample_ggx's + // TBN frame degenerates when N=(0,0,1), so we sample H inline here. + let a = roughness * roughness; + + var A = 0.0; + var B = 0.0; + for (var i = 0u; i < SAMPLES; i = i + 1u) { + let xi = hammersley(i, SAMPLES); + let phi = 2.0 * PI * xi.x; + let cosTheta = sqrt((1.0 - xi.y) / (1.0 + (a * a - 1.0) * xi.y)); + let sinTheta = sqrt(1.0 - cosTheta * cosTheta); + let H = vec3f(cos(phi) * sinTheta, sin(phi) * sinTheta, cosTheta); + let L = normalize(2.0 * dot(V, H) * H - V); + let nDotL = max(L.z, 0.0); + let nDotH = max(H.z, 0.0); + let vDotH = max(dot(V, H), 0.0); + if (nDotL > 0.0) { + let G = g_smith_ibl(nDotV, nDotL, roughness); + let g_vis = (G * vDotH) / max(nDotH * nDotV, 0.0001); + let fc = pow(1.0 - vDotH, 5.0); + A = A + (1.0 - fc) * g_vis; + B = B + fc * g_vis; + } + } + return vec2f(A, B) / f32(SAMPLES); +} + +@fragment +fn fs_main(@builtin(position) frag: vec4f) -> @location(0) vec4f { + let uv = frag.xy / f32(params.size); + let nDotV = max(uv.x, 0.001); + let roughness = max(uv.y, 0.04); + let r = integrate(nDotV, roughness); + return vec4f(r, 0.0, 1.0); +} +`; + +export function generateBrdfLut( + device: GPUDevice, + encoder: GPUCommandEncoder, + size = 256, +): GPUTexture { + const format: GPUTextureFormat = "rgba16float"; + const texture = device.createTexture({ + size: [size, size, 1], + format, + usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.RENDER_ATTACHMENT, + }); + + const module = device.createShaderModule({ code: FULLSCREEN_VS + iblMath + BRDF_LUT_FS }); + const bindGroupLayout = device.createBindGroupLayout({ + entries: [{ binding: 0, visibility: GPUShaderStage.FRAGMENT, buffer: { type: "uniform" } }], + }); + const pipeline = device.createRenderPipeline({ + layout: device.createPipelineLayout({ bindGroupLayouts: [bindGroupLayout] }), + vertex: { module, entryPoint: "vs_fullscreen" }, + fragment: { module, entryPoint: "fs_main", targets: [{ format }] }, + primitive: { topology: "triangle-list" }, + }); + + const ub = device.createBuffer({ + size: 16, + usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(ub, 0, new Uint32Array([size, 0, 0, 0])); + const bindGroup = device.createBindGroup({ + layout: bindGroupLayout, + entries: [{ binding: 0, resource: { buffer: ub } }], + }); + + const pass = encoder.beginRenderPass({ + colorAttachments: [{ + view: texture.createView(), + loadOp: "clear", + storeOp: "store", + clearValue: [0, 0, 0, 1], + }], + }); + pass.setPipeline(pipeline); + pass.setBindGroup(0, bindGroup); + pass.draw(3); + pass.end(); + + return texture; +} diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/build-ibl-resources.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/build-ibl-resources.ts new file mode 100644 index 00000000..2a1728d8 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/build-ibl-resources.ts @@ -0,0 +1,57 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { generateBrdfLut } from "./brdf-lut.js"; +import { generateEnvironment } from "./environment.js"; +import { generateIrradiance } from "./irradiance.js"; +import type { ParsedHdr } from "./parse-hdr.js"; +import { generatePrefiltered } from "./prefilter.js"; + +export interface IblResources { + environment: GPUTexture; + irradiance: GPUTexture; + prefiltered: GPUTexture; + prefilteredMipCount: number; + brdfLut: GPUTexture; +} + +export interface IblOptions { + environmentSize?: number; + irradianceSize?: number; + prefilteredSize?: number; + prefilteredMipCount?: number; + brdfLutSize?: number; + /** Equirectangular HDR source. When omitted, a procedural studio is used. */ + hdrSource?: ParsedHdr; +} + +/** + * Computes the four IBL maps used by the prefiltered split-sum approximation: + * source environment, diffuse irradiance, specular prefiltered (mip chain by + * roughness), and the BRDF integration LUT. Everything runs in one command + * buffer; the GPU sequences the dependencies internally. + */ +export function buildIblResources(device: GPUDevice, options: IblOptions = {}): IblResources { + const envSize = options.environmentSize ?? 512; + const irradianceSize = options.irradianceSize ?? 32; + const prefilteredSize = options.prefilteredSize ?? 256; + const prefilteredMipCount = options.prefilteredMipCount ?? 7; + const brdfLutSize = options.brdfLutSize ?? 256; + + const encoder = device.createCommandEncoder({ label: "ibl-precompute" }); + const environment = generateEnvironment(device, encoder, envSize, options.hdrSource); + const irradiance = generateIrradiance(device, encoder, environment, irradianceSize); + const prefiltered = generatePrefiltered(device, encoder, environment, prefilteredSize, prefilteredMipCount); + device.queue.submit([encoder.finish()]); + + const brdfEncoder = device.createCommandEncoder({ label: "ibl-brdf-lut" }); + const brdfLut = generateBrdfLut(device, brdfEncoder, brdfLutSize); + device.queue.submit([brdfEncoder.finish()]); + + return { + environment, + irradiance, + prefiltered: prefiltered.texture, + prefilteredMipCount: prefiltered.mipLevelCount, + brdfLut, + }; +} diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/create-cubemap.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/create-cubemap.ts new file mode 100644 index 00000000..49a4ac98 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/create-cubemap.ts @@ -0,0 +1,17 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export function createCubemap( + device: GPUDevice, + size: number, + format: GPUTextureFormat, + mipLevelCount = 1, +): GPUTexture { + return device.createTexture({ + size: [size, size, 6], + format, + mipLevelCount, + usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.COPY_DST, + dimension: "2d", + textureBindingViewDimension: "cube", + }); +} diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/cube-face-view.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/cube-face-view.ts new file mode 100644 index 00000000..172d3ff2 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/cube-face-view.ts @@ -0,0 +1,11 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export function cubeFaceView(texture: GPUTexture, face: number, mip = 0): GPUTextureView { + return texture.createView({ + dimension: "2d", + baseArrayLayer: face, + arrayLayerCount: 1, + baseMipLevel: mip, + mipLevelCount: 1, + }); +} diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/cubemap-sample-view.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/cubemap-sample-view.ts new file mode 100644 index 00000000..c228bfc7 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/cubemap-sample-view.ts @@ -0,0 +1,9 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export function cubemapSampleView(texture: GPUTexture): GPUTextureView { + return texture.createView({ + dimension: "cube", + baseArrayLayer: 0, + arrayLayerCount: 6, + }); +} diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/environment.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/environment.ts new file mode 100644 index 00000000..37cb3151 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/environment.ts @@ -0,0 +1,190 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { floatArrayToHalf } from "./float-to-half.js"; +import iblMath from "./ibl-math.wgsl.js"; +import type { ParsedHdr } from "./parse-hdr.js"; +import { createCubemap, cubeFaceView, FULLSCREEN_VS } from "./render-helpers.js"; + +const ENV_PROCEDURAL_FS = /* wgsl */ ` +struct Params { face: u32, size: u32 }; +@group(0) @binding(0) var params: Params; + +fn env_color(dir: vec3f) -> vec3f { + let sky_top = vec3f(0.45, 0.55, 0.75); + let horizon = vec3f(0.18, 0.20, 0.24); + let ground = vec3f(0.08, 0.08, 0.10); + var color = mix(horizon, sky_top, smoothstep(-0.05, 0.65, dir.y)); + color = mix(ground, color, smoothstep(-0.3, 0.0, dir.y)); + + let warm = vec3f(9.0, 7.2, 5.0); + let cool = vec3f(5.0, 7.5, 10.0); + for (var i = 0u; i < 4u; i = i + 1u) { + let a = f32(i) * 1.5707963; + let ld = normalize(vec3f(sin(a) * 0.55, 0.85, cos(a) * 0.55)); + let d = max(dot(dir, ld), 0.0); + let intensity = pow(d, 96.0); + let lc = select(warm, cool, i % 2u == 0u); + color = color + intensity * lc; + } + return color; +} + +@fragment +fn fs_main(@builtin(position) frag: vec4f) -> @location(0) vec4f { + let uv = frag.xy / f32(params.size); + let dir = cube_uv_to_dir(params.face, uv); + return vec4f(env_color(dir), 1.0); +} +`; + +const ENV_EQUIRECT_FS = /* wgsl */ ` +struct Params { face: u32, size: u32 }; +@group(0) @binding(0) var params: Params; +@group(0) @binding(1) var equirect: texture_2d; +@group(0) @binding(2) var equirectSampler: sampler; + +fn dir_to_equirect_uv(d: vec3f) -> vec2f { + let theta = atan2(d.z, d.x); + let phi = asin(clamp(d.y, -1.0, 1.0)); + return vec2f((theta + PI) / (2.0 * PI), 0.5 - phi / PI); +} + +@fragment +fn fs_main(@builtin(position) frag: vec4f) -> @location(0) vec4f { + let uv = frag.xy / f32(params.size); + let dir = cube_uv_to_dir(params.face, uv); + let env_uv = dir_to_equirect_uv(dir); + return textureSampleLevel(equirect, equirectSampler, env_uv, 0.0); +} +`; + +function uploadHdrAsTexture(device: GPUDevice, hdr: ParsedHdr): GPUTexture { + const halfData = floatArrayToHalf(hdr.rgba); + const texture = device.createTexture({ + size: [hdr.width, hdr.height, 1], + format: "rgba16float", + usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST, + }); + device.queue.writeTexture( + { texture }, + halfData.buffer, + { bytesPerRow: hdr.width * 8 }, + [hdr.width, hdr.height, 1], + ); + return texture; +} + +function makeProceduralPipeline(device: GPUDevice, format: GPUTextureFormat) { + const module = device.createShaderModule({ code: FULLSCREEN_VS + iblMath + ENV_PROCEDURAL_FS }); + const bindGroupLayout = device.createBindGroupLayout({ + entries: [{ binding: 0, visibility: GPUShaderStage.FRAGMENT, buffer: { type: "uniform" } }], + }); + const pipeline = device.createRenderPipeline({ + layout: device.createPipelineLayout({ bindGroupLayouts: [bindGroupLayout] }), + vertex: { module, entryPoint: "vs_fullscreen" }, + fragment: { module, entryPoint: "fs_main", targets: [{ format }] }, + primitive: { topology: "triangle-list" }, + }); + return { pipeline, bindGroupLayout }; +} + +function makeEquirectPipeline(device: GPUDevice, format: GPUTextureFormat) { + const module = device.createShaderModule({ code: FULLSCREEN_VS + iblMath + ENV_EQUIRECT_FS }); + const bindGroupLayout = device.createBindGroupLayout({ + entries: [ + { binding: 0, visibility: GPUShaderStage.FRAGMENT, buffer: { type: "uniform" } }, + { binding: 1, visibility: GPUShaderStage.FRAGMENT, texture: { sampleType: "float", viewDimension: "2d" } }, + { binding: 2, visibility: GPUShaderStage.FRAGMENT, sampler: { type: "filtering" } }, + ], + }); + const pipeline = device.createRenderPipeline({ + layout: device.createPipelineLayout({ bindGroupLayouts: [bindGroupLayout] }), + vertex: { module, entryPoint: "vs_fullscreen" }, + fragment: { module, entryPoint: "fs_main", targets: [{ format }] }, + primitive: { topology: "triangle-list" }, + }); + return { pipeline, bindGroupLayout }; +} + +/** + * Renders an environment cubemap. With no `hdr` source it falls back to the + * built-in procedural studio. Pass a parsed Radiance HDR for a real photoreal + * environment — used by the IBL plugin to enable Babylon-style glossy + * reflections. + */ +export function generateEnvironment( + device: GPUDevice, + encoder: GPUCommandEncoder, + size: number, + hdr?: ParsedHdr, +): GPUTexture { + const format: GPUTextureFormat = "rgba16float"; + const texture = createCubemap(device, size, format, 1); + + if (hdr) { + const equirectTexture = uploadHdrAsTexture(device, hdr); + const sampler = device.createSampler({ + magFilter: "linear", + minFilter: "linear", + addressModeU: "repeat", + addressModeV: "clamp-to-edge", + }); + const { pipeline, bindGroupLayout } = makeEquirectPipeline(device, format); + const equirectView = equirectTexture.createView(); + + for (let face = 0; face < 6; face++) { + const ub = device.createBuffer({ + size: 16, + usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(ub, 0, new Uint32Array([face, size, 0, 0])); + const bindGroup = device.createBindGroup({ + layout: bindGroupLayout, + entries: [ + { binding: 0, resource: { buffer: ub } }, + { binding: 1, resource: equirectView }, + { binding: 2, resource: sampler }, + ], + }); + const pass = encoder.beginRenderPass({ + colorAttachments: [{ + view: cubeFaceView(texture, face, 0), + loadOp: "clear", + storeOp: "store", + clearValue: [0, 0, 0, 1], + }], + }); + pass.setPipeline(pipeline); + pass.setBindGroup(0, bindGroup); + pass.draw(3); + pass.end(); + } + return texture; + } + + const { pipeline, bindGroupLayout } = makeProceduralPipeline(device, format); + for (let face = 0; face < 6; face++) { + const ub = device.createBuffer({ + size: 16, + usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(ub, 0, new Uint32Array([face, size, 0, 0])); + const bindGroup = device.createBindGroup({ + layout: bindGroupLayout, + entries: [{ binding: 0, resource: { buffer: ub } }], + }); + const pass = encoder.beginRenderPass({ + colorAttachments: [{ + view: cubeFaceView(texture, face, 0), + loadOp: "clear", + storeOp: "store", + clearValue: [0, 0, 0, 1], + }], + }); + pass.setPipeline(pipeline); + pass.setBindGroup(0, bindGroup); + pass.draw(3); + pass.end(); + } + return texture; +} diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/float-to-half.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/float-to-half.ts new file mode 100644 index 00000000..63b23333 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/float-to-half.ts @@ -0,0 +1,35 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +// IEEE 754 binary16 (half-float) encoder. WebGPU's rgba16float upload path +// expects Uint16-packed half-floats; Float32 source data has to be converted +// element-wise. Standard bit-twiddle, no infinity/NaN edge cases needed for +// well-formed HDRI input (subnormals are flushed to zero). + +const f32 = new Float32Array(1); +const u32 = new Uint32Array(f32.buffer); + +export function floatToHalf(value: number): number { + f32[0] = value; + const bits = u32[0]; + + const sign = (bits >> 16) & 0x8000; + const expRaw = (bits >> 23) & 0xff; + const mantissa = bits & 0x7fffff; + + if (expRaw === 0xff) { + // Infinity or NaN. + return sign | 0x7c00 | (mantissa ? 0x200 : 0); + } + const exp = expRaw - 127; + if (exp >= 16) return sign | 0x7c00; // overflow → ±Inf + if (exp < -14) return sign; // underflow → ±0 + return sign | ((exp + 15) << 10) | (mantissa >> 13); +} + +export function floatArrayToHalf(src: Float32Array): Uint16Array { + const out = new Uint16Array(src.length); + for (let i = 0; i < src.length; i++) { + out[i] = floatToHalf(src[i]); + } + return out; +} diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/ibl-math.wgsl.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/ibl-math.wgsl.ts new file mode 100644 index 00000000..5ed352cc --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/ibl-math.wgsl.ts @@ -0,0 +1,60 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +// Shared WGSL math for IBL precomputation passes. +// - cube_uv_to_dir: WebGPU cube layer (face 0..5) + 2D uv → world-space direction +// - hammersley / radical_inverse_vdc: low-discrepancy 2D sample sequence +// - importance_sample_ggx: GGX-distributed half-vector in tangent space, returned in world space + +export default /* wgsl */ ` +const PI: f32 = 3.14159265359; + +fn cube_uv_to_dir(face: u32, uv: vec2f) -> vec3f { + let p = uv * 2.0 - vec2f(1.0); + var dir: vec3f; + switch face { + case 0u: { dir = vec3f( 1.0, -p.y, -p.x); } + case 1u: { dir = vec3f(-1.0, -p.y, p.x); } + case 2u: { dir = vec3f( p.x, 1.0, p.y); } + case 3u: { dir = vec3f( p.x, -1.0, -p.y); } + case 4u: { dir = vec3f( p.x, -p.y, 1.0); } + default: { dir = vec3f(-p.x, -p.y, -1.0); } + } + return normalize(dir); +} + +fn radical_inverse_vdc(bitsIn: u32) -> f32 { + var bits = bitsIn; + bits = (bits << 16u) | (bits >> 16u); + bits = ((bits & 0x55555555u) << 1u) | ((bits & 0xAAAAAAAAu) >> 1u); + bits = ((bits & 0x33333333u) << 2u) | ((bits & 0xCCCCCCCCu) >> 2u); + bits = ((bits & 0x0F0F0F0Fu) << 4u) | ((bits & 0xF0F0F0F0u) >> 4u); + bits = ((bits & 0x00FF00FFu) << 8u) | ((bits & 0xFF00FF00u) >> 8u); + return f32(bits) * 2.3283064365386963e-10; +} + +fn hammersley(i: u32, n: u32) -> vec2f { + return vec2f(f32(i) / f32(n), radical_inverse_vdc(i)); +} + +fn importance_sample_ggx(xi: vec2f, N: vec3f, roughness: f32) -> vec3f { + let a = roughness * roughness; + let phi = 2.0 * PI * xi.x; + let cosTheta = sqrt((1.0 - xi.y) / (1.0 + (a * a - 1.0) * xi.y)); + let sinTheta = sqrt(1.0 - cosTheta * cosTheta); + let H_tangent = vec3f(cos(phi) * sinTheta, sin(phi) * sinTheta, cosTheta); + + let up = select(vec3f(0.0, 0.0, 1.0), vec3f(1.0, 0.0, 0.0), abs(N.z) < 0.999); + let tangent = normalize(cross(up, N)); + let bitangent = cross(N, tangent); + + return normalize(tangent * H_tangent.x + bitangent * H_tangent.y + N * H_tangent.z); +} + +fn g_smith_ibl(nDotV: f32, nDotL: f32, roughness: f32) -> f32 { + let a = roughness; + let k = (a * a) / 2.0; + let gv = nDotV / (nDotV * (1.0 - k) + k); + let gl = nDotL / (nDotL * (1.0 - k) + k); + return gv * gl; +} +`; diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/irradiance.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/irradiance.ts new file mode 100644 index 00000000..7ae4b25e --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/irradiance.ts @@ -0,0 +1,105 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import iblMath from "./ibl-math.wgsl.js"; +import { createCubemap, cubeFaceView, cubemapSampleView, FULLSCREEN_VS } from "./render-helpers.js"; + +// Cosine-weighted hemispherical Monte-Carlo convolution of the source +// environment. With cos-weighted PDF, irradiance/π = mean(L(ωi)) — no extra +// factor needed. 1024 samples is plenty for a 32x32 output. +const IRRADIANCE_FS = /* wgsl */ ` +struct Params { face: u32, size: u32 }; +@group(0) @binding(0) var params: Params; +@group(0) @binding(1) var envMap: texture_cube; +@group(0) @binding(2) var envSampler: sampler; + +const SAMPLES: u32 = 1024u; + +fn convolve(N: vec3f) -> vec3f { + let up = select(vec3f(0.0, 0.0, 1.0), vec3f(1.0, 0.0, 0.0), abs(N.z) < 0.999); + let tangent = normalize(cross(up, N)); + let bitangent = cross(N, tangent); + + var acc = vec3f(0.0); + for (var i = 0u; i < SAMPLES; i = i + 1u) { + let xi = hammersley(i, SAMPLES); + let phi = 2.0 * PI * xi.x; + let cosTheta = sqrt(xi.y); + let sinTheta = sqrt(1.0 - xi.y); + let local = vec3f(cos(phi) * sinTheta, sin(phi) * sinTheta, cosTheta); + let dir = normalize(tangent * local.x + bitangent * local.y + N * local.z); + acc = acc + textureSampleLevel(envMap, envSampler, dir, 0.0).rgb; + } + return acc / f32(SAMPLES); +} + +@fragment +fn fs_main(@builtin(position) frag: vec4f) -> @location(0) vec4f { + let uv = frag.xy / f32(params.size); + let N = cube_uv_to_dir(params.face, uv); + return vec4f(convolve(N), 1.0); +} +`; + +export function generateIrradiance( + device: GPUDevice, + encoder: GPUCommandEncoder, + envMap: GPUTexture, + size: number, +): GPUTexture { + const format: GPUTextureFormat = "rgba16float"; + const texture = createCubemap(device, size, format, 1); + + const module = device.createShaderModule({ code: FULLSCREEN_VS + iblMath + IRRADIANCE_FS }); + const bindGroupLayout = device.createBindGroupLayout({ + entries: [ + { binding: 0, visibility: GPUShaderStage.FRAGMENT, buffer: { type: "uniform" } }, + { binding: 1, visibility: GPUShaderStage.FRAGMENT, texture: { sampleType: "float", viewDimension: "cube" } }, + { binding: 2, visibility: GPUShaderStage.FRAGMENT, sampler: { type: "filtering" } }, + ], + }); + const pipeline = device.createRenderPipeline({ + layout: device.createPipelineLayout({ bindGroupLayouts: [bindGroupLayout] }), + vertex: { module, entryPoint: "vs_fullscreen" }, + fragment: { module, entryPoint: "fs_main", targets: [{ format }] }, + primitive: { topology: "triangle-list" }, + }); + + const sampler = device.createSampler({ + magFilter: "linear", + minFilter: "linear", + addressModeU: "clamp-to-edge", + addressModeV: "clamp-to-edge", + addressModeW: "clamp-to-edge", + }); + const envView = cubemapSampleView(envMap); + + for (let face = 0; face < 6; face++) { + const ub = device.createBuffer({ + size: 16, + usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(ub, 0, new Uint32Array([face, size, 0, 0])); + const bindGroup = device.createBindGroup({ + layout: bindGroupLayout, + entries: [ + { binding: 0, resource: { buffer: ub } }, + { binding: 1, resource: envView }, + { binding: 2, resource: sampler }, + ], + }); + const pass = encoder.beginRenderPass({ + colorAttachments: [{ + view: cubeFaceView(texture, face, 0), + loadOp: "clear", + storeOp: "store", + clearValue: [0, 0, 0, 1], + }], + }); + pass.setPipeline(pipeline); + pass.setBindGroup(0, bindGroup); + pass.draw(3); + pass.end(); + } + + return texture; +} diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/parse-hdr.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/parse-hdr.ts new file mode 100644 index 00000000..12d40359 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/parse-hdr.ts @@ -0,0 +1,108 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +// Minimal Radiance .hdr / RGBE parser. Decodes the run-length encoded form +// (standard for files written by HDRshop / Photosphere / Polyhaven) and +// returns Float32 RGBA in row-major order. A=1 for every pixel. + +export interface ParsedHdr { + width: number; + height: number; + rgba: Float32Array; +} + +export function parseHdr(buffer: ArrayBuffer): ParsedHdr { + const bytes = new Uint8Array(buffer); + + // Header: ASCII, terminated by a blank line; then the resolution line. + const text = new TextDecoder("latin1").decode(bytes); + const headerEnd = text.indexOf("\n\n"); + if (headerEnd < 0) throw new Error("HDR: missing header terminator"); + const header = text.slice(0, headerEnd); + if (!header.startsWith("#?RADIANCE") && !header.startsWith("#?RGBE")) { + throw new Error("HDR: unknown magic"); + } + if (!/FORMAT=32-bit_rle_rgbe/i.test(header)) { + throw new Error("HDR: only 32-bit_rle_rgbe is supported"); + } + + // Resolution: e.g. "-Y 512 +X 1024". Negative Y means top-to-bottom. + const resStart = headerEnd + 2; + const newline = bytes.indexOf(0x0a, resStart); + const resLine = new TextDecoder().decode(bytes.subarray(resStart, newline)); + const m = /([+-][YX])\s+(\d+)\s+([+-][YX])\s+(\d+)/.exec(resLine); + if (!m) throw new Error(`HDR: bad resolution line "${resLine}"`); + const axisA = m[1], dimA = +m[2], axisB = m[3], dimB = +m[4]; + // We only handle the common -Y +X / +Y +X orientations. + let width: number, height: number, flipY: boolean; + if (axisA[1] === "Y") { + height = dimA; width = dimB; + flipY = axisA[0] === "+"; + } else { + width = dimA; height = dimB; + flipY = axisB[0] === "+"; + } + if (axisA[1] === axisB[1]) throw new Error("HDR: redundant axes"); + + const data = bytes.subarray(newline + 1); + let p = 0; + const rgba = new Float32Array(width * height * 4); + + const scanline = new Uint8Array(width * 4); + for (let y = 0; y < height; y++) { + if (p + 4 > data.length) throw new Error("HDR: truncated scanline header"); + // RLE-encoded scanline: starts with 0x02 0x02 (hi lo) of width. + if ( + data[p] === 0x02 && data[p + 1] === 0x02 && + ((data[p + 2] << 8) | data[p + 3]) === width && + width >= 8 && width <= 0x7fff + ) { + p += 4; + for (let c = 0; c < 4; c++) { + let x = 0; + while (x < width) { + if (p >= data.length) throw new Error("HDR: truncated RLE channel"); + const len = data[p++]; + if (len > 128) { + const runLen = len - 128; + if (p >= data.length || x + runLen > width) throw new Error("HDR: bad run"); + const val = data[p++]; + for (let i = 0; i < runLen; i++) scanline[(x + i) * 4 + c] = val; + x += runLen; + } else { + if (p + len > data.length || x + len > width) throw new Error("HDR: bad literal"); + for (let i = 0; i < len; i++) scanline[(x + i) * 4 + c] = data[p + i]; + p += len; + x += len; + } + } + } + } else { + // Old-style: raw RGBE pixels in raster order. + if (p + width * 4 > data.length) throw new Error("HDR: truncated raw scanline"); + scanline.set(data.subarray(p, p + width * 4)); + p += width * 4; + } + + const dstY = flipY ? height - 1 - y : y; + const rowOffset = dstY * width * 4; + for (let x = 0; x < width; x++) { + const r = scanline[x * 4]; + const g = scanline[x * 4 + 1]; + const b = scanline[x * 4 + 2]; + const e = scanline[x * 4 + 3]; + if (e === 0) { + rgba[rowOffset + x * 4 + 0] = 0; + rgba[rowOffset + x * 4 + 1] = 0; + rgba[rowOffset + x * 4 + 2] = 0; + } else { + const scale = Math.pow(2, e - 128) / 256; + rgba[rowOffset + x * 4 + 0] = r * scale; + rgba[rowOffset + x * 4 + 1] = g * scale; + rgba[rowOffset + x * 4 + 2] = b * scale; + } + rgba[rowOffset + x * 4 + 3] = 1; + } + } + + return { width, height, rgba }; +} diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/prefilter.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/prefilter.ts new file mode 100644 index 00000000..c208271f --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/prefilter.ts @@ -0,0 +1,121 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import iblMath from "./ibl-math.wgsl.js"; +import { createCubemap, cubeFaceView, cubemapSampleView, FULLSCREEN_VS } from "./render-helpers.js"; + +// Split-sum prefiltered specular: each mip stores GGX importance-sampled +// reflections at a fixed roughness. Mip 0 = mirror (roughness 0), mip max = +// fully blurred (roughness 1). Uses Karis' simplification of N == V == R. +const PREFILTER_FS = /* wgsl */ ` +struct Params { face: u32, size: u32, roughness: f32, _pad: f32 }; +@group(0) @binding(0) var params: Params; +@group(0) @binding(1) var envMap: texture_cube; +@group(0) @binding(2) var envSampler: sampler; + +const SAMPLES: u32 = 1024u; + +fn prefilter(N: vec3f) -> vec3f { + if (params.roughness < 0.001) { + return textureSampleLevel(envMap, envSampler, N, 0.0).rgb; + } + let V = N; + var color = vec3f(0.0); + var weight = 0.0; + for (var i = 0u; i < SAMPLES; i = i + 1u) { + let xi = hammersley(i, SAMPLES); + let H = importance_sample_ggx(xi, N, params.roughness); + let L = normalize(2.0 * dot(V, H) * H - V); + let nDotL = max(dot(N, L), 0.0); + if (nDotL > 0.0) { + color = color + textureSampleLevel(envMap, envSampler, L, 0.0).rgb * nDotL; + weight = weight + nDotL; + } + } + return color / max(weight, 0.0001); +} + +@fragment +fn fs_main(@builtin(position) frag: vec4f) -> @location(0) vec4f { + let uv = frag.xy / f32(params.size); + let N = cube_uv_to_dir(params.face, uv); + return vec4f(prefilter(N), 1.0); +} +`; + +export interface PrefilteredResult { + texture: GPUTexture; + mipLevelCount: number; +} + +export function generatePrefiltered( + device: GPUDevice, + encoder: GPUCommandEncoder, + envMap: GPUTexture, + baseSize: number, + mipLevelCount: number, +): PrefilteredResult { + const format: GPUTextureFormat = "rgba16float"; + const texture = createCubemap(device, baseSize, format, mipLevelCount); + + const module = device.createShaderModule({ code: FULLSCREEN_VS + iblMath + PREFILTER_FS }); + const bindGroupLayout = device.createBindGroupLayout({ + entries: [ + { binding: 0, visibility: GPUShaderStage.FRAGMENT, buffer: { type: "uniform" } }, + { binding: 1, visibility: GPUShaderStage.FRAGMENT, texture: { sampleType: "float", viewDimension: "cube" } }, + { binding: 2, visibility: GPUShaderStage.FRAGMENT, sampler: { type: "filtering" } }, + ], + }); + const pipeline = device.createRenderPipeline({ + layout: device.createPipelineLayout({ bindGroupLayouts: [bindGroupLayout] }), + vertex: { module, entryPoint: "vs_fullscreen" }, + fragment: { module, entryPoint: "fs_main", targets: [{ format }] }, + primitive: { topology: "triangle-list" }, + }); + + const sampler = device.createSampler({ + magFilter: "linear", + minFilter: "linear", + addressModeU: "clamp-to-edge", + addressModeV: "clamp-to-edge", + addressModeW: "clamp-to-edge", + }); + const envView = cubemapSampleView(envMap); + + for (let mip = 0; mip < mipLevelCount; mip++) { + const mipSize = Math.max(1, baseSize >> mip); + const roughness = mipLevelCount > 1 ? mip / (mipLevelCount - 1) : 0; + for (let face = 0; face < 6; face++) { + const ub = device.createBuffer({ + size: 16, + usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST, + }); + const buf = new ArrayBuffer(16); + new Uint32Array(buf, 0, 2).set([face, mipSize]); + new Float32Array(buf, 8, 2).set([roughness, 0]); + device.queue.writeBuffer(ub, 0, buf); + + const bindGroup = device.createBindGroup({ + layout: bindGroupLayout, + entries: [ + { binding: 0, resource: { buffer: ub } }, + { binding: 1, resource: envView }, + { binding: 2, resource: sampler }, + ], + }); + const pass = encoder.beginRenderPass({ + colorAttachments: [{ + view: cubeFaceView(texture, face, mip), + loadOp: "clear", + storeOp: "store", + clearValue: [0, 0, 0, 1], + }], + }); + pass.setPipeline(pipeline); + pass.setBindGroup(0, bindGroup); + pass.draw(3); + pass.end(); + } + } + + return { texture, mipLevelCount }; +} diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/render-helpers.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/render-helpers.ts new file mode 100644 index 00000000..5f65c708 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/ibl/render-helpers.ts @@ -0,0 +1,17 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export { createCubemap } from "./create-cubemap.js"; +export { cubeFaceView } from "./cube-face-view.js"; +export { cubemapSampleView } from "./cubemap-sample-view.js"; + +// Fullscreen-triangle vertex shader: three vertices generate a clip-space +// triangle that covers the viewport. The fragment shader then uses +// @builtin(position) (framebuffer pixels) to derive per-fragment UVs. +export const FULLSCREEN_VS = /* wgsl */ ` +@vertex +fn vs_fullscreen(@builtin(vertex_index) i: u32) -> @builtin(position) vec4f { + let xs = array(-1.0, 3.0, -1.0); + let ys = array(-1.0, -1.0, 3.0); + return vec4f(xs[i], ys[i], 0.0, 1.0); +} +`; diff --git a/packages/data-gpu/src/graphics/rendering/ibl-render/skybox-shader.wgsl.ts b/packages/data-gpu/src/graphics/rendering/ibl-render/skybox-shader.wgsl.ts new file mode 100644 index 00000000..67c57742 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ibl-render/skybox-shader.wgsl.ts @@ -0,0 +1,51 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import brdf from "./brdf.wgsl.js"; + +// Skybox shader. Computes the view-ray direction in the vertex shader from the +// camera's orthonormal basis + half-FOV — bypasses the perspective matrix +// inverse, which would otherwise depend on the project's specific +// clip-space convention. Depth state is "always" with no write so primitives +// drawn afterward in the same render pass overlap correctly. +export default /* wgsl */ ` +struct SkyboxUniforms { + right: vec3f, + aspect: f32, + up: vec3f, + tanHalfFov: f32, + forward: vec3f, + _pad: f32, +} +@group(0) @binding(0) var sky: SkyboxUniforms; +@group(0) @binding(1) var skyCubemap: texture_cube; +@group(0) @binding(2) var skySampler: sampler; + +struct SkyboxOutput { + @builtin(position) clip: vec4f, + @location(0) dir: vec3f, +} + +@vertex +fn vs_skybox(@builtin(vertex_index) i: u32) -> SkyboxOutput { + let xs = array(-1.0, 3.0, -1.0); + let ys = array(-1.0, -1.0, 3.0); + let xn = xs[i]; + let yn = ys[i]; + var out: SkyboxOutput; + out.clip = vec4f(xn, yn, 0.0, 1.0); + out.dir = sky.forward + + sky.right * (xn * sky.aspect * sky.tanHalfFov) + + sky.up * (yn * sky.tanHalfFov); + return out; +} + +${brdf} + +@fragment +fn fs_skybox(in: SkyboxOutput) -> @location(0) vec4f { + let dir = normalize(in.dir); + let raw = textureSampleLevel(skyCubemap, skySampler, dir, 0.0).rgb; + let mapped = tone_map_aces(raw); + return vec4f(pow(mapped, vec3f(1.0 / 2.2)), 1.0); +} +`; diff --git a/packages/data-gpu/src/graphics/rendering/interpolation-plugin.ts b/packages/data-gpu/src/graphics/rendering/interpolation-plugin.ts new file mode 100644 index 00000000..4ba6ea4c --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/interpolation-plugin.ts @@ -0,0 +1,97 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Mat4x4, Quat } from "@adobe/data/math"; +import { physicsData } from "../../physics/physics-data-plugin.js"; +import { physicsClock } from "../../physics/physics-clock-plugin.js"; +import { transform } from "../scene/node/transform-plugin.js"; +import { displayTransform } from "./display-transform-plugin.js"; + +/** + * interpolation — the single pre-render pass that turns the fixed-step physics + * state into a smooth display pose. Because the solver steps on `physicsClock` + * (0..N steps/frame), the canonical `position`/`rotation` jump in fixed + * increments; rendering them raw judders when the render rate ≠ the sim rate. + * + * Each render frame this blends `_prevPosition`→`position` (and the rotations) + * by `physicsClock.alpha` into `_renderPosition`/`_renderRotation`, which the + * renderer draws. Doing it once here (not in each renderer) keeps render systems + * free of physics knowledge — they only read the display transform. + * + * Bodies gain `_renderPosition`/`_renderRotation` the first frame after the + * solver has mirrored them (i.e. once `_prevPosition` exists); the tag-exclude + * query means steady state never re-scans already-equipped bodies. + */ + +// Dynamic bodies that have been mirrored (so `_prevPosition` exists) but don't +// yet carry the display-pose columns — give them those columns once. +const NEEDS_RENDER_POSE = ["position", "rotation", "_prevPosition"] as const; +const WITHOUT_RENDER_POSE = { exclude: ["_renderPosition"] } as const; +// Fully-equipped bodies: blend prev→current into the display pose every frame. +const INTERPOLATED = ["position", "rotation", "_prevPosition", "_prevRotation", "_renderPosition", "_renderRotation"] as const; +// Interpolated bodies that render through the world-matrix path (models, which have +// a `parent`/`_worldMatrix`) rather than the flat instanced path. +const INTERPOLATED_MODEL = ["_renderPosition", "_renderRotation", "scale", "parent", "_worldMatrix"] as const; + +export const interpolation = Database.Plugin.create({ + extends: Database.Plugin.combine(physicsData, physicsClock, displayTransform, transform), + systems: { + interpolateDisplayPose: { + schedule: { during: ["preRender"] }, + create: db => () => { + // 1) Equip newly-mirrored bodies with display-pose columns (seed = current + // pose). Tail→head: every visited row migrates out on gaining the columns. + for (const arch of db.store.queryArchetypes(NEEDS_RENDER_POSE, WITHOUT_RENDER_POSE)) { + const ids = arch.columns.id, pos = arch.columns.position, ori = arch.columns.rotation; + for (let r = arch.rowCount - 1; r >= 0; r--) { + db.store.update(ids.get(r), { _renderPosition: pos.get(r), _renderRotation: ori.get(r) }); + } + } + + // 2) Blend prev→current by alpha into the display pose — every frame, in + // place (no migration), straight on the backing typed arrays. + const alpha = db.store.resources.physicsClock.alpha; + for (const arch of db.store.queryArchetypes(INTERPOLATED)) { + const pos = arch.columns.position.getTypedArray(), ori = arch.columns.rotation.getTypedArray(); + const pp = arch.columns._prevPosition.getTypedArray(), pr = arch.columns._prevRotation.getTypedArray(); + const rp = arch.columns._renderPosition.getTypedArray(), rr = arch.columns._renderRotation.getTypedArray(); + for (let r = 0; r < arch.rowCount; r++) { + const r3 = r * 3, r4 = r * 4; + rp[r3] = pp[r3] + (pos[r3] - pp[r3]) * alpha; + rp[r3 + 1] = pp[r3 + 1] + (pos[r3 + 1] - pp[r3 + 1]) * alpha; + rp[r3 + 2] = pp[r3 + 2] + (pos[r3 + 2] - pp[r3 + 2]) * alpha; + // nlerp with shortest-path sign flip — cheap, allocation-free, and + // visually exact for the small per-step rotations of interpolation. + let ax = pr[r4], ay = pr[r4 + 1], az = pr[r4 + 2], aw = pr[r4 + 3]; + const bx = ori[r4], by = ori[r4 + 1], bz = ori[r4 + 2], bw = ori[r4 + 3]; + if (ax * bx + ay * by + az * bz + aw * bw < 0) { ax = -ax; ay = -ay; az = -az; aw = -aw; } + const x = ax + (bx - ax) * alpha, y = ay + (by - ay) * alpha, z = az + (bz - az) * alpha, w = aw + (bw - aw) * alpha; + const inv = 1 / Math.hypot(x, y, z, w); + rr[r4] = x * inv; rr[r4 + 1] = y * inv; rr[r4 + 2] = z * inv; rr[r4 + 3] = w * inv; + } + } + }, + }, + // Models render from `_worldMatrix` (the transform path), not the flat display + // pose — so recompose their world matrix from the interpolated pose, after the + // blend above and after `transformSystem` (which wrote the raw, juddering pose). + interpolateWorldMatrix: { + schedule: { during: ["preRender"], after: ["interpolateDisplayPose", "transformSystem"] }, + create: db => () => { + for (const arch of db.store.queryArchetypes(INTERPOLATED_MODEL)) { + const rp = arch.columns._renderPosition.getTypedArray(), rr = arch.columns._renderRotation.getTypedArray(); + const scl = arch.columns.scale, parents = arch.columns.parent, world = arch.columns._worldMatrix; + for (let i = 0; i < arch.rowCount; i++) { + const r3 = i * 3, r4 = i * 4, s = scl.get(i), parentId = parents.get(i); + const local = Mat4x4.multiply( + Mat4x4.translation(rp[r3], rp[r3 + 1], rp[r3 + 2]), + Mat4x4.multiply(Quat.toMat4([rr[r4], rr[r4 + 1], rr[r4 + 2], rr[r4 + 3]]), Mat4x4.scaling(s[0], s[1], s[2])), + ); + const parentWorld = parentId === 0 ? Mat4x4.identity : db.store.get(parentId, "_worldMatrix") ?? Mat4x4.identity; + world.set(i, parentId === 0 ? local : Mat4x4.multiply(parentWorld, local)); + } + } + }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/interpolation.test.ts b/packages/data-gpu/src/graphics/rendering/interpolation.test.ts new file mode 100644 index 00000000..6ff8830b --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/interpolation.test.ts @@ -0,0 +1,52 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { describe, it, expect } from "vitest"; +import { Database } from "@adobe/data/ecs"; +import { interpolation } from "./interpolation-plugin.js"; +import type { PhysicsClock } from "../../physics/physics-clock-plugin.js"; + +/** + * The interpolation pass turns the fixed-step sim state into a smooth render pose. + * No solver here — we hand-seed the prev snapshot a solver would write, then assert + * the display pose is the alpha-blend of prev → current. Pure + deterministic + * (no WASM, no GPU), so it runs headless. + */ +describe("interpolation", () => { + // A created database carries a writable `store` (runtime invariant; not on the + // public type — the same loose access the solver benchmark uses). + interface InterpStore { + resources: { physicsClock: PhysicsClock }; + archetypes: { RigidBody: { insert(v: object): number } }; + read(id: number): { _renderPosition: readonly number[] }; + update(id: number, v: object): void; + } + + const make = () => { + const db = Database.create(interpolation); + const store = (db as unknown as { store: InterpStore }).store; + // a body the solver has already mirrored (so it carries a prev snapshot) + const id = store.archetypes.RigidBody.insert({ + bodyType: "dynamic", colliderShape: "box", halfExtents: [0.5, 0.5, 0.5], material: 0, + position: [0, 10, 0], rotation: [0, 0, 0, 1], linearVelocity: [0, 0, 0], angularVelocity: [0, 0, 0], + }); + store.update(id, { _prevPosition: [0, 10, 0], _prevRotation: [0, 0, 0, 1] }); + store.update(id, { position: [0, 6, 0] }); // a step landed: prev y=10, current y=6 + + const interpolateAt = (alpha: number): number => { + store.resources.physicsClock = { ...store.resources.physicsClock, alpha, steps: 0 }; + db.system.functions.interpolateDisplayPose?.(); + return store.read(id)._renderPosition[1]; + }; + return { interpolateAt }; + }; + + it("blends prev → current by the clock's alpha into the display pose", () => { + expect(make().interpolateAt(0.25)).toBeCloseTo(9); // 10 + (6 - 10) * 0.25 + }); + + it("at alpha→0 the display pose equals prev; at alpha→1 it equals current", () => { + const { interpolateAt } = make(); + expect(interpolateAt(0)).toBeCloseTo(10); // prev + expect(interpolateAt(1)).toBeCloseTo(6); // current + }); +}); diff --git a/packages/data-gpu/src/graphics/rendering/jolt-ragdoll-plugin.ts b/packages/data-gpu/src/graphics/rendering/jolt-ragdoll-plugin.ts new file mode 100644 index 00000000..4f583ce1 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/jolt-ragdoll-plugin.ts @@ -0,0 +1,174 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, type Entity } from "@adobe/data/ecs"; +import { Mat4x4, Quat } from "@adobe/data/math"; +import { joltSolver, type JoltContext } from "../../physics/solvers/jolt-solver-plugin.js"; +import { fitBoneCapsules } from "../../physics/ragdoll/fit-bone-capsules.js"; +import { ragdollTrigger } from "./ragdoll-trigger-plugin.js"; +import { pbrSkinning } from "./skinning/skinning-plugin.js"; +import { modelLoader } from "../scene/model/model-loader-plugin.js"; +import { transform } from "../scene/node/transform-plugin.js"; +import type { JointTemplate } from "../scene/model/gltf/parse-skin.js"; + +/** + * joltRagdoll — a **Jolt-native** ragdoll backend (the alternative to our generic + * `boneColliders`), using Jolt's `Skeleton` / `RagdollSettings` / `Ragdoll`. + * + * Once the skin loads, it builds a `Ragdoll` in the *solver's* physics system: one + * dynamic body per bone (capsule fitted by `fitBoneCapsules`, tiny sphere for + * capsule-less joints), a swing-twist constraint to each parent, and + * `DisableParentChildCollisions`. While alive it `DriveToPoseUsingKinematics` + * toward the animated pose each frame; on `triggerRagdoll` it stops driving (the + * bodies fall) and reads the pose back onto the skeleton so the skin flops. + * + * Jolt-only by construction — it needs the solver's `PhysicsSystem` (exposed as + * `_joltContext`). Pair it with `joltSolver`; use `boneColliders` for other solvers. + */ + +type Jolt = JoltContext["jolt"]; +interface SkinGeo { _cpuSkin?: { positions: Float32Array; joints: Uint32Array; weights: Float32Array } | null; _skinInverseBindMatrices?: Float32Array | null; _skinJointTemplate?: JointTemplate[] } + +/** Column-major rigid transform from a position + unit quaternion. */ +const compose = (px: number, py: number, pz: number, q: ArrayLike): Mat4x4 => + Mat4x4.multiply(Mat4x4.translation(px, py, pz), Quat.toMat4([q[0], q[1], q[2], q[3]])); + +/** Orthonormalised rotation of a column-major matrix → quaternion. */ +function rotationOf(m: Mat4x4): Quat { + let m00 = m[0], m10 = m[1], m20 = m[2], m01 = m[4], m11 = m[5], m21 = m[6], m02 = m[8], m12 = m[9], m22 = m[10]; + const sx = Math.hypot(m00, m10, m20) || 1, sy = Math.hypot(m01, m11, m21) || 1, sz = Math.hypot(m02, m12, m22) || 1; + m00 /= sx; m10 /= sx; m20 /= sx; m01 /= sy; m11 /= sy; m21 /= sy; m02 /= sz; m12 /= sz; m22 /= sz; + const tr = m00 + m11 + m22; + if (tr > 0) { const s = Math.sqrt(tr + 1) * 2; return [(m21 - m12) / s, (m02 - m20) / s, (m10 - m01) / s, 0.25 * s]; } + if (m00 > m11 && m00 > m22) { const s = Math.sqrt(1 + m00 - m11 - m22) * 2; return [0.25 * s, (m01 + m10) / s, (m02 + m20) / s, (m21 - m12) / s]; } + if (m11 > m22) { const s = Math.sqrt(1 + m11 - m00 - m22) * 2; return [(m01 + m10) / s, 0.25 * s, (m12 + m21) / s, (m02 - m20) / s]; } + const s = Math.sqrt(1 + m22 - m00 - m11) * 2; return [(m02 + m20) / s, (m12 + m21) / s, 0.25 * s, (m10 - m01) / s]; +} + +const SWING = 0.9, TWIST = 0.5; // ~52° swing cone, ~±29° twist + +export const joltRagdoll = Database.Plugin.create({ + extends: Database.Plugin.combine(joltSolver, ragdollTrigger, pbrSkinning, modelLoader, transform), + systems: { + joltRagdollSystem: { + schedule: { during: ["postUpdate"], after: ["transformSystem"] }, + create: db => { + let built = false, dead = false; + // Jolt objects (created at build): the ragdoll, a reusable pose, scratch. + let ragdoll: InstanceType | null = null; + let pose: InstanceType | null = null; + let joints: readonly Entity[] = []; + let modelEntity: Entity = 0 as Entity; // the skeleton's root parent (the Model); moved to follow the ragdoll root when limp + let tv: InstanceType | null = null, tq: InstanceType | null = null, troot: InstanceType | null = null; + + const jointWorld = (j: Entity): Mat4x4 => (db.store.get(j, "_worldMatrix") as Mat4x4 | undefined) ?? Mat4x4.identity; + + const build = (ctx: JoltContext, jointIds: readonly Entity[], g: SkinGeo): void => { + const jolt = ctx.jolt, template = g._skinJointTemplate!; + const caps = new Map(); + for (const c of fitBoneCapsules({ jointCount: jointIds.length, inverseBindMatrices: g._skinInverseBindMatrices!, skin: g._cpuSkin! })) + caps.set(c.jointIndex, { radius: c.radius, halfHeight: c.halfHeight, offPos: c.offsetPosition, offRot: c.offsetRotation }); + + const skel = new jolt.Skeleton(); + for (let i = 0; i < jointIds.length; i++) { const s = new jolt.JPHString(template[i]?.name ?? `j${i}`, (template[i]?.name ?? `j${i}`).length); skel.AddJoint(s, template[i]?.parentJointIndex ?? -1); jolt.destroy(s); } + + const settings = new jolt.RagdollSettings(); + settings.mSkeleton = skel; + // RagdollPart has no constructor in the IDL — resize the parts vector and + // configure each element in place, then write the vector back. + const parts = settings.mParts; + parts.resize(jointIds.length); + for (let i = 0; i < jointIds.length; i++) { + const jw = jointWorld(jointIds[i]), cap = caps.get(i); + const part = parts.at(i); + // body bind pose: capsule sits at jointWorld · offset; sphere at the joint + const bodyW = cap ? Mat4x4.multiply(jw, compose(cap.offPos[0], cap.offPos[1], cap.offPos[2], cap.offRot)) : jw; + const shp = cap ? new jolt.CapsuleShape(Math.max(cap.halfHeight, 1e-3), cap.radius) : new jolt.SphereShape(0.03); + part.SetShape(shp); + const bp = new jolt.RVec3(bodyW[12], bodyW[13], bodyW[14]); const br = rotationOf(bodyW); const bq = new jolt.Quat(br[0], br[1], br[2], br[3]); + part.set_mPosition(bp); part.set_mRotation(bq); + part.set_mMotionType(jolt.EMotionType_Dynamic); part.set_mObjectLayer(ctx.ragdollLayer); + const pj = template[i]?.parentJointIndex ?? -1; + if (pj >= 0) { + // swing-twist to the parent, anchored at this joint, bone axis = parent→this + const pw = jointWorld(jointIds[pj]); + let ax = jw[12] - pw[12], ay = jw[13] - pw[13], az = jw[14] - pw[14]; + const al = Math.hypot(ax, ay, az) || 1; ax /= al; ay /= al; az /= al; + let nx = 0, ny = 0, nz = 0; if (Math.abs(ax) <= Math.abs(ay) && Math.abs(ax) <= Math.abs(az)) { ny = az; nz = -ay; } else { nx = -az; nz = ax; } + const nl = Math.hypot(nx, ny, nz) || 1; nx /= nl; ny /= nl; nz /= nl; + const st = new jolt.SwingTwistConstraintSettings(); + st.mSpace = jolt.EConstraintSpace_WorldSpace; + const p1 = new jolt.RVec3(jw[12], jw[13], jw[14]), p2 = new jolt.RVec3(jw[12], jw[13], jw[14]); + const t1 = new jolt.Vec3(ax, ay, az), t2 = new jolt.Vec3(ax, ay, az), n1 = new jolt.Vec3(nx, ny, nz), n2 = new jolt.Vec3(nx, ny, nz); + st.mPosition1 = p1; st.mPosition2 = p2; st.mTwistAxis1 = t1; st.mTwistAxis2 = t2; st.mPlaneAxis1 = n1; st.mPlaneAxis2 = n2; + st.mSwingType = jolt.ESwingType_Cone; st.mNormalHalfConeAngle = SWING; st.mPlaneHalfConeAngle = SWING; st.mTwistMinAngle = -TWIST; st.mTwistMaxAngle = TWIST; + part.set_mToParent(st); + jolt.destroy(p1); jolt.destroy(p2); jolt.destroy(t1); jolt.destroy(t2); jolt.destroy(n1); jolt.destroy(n2); + } + jolt.destroy(bp); jolt.destroy(bq); + } + settings.mParts = parts; + settings.DisableParentChildCollisions(); + settings.Stabilize(); + ragdoll = settings.CreateRagdoll(0, 0, ctx.physicsSystem) as typeof ragdoll; + ragdoll!.AddToPhysicsSystem(jolt.EActivation_Activate); + pose = new jolt.SkeletonPose(); pose.SetSkeleton(skel); + tv = new jolt.Vec3(0, 0, 0); tq = new jolt.Quat(0, 0, 0, 1); troot = new jolt.RVec3(0, 0, 0); + joints = jointIds; + let rootIdx = template.findIndex(t => (t.parentJointIndex ?? -1) < 0); if (rootIdx < 0) rootIdx = 0; + modelEntity = (db.store.get(jointIds[rootIdx], "parent") as Entity | undefined) ?? (0 as Entity); + jolt.destroy(settings); + }; + + return () => { + const ctx = db.store.resources._joltContext; + if (!ctx) return; + if (!built) { + for (const arch of db.store.queryArchetypes(["_skeletonJoints", "_skeletonGeometry"])) { + const jc = arch.columns._skeletonJoints, gc = arch.columns._skeletonGeometry; + for (let i = 0; i < arch.rowCount; i++) { + const g = db.store.read(gc.get(i)) as SkinGeo | null; + if (!g?._cpuSkin || !g._skinInverseBindMatrices || !g._skinJointTemplate?.length) continue; + build(ctx, jc.get(i), g); built = true; break; + } + if (built) break; + } + if (!built) return; + } + const jolt = ctx.jolt, rd = ragdoll!, p = pose!; + if (!dead && db.store.resources._ragdollTrigger) { + dead = true; + for (const a of db.store.queryArchetypes(["animationPlaying"])) for (let i = 0; i < a.rowCount; i++) db.store.update(a.columns.id.get(i), { animationPlaying: false }); + } + if (!dead) { + // alive: drive the ragdoll toward the animated pose. Root offset = + // the model's world translation; joint states = the animated locals. + const rm = jointWorld(modelEntity); + troot!.Set(rm[12], rm[13], rm[14]); p.SetRootOffset(troot!); + for (let i = 0; i < joints.length; i++) { + const pos = db.store.get(joints[i], "position") as readonly number[] | undefined; + const rot = db.store.get(joints[i], "rotation") as readonly number[] | undefined; + if (!pos || !rot) continue; + const st = p.GetJoint(i); + tv!.Set(pos[0], pos[1], pos[2]); tq!.Set(rot[0], rot[1], rot[2], rot[3]); + st.set_mTranslation(tv!); st.set_mRotation(tq!); + } + p.CalculateJointMatrices(); + rd.DriveToPoseUsingKinematics(p, db.store.resources.physicsClock.fixedDt * Math.max(1, db.store.resources.physicsClock.steps)); + } else { + // limp: read the physics pose back. The ragdoll's root has fallen, so move + // the model to the pose's root offset and apply the local joint states — + // the whole skin follows the fall, the limbs flop. + rd.GetPose(p, false); + p.CalculateJointStates(); + const ro = p.GetRootOffset(); + db.store.update(modelEntity, { position: [ro.GetX(), ro.GetY(), ro.GetZ()] }); + for (let i = 0; i < joints.length; i++) { + const st = p.GetJoint(i), t = st.get_mTranslation(), r = st.get_mRotation(); + db.store.update(joints[i], { position: [t.GetX(), t.GetY(), t.GetZ()], rotation: [r.GetX(), r.GetY(), r.GetZ(), r.GetW()] }); + } + } + }; + }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/material-gpu/material-arrays.ts b/packages/data-gpu/src/graphics/rendering/material-gpu/material-arrays.ts new file mode 100644 index 00000000..cd3b00d4 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/material-gpu/material-arrays.ts @@ -0,0 +1,113 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +/** + * The shared GPU material set: five `texture_2d_array`s (one per PBR map) plus + * a palette storage buffer of per-layer factors. Every material occupies one + * layer across all five arrays; a drawable selects its material with a single + * per-instance layer index. Built once and grown by layer — materials are + * added rarely and edited never, so nothing here is rebuilt per frame. + */ +export interface MaterialArrays { + baseColor: GPUTexture; // rgba8unorm-srgb + metallicRoughness: GPUTexture; // rgba8unorm (G = roughness, B = metalness) + normal: GPUTexture; // rgba8unorm + occlusion: GPUTexture; // rgba8unorm (R = AO) + emissive: GPUTexture; // rgba8unorm-srgb +} + +/** Fixed layer size every material map is resampled to. */ +export const MATERIAL_TEXTURE_SIZE = 512; +/** Maximum distinct materials (array layers). 16 × 512² × 5 maps ≈ 80 MB. */ +export const MAX_MATERIAL_LAYERS = 16; +/** Bytes per palette entry: baseColor(vec4) + emissive/metallic(vec4) + rough/normal/occlusion(vec4). */ +export const PALETTE_STRIDE = 48; + +export function createMaterialArrays(device: GPUDevice): MaterialArrays { + const make = (format: GPUTextureFormat): GPUTexture => + device.createTexture({ + size: [MATERIAL_TEXTURE_SIZE, MATERIAL_TEXTURE_SIZE, MAX_MATERIAL_LAYERS], + dimension: "2d", + format, + // RENDER_ATTACHMENT is required by copyExternalImageToTexture. + usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST | GPUTextureUsage.RENDER_ATTACHMENT, + }); + return { + baseColor: make("rgba8unorm-srgb"), + metallicRoughness: make("rgba8unorm"), + normal: make("rgba8unorm"), + occlusion: make("rgba8unorm"), + emissive: make("rgba8unorm-srgb"), + }; +} + +/** Per-material visible factors written into one palette layer. */ +export interface MaterialFactors { + baseColorFactor: readonly number[]; + emissiveFactor: readonly number[]; + metallicFactor: number; + roughnessFactor: number; + normalScale: number; + occlusionStrength: number; +} + +const _palette = new Float32Array(PALETTE_STRIDE / 4); + +export function writePaletteLayer(device: GPUDevice, palette: GPUBuffer, layer: number, f: MaterialFactors): void { + _palette[0] = f.baseColorFactor[0]; _palette[1] = f.baseColorFactor[1]; _palette[2] = f.baseColorFactor[2]; _palette[3] = f.baseColorFactor[3]; + _palette[4] = f.emissiveFactor[0]; _palette[5] = f.emissiveFactor[1]; _palette[6] = f.emissiveFactor[2]; _palette[7] = f.metallicFactor; + _palette[8] = f.roughnessFactor; _palette[9] = f.normalScale; _palette[10] = f.occlusionStrength; _palette[11] = 0; + device.queue.writeBuffer(palette, layer * PALETTE_STRIDE, _palette); +} + +/** The five map source URLs for one material ("" = leave the neutral layer). */ +export interface MaterialMapUrls { + baseColorUrl: string; + metallicRoughnessUrl: string; + normalUrl: string; + occlusionUrl: string; + emissiveUrl: string; +} + +/** + * Fetches a material's maps and blits them (resampled to the layer size) into + * its array layer. Fire-and-forget: each map resolves independently, so a body + * renders (neutral) immediately and sharpens as textures arrive. URLs shared + * across maps (e.g. an ARM image used for both metallicRoughness and occlusion) + * are fetched once and copied to each target. + */ +export function loadMaterialMaps(device: GPUDevice, arrays: MaterialArrays, layer: number, urls: MaterialMapUrls): void { + const targets: { url: string; tex: GPUTexture }[] = [ + { url: urls.baseColorUrl, tex: arrays.baseColor }, + { url: urls.metallicRoughnessUrl, tex: arrays.metallicRoughness }, + { url: urls.normalUrl, tex: arrays.normal }, + { url: urls.occlusionUrl, tex: arrays.occlusion }, + { url: urls.emissiveUrl, tex: arrays.emissive }, + ]; + const byUrl = new Map(); + for (const t of targets) { + if (!t.url) continue; + const list = byUrl.get(t.url); + if (list) list.push(t.tex); + else byUrl.set(t.url, [t.tex]); + } + for (const [url, texes] of byUrl) { + fetch(url) + .then(r => { if (!r.ok) throw new Error(`material map ${r.status}: ${url}`); return r.blob(); }) + .then(blob => createImageBitmap(blob, { + resizeWidth: MATERIAL_TEXTURE_SIZE, + resizeHeight: MATERIAL_TEXTURE_SIZE, + colorSpaceConversion: "none", + })) + .then(bitmap => { + for (const tex of texes) { + device.queue.copyExternalImageToTexture( + { source: bitmap }, + { texture: tex, origin: [0, 0, layer] }, + [MATERIAL_TEXTURE_SIZE, MATERIAL_TEXTURE_SIZE, 1], + ); + } + bitmap.close(); + }) + .catch(err => console.warn("[materialGpu] map load failed", err)); + } +} diff --git a/packages/data-gpu/src/graphics/rendering/material-gpu/material-bind-group.ts b/packages/data-gpu/src/graphics/rendering/material-gpu/material-bind-group.ts new file mode 100644 index 00000000..c894dd13 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/material-gpu/material-bind-group.ts @@ -0,0 +1,61 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { MaterialArrays } from "./material-arrays.js"; + +interface DeviceCache { + layout: GPUBindGroupLayout; + sampler: GPUSampler; +} +const cacheByDevice = new WeakMap(); + +function deviceCache(device: GPUDevice): DeviceCache { + let c = cacheByDevice.get(device); + if (!c) { + const tex = (binding: number): GPUBindGroupLayoutEntry => ({ + binding, visibility: GPUShaderStage.FRAGMENT, + texture: { sampleType: "float", viewDimension: "2d-array" }, + }); + c = { + layout: device.createBindGroupLayout({ + entries: [ + { binding: 0, visibility: GPUShaderStage.FRAGMENT, buffer: { type: "read-only-storage" } }, + tex(1), tex(2), tex(3), tex(4), tex(5), + { binding: 6, visibility: GPUShaderStage.FRAGMENT, sampler: { type: "filtering" } }, + ], + }), + sampler: device.createSampler({ + magFilter: "linear", minFilter: "linear", mipmapFilter: "linear", + addressModeU: "repeat", addressModeV: "repeat", + }), + }; + cacheByDevice.set(device, c); + } + return c; +} + +/** + * Bind-group layout for the shared material set: palette storage buffer + + * five `texture_2d_array`s + a filtering sampler. The renderer's primitive + * pipeline builds its pipeline layout from this; `createMaterialBindGroup` + * builds the matching bind group. Cached per device. + */ +export function createMaterialBindGroupLayout(device: GPUDevice): GPUBindGroupLayout { + return deviceCache(device).layout; +} + +export function createMaterialBindGroup(device: GPUDevice, arrays: MaterialArrays, palette: GPUBuffer): GPUBindGroup { + const { layout, sampler } = deviceCache(device); + const view = (t: GPUTexture): GPUTextureView => t.createView({ dimension: "2d-array" }); + return device.createBindGroup({ + layout, + entries: [ + { binding: 0, resource: { buffer: palette } }, + { binding: 1, resource: view(arrays.baseColor) }, + { binding: 2, resource: view(arrays.metallicRoughness) }, + { binding: 3, resource: view(arrays.normal) }, + { binding: 4, resource: view(arrays.occlusion) }, + { binding: 5, resource: view(arrays.emissive) }, + { binding: 6, resource: sampler }, + ], + }); +} diff --git a/packages/data-gpu/src/graphics/rendering/material-gpu/material-gpu-plugin.ts b/packages/data-gpu/src/graphics/rendering/material-gpu/material-gpu-plugin.ts new file mode 100644 index 00000000..9ee1cfe2 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/material-gpu/material-gpu-plugin.ts @@ -0,0 +1,95 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { U32 } from "@adobe/data/math"; +import { core } from "../../../core/core-plugin.js"; +import { Material } from "../../../material/material.js"; +import { + type MaterialArrays, + MAX_MATERIAL_LAYERS, PALETTE_STRIDE, + createMaterialArrays, writePaletteLayer, loadMaterialMaps, +} from "./material-arrays.js"; +import { createMaterialBindGroup } from "./material-bind-group.js"; + +/** Material rows the builder reads to fill a layer (factors + map URLs). */ +const MATERIAL_COMPONENTS = [ + "baseColorFactor", "emissiveFactor", "metallicFactor", "roughnessFactor", "normalScale", "occlusionStrength", + "baseColorUrl", "metallicRoughnessUrl", "normalUrl", "occlusionUrl", "emissiveUrl", +] as const; + +/** + * materialGpu — derives the shared GPU material set from the `Material` + * registry. Each authored material is assigned one array layer; its factors go + * into the palette buffer and its maps are fetched + blitted into that layer. + * + * Incremental and cached: the arrays / palette / bind group are built once, and + * a layer is assigned only to a material that doesn't have one yet (`_layerIndex` + * excluded). Materials are added rarely and edited never, so steady state does + * no work. The renderer (P3) binds `_materialBindGroup` once and selects a + * material per instance by `_layerIndex`. + */ +export const materialGpu = Database.Plugin.create({ + extends: Database.Plugin.combine(Material.plugin, core), + components: { + _layerIndex: U32.schema, + }, + resources: { + _materialArrays: { default: null as MaterialArrays | null, transient: true }, + _materialPalette: { default: null as GPUBuffer | null, transient: true }, + _materialBindGroup: { default: null as GPUBindGroup | null, transient: true }, + }, + systems: { + materialGpuBuilder: { + schedule: { during: ["preRender"] }, + create: db => { + let arrays: MaterialArrays | null = null; + let palette: GPUBuffer | null = null; + let nextLayer = 0; + return () => { + const { device } = db.store.resources; + if (!device) return; + + // Assign a layer to each material that lacks one. Tail→head: + // every visited row migrates out (gains _layerIndex), so the + // removal is always from the tail — no hole-fill shift. The GPU + // arrays are created lazily on the first material, so glTF-only + // scenes (no materials) allocate nothing. + for (const arch of db.store.queryArchetypes(["name", ...MATERIAL_COMPONENTS], { exclude: ["_layerIndex"] })) { + const c = arch.columns; + for (let i = arch.rowCount - 1; i >= 0; i--) { + if (nextLayer >= MAX_MATERIAL_LAYERS) break; + if (!arrays || !palette) { + arrays = createMaterialArrays(device); + palette = device.createBuffer({ + size: MAX_MATERIAL_LAYERS * PALETTE_STRIDE, + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + db.store.resources._materialArrays = arrays; + db.store.resources._materialPalette = palette; + db.store.resources._materialBindGroup = createMaterialBindGroup(device, arrays, palette); + } + const id = c.id.get(i); + const layer = nextLayer++; + writePaletteLayer(device, palette, layer, { + baseColorFactor: c.baseColorFactor.get(i), + emissiveFactor: c.emissiveFactor.get(i), + metallicFactor: c.metallicFactor.get(i), + roughnessFactor: c.roughnessFactor.get(i), + normalScale: c.normalScale.get(i), + occlusionStrength: c.occlusionStrength.get(i), + }); + loadMaterialMaps(device, arrays, layer, { + baseColorUrl: c.baseColorUrl.get(i), + metallicRoughnessUrl: c.metallicRoughnessUrl.get(i), + normalUrl: c.normalUrl.get(i), + occlusionUrl: c.occlusionUrl.get(i), + emissiveUrl: c.emissiveUrl.get(i), + }); + db.store.update(id, { _layerIndex: layer }); + } + } + }; + }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/model-collider-plugin.ts b/packages/data-gpu/src/graphics/rendering/model-collider-plugin.ts new file mode 100644 index 00000000..a2dd6109 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/model-collider-plugin.ts @@ -0,0 +1,77 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, Entity } from "@adobe/data/ecs"; +import { physicsData } from "../../physics/physics-data-plugin.js"; +import type { ColliderMesh } from "../../physics/body/collider-mesh.js"; +import { modelLoader } from "../scene/model/model-loader-plugin.js"; +import { hullVertices } from "../scene/model/shape/convex-hull.js"; + +/** + * modelCollider — lets a rendered model double as a physics body whose collider is + * **auto-generated from its mesh**. A `ModelBody` (or `StaticModelCollider`) carries + * a `geometry` + a `colliderShape` of `"hull"` or `"mesh"` but *no* collision data; + * this fills it once the mesh has loaded: + * + * hull → `convexPoints` = the simplified hull vertices of the mesh (the engine + * rebuilds the hull) — a coarse, convex stand-in for a detailed model. + * mesh → `colliderMesh` = the mesh triangles verbatim (static trimesh). + * + * The source defaults to the body's own `geometry` but can be overridden with + * `collisionGeometry` (point it at a low-poly mesh for a cheaper collider). The + * model's `scale` is baked into the collision geometry so it matches the render. + * Generated data is cached per (source geometry, shape, scale), so many instances + * of one model share it. Authoring the collider by hand (supplying `convexPoints` + * / `colliderMesh` directly, as `ConvexBody` / `MeshCollider` do) still works and + * simply skips generation. + * + * Because the body already has a `geometry`, the physics render bridge leaves it + * alone — it renders as the detailed model and collides as the generated shape. + */ +export const modelCollider = Database.Plugin.create({ + extends: Database.Plugin.combine(physicsData, modelLoader), + components: { + collisionGeometry: Entity.schema, // optional override: generate the collider from this Geometry instead of `geometry` + }, + archetypes: { + // A model that is also a dynamic/kinematic body; collider auto-generated. + ModelBody: ["geometry", "scale", "visible", "parent", "bodyType", "colliderShape", "halfExtents", "material", "position", "rotation", "linearVelocity", "angularVelocity"], + // A static model collider (immovable level geometry that renders as a model). + StaticModelCollider: ["geometry", "scale", "visible", "parent", "colliderShape", "halfExtents", "material", "position", "rotation"], + }, + systems: { + generateModelColliders: { + schedule: { during: ["postUpdate"] }, + create: db => { + // generated collider data, shared across instances of the same (source mesh, shape, scale) + const cache = new Map(); + return () => { + // model-bodies (geometry + scale) whose collider data isn't generated yet + for (const arch of db.store.queryArchetypes(["colliderShape", "geometry", "scale"], { exclude: ["convexPoints", "colliderMesh"] })) { + const ids = arch.columns.id, css = arch.columns.colliderShape, scl = arch.columns.scale; + // tail→head: filling the data migrates the row out of this archetype + for (let i = arch.rowCount - 1; i >= 0; i--) { + const shape = css.get(i); + if (shape !== "hull" && shape !== "mesh") continue; // primitive-shaped model-bodies need no generation + const id = ids.get(i); + const src = (db.store.read(id) as { collisionGeometry?: Entity }).collisionGeometry || arch.columns.geometry.get(i); + const mesh = db.store.read(src) as { _cpuPositions?: Float32Array | null; _cpuIndices?: Uint32Array | null } | null; + const positions = mesh?._cpuPositions; + if (!positions) continue; // source mesh not loaded yet — retry next frame + const s = scl.get(i), key = `${src}:${shape}:${s[0]}:${s[1]}:${s[2]}`; + let data = cache.get(key); + if (!data) { + const scaled = new Float32Array(positions.length); + for (let k = 0; k < positions.length; k += 3) { scaled[k] = positions[k] * s[0]; scaled[k + 1] = positions[k + 1] * s[1]; scaled[k + 2] = positions[k + 2] * s[2]; } + data = shape === "hull" + ? { convexPoints: hullVertices(scaled) } + : { colliderMesh: { positions: scaled, indices: mesh!._cpuIndices ?? new Uint32Array(0) } }; + cache.set(key, data); + } + db.store.update(id, data); + } + } + }; + }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/pbr-core-plugin.ts b/packages/data-gpu/src/graphics/rendering/pbr-core-plugin.ts new file mode 100644 index 00000000..8dc3b8aa --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/pbr-core-plugin.ts @@ -0,0 +1,40 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, Entity } from "@adobe/data/ecs"; +import { Mat4x4 } from "@adobe/data/math"; +import { U32 } from "@adobe/data/math"; + +/** + * Shape declarations shared between asset producers (model loader, shape + * generators) and consumers (renderers). Every component and archetype is + * ephemeral — not part of the user's data model. Pair with an asset + * producer plus a renderer aggregator (`pbrIblRender`) + * to get drawable scenes. + */ +export const pbrCore = Database.Plugin.create({ + components: { + _vertexBuffer: { default: null as GPUBuffer | null }, + /** Secondary VBO with skinning attributes (joints u32×4, weights f32×4), + * null for static primitives. Drives skinned-pipeline dispatch. */ + _skinVertexBuffer: { default: null as GPUBuffer | null }, + _indexBuffer: { default: null as GPUBuffer | null }, + _indexCount: U32.schema, + _indexFormat: { default: "uint16" as GPUIndexFormat }, + _materialBindGroup: { default: null as GPUBindGroup | null }, + /** Back-reference from a _PbrPrimitive / _VisibleMaterial to the + * authored Geometry that owns it. */ + _geometry: Entity.schema, + _material: Entity.schema, + /** Node-local-to-model-root matrix baked at load time. The renderer + * pre-multiplies it with the per-instance model-root world matrix. */ + _nodeLocalMatrix: Mat4x4.schema, + /** Skeleton-side back-references the renderer reads when dispatching + * the skinned pipeline. Populated by `pbrSkinning`. */ + _skeletonModelRef: Entity.schema, + _skeletonJointMatrixBindGroup: { default: null as GPUBindGroup | null }, + }, + archetypes: { + _VisibleMaterial: ["ephemeral", "_materialBindGroup", "_geometry"], + _PbrPrimitive: ["ephemeral", "_vertexBuffer", "_skinVertexBuffer", "_indexBuffer", "_indexCount", "_indexFormat", "_material", "_geometry", "_nodeLocalMatrix"], + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/pbr-render/pbr-array-shader.wgsl.ts b/packages/data-gpu/src/graphics/rendering/pbr-render/pbr-array-shader.wgsl.ts new file mode 100644 index 00000000..d668e374 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/pbr-render/pbr-array-shader.wgsl.ts @@ -0,0 +1,146 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { wgslStructFields } from "@adobe/data/typed-buffer"; +import { schema as sceneUniformsSchema } from "../../scene/scene-uniforms/schema.js"; +import brdf from "../ibl-render/brdf.wgsl.js"; + +/** + * Unified PBR shader for instanced primitives (and, after convergence, glTF). + * Materials come from the shared `materialGpu` set — five `texture_2d_array`s + + * a palette of per-layer factors (group 1) — selected by a per-instance layer + * index (group 3, binding 1). Shading is identical Cook-Torrance + split-sum + * IBL to the per-material `ibl-shader`; only the material *source* differs. + */ +export function buildPbrArrayShader(options: { prefilteredMipCount: number }): string { + return /* wgsl */ ` +struct SceneUniforms { +${wgslStructFields(sceneUniformsSchema)} +} + +struct MaterialFactors { + baseColorFactor: vec4f, + emissiveMetallic: vec4f, // emissive.rgb, metallicFactor + roughNormalOcc: vec4f, // roughnessFactor, normalScale, occlusionStrength, _ +} + +@group(0) @binding(0) var scene: SceneUniforms; + +@group(1) @binding(0) var palette: array; +@group(1) @binding(1) var baseColorArray: texture_2d_array; +@group(1) @binding(2) var metallicRoughnessArray: texture_2d_array; +@group(1) @binding(3) var normalArray: texture_2d_array; +@group(1) @binding(4) var occlusionArray: texture_2d_array; +@group(1) @binding(5) var emissiveArray: texture_2d_array; +@group(1) @binding(6) var matSampler: sampler; + +@group(2) @binding(0) var iblIrradiance: texture_cube; +@group(2) @binding(1) var iblPrefiltered: texture_cube; +@group(2) @binding(2) var iblBrdfLut: texture_2d; +@group(2) @binding(3) var iblSampler: sampler; + +@group(3) @binding(0) var instances: array>; +@group(3) @binding(1) var layers: array; + +const PREFILTERED_MIP_COUNT: f32 = ${options.prefilteredMipCount.toFixed(1)}; + +struct VertexInput { + @location(0) position: vec3f, + @location(1) normal: vec3f, + @location(2) tangent: vec4f, + @location(3) uv: vec2f, +} + +struct VertexOutput { + @builtin(position) clipPosition: vec4f, + @location(0) worldPosition: vec3f, + @location(1) normal: vec3f, + @location(2) tangent: vec3f, + @location(3) bitangent: vec3f, + @location(4) uv: vec2f, + @location(5) @interpolate(flat) layer: u32, +} + +@vertex +fn vs_main(@builtin(instance_index) instanceIndex: u32, in: VertexInput) -> VertexOutput { + let m = instances[instanceIndex]; + let m3 = mat3x3(m[0].xyz, m[1].xyz, m[2].xyz); + let normalMat = mat3x3( + cross(m3[1], m3[2]), + cross(m3[2], m3[0]), + cross(m3[0], m3[1]), + ); + let worldPos = m * vec4f(in.position, 1.0); + var out: VertexOutput; + out.clipPosition = scene.viewProjectionMatrix * worldPos; + out.worldPosition = worldPos.xyz; + out.normal = normalize(normalMat * in.normal); + out.tangent = normalize(m3 * in.tangent.xyz); + out.bitangent = normalize(cross(out.normal, out.tangent) * in.tangent.w); + out.uv = in.uv; + out.layer = layers[instanceIndex]; + return out; +} + +${brdf} + +@fragment +fn fs_main(in: VertexOutput) -> @location(0) vec4f { + let f = palette[in.layer]; + let li = i32(in.layer); + + let baseColor = textureSample(baseColorArray, matSampler, in.uv, li) * f.baseColorFactor; + + let mr = textureSample(metallicRoughnessArray, matSampler, in.uv, li); + let metallic = mr.b * f.emissiveMetallic.w; + var roughness = mr.g * f.roughNormalOcc.x; + roughness = max(roughness, MIN_ROUGHNESS); + let alpha = roughness * roughness; + + let occlusion = textureSample(occlusionArray, matSampler, in.uv, li).r; + let emissive = textureSample(emissiveArray, matSampler, in.uv, li).rgb * f.emissiveMetallic.xyz; + + let nSampled = textureSample(normalArray, matSampler, in.uv, li).rgb * 2.0 - vec3f(1.0); + let nScaled = vec3f(nSampled.xy * f.roughNormalOcc.y, nSampled.z); + let tbn = mat3x3(in.tangent, in.bitangent, in.normal); + let N = normalize(tbn * nScaled); + + let V = normalize(scene.cameraPosition - in.worldPosition); + let nDotV = max(dot(N, V), 0.001); + let R = reflect(-V, N); + + let f0 = mix(vec3f(0.04), baseColor.rgb, metallic); + + // --- IBL contribution (split-sum) --- + let F_ibl = f_schlick_roughness(nDotV, f0, roughness); + let kD_ibl = (vec3f(1.0) - F_ibl) * (1.0 - metallic); + + let irradiance = textureSampleLevel(iblIrradiance, iblSampler, N, 0.0).rgb; + let diffuseIbl = kD_ibl * irradiance * baseColor.rgb; + + let mipLevel = roughness * (PREFILTERED_MIP_COUNT - 1.0); + let prefiltered = textureSampleLevel(iblPrefiltered, iblSampler, R, mipLevel).rgb; + let envBrdf = textureSampleLevel(iblBrdfLut, iblSampler, vec2f(nDotV, roughness), 0.0).rg; + let specularIbl = prefiltered * (F_ibl * envBrdf.x + envBrdf.y); + + let ambient = (diffuseIbl + specularIbl) * mix(1.0, occlusion, f.roughNormalOcc.z); + + // --- Direct light (parity with the single scene light) --- + let L = normalize(-scene.lightDirection); + let H = normalize(V + L); + let nDotL = max(dot(N, L), 0.0); + let nDotH = max(dot(N, H), 0.0); + let vDotH = max(dot(V, H), 0.0); + let D_d = d_ggx(nDotH, alpha); + let G_d = g_smith(nDotV, nDotL, alpha); + let F_d = f_schlick(vDotH, f0); + let spec_d = (D_d * G_d * F_d) / (4.0 * nDotV * nDotL + 0.0001); + let kD_d = (vec3f(1.0) - F_d) * (1.0 - metallic); + let direct = (kD_d * baseColor.rgb / PI + spec_d) * scene.lightColor * nDotL; + + let color = direct + ambient + emissive; + let mapped = tone_map_aces(color); + let gamma = pow(mapped, vec3f(1.0 / 2.2)); + return vec4f(gamma, baseColor.a); +} +`; +} diff --git a/packages/data-gpu/src/graphics/rendering/pbr-render/pbr-render-plugin.ts b/packages/data-gpu/src/graphics/rendering/pbr-render/pbr-render-plugin.ts new file mode 100644 index 00000000..3c3c12f6 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/pbr-render/pbr-render-plugin.ts @@ -0,0 +1,247 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, type Entity } from "@adobe/data/ecs"; +import { pbrIblRender } from "../ibl-render/ibl-render-plugin.js"; +import { materialGpu } from "../material-gpu/material-gpu-plugin.js"; +import { displayTransform } from "../display-transform-plugin.js"; +import { SceneUniforms } from "../../scene/scene-uniforms/scene-uniforms.js"; +import { StandardVertex } from "../standard-vertex/standard-vertex.js"; +import { createMaterialBindGroupLayout } from "../material-gpu/material-bind-group.js"; +import { buildPbrArrayShader } from "./pbr-array-shader.wgsl.js"; + +// Must match pbrIblRender's bake so the array shader's prefiltered-mip math agrees. +const PREFILTERED_MIP_COUNT = 7; + +/** Drawable: any visible entity with a geometry ref, a transform, and a material. */ +const DRAWABLE = ["geometry", "visible", "position", "rotation", "scale", "material"] as const; +// Bodies carrying a derived display pose (interpolation-plugin) draw at it instead +// of the canonical pose; everything else (statics, props, non-physics instances) +// draws straight from position/rotation. Splitting the gather by archetype shape +// avoids a per-row "has display pose?" branch. +const DRAWABLE_INTERP = [...DRAWABLE, "_renderPosition", "_renderRotation"] as const; +const DRAWABLE_DIRECT = { exclude: ["_renderPosition"] } as const; +const PRIMITIVE = ["_vertexBuffer", "_indexBuffer", "_indexCount", "_indexFormat", "_geometry", "_nodeLocalMatrix"] as const; + +/** Minimal column shape the gather reads — satisfied by either query's columns. */ +interface Col { get(i: number): T } + +function createIblBindGroupLayout(device: GPUDevice): GPUBindGroupLayout { + return device.createBindGroupLayout({ + entries: [ + { binding: 0, visibility: GPUShaderStage.FRAGMENT, texture: { sampleType: "float", viewDimension: "cube" } }, + { binding: 1, visibility: GPUShaderStage.FRAGMENT, texture: { sampleType: "float", viewDimension: "cube" } }, + { binding: 2, visibility: GPUShaderStage.FRAGMENT, texture: { sampleType: "float", viewDimension: "2d" } }, + { binding: 3, visibility: GPUShaderStage.FRAGMENT, sampler: { type: "filtering" } }, + ], + }); +} + +/** + * Compose a column-major TRS matrix directly into `out[o..o+15]` — no + * allocation, reading position/rotation/scale straight from their column typed + * arrays. Matches `translation · Quat.toMat4 · scaling`. Runs once per visible + * drawable per frame, so the array churn of the math-helper form matters. + */ +function composeTrs( + out: Float32Array, o: number, + p: ArrayLike, pi: number, q: ArrayLike, qi: number, s: ArrayLike, si: number, +): void { + const qx = q[qi], qy = q[qi + 1], qz = q[qi + 2], qw = q[qi + 3]; + const sx = s[si], sy = s[si + 1], sz = s[si + 2]; + const xx = qx * qx, yy = qy * qy, zz = qz * qz; + const xy = qx * qy, xz = qx * qz, yz = qy * qz; + const wx = qw * qx, wy = qw * qy, wz = qw * qz; + out[o] = (1 - 2 * (yy + zz)) * sx; out[o + 1] = (2 * (xy + wz)) * sx; out[o + 2] = (2 * (xz - wy)) * sx; out[o + 3] = 0; + out[o + 4] = (2 * (xy - wz)) * sy; out[o + 5] = (1 - 2 * (xx + zz)) * sy; out[o + 6] = (2 * (yz + wx)) * sy; out[o + 7] = 0; + out[o + 8] = (2 * (xz + wy)) * sz; out[o + 9] = (2 * (yz - wx)) * sz; out[o + 10] = (1 - 2 * (xx + yy)) * sz; out[o + 11] = 0; + out[o + 12] = p[pi]; out[o + 13] = p[pi + 1]; out[o + 14] = p[pi + 2]; out[o + 15] = 1; +} + +/** + * pbrRender — the single PBR + IBL renderer. It extends `pbrIblRender`, reusing + * its IBL bake, skybox, scene/material bind groups and glTF draw path verbatim, + * and adds one extra draw query for **instanced primitives** whose material + * comes from the shared `materialGpu` arrays (selected per instance by the + * material entity's `_layerIndex`). Two efficient queries, one render plugin: + * + * - glTF models → per-material bind group, native-resolution textures (pbrIblRender) + * - primitives → shared texture_2d_array + per-instance layer (this system) + * + * The primitive system runs after `pbrIblRenderSystem` so the skybox is drawn + * first. World matrices are computed directly from `position`/`rotation`/`scale` + * (primitives are flat — no hierarchy), so it's independent of `transform`. + */ +export const pbrRender = Database.Plugin.create({ + extends: Database.Plugin.combine(pbrIblRender, materialGpu, displayTransform), + systems: { + pbrPrimitiveRenderSystem: { + schedule: { during: ["render"], after: ["beginRenderPass", "pbrIblRenderSystem"], before: ["endRenderPass"] }, + create: db => { + let pipeline: GPURenderPipeline | null = null; + let sceneLayout: GPUBindGroupLayout | null = null; + let iblLayout: GPUBindGroupLayout | null = null; + let instanceLayout: GPUBindGroupLayout | null = null; + let iblSampler: GPUSampler | null = null; + let sceneBindGroup: GPUBindGroup | null = null; + let iblBindGroup: GPUBindGroup | null = null; + let cachedSceneBuffer: GPUBuffer | null = null; + let cachedIrradiance: GPUTexture | null = null; + + let matLayer = new Map(); + let matCount = -1; + + interface Inst { buffer: GPUBuffer; layerBuffer: GPUBuffer; bindGroup: GPUBindGroup; capacity: number } + const instCache = new Map(); + let matScratch = new Float32Array(16); + let layerScratch = new Uint32Array(1); + // Per-frame matrix arena (composed TRS, one mat4 per drawable) + per-geometry + // draw lists (matrix offset + layer). All pooled and reused across frames — + // the gather runs over every visible body each frame, so zero allocation here. + let mats = new Float32Array(64 * 16); + const batches = new Map(); + + return () => { + const { + device, renderPassEncoder, canvasFormat, depthFormat, _sceneUniformsBuffer, + _materialBindGroup, _iblIrradiance, _iblPrefiltered, _iblBrdfLut, + } = db.store.resources; + if (!device || !renderPassEncoder || !_sceneUniformsBuffer || !_materialBindGroup) return; + if (!_iblIrradiance || !_iblPrefiltered || !_iblBrdfLut) return; + + if (!sceneLayout) sceneLayout = SceneUniforms.createBindGroupLayout(device); + if (!iblLayout) iblLayout = createIblBindGroupLayout(device); + if (!instanceLayout) instanceLayout = device.createBindGroupLayout({ + entries: [ + { binding: 0, visibility: GPUShaderStage.VERTEX, buffer: { type: "read-only-storage" } }, + { binding: 1, visibility: GPUShaderStage.VERTEX, buffer: { type: "read-only-storage" } }, + ], + }); + if (!iblSampler) iblSampler = device.createSampler({ + magFilter: "linear", minFilter: "linear", mipmapFilter: "linear", + addressModeU: "clamp-to-edge", addressModeV: "clamp-to-edge", addressModeW: "clamp-to-edge", + }); + if (!pipeline) { + const module = device.createShaderModule({ code: buildPbrArrayShader({ prefilteredMipCount: PREFILTERED_MIP_COUNT }) }); + pipeline = device.createRenderPipeline({ + layout: device.createPipelineLayout({ bindGroupLayouts: [sceneLayout, createMaterialBindGroupLayout(device), iblLayout, instanceLayout] }), + vertex: { module, entryPoint: "vs_main", buffers: [StandardVertex.layout] }, + fragment: { module, entryPoint: "fs_main", targets: [{ format: canvasFormat }] }, + primitive: { topology: "triangle-list", cullMode: "back" }, + depthStencil: { format: depthFormat, depthWriteEnabled: true, depthCompare: "less" }, + }); + } + + if (_sceneUniformsBuffer !== cachedSceneBuffer || !sceneBindGroup) { + sceneBindGroup = device.createBindGroup({ layout: sceneLayout, entries: [{ binding: 0, resource: { buffer: _sceneUniformsBuffer } }] }); + cachedSceneBuffer = _sceneUniformsBuffer; + } + if (_iblIrradiance !== cachedIrradiance || !iblBindGroup) { + iblBindGroup = device.createBindGroup({ + layout: iblLayout, + entries: [ + { binding: 0, resource: _iblIrradiance.createView({ dimension: "cube" }) }, + { binding: 1, resource: _iblPrefiltered.createView({ dimension: "cube" }) }, + { binding: 2, resource: _iblBrdfLut.createView() }, + { binding: 3, resource: iblSampler }, + ], + }); + cachedIrradiance = _iblIrradiance; + } + + // material → layer, refreshed only when the material set grows + let mc = 0; + for (const arch of db.store.queryArchetypes(["_layerIndex"])) mc += arch.rowCount; + if (mc !== matCount) { + matLayer = new Map(); + for (const arch of db.store.queryArchetypes(["_layerIndex"])) { + const id = arch.columns.id, li = arch.columns._layerIndex; + for (let i = 0; i < arch.rowCount; i++) matLayer.set(id.get(i), li.get(i)); + } + matCount = mc; + } + + // Gather visible primitive drawables grouped by geometry. Compose each + // world matrix straight from the chosen pose/scale column typed arrays + // into the pooled `mats` arena — no per-row array allocation. Two passes: + // bodies with a display pose use it; everything else uses position/rotation. + for (const b of batches.values()) { b.off.length = 0; b.layer.length = 0; } + let drawCount = 0; + const gather = ( + rowCount: number, vis: Col, geo: Col, mat: Col, + posArr: ArrayLike, rotArr: ArrayLike, sclArr: ArrayLike, + ): void => { + for (let i = 0; i < rowCount; i++) { + if (!vis.get(i)) continue; + const layer = matLayer.get(mat.get(i)); + if (layer === undefined) continue; + if ((drawCount + 1) * 16 > mats.length) { const grown = new Float32Array(mats.length * 2); grown.set(mats); mats = grown; } + composeTrs(mats, drawCount * 16, posArr, i * 3, rotArr, i * 4, sclArr, i * 3); + const g = geo.get(i); + let b = batches.get(g); + if (!b) { b = { off: [], layer: [] }; batches.set(g, b); } + b.off.push(drawCount); b.layer.push(layer); + drawCount++; + } + }; + for (const arch of db.store.queryArchetypes(DRAWABLE_INTERP)) { + gather(arch.rowCount, arch.columns.visible, arch.columns.geometry, arch.columns.material, + arch.columns._renderPosition.getTypedArray(), arch.columns._renderRotation.getTypedArray(), arch.columns.scale.getTypedArray()); + } + for (const arch of db.store.queryArchetypes(DRAWABLE, DRAWABLE_DIRECT)) { + gather(arch.rowCount, arch.columns.visible, arch.columns.geometry, arch.columns.material, + arch.columns.position.getTypedArray(), arch.columns.rotation.getTypedArray(), arch.columns.scale.getTypedArray()); + } + if (drawCount === 0) return; + + renderPassEncoder.setPipeline(pipeline); + renderPassEncoder.setBindGroup(0, sceneBindGroup); + renderPassEncoder.setBindGroup(1, _materialBindGroup); + renderPassEncoder.setBindGroup(2, iblBindGroup); + + for (const arch of db.store.queryArchetypes(PRIMITIVE)) { + const vbs = arch.columns._vertexBuffer, ibs = arch.columns._indexBuffer; + const counts = arch.columns._indexCount, formats = arch.columns._indexFormat; + const geoRefs = arch.columns._geometry, primIds = arch.columns.id; + for (let i = 0; i < arch.rowCount; i++) { + const b = batches.get(geoRefs.get(i)); + if (!b || b.off.length === 0) continue; + const n = b.off.length; + const primId = primIds.get(i); + + let entry = instCache.get(primId); + if (!entry || entry.capacity < n) { + entry?.buffer.destroy(); + entry?.layerBuffer.destroy(); + const buffer = device.createBuffer({ size: Math.max(n * 64, 64), usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST }); + const layerBuffer = device.createBuffer({ size: Math.max(n * 4, 4), usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST }); + const bindGroup = device.createBindGroup({ + layout: instanceLayout, + entries: [{ binding: 0, resource: { buffer } }, { binding: 1, resource: { buffer: layerBuffer } }], + }); + entry = { buffer, layerBuffer, bindGroup, capacity: n }; + instCache.set(primId, entry); + } + if (matScratch.length < n * 16) matScratch = new Float32Array(n * 16); + if (layerScratch.length < n) layerScratch = new Uint32Array(n); + // Copy this geometry's matrices out of the arena into a contiguous + // upload buffer. Primitive node matrices are identity (flat shapes; + // glTF hierarchy goes through pbrIblRender), so no extra multiply. + for (let j = 0; j < n; j++) { + const src = b.off[j] * 16, dst = j * 16; + for (let k = 0; k < 16; k++) matScratch[dst + k] = mats[src + k]; + layerScratch[j] = b.layer[j]; + } + device.queue.writeBuffer(entry.buffer, 0, matScratch, 0, n * 16); + device.queue.writeBuffer(entry.layerBuffer, 0, layerScratch, 0, n); + + renderPassEncoder.setBindGroup(3, entry.bindGroup); + renderPassEncoder.setVertexBuffer(0, vbs.get(i)!); // _PbrPrimitive guarantees non-null + renderPassEncoder.setIndexBuffer(ibs.get(i)!, formats.get(i)); // _PbrPrimitive guarantees non-null + renderPassEncoder.drawIndexed(counts.get(i), n); + } + } + }; + }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/pbr-render/physics-bridge-plugin.ts b/packages/data-gpu/src/graphics/rendering/pbr-render/physics-bridge-plugin.ts new file mode 100644 index 00000000..6375def6 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/pbr-render/physics-bridge-plugin.ts @@ -0,0 +1,106 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, type Entity } from "@adobe/data/ecs"; +import { physicsData } from "../../../physics/physics-data-plugin.js"; +import { model } from "../../scene/model/model-plugin.js"; +import { shapeGeometry } from "../../scene/model/shape/shape-geometry-plugin.js"; +import { capsuleMesh, flatShadedMesh } from "../../scene/model/shape/shape-mesh.js"; +import { convexHullMesh } from "../../scene/model/shape/convex-hull.js"; +import { uploadShapeMesh } from "../../scene/model/shape/upload-shape-mesh.js"; +import type { ColliderMesh } from "../../../physics/body/collider-mesh.js"; +import { interpolation } from "../interpolation-plugin.js"; + +/** + * physicsRenderBridge — makes colliders renderable by `pbrRender`. Once the + * shape geometries exist, every body with a collider shape (dynamic RigidBody + * or immovable StaticCollider alike) gains a `geometry` ref (sphere / cube by + * collider shape), a `scale` (its half-extents — the unit shapes are size-1), + * and `visible`. Because physics `rotation`/`position` are the same components + * the renderer reads, there is no per-frame sync: this only migrates new bodies. + * + * It also folds in `interpolation`, so dynamic bodies render at the smooth, + * render-rate display pose for free (the solver steps on a fixed clock that need + * not match the render rate) — bridging physics to the renderer is exactly when + * you want that. Use `interpolation` directly only with a custom (non-bridge) path. + * + * Also declares `Prop` — a render-only placed geometry with a material that is + * not a physics body at all (decorative scenery with no collider). + */ +export const physicsRenderBridge = Database.Plugin.create({ + extends: Database.Plugin.combine(physicsData, model, shapeGeometry, interpolation), + archetypes: { + Prop: ["geometry", "position", "rotation", "scale", "visible", "material"], + }, + systems: { + physicsBridge: { + schedule: { during: ["postUpdate"] }, + create: db => { + // Sphere/cube are unit meshes scaled by half-extents. A capsule can't + // be non-uniformly scaled (its caps would distort), so it's built at its + // real size and drawn at unit scale — one mesh per distinct (radius, + // half-height), cached here (scenes use very few capsule sizes). + const capsuleGeometry = new Map(); + const ensureCapsule = (device: GPUDevice, r: number, hy: number): Entity => { + const key = `${r}:${hy}`; + let geo = capsuleGeometry.get(key); + if (geo === undefined) { + const m = uploadShapeMesh(device, capsuleMesh(r, hy)); + geo = db.transactions.insertShapePrimitive({ vertexBuffer: m.vb, indexBuffer: m.ib, indexCount: m.count }); + capsuleGeometry.set(key, geo); + } + return geo; + }; + // Hull render mesh = triangulated convex hull of the point cloud, cached + // by the point-array reference (bodies sharing a cloud share one mesh + draw). + const hullGeometry = new Map(); + const ensureHull = (device: GPUDevice, points: Float32Array): Entity => { + let geo = hullGeometry.get(points); + if (geo === undefined) { + const m = uploadShapeMesh(device, convexHullMesh(points)); + geo = db.transactions.insertShapePrimitive({ vertexBuffer: m.vb, indexBuffer: m.ib, indexCount: m.count }); + hullGeometry.set(points, geo); + } + return geo; + }; + // Static-mesh render = flat-shaded triangle soup, cached by the colliderMesh ref. + const meshGeometry = new Map(); + const ensureMesh = (device: GPUDevice, cm: ColliderMesh): Entity => { + let geo = meshGeometry.get(cm); + if (geo === undefined) { + const m = uploadShapeMesh(device, flatShadedMesh(cm.positions, cm.indices)); + geo = db.transactions.insertShapePrimitive({ vertexBuffer: m.vb, indexBuffer: m.ib, indexCount: m.count }); + meshGeometry.set(cm, geo); + } + return geo; + }; + return () => { + const shapes = db.store.resources._shapeGeometry; + const device = db.store.resources.device; + if (!shapes || !device) return; + for (const arch of db.store.queryArchetypes(["colliderShape", "halfExtents"], { exclude: ["geometry"] })) { + const ids = arch.columns.id, css = arch.columns.colliderShape, hes = arch.columns.halfExtents; + // Tail→head: every visited body migrates out (gains geometry). + for (let i = arch.rowCount - 1; i >= 0; i--) { + const id = ids.get(i), shape = css.get(i), he = hes.get(i); + // capsule + hull render at unit scale (their meshes are real-size); + // sphere/cube are unit meshes scaled by half-extents. + let geometry: Entity, scale: [number, number, number]; + if (shape === "box") { geometry = shapes.cube; scale = [he[0], he[1], he[2]]; } + else if (shape === "capsule") { geometry = ensureCapsule(device, he[0], he[1]); scale = [1, 1, 1]; } + else if (shape === "hull") { + const pts = (db.store.read(id) as { convexPoints?: Float32Array | null }).convexPoints; + geometry = pts ? ensureHull(device, pts) : shapes.sphere; + scale = [1, 1, 1]; + } else if (shape === "mesh") { + const cm = (db.store.read(id) as { colliderMesh?: ColliderMesh | null }).colliderMesh; + geometry = cm ? ensureMesh(device, cm) : shapes.cube; + scale = [1, 1, 1]; + } else { geometry = shapes.sphere; scale = [he[0], he[0], he[0]]; } + db.store.update(id, { geometry, scale, visible: true }); + } + } + }; + }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/ragdoll-trigger-plugin.ts b/packages/data-gpu/src/graphics/rendering/ragdoll-trigger-plugin.ts new file mode 100644 index 00000000..8564695d --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/ragdoll-trigger-plugin.ts @@ -0,0 +1,21 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; + +/** + * ragdollTrigger — the backend-agnostic "go limp" signal. A scene calls + * `triggerRagdoll()`; whichever ragdoll backend is combined in (our generic + * `boneColliders`, or the Jolt-native `joltRagdoll`) consumes `_ragdollTrigger` + * once and switches its bones to dynamic. Sharing the flag lets the same scene + * drive either backend, so a sample can run our ragdoll on one solver and Jolt's + * on another, side by side. + */ +export const ragdollTrigger = Database.Plugin.create({ + resources: { + _ragdollTrigger: { default: false as boolean, transient: true }, + }, + transactions: { + /** Go limp: the active ragdoll backend flips its bones to dynamic. */ + triggerRagdoll(t) { t.resources._ragdollTrigger = true; }, + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/rendering-plugin.ts b/packages/data-gpu/src/graphics/rendering/rendering-plugin.ts new file mode 100644 index 00000000..c8a68960 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/rendering-plugin.ts @@ -0,0 +1,3 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export { pbrIblRender as rendering } from "./ibl-render/ibl-render-plugin.js"; diff --git a/packages/data-gpu/src/graphics/rendering/skinning/skinning-attributes/layout.ts b/packages/data-gpu/src/graphics/rendering/skinning/skinning-attributes/layout.ts new file mode 100644 index 00000000..92cc065d --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/skinning/skinning-attributes/layout.ts @@ -0,0 +1,20 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +// Skinning attributes live in a separate vertex buffer from the StandardVertex +// data so static meshes pay no per-vertex cost. Encoded as: +// +// joints (uint32 × 4) — joint indices into the skeleton (16 bytes) +// weights (float32 × 4) — skinning weights, sum to 1.0 (16 bytes) +// +// Total stride: 32 bytes / vertex. + +export const SKINNING_STRIDE = 32; + +export const layout: GPUVertexBufferLayout = { + arrayStride: SKINNING_STRIDE, + stepMode: "vertex", + attributes: [ + { format: "uint32x4", offset: 0, shaderLocation: 4 }, + { format: "float32x4", offset: 16, shaderLocation: 5 }, + ], +}; diff --git a/packages/data-gpu/src/graphics/rendering/skinning/skinning-attributes/public.ts b/packages/data-gpu/src/graphics/rendering/skinning/skinning-attributes/public.ts new file mode 100644 index 00000000..1e33331b --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/skinning/skinning-attributes/public.ts @@ -0,0 +1,3 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export { layout, SKINNING_STRIDE } from "./layout.js"; diff --git a/packages/data-gpu/src/graphics/rendering/skinning/skinning-attributes/skinning-attributes.ts b/packages/data-gpu/src/graphics/rendering/skinning/skinning-attributes/skinning-attributes.ts new file mode 100644 index 00000000..be61b869 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/skinning/skinning-attributes/skinning-attributes.ts @@ -0,0 +1,3 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export * as SkinningAttributes from "./public.js"; diff --git a/packages/data-gpu/src/graphics/rendering/skinning/skinning-plugin.ts b/packages/data-gpu/src/graphics/rendering/skinning/skinning-plugin.ts new file mode 100644 index 00000000..36c15431 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/skinning/skinning-plugin.ts @@ -0,0 +1,213 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, Entity } from "@adobe/data/ecs"; +import { Mat4x4 } from "@adobe/data/math"; +import { modelLoader } from "../../scene/model/model-loader-plugin.js"; +import { pbrCore } from "../pbr-core-plugin.js"; +import { transform } from "../../scene/node/transform-plugin.js"; +import { animation } from "../../animation/animation-plugin.js"; +import type { JointTemplate } from "../../scene/model/gltf/parse-skin.js"; + +/** + * pbrSkinning + * query: Model+geometry, _Skeleton + * read: + * geometry + * _skinJointTemplate (from Geometry) + * _skinInverseBindMatrices (from Geometry) + * _animationClipRefs (from Geometry) + * _worldMatrix (from Model + joint Nodes) + * write: + * _Skeleton // new ephemeral entity, one per skinned Model + * Node // joint entities (TRS hierarchy) + * Animation // optional, when geometry has clips + * _skeletonInstanceBuffer + * _skeletonJointMatrixBuffer + * _skeletonJointMatrixBindGroup + */ +export const pbrSkinning = Database.Plugin.create({ + extends: Database.Plugin.combine(modelLoader, pbrCore, transform, animation), + components: { + _skeletonJoints: { default: [] as number[] }, + _skeletonGeometry: Entity.schema, + /** 64 bytes — the Model's world matrix, refreshed each frame. */ + _skeletonInstanceBuffer: { default: null as GPUBuffer | null }, + /** N × 64 bytes — joint skinning matrices, refreshed each frame. */ + _skeletonJointMatrixBuffer: { default: null as GPUBuffer | null }, + }, + archetypes: { + _Skeleton: [ + "_skeletonJoints", + "_skeletonGeometry", + "_skeletonModelRef", + "_skeletonInstanceBuffer", + "_skeletonJointMatrixBuffer", + "_skeletonJointMatrixBindGroup", + ], + }, + transactions: { + initSkeleton(t, args: { + modelRef: number; + geometryRef: number; + jointTemplate: readonly JointTemplate[]; + instanceBuffer: GPUBuffer; + jointMatrixBuffer: GPUBuffer; + jointMatrixBindGroup: GPUBindGroup; + clipRef: number | null; + }) { + const jointIds: number[] = new Array(args.jointTemplate.length); + // jointTemplate is in glTF skin.joints[] order. Parents can come + // after children in glTF, so resolve parent entity ids in two passes: + // first insert with parent=0, then update each joint's parent. + for (let i = 0; i < args.jointTemplate.length; i++) { + const j = args.jointTemplate[i]; + jointIds[i] = t.archetypes.Node.insert({ + position: j.position, + rotation: j.rotation, + scale: j.scale, + visible: true, + parent: 0, + }); + } + for (let i = 0; i < args.jointTemplate.length; i++) { + const parentJoint = args.jointTemplate[i].parentJointIndex; + const parentId = parentJoint >= 0 ? jointIds[parentJoint] : args.modelRef; + t.update(jointIds[i], { parent: parentId }); + } + t.archetypes._Skeleton.insert({ + _skeletonJoints: jointIds, + _skeletonGeometry: args.geometryRef, + _skeletonModelRef: args.modelRef, + _skeletonInstanceBuffer: args.instanceBuffer, + _skeletonJointMatrixBuffer: args.jointMatrixBuffer, + _skeletonJointMatrixBindGroup: args.jointMatrixBindGroup, + }); + if (args.clipRef !== null) { + t.archetypes.Animation.insert({ + animationClipRef: args.clipRef, + animationTargets: jointIds, + animationTime: 0, + animationSpeed: 1, + animationLoop: true, + animationPlaying: true, + }); + } + }, + }, + systems: { + skinningInitSystem: { + create: db => { + const initialized = new Set(); + let layoutCache: GPUBindGroupLayout | null = null; + return () => { + const { device } = db.store.resources; + if (!device) return; + if (!layoutCache) { + layoutCache = device.createBindGroupLayout({ + entries: [ + { binding: 0, visibility: GPUShaderStage.VERTEX, buffer: { type: "read-only-storage" } }, + { binding: 1, visibility: GPUShaderStage.VERTEX, buffer: { type: "read-only-storage" } }, + ], + }); + } + for (const arch of db.store.queryArchetypes(["geometry"])) { + const ids = arch.columns.id; + const geoRefs = arch.columns.geometry; + for (let i = 0; i < arch.rowCount; i++) { + const modelId = ids.get(i); + if (initialized.has(modelId)) continue; + const geoId = geoRefs.get(i); + const geo = db.store.read(geoId); + const template = geo?._skinJointTemplate; + if (!template || template.length === 0) continue; + initialized.add(modelId); + + const instanceBuffer = device.createBuffer({ + size: 64, + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + const matrixBuffer = device.createBuffer({ + size: template.length * 64, + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + const matrixBindGroup = device.createBindGroup({ + layout: layoutCache, + entries: [ + { binding: 0, resource: { buffer: instanceBuffer } }, + { binding: 1, resource: { buffer: matrixBuffer } }, + ], + }); + db.transactions.initSkeleton({ + modelRef: modelId, + geometryRef: geoId, + jointTemplate: template, + instanceBuffer, + jointMatrixBuffer: matrixBuffer, + jointMatrixBindGroup: matrixBindGroup, + clipRef: geo?._animationClipRefs?.[0] ?? null, + }); + } + } + }; + }, + schedule: { during: ["preUpdate"] }, + }, + skinningMatrixSystem: { + create: db => { + let scratch = new Float32Array(0); + return () => { + const { device } = db.store.resources; + if (!device) return; + for (const arch of db.store.queryArchetypes([ + "_skeletonJoints", + "_skeletonGeometry", + "_skeletonModelRef", + "_skeletonInstanceBuffer", + "_skeletonJointMatrixBuffer", + ])) { + const jointsCol = arch.columns._skeletonJoints; + const geoCol = arch.columns._skeletonGeometry; + const modelCol = arch.columns._skeletonModelRef; + const instBufCol = arch.columns._skeletonInstanceBuffer; + const bufCol = arch.columns._skeletonJointMatrixBuffer; + for (let i = 0; i < arch.rowCount; i++) { + const joints = jointsCol.get(i); + const geoId = geoCol.get(i); + const modelId = modelCol.get(i); + const instanceBuffer = instBufCol.get(i); + const buffer = bufCol.get(i); + if (!buffer || !instanceBuffer) continue; + const ibm = db.store.get(geoId, "_skinInverseBindMatrices"); + if (!ibm) continue; + const modelWorld = db.store.get(modelId, "_worldMatrix"); + if (!modelWorld) continue; + + // The vertex shader's `instances[0]` slot. + device.queue.writeBuffer(instanceBuffer, 0, new Float32Array(modelWorld)); + + const invModelWorld = Mat4x4.inverse(modelWorld); + if (scratch.length < joints.length * 16) { + scratch = new Float32Array(joints.length * 16); + } + for (let j = 0; j < joints.length; j++) { + const jw = db.store.get(joints[j], "_worldMatrix") ?? Mat4x4.identity; + // inverse(modelWorld) × jointWorld × IBM — leaves vertices in + // model-local space; vertex shader applies modelWorld separately. + const ibmJoint: Mat4x4 = [ + ibm[j * 16 + 0], ibm[j * 16 + 1], ibm[j * 16 + 2], ibm[j * 16 + 3], + ibm[j * 16 + 4], ibm[j * 16 + 5], ibm[j * 16 + 6], ibm[j * 16 + 7], + ibm[j * 16 + 8], ibm[j * 16 + 9], ibm[j * 16 + 10], ibm[j * 16 + 11], + ibm[j * 16 + 12], ibm[j * 16 + 13], ibm[j * 16 + 14], ibm[j * 16 + 15], + ]; + const m = Mat4x4.multiply(Mat4x4.multiply(invModelWorld, jw), ibmJoint); + scratch.set(m, j * 16); + } + device.queue.writeBuffer(buffer, 0, scratch, 0, joints.length * 16); + } + } + }; + }, + schedule: { during: ["preRender"], after: ["transformSystem"] }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/rendering/standard-vertex/layout.ts b/packages/data-gpu/src/graphics/rendering/standard-vertex/layout.ts new file mode 100644 index 00000000..376da5ef --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/standard-vertex/layout.ts @@ -0,0 +1,19 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { getStructLayout } from "@adobe/data/typed-buffer"; +import { schema } from "./schema.js"; + +const sl = getStructLayout(schema); + +export const layout: GPUVertexBufferLayout = { + arrayStride: sl.size, + stepMode: "vertex", + attributes: [ + { format: "float32x3", offset: sl.fields["position"]!.offset, shaderLocation: 0 }, + { format: "float32x3", offset: sl.fields["normal"]!.offset, shaderLocation: 1 }, + { format: "float32x4", offset: sl.fields["tangent"]!.offset, shaderLocation: 2 }, + { format: "float32x2", offset: sl.fields["uv"]!.offset, shaderLocation: 3 }, + ], +}; + +export const stride = sl.size; diff --git a/packages/data-gpu/src/graphics/rendering/standard-vertex/public.ts b/packages/data-gpu/src/graphics/rendering/standard-vertex/public.ts new file mode 100644 index 00000000..fbcfe4cc --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/standard-vertex/public.ts @@ -0,0 +1,4 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export * from "./schema.js"; +export * from "./layout.js"; diff --git a/packages/data-gpu/src/graphics/rendering/standard-vertex/schema.ts b/packages/data-gpu/src/graphics/rendering/standard-vertex/schema.ts new file mode 100644 index 00000000..79b87e26 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/standard-vertex/schema.ts @@ -0,0 +1,17 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Vec2, Vec3, Vec4 } from "@adobe/data/math"; +import { Schema } from "@adobe/data/schema"; + +export const schema = { + type: "object", + properties: { + position: Vec3.schema, + normal: Vec3.schema, + tangent: Vec4.schema, + uv: Vec2.schema, + }, + required: ["position", "normal", "tangent", "uv"], + additionalProperties: false, + layout: "packed", +} as const satisfies Schema; diff --git a/packages/data-gpu/src/graphics/rendering/standard-vertex/standard-vertex.ts b/packages/data-gpu/src/graphics/rendering/standard-vertex/standard-vertex.ts new file mode 100644 index 00000000..37ad1eab --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/standard-vertex/standard-vertex.ts @@ -0,0 +1,8 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; +import { schema } from "./schema.js"; + +export type StandardVertex = Schema.ToType; + +export * as StandardVertex from "./public.js"; diff --git a/packages/data-gpu/src/graphics/rendering/visible-material/color-material-options.ts b/packages/data-gpu/src/graphics/rendering/visible-material/color-material-options.ts new file mode 100644 index 00000000..e7be629c --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/visible-material/color-material-options.ts @@ -0,0 +1,10 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Vec3 } from "@adobe/data/math"; + +export interface ColorMaterialOptions { + color: readonly [number, number, number, number]; + emissive?: Vec3; + metallic?: number; + roughness?: number; +} diff --git a/packages/data-gpu/src/graphics/rendering/visible-material/create-bind-group-layout.ts b/packages/data-gpu/src/graphics/rendering/visible-material/create-bind-group-layout.ts new file mode 100644 index 00000000..503c2594 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/visible-material/create-bind-group-layout.ts @@ -0,0 +1,35 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +/** + * Bind group layout for a visible (PBR-style) material: a uniform buffer + * with the material factors, five 2D textures (baseColor, metallicRoughness, + * normal, occlusion, emissive), and a filtering sampler. Stable across + * renderer pipeline creation and the bind-group builders in `_modelLoader` + * and `VisibleMaterial.createColorBindGroup`. + */ +export function createBindGroupLayout(device: GPUDevice): GPUBindGroupLayout { + const textureEntry = (binding: number): GPUBindGroupLayoutEntry => ({ + binding, + visibility: GPUShaderStage.FRAGMENT, + texture: { sampleType: "float", viewDimension: "2d" }, + }); + return device.createBindGroupLayout({ + entries: [ + { + binding: 0, + visibility: GPUShaderStage.FRAGMENT, + buffer: { type: "uniform" }, + }, + textureEntry(1), + textureEntry(2), + textureEntry(3), + textureEntry(4), + textureEntry(5), + { + binding: 6, + visibility: GPUShaderStage.FRAGMENT, + sampler: { type: "filtering" }, + }, + ], + }); +} diff --git a/packages/data-gpu/src/graphics/rendering/visible-material/create-color-bind-group.ts b/packages/data-gpu/src/graphics/rendering/visible-material/create-color-bind-group.ts new file mode 100644 index 00000000..dc8ca92d --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/visible-material/create-color-bind-group.ts @@ -0,0 +1,40 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { createBindGroupLayout } from "./create-bind-group-layout.js"; +import type { ColorMaterialOptions } from "./color-material-options.js"; +import { createColorMaterial } from "../../scene/model/gltf/create-color-material.js"; +import { createFallbackTextures } from "../../scene/model/gltf/decode-images.js"; +import type { FallbackViews } from "../../scene/model/gltf/build-material-bind-group.js"; + +interface DeviceCache { + layout: GPUBindGroupLayout; + sampler: GPUSampler; + fallback: FallbackViews; +} + +const cacheByDevice = new WeakMap(); + +/** + * Builds a PBR material bind group from a flat color spec. The layout, + * sampler, and fallback textures are cached per-device, so calling this + * once per sphere/cube is cheap. + * + * The returned bind group is layout-compatible with the standard PBR + * pipeline (same layout structure as glTF-loaded materials). + */ +export function createColorBindGroup( + device: GPUDevice, + options: ColorMaterialOptions, +): GPUBindGroup { + let cache = cacheByDevice.get(device); + if (!cache) { + cache = { + layout: createBindGroupLayout(device), + sampler: device.createSampler({ magFilter: "linear", minFilter: "linear", mipmapFilter: "linear" }), + fallback: createFallbackTextures(device), + }; + cacheByDevice.set(device, cache); + } + return createColorMaterial(device, cache.layout, cache.sampler, cache.fallback, options); +} + diff --git a/packages/data-gpu/src/graphics/rendering/visible-material/public.ts b/packages/data-gpu/src/graphics/rendering/visible-material/public.ts new file mode 100644 index 00000000..de419d36 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/visible-material/public.ts @@ -0,0 +1,6 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export * from "./schema.js"; +export { createBindGroupLayout } from "./create-bind-group-layout.js"; +export { createColorBindGroup } from "./create-color-bind-group.js"; +export type { ColorMaterialOptions } from "./color-material-options.js"; diff --git a/packages/data-gpu/src/graphics/rendering/visible-material/schema.ts b/packages/data-gpu/src/graphics/rendering/visible-material/schema.ts new file mode 100644 index 00000000..4a54237a --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/visible-material/schema.ts @@ -0,0 +1,25 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { F32, Vec3, Vec4 } from "@adobe/data/math"; +import { Schema } from "@adobe/data/schema"; + +export const schema = { + type: "object", + properties: { + baseColorFactor: Vec4.schema, + emissiveFactor: Vec3.schema, + metallicFactor: F32.schema, + roughnessFactor: F32.schema, + normalScale: F32.schema, + occlusionStrength: F32.schema, + }, + required: [ + "baseColorFactor", + "emissiveFactor", + "metallicFactor", + "roughnessFactor", + "normalScale", + "occlusionStrength", + ], + additionalProperties: false, +} as const satisfies Schema; diff --git a/packages/data-gpu/src/graphics/rendering/visible-material/visible-material.ts b/packages/data-gpu/src/graphics/rendering/visible-material/visible-material.ts new file mode 100644 index 00000000..e74cb872 --- /dev/null +++ b/packages/data-gpu/src/graphics/rendering/visible-material/visible-material.ts @@ -0,0 +1,8 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; +import { schema } from "./schema.js"; + +export type VisibleMaterial = Schema.ToType; + +export * as VisibleMaterial from "./public.js"; diff --git a/packages/data-gpu/src/graphics/scene/light/light-plugin.ts b/packages/data-gpu/src/graphics/scene/light/light-plugin.ts new file mode 100644 index 00000000..d9aba525 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/light/light-plugin.ts @@ -0,0 +1,37 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Vec3 } from "@adobe/data/math"; +import type { Light } from "./light.js"; + +/** + * Authored lighting state — a directional light plus optional image-based + * lighting from an HDR environment. + * + * The `sceneUniforms` system packs these into the GPU uniform buffer the + * renderers consume; the `iblInitSystem` fetches and bakes the IBL textures + * from `light.environmentUrl`. + */ +export const light = Database.Plugin.create({ + resources: { + light: { + default: { + direction: Vec3.normalize([-1, -3, -10]), + color: [1.0, 1.0, 1.0], + ambientStrength: 0.5, + environmentUrl: null, + } satisfies Light as Light, + }, + }, + transactions: { + setLight(t, args: Partial) { + const cur = t.resources.light; + t.resources.light = { + direction: args.direction !== undefined ? Vec3.normalize(args.direction) : cur.direction, + color: args.color ?? cur.color, + ambientStrength: args.ambientStrength ?? cur.ambientStrength, + environmentUrl: args.environmentUrl !== undefined ? args.environmentUrl : cur.environmentUrl, + }; + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/scene/light/light.ts b/packages/data-gpu/src/graphics/scene/light/light.ts new file mode 100644 index 00000000..a99086b4 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/light/light.ts @@ -0,0 +1,18 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Vec3 } from "@adobe/data/math"; + +/** + * Lighting state — one directional light plus optional image-based lighting + * from an HDR environment map. Setting `environmentUrl` triggers the IBL + * bake; it's the only field consumed for setup rather than as direct shader + * input. + */ +export interface Light { + direction: Vec3; + color: Vec3; + ambientStrength: number; + environmentUrl: string | null; +} + +export * as Light from "./public.js"; diff --git a/packages/data-gpu/src/graphics/scene/light/public.ts b/packages/data-gpu/src/graphics/scene/light/public.ts new file mode 100644 index 00000000..bc453f4c --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/light/public.ts @@ -0,0 +1 @@ +export { light as plugin } from "./light-plugin.js"; diff --git a/packages/data-gpu/src/graphics/scene/model/geometry/geometry.ts b/packages/data-gpu/src/graphics/scene/model/geometry/geometry.ts new file mode 100644 index 00000000..2d51a68d --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/geometry/geometry.ts @@ -0,0 +1,12 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +/** + * One row of the `Geometry` archetype — an asset identified by URL. The + * model loader fetches the URL, then writes derived state (`_bounds`, + * primitive entities, animation clips) onto the same row. + */ +export interface Geometry { + modelUrl: string; +} + +export * as Geometry from "./public.js"; diff --git a/packages/data-gpu/src/graphics/scene/model/geometry/public.ts b/packages/data-gpu/src/graphics/scene/model/geometry/public.ts new file mode 100644 index 00000000..4aa281a4 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/geometry/public.ts @@ -0,0 +1,4 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +// Reserved for future Geometry helpers (e.g. bounds queries). +export {}; diff --git a/packages/data-gpu/src/graphics/scene/model/gltf/accessor-view.ts b/packages/data-gpu/src/graphics/scene/model/gltf/accessor-view.ts new file mode 100644 index 00000000..fb1b1d0a --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/gltf/accessor-view.ts @@ -0,0 +1,53 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { + ACCESSOR_TYPE_COMPONENTS, + COMPONENT_TYPE, + type GltfAccessor, + type GltfAsset, +} from "./gltf-schema.js"; + +/** + * Returns a typed-array view over the buffer slice referenced by an accessor. + * Assumes tightly packed data (no byteStride). For interleaved attributes a + * different reader would be needed — glTF primitives commonly use separate buffer + * views per attribute so this is sufficient for the standard case. + */ +export function readAccessor( + gltf: GltfAsset, + bin: ArrayBuffer, + accessorIndex: number, +): Float32Array | Uint16Array | Uint32Array | Uint8Array { + const accessor = gltf.accessors?.[accessorIndex]; + if (!accessor) throw new Error(`Accessor ${accessorIndex} not found`); + const bv = gltf.bufferViews?.[accessor.bufferView ?? -1]; + if (!bv) throw new Error(`BufferView for accessor ${accessorIndex} not found`); + + const componentCount = ACCESSOR_TYPE_COMPONENTS[accessor.type]; + const totalElements = accessor.count * componentCount; + const offset = (bv.byteOffset ?? 0) + (accessor.byteOffset ?? 0); + + switch (accessor.componentType) { + case COMPONENT_TYPE.FLOAT: + return new Float32Array(bin, offset, totalElements); + case COMPONENT_TYPE.UNSIGNED_SHORT: + return new Uint16Array(bin, offset, totalElements); + case COMPONENT_TYPE.UNSIGNED_INT: + return new Uint32Array(bin, offset, totalElements); + case COMPONENT_TYPE.UNSIGNED_BYTE: + return new Uint8Array(bin, offset, totalElements); + default: + throw new Error(`Unsupported componentType ${accessor.componentType}`); + } +} + +export function readImageBytes(gltf: GltfAsset, bin: ArrayBuffer, imageIndex: number): Uint8Array { + const image = gltf.images?.[imageIndex]; + if (!image) throw new Error(`Image ${imageIndex} not found`); + if (image.bufferView === undefined) { + throw new Error(`Image ${imageIndex} has no bufferView (external URIs not supported)`); + } + const bv = gltf.bufferViews?.[image.bufferView]; + if (!bv) throw new Error(`BufferView for image ${imageIndex} not found`); + return new Uint8Array(bin, bv.byteOffset ?? 0, bv.byteLength); +} diff --git a/packages/data-gpu/src/graphics/scene/model/gltf/build-material-bind-group.ts b/packages/data-gpu/src/graphics/scene/model/gltf/build-material-bind-group.ts new file mode 100644 index 00000000..e7aaa718 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/gltf/build-material-bind-group.ts @@ -0,0 +1,110 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { createStructBuffer, copyToGPUBuffer, getStructLayout, type TypedBuffer } from "@adobe/data/typed-buffer"; +import { schema as visibleMaterialSchema } from "../../../rendering/visible-material/schema.js"; +import type { VisibleMaterial } from "../../../rendering/visible-material/visible-material.js"; +import type { GltfAsset, GltfMaterial } from "./gltf-schema.js"; + +const layout = getStructLayout(visibleMaterialSchema); + +export interface MaterialTextures { + baseColor: GPUTextureView; + metallicRoughness: GPUTextureView; + normal: GPUTextureView; + occlusion: GPUTextureView; + emissive: GPUTextureView; +} + +export interface FallbackViews { + white: GPUTextureView; + black: GPUTextureView; + flatNormal: GPUTextureView; +} + +function defaultMaterial(): VisibleMaterial { + return { + baseColorFactor: [1, 1, 1, 1], + emissiveFactor: [0, 0, 0], + metallicFactor: 1, + roughnessFactor: 1, + normalScale: 1, + occlusionStrength: 1, + }; +} + +function gltfToVisibleMaterial(m: GltfMaterial): VisibleMaterial { + const pbr = m.pbrMetallicRoughness; + return { + baseColorFactor: pbr?.baseColorFactor ?? [1, 1, 1, 1], + emissiveFactor: m.emissiveFactor ?? [0, 0, 0], + metallicFactor: pbr?.metallicFactor ?? 1, + roughnessFactor: pbr?.roughnessFactor ?? 1, + normalScale: m.normalTexture?.scale ?? 1, + occlusionStrength: m.occlusionTexture?.strength ?? 1, + }; +} + +function viewFor( + textures: GPUTexture[], + gltf: GltfAsset, + textureIndex: number | undefined, + fallback: GPUTextureView, +): GPUTextureView { + if (textureIndex === undefined) return fallback; + const tex = gltf.textures?.[textureIndex]; + if (!tex || tex.source === undefined) return fallback; + const gpuTexture = textures[tex.source]; + if (!gpuTexture) return fallback; + return gpuTexture.createView(); +} + +/** + * Builds the per-primitive material bind group: a small uniform buffer holding + * the PBR factors plus the five sampled textures and a shared sampler. + * + * Missing textures fall back to neutral 1x1 textures so the shader always has + * a valid binding (white for baseColor/occlusion, black for emissive, flat + * (0.5, 0.5, 1) for the normal map). + */ +export function buildMaterialBindGroup( + device: GPUDevice, + gltf: GltfAsset, + sourceTextures: GPUTexture[], + fallback: FallbackViews, + sampler: GPUSampler, + layoutGpu: GPUBindGroupLayout, + materialIndex: number | undefined, +): GPUBindGroup { + const material = materialIndex !== undefined && gltf.materials?.[materialIndex] + ? gltfToVisibleMaterial(gltf.materials[materialIndex]) + : defaultMaterial(); + const gltfMat = materialIndex !== undefined ? gltf.materials?.[materialIndex] : undefined; + + const structBuffer: TypedBuffer = createStructBuffer(visibleMaterialSchema, layout.size); + structBuffer.set(0, material); + + let uniformBuffer = device.createBuffer({ + size: layout.size, + usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST, + }); + uniformBuffer = copyToGPUBuffer(structBuffer, device, uniformBuffer); + + const baseColorView = viewFor(sourceTextures, gltf, gltfMat?.pbrMetallicRoughness?.baseColorTexture?.index, fallback.white); + const mrView = viewFor(sourceTextures, gltf, gltfMat?.pbrMetallicRoughness?.metallicRoughnessTexture?.index, fallback.white); + const normalView = viewFor(sourceTextures, gltf, gltfMat?.normalTexture?.index, fallback.flatNormal); + const occlusionView = viewFor(sourceTextures, gltf, gltfMat?.occlusionTexture?.index, fallback.white); + const emissiveView = viewFor(sourceTextures, gltf, gltfMat?.emissiveTexture?.index, fallback.black); + + return device.createBindGroup({ + layout: layoutGpu, + entries: [ + { binding: 0, resource: { buffer: uniformBuffer } }, + { binding: 1, resource: baseColorView }, + { binding: 2, resource: mrView }, + { binding: 3, resource: normalView }, + { binding: 4, resource: occlusionView }, + { binding: 5, resource: emissiveView }, + { binding: 6, resource: sampler }, + ], + }); +} diff --git a/packages/data-gpu/src/graphics/scene/model/gltf/compute-world-matrices.ts b/packages/data-gpu/src/graphics/scene/model/gltf/compute-world-matrices.ts new file mode 100644 index 00000000..6b07529b --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/gltf/compute-world-matrices.ts @@ -0,0 +1,43 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Mat4x4, Quat } from "@adobe/data/math"; +import type { GltfAsset, GltfNode } from "./gltf-schema.js"; + +function localMatrix(node: GltfNode): Mat4x4 { + if (node.matrix && node.matrix.length === 16) { + // glTF spec guarantees a column-major 16-element float array when present. + return node.matrix as unknown as Mat4x4; + } + const t = node.translation ?? [0, 0, 0]; + const r = node.rotation ?? [0, 0, 0, 1]; + const s = node.scale ?? [1, 1, 1]; + const T = Mat4x4.translation(t[0], t[1], t[2]); + // glTF rotation is a unit quaternion [x, y, z, w]. + const R = Quat.toMat4(r as Quat); + const S = Mat4x4.scaling(s[0], s[1], s[2]); + return Mat4x4.multiply(Mat4x4.multiply(T, R), S); +} + +/** + * Walks the scene graph and returns a world matrix for every node index. + * Nodes not reachable from a scene root get the identity matrix. + */ +export function computeWorldMatrices(gltf: GltfAsset): Mat4x4[] { + const result: Mat4x4[] = new Array((gltf.nodes ?? []).length).fill(Mat4x4.identity); + + const visit = (nodeIdx: number, parent: Mat4x4): void => { + const node = gltf.nodes![nodeIdx]; + const world = Mat4x4.multiply(parent, localMatrix(node)); + result[nodeIdx] = world; + for (const child of node.children ?? []) { + visit(child, world); + } + }; + + const sceneIdx = gltf.scene ?? 0; + const roots = gltf.scenes?.[sceneIdx]?.nodes ?? []; + for (const r of roots) { + visit(r, Mat4x4.identity); + } + return result; +} diff --git a/packages/data-gpu/src/graphics/scene/model/gltf/create-color-material.ts b/packages/data-gpu/src/graphics/scene/model/gltf/create-color-material.ts new file mode 100644 index 00000000..61fb967f --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/gltf/create-color-material.ts @@ -0,0 +1,47 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { createStructBuffer, copyToGPUBuffer, getStructLayout } from "@adobe/data/typed-buffer"; +import { schema as visibleMaterialSchema } from "../../../rendering/visible-material/schema.js"; +import type { VisibleMaterial } from "../../../rendering/visible-material/visible-material.js"; +import type { ColorMaterialOptions } from "../../../rendering/visible-material/color-material-options.js"; +import type { FallbackViews } from "./build-material-bind-group.js"; + +const layout = getStructLayout(visibleMaterialSchema); + +export function createColorMaterial( + device: GPUDevice, + bindGroupLayout: GPUBindGroupLayout, + sampler: GPUSampler, + fallback: FallbackViews, + options: ColorMaterialOptions, +): GPUBindGroup { + const material: VisibleMaterial = { + baseColorFactor: options.color, + emissiveFactor: options.emissive ?? [0, 0, 0], + metallicFactor: options.metallic ?? 0, + roughnessFactor: options.roughness ?? 0.8, + normalScale: 1, + occlusionStrength: 1, + }; + + const structBuffer = createStructBuffer(visibleMaterialSchema, layout.size); + structBuffer.set(0, material); + let uniformBuffer = device.createBuffer({ + size: layout.size, + usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST, + }); + uniformBuffer = copyToGPUBuffer(structBuffer, device, uniformBuffer); + + return device.createBindGroup({ + layout: bindGroupLayout, + entries: [ + { binding: 0, resource: { buffer: uniformBuffer } }, + { binding: 1, resource: fallback.white }, + { binding: 2, resource: fallback.white }, + { binding: 3, resource: fallback.flatNormal }, + { binding: 4, resource: fallback.white }, + { binding: 5, resource: fallback.white }, + { binding: 6, resource: sampler }, + ], + }); +} diff --git a/packages/data-gpu/src/graphics/scene/model/gltf/decode-images.ts b/packages/data-gpu/src/graphics/scene/model/gltf/decode-images.ts new file mode 100644 index 00000000..47cdc4c3 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/gltf/decode-images.ts @@ -0,0 +1,93 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { readImageBytes } from "./accessor-view.js"; +import type { GltfAsset } from "./gltf-schema.js"; + +/** + * Categorizes each image index as either "color" (sRGB) or "data" (linear). + * glTF spec: baseColor and emissive textures are in sRGB color space; normal, + * metallic-roughness, and occlusion textures are in linear color space. + * + * When an image is referenced from both kinds of slot, "color" wins because + * losing precision in a data texture is more visible than gamma drift in a + * color one. + */ +function categorizeImages(gltf: GltfAsset): ("color" | "data")[] { + const out: ("color" | "data")[] = new Array((gltf.images ?? []).length).fill("data"); + + const markColor = (textureIndex: number | undefined): void => { + if (textureIndex === undefined) return; + const t = gltf.textures?.[textureIndex]; + if (t?.source !== undefined) out[t.source] = "color"; + }; + + for (const mat of gltf.materials ?? []) { + markColor(mat.pbrMetallicRoughness?.baseColorTexture?.index); + markColor(mat.emissiveTexture?.index); + } + + return out; +} + +async function decodeOne( + device: GPUDevice, + bytes: Uint8Array, + mimeType: string | undefined, + colorSpace: "color" | "data", +): Promise { + const blob = new Blob([bytes], { type: mimeType ?? "image/png" }); + const bitmap = await createImageBitmap(blob, { colorSpaceConversion: "none" }); + const format: GPUTextureFormat = colorSpace === "color" ? "rgba8unorm-srgb" : "rgba8unorm"; + const texture = device.createTexture({ + size: [bitmap.width, bitmap.height, 1], + format, + usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST | GPUTextureUsage.RENDER_ATTACHMENT, + }); + device.queue.copyExternalImageToTexture( + { source: bitmap }, + { texture }, + [bitmap.width, bitmap.height, 1], + ); + bitmap.close(); + return texture; +} + +export async function decodeAllImages( + device: GPUDevice, + gltf: GltfAsset, + bin: ArrayBuffer, +): Promise { + const kinds = categorizeImages(gltf); + return Promise.all( + (gltf.images ?? []).map((image, idx) => { + const bytes = readImageBytes(gltf, bin, idx); + return decodeOne(device, bytes, image.mimeType, kinds[idx]); + }), + ); +} + +/** + * Creates 1x1 fallback textures that the renderer can bind when a material + * doesn't supply a particular slot. Avoids "if texture exists" branches in + * the shader. + */ +export function createFallbackTextures(device: GPUDevice): { + white: GPUTextureView; + black: GPUTextureView; + flatNormal: GPUTextureView; +} { + const make = (pixel: Uint8Array, format: GPUTextureFormat): GPUTextureView => { + const tex = device.createTexture({ + size: [1, 1, 1], + format, + usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST, + }); + device.queue.writeTexture({ texture: tex }, pixel, { bytesPerRow: 4 }, [1, 1, 1]); + return tex.createView(); + }; + return { + white: make(new Uint8Array([255, 255, 255, 255]), "rgba8unorm-srgb"), + black: make(new Uint8Array([0, 0, 0, 255]), "rgba8unorm-srgb"), + flatNormal: make(new Uint8Array([128, 128, 255, 255]), "rgba8unorm"), + }; +} diff --git a/packages/data-gpu/src/graphics/scene/model/gltf/gltf-schema.ts b/packages/data-gpu/src/graphics/scene/model/gltf/gltf-schema.ts new file mode 100644 index 00000000..856e03e7 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/gltf/gltf-schema.ts @@ -0,0 +1,161 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +// Minimal subset of glTF 2.0 schema fields we actually consume. +// Spec: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html + +export interface GltfAsset { + asset: { version: string }; + scene?: number; + scenes?: GltfScene[]; + nodes?: GltfNode[]; + meshes?: GltfMesh[]; + skins?: GltfSkin[]; + materials?: GltfMaterial[]; + textures?: GltfTexture[]; + images?: GltfImage[]; + samplers?: GltfSampler[]; + accessors?: GltfAccessor[]; + bufferViews?: GltfBufferView[]; + buffers?: GltfBuffer[]; + animations?: GltfAnimation[]; +} + +export interface GltfAnimation { + name?: string; + channels: GltfAnimationChannel[]; + samplers: GltfAnimationSampler[]; +} + +export interface GltfAnimationChannel { + sampler: number; + target: { node: number; path: "translation" | "rotation" | "scale" | "weights" }; +} + +export interface GltfAnimationSampler { + input: number; // accessor → keyframe times (Float32Array) + output: number; // accessor → keyframe values + interpolation?: "LINEAR" | "STEP" | "CUBICSPLINE"; +} + +export interface GltfScene { + nodes?: number[]; +} + +export interface GltfNode { + name?: string; + mesh?: number; + skin?: number; + children?: number[]; + translation?: [number, number, number]; + rotation?: [number, number, number, number]; + scale?: [number, number, number]; + matrix?: number[]; +} + +export interface GltfMesh { + name?: string; + primitives: GltfPrimitive[]; +} + +export interface GltfPrimitive { + attributes: { + POSITION: number; + NORMAL?: number; + TANGENT?: number; + TEXCOORD_0?: number; + JOINTS_0?: number; + WEIGHTS_0?: number; + [key: string]: number | undefined; + }; + indices?: number; + material?: number; + mode?: number; // default 4 (triangles) +} + +export interface GltfSkin { + name?: string; + inverseBindMatrices?: number; // accessor index → array of mat4 + skeleton?: number; // root joint node index + joints: number[]; // ordered list of node indices that are joints +} + +export interface GltfMaterial { + name?: string; + pbrMetallicRoughness?: { + baseColorFactor?: [number, number, number, number]; + baseColorTexture?: { index: number; texCoord?: number }; + metallicFactor?: number; + roughnessFactor?: number; + metallicRoughnessTexture?: { index: number; texCoord?: number }; + }; + normalTexture?: { index: number; texCoord?: number; scale?: number }; + occlusionTexture?: { index: number; texCoord?: number; strength?: number }; + emissiveTexture?: { index: number; texCoord?: number }; + emissiveFactor?: [number, number, number]; + alphaMode?: "OPAQUE" | "MASK" | "BLEND"; + alphaCutoff?: number; + doubleSided?: boolean; +} + +export interface GltfTexture { + sampler?: number; + source?: number; +} + +export interface GltfImage { + name?: string; + mimeType?: string; + bufferView?: number; + uri?: string; +} + +export interface GltfSampler { + magFilter?: number; + minFilter?: number; + wrapS?: number; + wrapT?: number; +} + +export interface GltfAccessor { + bufferView?: number; + byteOffset?: number; + componentType: number; // 5120..5126 + count: number; + type: "SCALAR" | "VEC2" | "VEC3" | "VEC4" | "MAT2" | "MAT3" | "MAT4"; + normalized?: boolean; + min?: number[]; + max?: number[]; +} + +export interface GltfBufferView { + buffer: number; + byteOffset?: number; + byteLength: number; + byteStride?: number; + target?: number; +} + +export interface GltfBuffer { + byteLength: number; + uri?: string; +} + +// glTF componentType constants +export const COMPONENT_TYPE = { + BYTE: 5120, + UNSIGNED_BYTE: 5121, + SHORT: 5122, + UNSIGNED_SHORT: 5123, + UNSIGNED_INT: 5125, + FLOAT: 5126, +} as const; + +export const ACCESSOR_TYPE_COMPONENTS: Record = { + SCALAR: 1, + VEC2: 2, + VEC3: 3, + VEC4: 4, + MAT2: 4, + MAT3: 9, + MAT4: 16, +}; diff --git a/packages/data-gpu/src/graphics/scene/model/gltf/load-gltf-model.ts b/packages/data-gpu/src/graphics/scene/model/gltf/load-gltf-model.ts new file mode 100644 index 00000000..0993b328 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/gltf/load-gltf-model.ts @@ -0,0 +1,220 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Mat4x4, type Aabb } from "@adobe/data/math"; +import { VisibleMaterial } from "../../../rendering/visible-material/visible-material.js"; +import { readAccessor } from "./accessor-view.js"; +import { buildMaterialBindGroup, type FallbackViews } from "./build-material-bind-group.js"; +import { computeWorldMatrices } from "./compute-world-matrices.js"; +import { createFallbackTextures, decodeAllImages } from "./decode-images.js"; +import { packPrimitiveVertices } from "./pack-vertex-buffer.js"; +import { parseGlb } from "./parse-glb.js"; +import { parseGltfSkin, type LoadedSkin } from "./parse-skin.js"; +import { parseGltfAnimations, type LoadedAnimation } from "./parse-animations.js"; + +export interface GpuPrimitiveData { + pbrVertexBuffer: GPUBuffer; + /** Secondary VBO with skinning attributes (joints u32×4, weights f32×4). + * Null for non-skinned primitives. Drives the renderer's pipeline choice. */ + pbrSkinVertexBuffer: GPUBuffer | null; + pbrIndexBuffer: GPUBuffer; + pbrIndexCount: number; + pbrIndexFormat: GPUIndexFormat; + pbrMaterialBindGroup: GPUBindGroup; + /** Node's model-root-local matrix. Renderers pre-multiply this with the + * per-instance model-root world matrix at draw time. Identity for + * single-node and skinned-mesh primitives (skin owns the deformation). */ + pbrNodeLocalMatrix: Mat4x4; +} + +export interface LoadedGltfData { + primitives: GpuPrimitiveData[]; + bounds: Aabb; + /** Present when the glTF declares a `skins[0]`. */ + skin: LoadedSkin | null; + /** Parsed `animations[]`; jointIndex on each track is into `skin.jointTemplate`. */ + animations: LoadedAnimation[]; + /** Model-space collision geometry retained on the CPU for auto-generating + * physics colliders (convex hull / trimesh) — the non-skinned primitives' + * positions (each baked by its node matrix) + indices, aggregated. Null when + * the model has no static geometry (e.g. skin-only). */ + collision: { positions: Float32Array; indices: Uint32Array } | null; + /** Skinned vertices retained on the CPU (mesh-bind positions + 4 joint + * indices + 4 weights per vertex), for fitting per-bone ragdoll capsules. + * Null for non-skinned models. */ + skinVertices: { positions: Float32Array; joints: Uint32Array; weights: Float32Array } | null; +} + +function expandBounds( + m: Mat4x4, + localMin: readonly [number, number, number], + localMax: readonly [number, number, number], + outMin: [number, number, number], + outMax: [number, number, number], +): void { + const xs = [localMin[0], localMax[0]]; + const ys = [localMin[1], localMax[1]]; + const zs = [localMin[2], localMax[2]]; + for (const lx of xs) for (const ly of ys) for (const lz of zs) { + const wx = m[0] * lx + m[4] * ly + m[8] * lz + m[12]; + const wy = m[1] * lx + m[5] * ly + m[9] * lz + m[13]; + const wz = m[2] * lx + m[6] * ly + m[10] * lz + m[14]; + if (wx < outMin[0]) outMin[0] = wx; if (wx > outMax[0]) outMax[0] = wx; + if (wy < outMin[1]) outMin[1] = wy; if (wy > outMax[1]) outMax[1] = wy; + if (wz < outMin[2]) outMin[2] = wz; if (wz > outMax[2]) outMax[2] = wz; + } +} + +function pickIndexFormat(raw: Uint16Array | Uint32Array | Uint8Array | Float32Array): { + indices: Uint16Array | Uint32Array; + format: GPUIndexFormat; +} { + if (raw instanceof Uint32Array) return { indices: raw, format: "uint32" }; + if (raw instanceof Uint16Array) return { indices: raw, format: "uint16" }; + if (raw instanceof Uint8Array) { + const u16 = new Uint16Array(raw.length); + for (let i = 0; i < raw.length; i++) u16[i] = raw[i]; + return { indices: u16, format: "uint16" }; + } + throw new Error("Index accessor must be an integer type"); +} + +export async function loadGltfPrimitives(device: GPUDevice, url: string): Promise { + const response = await fetch(url); + if (!response.ok) throw new Error(`Failed to fetch ${url}: ${response.status}`); + const buffer = await response.arrayBuffer(); + + const { json, bin } = parseGlb(buffer); + + const materialLayout = VisibleMaterial.createBindGroupLayout(device); + const sourceTextures = await decodeAllImages(device, json, bin); + const fallback: FallbackViews = createFallbackTextures(device); + + const sampler = device.createSampler({ + magFilter: "linear", + minFilter: "linear", + addressModeU: "repeat", + addressModeV: "repeat", + }); + + const worldMatrices = computeWorldMatrices(json); + const skin = parseGltfSkin(json, bin); + const animations = skin ? parseGltfAnimations(json, bin, json.skins![0].joints) : []; + + const boundsMin: [number, number, number] = [Infinity, Infinity, Infinity]; + const boundsMax: [number, number, number] = [-Infinity, -Infinity, -Infinity]; + const primitives: GpuPrimitiveData[] = []; + // CPU-retained collision geometry (model space, non-skinned primitives only). + const collPositions: number[] = [], collIndices: number[] = []; + // CPU-retained skin vertices (mesh-bind space, skinned primitives only). + const skinPos: number[] = [], skinJoints: number[] = [], skinWeights: number[] = []; + + for (let nodeIdx = 0; nodeIdx < (json.nodes ?? []).length; nodeIdx++) { + const node = json.nodes![nodeIdx]; + if (node.mesh === undefined) continue; + const mesh = json.meshes![node.mesh]; + // For skinned primitives the joint transforms own the deformation; + // the mesh node's own transform must not be baked in. For non-skinned + // primitives we pre-bake the node's world matrix as before. + const skinned = node.skin !== undefined; + const pbrNodeLocalMatrix = skinned ? Mat4x4.identity : worldMatrices[nodeIdx]; + + for (const prim of mesh.primitives) { + const packed = packPrimitiveVertices(json, bin, prim); + + expandBounds(pbrNodeLocalMatrix, packed.boundsMin, packed.boundsMax, boundsMin, boundsMax); + + const vertexBuffer = device.createBuffer({ + size: packed.vertices.byteLength, + usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(vertexBuffer, 0, packed.vertices); + + let skinVertexBuffer: GPUBuffer | null = null; + if (packed.skinningAttributes) { + skinVertexBuffer = device.createBuffer({ + size: packed.skinningAttributes.byteLength, + usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(skinVertexBuffer, 0, packed.skinningAttributes); + } + + // Non-indexed primitives are valid in glTF — synthesize sequential indices. + let indices: Uint16Array | Uint32Array; + let format: GPUIndexFormat; + if (prim.indices === undefined) { + if (packed.vertexCount <= 0xffff) { + indices = new Uint16Array(packed.vertexCount); + for (let k = 0; k < packed.vertexCount; k++) indices[k] = k; + format = "uint16"; + } else { + indices = new Uint32Array(packed.vertexCount); + for (let k = 0; k < packed.vertexCount; k++) indices[k] = k; + format = "uint32"; + } + } else { + const raw = readAccessor(json, bin, prim.indices); + ({ indices, format } = pickIndexFormat(raw)); + } + + const indexBuffer = device.createBuffer({ + size: indices.byteLength, + usage: GPUBufferUsage.INDEX | GPUBufferUsage.COPY_DST, + }); + device.queue.writeBuffer(indexBuffer, 0, indices); + + // Retain collision geometry for non-skinned primitives: deinterleave the + // positions (stride = floats/vertex, position at offset 0), bake the node + // matrix into model space, and append with index offset. Skinned primitives + // deform at runtime, so a static collider from their bind pose is meaningless. + if (!skinned) { + const m = pbrNodeLocalMatrix, verts = packed.vertices; + const stride = packed.vertices.length / packed.vertexCount; + const vbase = collPositions.length / 3; + for (let v = 0; v < packed.vertexCount; v++) { + const o = v * stride, x = verts[o], y = verts[o + 1], z = verts[o + 2]; + collPositions.push( + m[0] * x + m[4] * y + m[8] * z + m[12], + m[1] * x + m[5] * y + m[9] * z + m[13], + m[2] * x + m[6] * y + m[10] * z + m[14], + ); + } + for (let k = 0; k < indices.length; k++) collIndices.push(indices[k] + vbase); + } else if (packed.skinningAttributes) { + // Retain skin vertices (mesh-bind positions + joints/weights) for + // fitting per-bone ragdoll capsules. Skinning attributes pack 8 words + // per vertex: joints (u32×4) in words 0–3, weights (f32×4) in 4–7. + const verts = packed.vertices, stride = packed.vertices.length / packed.vertexCount; + const sj = new Uint32Array(packed.skinningAttributes), sw = new Float32Array(packed.skinningAttributes); + for (let v = 0; v < packed.vertexCount; v++) { + const o = v * stride, s = v * 8; + skinPos.push(verts[o], verts[o + 1], verts[o + 2]); + skinJoints.push(sj[s], sj[s + 1], sj[s + 2], sj[s + 3]); + skinWeights.push(sw[s + 4], sw[s + 5], sw[s + 6], sw[s + 7]); + } + } + + const materialBindGroup = buildMaterialBindGroup( + device, json, sourceTextures, fallback, sampler, materialLayout, prim.material, + ); + + primitives.push({ + pbrVertexBuffer: vertexBuffer, + pbrSkinVertexBuffer: skinVertexBuffer, + pbrIndexBuffer: indexBuffer, + pbrIndexCount: indices.length, + pbrIndexFormat: format, + pbrMaterialBindGroup: materialBindGroup, + pbrNodeLocalMatrix, + }); + } + } + + return { + primitives, + bounds: { min: boundsMin as [number, number, number], max: boundsMax as [number, number, number] }, + skin, + animations, + collision: collPositions.length ? { positions: new Float32Array(collPositions), indices: new Uint32Array(collIndices) } : null, + skinVertices: skinPos.length ? { positions: new Float32Array(skinPos), joints: new Uint32Array(skinJoints), weights: new Float32Array(skinWeights) } : null, + }; +} diff --git a/packages/data-gpu/src/graphics/scene/model/gltf/pack-vertex-buffer.ts b/packages/data-gpu/src/graphics/scene/model/gltf/pack-vertex-buffer.ts new file mode 100644 index 00000000..3136d375 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/gltf/pack-vertex-buffer.ts @@ -0,0 +1,116 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Vec3 } from "@adobe/data/math"; +import { stride as vertexStride } from "../../../rendering/standard-vertex/layout.js"; +import { SKINNING_STRIDE } from "../../../rendering/skinning/skinning-attributes/layout.js"; +import { readAccessor } from "./accessor-view.js"; +import type { GltfAsset, GltfPrimitive } from "./gltf-schema.js"; + +const FLOATS_PER_VERTEX = vertexStride / 4; // 48 / 4 = 12 + +type MutVec3 = [number, number, number]; + +export interface PackedPrimitive { + vertices: Float32Array; + /** Packed skinning attributes (uint32×4 joints, float32×4 weights). Null + * for non-skinned primitives. Size = vertexCount × {@link SKINNING_STRIDE}. */ + skinningAttributes: ArrayBuffer | null; + vertexCount: number; + boundsMin: Vec3; + boundsMax: Vec3; +} + +/** + * Reads POSITION/NORMAL/TANGENT/TEXCOORD_0 from a primitive and interleaves + * them into the packed StandardVertex layout (48 bytes / vertex). + * + * If JOINTS_0 and WEIGHTS_0 are present, also packs them into a separate + * skinning attribute buffer with stride {@link SKINNING_STRIDE}. + * + * Vertices are kept in node-local space — no world transform is applied here. + */ +export function packPrimitiveVertices( + gltf: GltfAsset, + bin: ArrayBuffer, + prim: GltfPrimitive, +): PackedPrimitive { + const positions = readAccessor(gltf, bin, prim.attributes.POSITION) as Float32Array; + // Normals and UVs are optional in glTF — fall back to defaults when absent. + // Skinned demo meshes (e.g. Fox) frequently omit NORMAL. + const normals = prim.attributes.NORMAL !== undefined + ? readAccessor(gltf, bin, prim.attributes.NORMAL) as Float32Array + : null; + const uvs = prim.attributes.TEXCOORD_0 !== undefined + ? readAccessor(gltf, bin, prim.attributes.TEXCOORD_0) as Float32Array + : null; + const tangents = prim.attributes.TANGENT !== undefined + ? readAccessor(gltf, bin, prim.attributes.TANGENT) as Float32Array + : null; + // glTF JOINTS_0 is unsigned byte vec4 or unsigned short vec4; readAccessor + // returns either Uint8Array or Uint16Array. WEIGHTS_0 is usually float32x4. + const joints = prim.attributes.JOINTS_0 !== undefined + ? readAccessor(gltf, bin, prim.attributes.JOINTS_0) as Uint8Array | Uint16Array | Uint32Array + : null; + const weights = prim.attributes.WEIGHTS_0 !== undefined + ? readAccessor(gltf, bin, prim.attributes.WEIGHTS_0) as Float32Array + : null; + const skinned = joints !== null && weights !== null; + + const vertexCount = positions.length / 3; + const out = new Float32Array(vertexCount * FLOATS_PER_VERTEX); + const skin = skinned ? new ArrayBuffer(vertexCount * SKINNING_STRIDE) : null; + const skinJointsView = skin ? new Uint32Array(skin) : null; + const skinWeightsView = skin ? new Float32Array(skin) : null; + + const min: MutVec3 = [Infinity, Infinity, Infinity]; + const max: MutVec3 = [-Infinity, -Infinity, -Infinity]; + + for (let i = 0; i < vertexCount; i++) { + const px = positions[i * 3], py = positions[i * 3 + 1], pz = positions[i * 3 + 2]; + const nx = normals ? normals[i * 3] : 0; + const ny = normals ? normals[i * 3 + 1] : 1; + const nz = normals ? normals[i * 3 + 2] : 0; + + let tx = 1, ty = 0, tz = 0, tw = 1; + if (tangents) { + tx = tangents[i * 4]; ty = tangents[i * 4 + 1]; + tz = tangents[i * 4 + 2]; tw = tangents[i * 4 + 3]; + } + + const u = uvs ? uvs[i * 2] : 0; + const v = uvs ? uvs[i * 2 + 1] : 0; + + const o = i * FLOATS_PER_VERTEX; + out[o + 0] = px; out[o + 1] = py; out[o + 2] = pz; + out[o + 3] = nx; out[o + 4] = ny; out[o + 5] = nz; + out[o + 6] = tx; out[o + 7] = ty; out[o + 8] = tz; out[o + 9] = tw; + out[o + 10] = u; out[o + 11] = v; + + if (skinned) { + // 32-byte stride / 4-byte words = 8 words per skinned vertex: + // words 0..3: joints (u32 each) + // words 4..7: weights (f32 each) + const so = i * 8; + skinJointsView![so + 0] = joints![i * 4 + 0]; + skinJointsView![so + 1] = joints![i * 4 + 1]; + skinJointsView![so + 2] = joints![i * 4 + 2]; + skinJointsView![so + 3] = joints![i * 4 + 3]; + skinWeightsView![so + 4] = weights![i * 4 + 0]; + skinWeightsView![so + 5] = weights![i * 4 + 1]; + skinWeightsView![so + 6] = weights![i * 4 + 2]; + skinWeightsView![so + 7] = weights![i * 4 + 3]; + } + + if (px < min[0]) min[0] = px; if (px > max[0]) max[0] = px; + if (py < min[1]) min[1] = py; if (py > max[1]) max[1] = py; + if (pz < min[2]) min[2] = pz; if (pz > max[2]) max[2] = pz; + } + + return { + vertices: out, + skinningAttributes: skin, + vertexCount, + boundsMin: min as Vec3, + boundsMax: max as Vec3, + }; +} diff --git a/packages/data-gpu/src/graphics/scene/model/gltf/parse-animations.ts b/packages/data-gpu/src/graphics/scene/model/gltf/parse-animations.ts new file mode 100644 index 00000000..aab9c1f7 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/gltf/parse-animations.ts @@ -0,0 +1,66 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { AnimationTrack } from "../../../animation/animation-track/animation-track.js"; +import type { InterpolationMode } from "../../../animation/interpolation-mode/interpolation-mode.js"; +import { readAccessor } from "./accessor-view.js"; +import type { GltfAsset } from "./gltf-schema.js"; + +export interface LoadedAnimation { + name?: string; + duration: number; + tracks: AnimationTrack[]; +} + +const PATH_TO_COMPONENT: Record = { + translation: "position", + rotation: "rotation", + scale: "scale", +}; + +const INTERP_MAP: Record = { + LINEAR: "linear", + STEP: "step", + CUBICSPLINE: "cubicSpline", +}; + +/** + * Parses glTF `animations[]` into clip data sized for our AnimationClip + * archetype. Each `track.targetIndex` is the **joint index** (position in + * `skin.joints`) — the skinning init system places joint entity IDs at those + * indices in the AnimationPlayer's `animationTargets` so the clip is portable + * across instances. + * + * Tracks targeting non-joint nodes or unsupported paths (e.g. morph weights) + * are skipped. + */ +export function parseGltfAnimations( + gltf: GltfAsset, + bin: ArrayBuffer, + jointNodeIndices: readonly number[], +): LoadedAnimation[] { + if (!gltf.animations || gltf.animations.length === 0) return []; + + const nodeToJoint = new Map(); + for (let j = 0; j < jointNodeIndices.length; j++) nodeToJoint.set(jointNodeIndices[j], j); + + return gltf.animations.map(anim => { + const tracks: AnimationTrack[] = []; + let duration = 0; + for (const channel of anim.channels) { + const component = PATH_TO_COMPONENT[channel.target.path]; + if (!component) continue; // skip morph "weights" etc. + const jointIdx = nodeToJoint.get(channel.target.node); + if (jointIdx === undefined) continue; + + const sampler = anim.samplers[channel.sampler]; + const times = new Float32Array(readAccessor(gltf, bin, sampler.input) as Float32Array); + const values = new Float32Array(readAccessor(gltf, bin, sampler.output) as Float32Array); + const interpolation = INTERP_MAP[sampler.interpolation ?? "LINEAR"] ?? "linear"; + + tracks.push({ targetIndex: jointIdx, component, times, values, interpolation }); + const lastTime = times[times.length - 1] ?? 0; + if (lastTime > duration) duration = lastTime; + } + return { name: anim.name, duration, tracks }; + }); +} diff --git a/packages/data-gpu/src/graphics/scene/model/gltf/parse-glb.ts b/packages/data-gpu/src/graphics/scene/model/gltf/parse-glb.ts new file mode 100644 index 00000000..fabd8112 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/gltf/parse-glb.ts @@ -0,0 +1,51 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { GltfAsset } from "./gltf-schema.js"; + +const MAGIC_GLTF = 0x46546c67; // "glTF" +const CHUNK_JSON = 0x4e4f534a; // "JSON" +const CHUNK_BIN = 0x004e4942; // "BIN\0" + +export interface ParsedGlb { + json: GltfAsset; + bin: ArrayBuffer; +} + +export function parseGlb(buffer: ArrayBuffer): ParsedGlb { + if (buffer.byteLength < 12) { + throw new Error("GLB too short to contain header"); + } + const view = new DataView(buffer); + const magic = view.getUint32(0, true); + if (magic !== MAGIC_GLTF) { + throw new Error(`Not a GLB file (magic = 0x${magic.toString(16)})`); + } + const version = view.getUint32(4, true); + if (version !== 2) { + throw new Error(`Unsupported GLB version ${version}`); + } + + let offset = 12; + let json: GltfAsset | null = null; + let bin: ArrayBuffer | null = null; + + while (offset < buffer.byteLength) { + const chunkLength = view.getUint32(offset, true); + const chunkType = view.getUint32(offset + 4, true); + const chunkStart = offset + 8; + const chunkEnd = chunkStart + chunkLength; + + if (chunkType === CHUNK_JSON) { + const bytes = new Uint8Array(buffer, chunkStart, chunkLength); + const text = new TextDecoder().decode(bytes); + json = JSON.parse(text) as GltfAsset; + } else if (chunkType === CHUNK_BIN) { + bin = buffer.slice(chunkStart, chunkEnd); + } + + offset = chunkEnd; + } + + if (!json) throw new Error("GLB missing JSON chunk"); + return { json, bin: bin ?? new ArrayBuffer(0) }; +} diff --git a/packages/data-gpu/src/graphics/scene/model/gltf/parse-skin.ts b/packages/data-gpu/src/graphics/scene/model/gltf/parse-skin.ts new file mode 100644 index 00000000..8af64b53 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/gltf/parse-skin.ts @@ -0,0 +1,67 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Quat, Vec3 } from "@adobe/data/math"; +import { readAccessor } from "./accessor-view.js"; +import type { GltfAsset } from "./gltf-schema.js"; + +export interface JointTemplate { + /** Local-space TRS in glTF node space (relative to joint parent). */ + position: Vec3; + rotation: Quat; + scale: Vec3; + /** Index into the same joint-template array of this joint's parent joint, + * or -1 when the joint is the rig root (parented to the Model entity). */ + parentJointIndex: number; + /** Human-readable name from the glTF (debug aid). */ + name?: string; +} + +export interface LoadedSkin { + jointTemplate: JointTemplate[]; + /** Flat N × 16 floats — bind-pose inverse matrices in joint order. */ + inverseBindMatrices: Float32Array; +} + +/** + * Builds a node-index → joint-index map and walks the glTF node hierarchy to + * find each joint's parent joint (or -1 if its parent is not itself a joint). + */ +function buildJointParentMap(gltf: GltfAsset, jointNodeIndices: readonly number[]): number[] { + const nodeToJoint = new Map(); + for (let j = 0; j < jointNodeIndices.length; j++) nodeToJoint.set(jointNodeIndices[j], j); + + const nodeParent = new Map(); // child node → parent node + for (let n = 0; n < (gltf.nodes ?? []).length; n++) { + for (const child of gltf.nodes![n].children ?? []) nodeParent.set(child, n); + } + + return jointNodeIndices.map(nodeIdx => { + const parentNode = nodeParent.get(nodeIdx); + if (parentNode === undefined) return -1; + return nodeToJoint.get(parentNode) ?? -1; + }); +} + +export function parseGltfSkin(gltf: GltfAsset, bin: ArrayBuffer): LoadedSkin | null { + const skin = gltf.skins?.[0]; + if (!skin) return null; + + const parentIndices = buildJointParentMap(gltf, skin.joints); + + const jointTemplate: JointTemplate[] = skin.joints.map((nodeIdx, jointIdx) => { + const node = gltf.nodes![nodeIdx]; + return { + position: (node.translation ?? [0, 0, 0]) as Vec3, + rotation: (node.rotation ?? [0, 0, 0, 1]) as Quat, + scale: (node.scale ?? [1, 1, 1]) as Vec3, + parentJointIndex: parentIndices[jointIdx], + name: node.name, + }; + }); + + const inverseBindMatrices = skin.inverseBindMatrices !== undefined + ? new Float32Array(readAccessor(gltf, bin, skin.inverseBindMatrices) as Float32Array) + : new Float32Array(skin.joints.length * 16).fill(0).map((_, i) => i % 17 === 0 ? 1 : 0); + + return { jointTemplate, inverseBindMatrices }; +} diff --git a/packages/data-gpu/src/graphics/scene/model/model-loader-plugin.ts b/packages/data-gpu/src/graphics/scene/model/model-loader-plugin.ts new file mode 100644 index 00000000..b468d43d --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/model-loader-plugin.ts @@ -0,0 +1,131 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import type { Aabb } from "@adobe/data/math"; +import { pbrCore } from "../../rendering/pbr-core-plugin.js"; +import { animation } from "../../animation/animation-plugin.js"; +import { core } from "../../../core/core-plugin.js"; +import { Model } from "./model.js"; +import { loadGltfPrimitives, type GpuPrimitiveData } from "./gltf/load-gltf-model.js"; +import type { LoadedAnimation } from "./gltf/parse-animations.js"; +import type { JointTemplate } from "./gltf/parse-skin.js"; + +export interface LoadedArgs { + geometry: number; + bounds: Aabb; + primitives: GpuPrimitiveData[]; + skinJointTemplate: JointTemplate[]; + skinInverseBindMatrices: Float32Array | null; + animations: LoadedAnimation[]; + collision: { positions: Float32Array; indices: Uint32Array } | null; + skinVertices: { positions: Float32Array; joints: Uint32Array; weights: Float32Array } | null; +} + +/** + * modelLoader + * query: Geometry-_bounds + * read: + * modelUrl + * write: + * _bounds: Aabb + * _skinJointTemplate: JointTemplate[] + * _skinInverseBindMatrices: Float32Array | null + * _animationClipRefs: EntityId[] + * _VisibleMaterial + * _PbrPrimitive + * AnimationClip // when the glTF carries animations + */ +export const modelLoader = Database.Plugin.create({ + extends: Database.Plugin.combine(pbrCore, Model.plugin, core, animation), + components: { + _bounds: { default: null as Aabb | null }, + _skinJointTemplate: { default: [] as JointTemplate[] }, + _skinInverseBindMatrices: { default: null as Float32Array | null }, + _animationClipRefs: { default: [] as number[] }, + // CPU-retained model-space collision geometry (auto-collider source). + _cpuPositions: { default: null as Float32Array | null }, + _cpuIndices: { default: null as Uint32Array | null }, + // CPU-retained skin (mesh-bind positions + 4 joints + 4 weights per vertex), + // for fitting per-bone ragdoll capsules. + _cpuSkin: { default: null as { positions: Float32Array; joints: Uint32Array; weights: Float32Array } | null }, + }, + transactions: { + insertLoadedPrimitives(t, args: LoadedArgs) { + for (const p of args.primitives) { + const materialId = t.archetypes._VisibleMaterial.insert({ + ephemeral: true, + _materialBindGroup: p.pbrMaterialBindGroup, + _geometry: args.geometry, + }); + t.archetypes._PbrPrimitive.insert({ + ephemeral: true, + _geometry: args.geometry, + _material: materialId, + _vertexBuffer: p.pbrVertexBuffer, + _skinVertexBuffer: p.pbrSkinVertexBuffer, + _indexBuffer: p.pbrIndexBuffer, + _indexCount: p.pbrIndexCount, + _indexFormat: p.pbrIndexFormat, + _nodeLocalMatrix: p.pbrNodeLocalMatrix, + }); + } + const clipRefs: number[] = []; + for (const anim of args.animations) { + if (anim.tracks.length === 0) continue; + const clipId = t.archetypes.AnimationClip.insert({ + animationClipTracks: anim.tracks, + animationClipDuration: anim.duration, + }); + clipRefs.push(clipId); + } + t.update(args.geometry, { + _bounds: args.bounds, + _skinJointTemplate: args.skinJointTemplate, + _skinInverseBindMatrices: args.skinInverseBindMatrices, + _animationClipRefs: clipRefs, + _cpuPositions: args.collision?.positions ?? null, + _cpuIndices: args.collision?.indices ?? null, + _cpuSkin: args.skinVertices, + }); + }, + }, + systems: { + modelLoadSystem: { + create: db => { + const inFlight = new Set(); + return () => { + const { device } = db.store.resources; + if (!device) return; + for (const arch of db.store.queryArchetypes(["modelUrl"])) { + const ids = arch.columns.id; + const urls = arch.columns.modelUrl; + for (let i = 0; i < arch.rowCount; i++) { + const id = ids.get(i); + if (inFlight.has(id)) continue; + const url = urls.get(i); + if (!url) continue; // procedural geometries (shapes) carry no URL + inFlight.add(id); + loadGltfPrimitives(device, url) + .then(loaded => { + db.transactions.insertLoadedPrimitives({ + geometry: id, + bounds: loaded.bounds, + primitives: loaded.primitives, + skinJointTemplate: loaded.skin?.jointTemplate ?? [], + skinInverseBindMatrices: loaded.skin?.inverseBindMatrices ?? null, + animations: loaded.animations, + collision: loaded.collision, + skinVertices: loaded.skinVertices, + }); + }) + .catch(err => { + console.error("[modelLoader] Failed to load model", urls.get(i), err); + }); + } + } + }; + }, + schedule: { during: ["preUpdate"] }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/scene/model/model-plugin.ts b/packages/data-gpu/src/graphics/scene/model/model-plugin.ts new file mode 100644 index 00000000..30bbfcbf --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/model-plugin.ts @@ -0,0 +1,46 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, Entity } from "@adobe/data/ecs"; +import { Quat, type Vec3 } from "@adobe/data/math"; +import { Node } from "../node/node.js"; + +/** + * Authored renderable scene. A `Geometry` is an asset identified by URL; a + * `Model` is a placed instance of a Geometry — a Node plus a reference to + * the Geometry it draws. + * + * Loading is performed by `modelLoader`, which reads the `modelUrl` and + * produces the GPU primitives the renderers consume. + */ +export const model = Database.Plugin.create({ + extends: Node.plugin, + components: { + modelUrl: { type: "string" }, + geometry: Entity.schema, + }, + archetypes: { + Geometry: ["modelUrl"], + Model: ["geometry", "position", "rotation", "scale", "visible", "parent"], + }, + transactions: { + insertGeometry(t, args: { modelUrl: string }): number { + return t.archetypes.Geometry.insert({ modelUrl: args.modelUrl }); + }, + insertModel(t, args: { + geometry: number; + position?: Vec3; + rotation?: Quat; + scale?: Vec3; + parent?: number; + }): number { + return t.archetypes.Model.insert({ + geometry: args.geometry, + position: args.position ?? [0, 0, 0], + rotation: args.rotation ?? Quat.identity, + scale: args.scale ?? [1, 1, 1], + visible: true, + parent: args.parent ?? 0, + }); + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/scene/model/model.ts b/packages/data-gpu/src/graphics/scene/model/model.ts new file mode 100644 index 00000000..96525a5d --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/model.ts @@ -0,0 +1,15 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Entity } from "@adobe/data/ecs"; +import type { Node } from "../node/node.js"; + +/** + * One row of the `Model` archetype — a placed instance of a Geometry. A + * Node (transform + visibility + parent) plus a reference to the Geometry + * the renderer should draw. + */ +export interface Model extends Node { + geometry: Entity; +} + +export * as Model from "./public.js"; diff --git a/packages/data-gpu/src/graphics/scene/model/public.ts b/packages/data-gpu/src/graphics/scene/model/public.ts new file mode 100644 index 00000000..71684ded --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/public.ts @@ -0,0 +1 @@ +export { model as plugin } from "./model-plugin.js"; diff --git a/packages/data-gpu/src/graphics/scene/model/shape/convex-hull.test.ts b/packages/data-gpu/src/graphics/scene/model/shape/convex-hull.test.ts new file mode 100644 index 00000000..cb840371 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/shape/convex-hull.test.ts @@ -0,0 +1,72 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { describe, it, expect } from "vitest"; +import { hullFaces, convexHullMesh } from "./convex-hull.js"; + +const f32 = (pts: number[][]) => new Float32Array(pts.flat()); + +describe("convexHull", () => { + const px = (p: Float32Array, i: number) => [p[i * 3], p[i * 3 + 1], p[i * 3 + 2]] as const; + + // every input point lies on the inner side of every face plane (within tol), + // and every face normal points away from the cloud centroid (outward). + const assertValidHull = (points: Float32Array) => { + const faces = hullFaces(points); + expect(faces).not.toBeNull(); + const n = points.length / 3; + let gx = 0, gy = 0, gz = 0; + for (let i = 0; i < n; i++) { const [x, y, z] = px(points, i); gx += x; gy += y; gz += z; } + gx /= n; gy /= n; gz /= n; + for (const fc of faces!) { + const [ax, ay, az] = px(points, fc.a); + for (let p = 0; p < n; p++) { // no point strictly outside this face + const [x, y, z] = px(points, p); + expect(fc.nx * (x - ax) + fc.ny * (y - ay) + fc.nz * (z - az)).toBeLessThan(1e-4); + } + // outward: normal agrees with (face vertex − cloud centroid) + expect(fc.nx * (ax - gx) + fc.ny * (ay - gy) + fc.nz * (az - gz)).toBeGreaterThan(0); + } + return faces!; + }; + + it("tetrahedron → 4 faces", () => { + expect(assertValidHull(f32([[0, 0, 0], [1, 0, 0], [0, 1, 0], [0, 0, 1]])).length).toBe(4); + }); + + it("octahedron → 8 faces", () => { + expect(assertValidHull(f32([[1, 0, 0], [-1, 0, 0], [0, 1, 0], [0, -1, 0], [0, 0, 1], [0, 0, -1]])).length).toBe(8); + }); + + it("cube → a valid closed hull using all 8 corners", () => { + const cube = f32([[-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1], [-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1]]); + const faces = assertValidHull(cube); + const used = new Set(); + for (const f of faces) { used.add(f.a); used.add(f.b); used.add(f.c); } + expect(used.size).toBe(8); // every corner is a hull vertex + // V − E + F = 2 (Euler), with all triangular faces ⇒ E = 3F/2 ⇒ F = 12 + expect(faces.length).toBe(12); + }); + + it("interior points are discarded", () => { + const withInterior = f32([ + [-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1], [-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1], + [0, 0, 0], [0.5, 0.2, -0.3], // inside — must not appear on the hull + ]); + const faces = assertValidHull(withInterior); + const used = new Set(); + for (const f of faces) { used.add(f.a); used.add(f.b); used.add(f.c); } + expect(used.has(8)).toBe(false); + expect(used.has(9)).toBe(false); + }); + + it("degenerate clouds → null (< 4 points, or coplanar)", () => { + expect(hullFaces(f32([[0, 0, 0], [1, 0, 0], [0, 1, 0]]))).toBeNull(); // 3 points + expect(hullFaces(f32([[0, 0, 0], [1, 0, 0], [0, 1, 0], [1, 1, 0]]))).toBeNull(); // coplanar (z=0) + }); + + it("convexHullMesh emits flat-shaded StandardVertex triangles (12 floats/vert, 3 verts/tri)", () => { + const mesh = convexHullMesh(f32([[0, 0, 0], [1, 0, 0], [0, 1, 0], [0, 0, 1]])); + expect(mesh.indices.length).toBe(12); // 4 faces × 3 + expect(mesh.vertices.length).toBe(12 * 12); // 12 verts × 12 floats + }); +}); diff --git a/packages/data-gpu/src/graphics/scene/model/shape/convex-hull.ts b/packages/data-gpu/src/graphics/scene/model/shape/convex-hull.ts new file mode 100644 index 00000000..65aafb7f --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/shape/convex-hull.ts @@ -0,0 +1,141 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { ShapeMesh } from "./shape-mesh.js"; + +/** + * Faceted render mesh of the convex hull of a point cloud (flat per-face normals, + * StandardVertex layout). A convex collider is *authored* as a point cloud — the + * physics engine builds the collision hull from the same points — but rendering + * needs a triangulated surface, which this computes. Phase-2 auto-hull (hull of a + * render mesh's vertices) reuses it. + * + * Incremental 3D hull: seed a tetrahedron, then fold each remaining point in by + * deleting the faces it can see and bridging the horizon to it. Every face's + * normal is oriented away from a fixed interior point (the seed centroid, always + * strictly inside a convex hull) — so orientation is robust without half-edge + * bookkeeping. Authored hulls are small (tens of points), so the simple O(n·f) + * form is plenty fast. Degenerate input (< 4 points or coplanar) → empty mesh. + */ + +interface Face { a: number; b: number; c: number; nx: number; ny: number; nz: number } + +const EPS = 1e-7; + +export function convexHullMesh(points: Float32Array): ShapeMesh { + const faces = hullFaces(points); + if (!faces) return { vertices: new Float32Array(0), indices: new Uint16Array(0) }; + const px = (i: number) => points[i * 3], py = (i: number) => points[i * 3 + 1], pz = (i: number) => points[i * 3 + 2]; + + // flat-shaded: three unique verts per face, all sharing the face normal. + const verts: number[] = [], indices: number[] = []; + for (const f of faces) { + const len = Math.hypot(f.nx, f.ny, f.nz) || 1; + const nx = f.nx / len, ny = f.ny / len, nz = f.nz / len; + // any in-plane unit vector for the tangent (flat shading ignores its sign) + let tx = px(f.b) - px(f.a), ty = py(f.b) - py(f.a), tz = pz(f.b) - pz(f.a); + const tl = Math.hypot(tx, ty, tz) || 1; tx /= tl; ty /= tl; tz /= tl; + const base = verts.length / 12; + const corners = [f.a, f.b, f.c], uv = [[0, 0], [1, 0], [0, 1]]; + for (let i = 0; i < 3; i++) { + const v = corners[i]; + verts.push(px(v), py(v), pz(v), nx, ny, nz, tx, ty, tz, 1, uv[i][0], uv[i][1]); + } + indices.push(base, base + 1, base + 2); + } + return { vertices: new Float32Array(verts), indices: new Uint16Array(indices) }; +} + +/** + * The deduplicated hull vertices (the extreme points) of a cloud — a minimal + * point set whose convex hull equals the cloud's. Used to feed a physics engine's + * convex-hull builder a *simplified* collider from a detailed mesh (thousands of + * verts → a few dozen). Empty for degenerate input. + */ +export function hullVertices(points: Float32Array): Float32Array { + const faces = hullFaces(points); + if (!faces) return new Float32Array(0); + const used = new Set(); + for (const f of faces) { used.add(f.a); used.add(f.b); used.add(f.c); } + const out = new Float32Array(used.size * 3); + let i = 0; + for (const idx of used) { out[i * 3] = points[idx * 3]; out[i * 3 + 1] = points[idx * 3 + 1]; out[i * 3 + 2] = points[idx * 3 + 2]; i++; } + return out; +} + +/** The hull's triangular faces, each normal oriented outward, or null if the + * point cloud is degenerate (fewer than 4 points, or all coplanar). Exported + * for tests; rendering goes through {@link convexHullMesh}. */ +export function hullFaces(points: Float32Array): Face[] | null { + const n = points.length / 3; + if (n < 4) return null; + const px = (i: number) => points[i * 3], py = (i: number) => points[i * 3 + 1], pz = (i: number) => points[i * 3 + 2]; + + const seed = seedTetra(n, px, py, pz); + if (!seed) return null; + // centroid of the seed tetra — strictly inside the hull for its whole life, so + // "normal points away from this" is always the correct outward test. + const [i0, i1, i2, i3] = seed; + const cx = (px(i0) + px(i1) + px(i2) + px(i3)) / 4; + const cy = (py(i0) + py(i1) + py(i2) + py(i3)) / 4; + const cz = (pz(i0) + pz(i1) + pz(i2) + pz(i3)) / 4; + + const makeFace = (a: number, b: number, c: number): Face => { + const ux = px(b) - px(a), uy = py(b) - py(a), uz = pz(b) - pz(a); + const vx = px(c) - px(a), vy = py(c) - py(a), vz = pz(c) - pz(a); + let nx = uy * vz - uz * vy, ny = uz * vx - ux * vz, nz = ux * vy - uy * vx; + if (nx * (px(a) - cx) + ny * (py(a) - cy) + nz * (pz(a) - cz) < 0) { // points inward → flip + return { a, b: c, c: b, nx: -nx, ny: -ny, nz: -nz }; + } + return { a, b, c, nx, ny, nz }; + }; + const above = (f: Face, p: number): number => + f.nx * (px(p) - px(f.a)) + f.ny * (py(p) - py(f.a)) + f.nz * (pz(p) - pz(f.a)); + + const faces: Face[] = [makeFace(i0, i1, i2), makeFace(i0, i1, i3), makeFace(i0, i2, i3), makeFace(i1, i2, i3)]; + for (let p = 0; p < n; p++) { + if (faces.every(f => above(f, p) <= EPS)) continue; // inside the current hull + + // horizon = edges on exactly one visible face (the toggle leaves only those). + const horizon = new Map(); + for (const f of faces) { + if (above(f, p) <= EPS) continue; + for (const [u, v] of [[f.a, f.b], [f.b, f.c], [f.c, f.a]] as const) { + const k = u < v ? `${u}_${v}` : `${v}_${u}`; + if (horizon.has(k)) horizon.delete(k); else horizon.set(k, [u, v]); + } + } + for (let i = faces.length - 1; i >= 0; i--) if (above(faces[i], p) > EPS) faces.splice(i, 1); + for (const [u, v] of horizon.values()) faces.push(makeFace(u, v, p)); + } + return faces; +} + +/** Indices of 4 non-coplanar seed points (max-spread pair, point farthest from + * that line, point farthest from that plane), or null if all points are coplanar. */ +function seedTetra(n: number, px: (i: number) => number, py: (i: number) => number, pz: (i: number) => number): [number, number, number, number] | null { + let i0 = 0, i1 = 0, best = -1; + for (let i = 0; i < n; i++) for (let j = i + 1; j < n; j++) { + const d = (px(i) - px(j)) ** 2 + (py(i) - py(j)) ** 2 + (pz(i) - pz(j)) ** 2; + if (d > best) { best = d; i0 = i; i1 = j; } + } + if (best <= EPS) return null; + const ex = px(i1) - px(i0), ey = py(i1) - py(i0), ez = pz(i1) - pz(i0); + let i2 = -1; best = EPS; + for (let i = 0; i < n; i++) { // area² of triangle (i0,i1,i) ∝ |edge × (i0→i)|² + const wx = px(i) - px(i0), wy = py(i) - py(i0), wz = pz(i) - pz(i0); + const cx = ey * wz - ez * wy, cy = ez * wx - ex * wz, cz = ex * wy - ey * wx; + const a = cx * cx + cy * cy + cz * cz; + if (a > best) { best = a; i2 = i; } + } + if (i2 < 0) return null; + const ux = px(i1) - px(i0), uy = py(i1) - py(i0), uz = pz(i1) - pz(i0); + const vx = px(i2) - px(i0), vy = py(i2) - py(i0), vz = pz(i2) - pz(i0); + const pnx = uy * vz - uz * vy, pny = uz * vx - ux * vz, pnz = ux * vy - uy * vx; + let i3 = -1; best = EPS; + for (let i = 0; i < n; i++) { + const d = Math.abs(pnx * (px(i) - px(i0)) + pny * (py(i) - py(i0)) + pnz * (pz(i) - pz(i0))); + if (d > best) { best = d; i3 = i; } + } + if (i3 < 0) return null; // coplanar + return [i0, i1, i2, i3]; +} diff --git a/packages/data-gpu/src/graphics/scene/model/shape/shape-geometry-plugin.ts b/packages/data-gpu/src/graphics/scene/model/shape/shape-geometry-plugin.ts new file mode 100644 index 00000000..48da30e8 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/shape/shape-geometry-plugin.ts @@ -0,0 +1,61 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, type Entity } from "@adobe/data/ecs"; +import { Mat4x4 } from "@adobe/data/math"; +import { pbrCore } from "../../../rendering/pbr-core-plugin.js"; +import { core } from "../../../../core/core-plugin.js"; +import { model } from "../model-plugin.js"; +import { unitSphere, unitCube } from "./shape-mesh.js"; +import { uploadShapeMesh } from "./upload-shape-mesh.js"; + +/** + * shapeGeometry — registers procedural unit-sphere and unit-cube geometries as + * `Geometry` + `_PbrPrimitive` entities (StandardVertex layout), exactly like + * the model loader does for glTF, so primitives and loaded models share one + * render path. The two geometry entity ids are published in `_shapeGeometry` + * for the physics render bridge to reference (sphere / cuboid bodies). Built + * once when the device is ready. + */ +export const shapeGeometry = Database.Plugin.create({ + extends: Database.Plugin.combine(pbrCore, model, core), + resources: { + _shapeGeometry: { default: null as { sphere: Entity; cube: Entity } | null, transient: true }, + }, + transactions: { + insertShapePrimitive(t, args: { vertexBuffer: GPUBuffer; indexBuffer: GPUBuffer; indexCount: number }): Entity { + const geometry = t.archetypes.Geometry.insert({ modelUrl: "" }); + t.archetypes._PbrPrimitive.insert({ + ephemeral: true, + _geometry: geometry, + _material: 0, // primitives carry material per-instance, not here + _vertexBuffer: args.vertexBuffer, + _skinVertexBuffer: null, + _indexBuffer: args.indexBuffer, + _indexCount: args.indexCount, + _indexFormat: "uint16", + _nodeLocalMatrix: Mat4x4.identity, + }); + return geometry; + }, + }, + systems: { + shapeGeometryInit: { + schedule: { during: ["preUpdate"] }, + create: db => { + let done = false; + return () => { + if (done) return; + const { device } = db.store.resources; + if (!device) return; + + const s = uploadShapeMesh(device, unitSphere()); + const c = uploadShapeMesh(device, unitCube()); + const sphere = db.transactions.insertShapePrimitive({ vertexBuffer: s.vb, indexBuffer: s.ib, indexCount: s.count }); + const cube = db.transactions.insertShapePrimitive({ vertexBuffer: c.vb, indexBuffer: c.ib, indexCount: c.count }); + db.store.resources._shapeGeometry = { sphere, cube }; + done = true; + }; + }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/scene/model/shape/shape-mesh.ts b/packages/data-gpu/src/graphics/scene/model/shape/shape-mesh.ts new file mode 100644 index 00000000..22708244 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/shape/shape-mesh.ts @@ -0,0 +1,150 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +/** + * Procedural unit-shape meshes in the StandardVertex packed layout + * (position vec3, normal vec3, tangent vec4, uv vec2 — 12 floats/vertex), so + * they feed the same PBR pipeline and `StandardVertex.layout` as glTF meshes. + * Unit-sized (radius / half-extent 1): a Node `scale` sets the real size, so a + * cuboid is the unit cube scaled by its half-extents and a sphere is the unit + * sphere scaled by its radius. + */ +export interface ShapeMesh { + vertices: Float32Array; + indices: Uint16Array; +} + +function push(out: number[], px: number, py: number, pz: number, nx: number, ny: number, nz: number, tx: number, ty: number, tz: number, tw: number, u: number, v: number): void { + out.push(px, py, pz, nx, ny, nz, tx, ty, tz, tw, u, v); +} + +/** UV sphere, radius 1, centred at origin. */ +export function unitSphere(rings = 24, segments = 48): ShapeMesh { + const verts: number[] = []; + for (let ring = 0; ring <= rings; ring++) { + const theta = (ring / rings) * Math.PI; + const st = Math.sin(theta), ct = Math.cos(theta); + for (let seg = 0; seg <= segments; seg++) { + const phi = (seg / segments) * Math.PI * 2; + const sp = Math.sin(phi), cp = Math.cos(phi); + const x = st * cp, y = ct, z = st * sp; + // tangent = d(pos)/dphi, normalized; degenerate at the poles. + let tx = -sp, ty = 0, tz = cp; + const tl = Math.hypot(tx, ty, tz); + if (tl < 1e-5) { tx = 1; ty = 0; tz = 0; } else { tx /= tl; tz /= tl; } + push(verts, x, y, z, x, y, z, tx, ty, tz, 1, seg / segments, ring / rings); + } + } + const indices: number[] = []; + const stride = segments + 1; + for (let ring = 0; ring < rings; ring++) { + for (let seg = 0; seg < segments; seg++) { + const a = ring * stride + seg, b = a + stride; + indices.push(a, b, a + 1, a + 1, b, b + 1); + } + } + return { vertices: new Float32Array(verts), indices: new Uint16Array(indices) }; +} + +interface Face { n: [number, number, number]; u: [number, number, number]; v: [number, number, number] } + +// u × v = n for every face, so [c0,c1,c2,c0,c2,c3] winds CCW outward (back-cull). +const CUBE_FACES: Face[] = [ + { n: [1, 0, 0], u: [0, 0, -1], v: [0, 1, 0] }, + { n: [-1, 0, 0], u: [0, 0, 1], v: [0, 1, 0] }, + { n: [0, 1, 0], u: [1, 0, 0], v: [0, 0, -1] }, + { n: [0, -1, 0], u: [1, 0, 0], v: [0, 0, 1] }, + { n: [0, 0, 1], u: [1, 0, 0], v: [0, 1, 0] }, + { n: [0, 0, -1], u: [-1, 0, 0], v: [0, 1, 0] }, +]; + +/** + * Flat-shaded StandardVertex mesh from a triangle soup (positions + indices) — + * one face normal per triangle, three unique verts per triangle. Used to render + * an authored static-mesh collider. Indices are emitted as `uint16`, so this is + * for the modest authored meshes (ramps, props) of the current shape path, not + * dense terrain (which would need a `uint32`-indexed primitive). + */ +export function flatShadedMesh(positions: Float32Array, indices: ArrayLike): ShapeMesh { + const verts: number[] = [], out: number[] = []; + for (let t = 0; t < indices.length; t += 3) { + const ia = indices[t] * 3, ib = indices[t + 1] * 3, ic = indices[t + 2] * 3; + const ax = positions[ia], ay = positions[ia + 1], az = positions[ia + 2]; + const bx = positions[ib], by = positions[ib + 1], bz = positions[ib + 2]; + const cx = positions[ic], cy = positions[ic + 1], cz = positions[ic + 2]; + let nx = (by - ay) * (cz - az) - (bz - az) * (cy - ay); + let ny = (bz - az) * (cx - ax) - (bx - ax) * (cz - az); + let nz = (bx - ax) * (cy - ay) - (by - ay) * (cx - ax); + const nl = Math.hypot(nx, ny, nz) || 1; nx /= nl; ny /= nl; nz /= nl; + let tx = bx - ax, ty = by - ay, tz = bz - az; + const tl = Math.hypot(tx, ty, tz) || 1; tx /= tl; ty /= tl; tz /= tl; + const base = verts.length / 12; + push(verts, ax, ay, az, nx, ny, nz, tx, ty, tz, 1, 0, 0); + push(verts, bx, by, bz, nx, ny, nz, tx, ty, tz, 1, 1, 0); + push(verts, cx, cy, cz, nx, ny, nz, tx, ty, tz, 1, 0, 1); + out.push(base, base + 1, base + 2); + } + return { vertices: new Float32Array(verts), indices: new Uint16Array(out) }; +} + +/** + * Y-aligned capsule: a cylinder (radius `r`, half-height `halfHeight`) capped by + * two hemispheres of radius `r`, centred at the origin (total height + * 2·(halfHeight + r)). Unlike the sphere/cube, a capsule has two independent + * dimensions and spherical caps that must not distort under non-uniform scale — + * so it is built at its real size and rendered with unit scale (the bridge caches + * one mesh per distinct radius/half-height). Rings run top pole → bottom pole; + * the two equator rings (`ny = 0`, full radius) bound the straight cylinder wall. + */ +export function capsuleMesh(r: number, halfHeight: number, capRings = 8, segments = 24): ShapeMesh { + const rows: { y: number; rr: number; ny: number }[] = []; + for (let i = 0; i <= capRings; i++) { // top cap: pole → equator + const a = (i / capRings) * (Math.PI / 2); + rows.push({ y: halfHeight + r * Math.cos(a), rr: r * Math.sin(a), ny: Math.cos(a) }); + } + rows.push({ y: -halfHeight, rr: r, ny: 0 }); // cylinder bottom rim + for (let i = 1; i <= capRings; i++) { // bottom cap: equator → pole + const b = (i / capRings) * (Math.PI / 2); + rows.push({ y: -halfHeight - r * Math.sin(b), rr: r * Math.cos(b), ny: -Math.sin(b) }); + } + const verts: number[] = []; + const stride = segments + 1; + for (let row = 0; row < rows.length; row++) { + const { y, rr, ny } = rows[row]; + const rad = Math.sqrt(Math.max(0, 1 - ny * ny)); // radial component of the unit normal + for (let seg = 0; seg <= segments; seg++) { + const phi = (seg / segments) * Math.PI * 2; + const cp = Math.cos(phi), sp = Math.sin(phi); + push(verts, rr * cp, y, rr * sp, rad * cp, ny, rad * sp, -sp, 0, cp, 1, seg / segments, row / (rows.length - 1)); + } + } + const indices: number[] = []; + for (let row = 0; row < rows.length - 1; row++) { + for (let seg = 0; seg < segments; seg++) { + const a = row * stride + seg, b = a + stride; + indices.push(a, b, a + 1, a + 1, b, b + 1); + } + } + return { vertices: new Float32Array(verts), indices: new Uint16Array(indices) }; +} + +/** Cube spanning [-1, 1] on each axis, one material-space UV per face. */ +export function unitCube(): ShapeMesh { + const verts: number[] = []; + const indices: number[] = []; + let base = 0; + for (const f of CUBE_FACES) { + const [nx, ny, nz] = f.n, [ux, uy, uz] = f.u, [vx, vy, vz] = f.v; + const corner = (su: number, sv: number, tu: number, tv: number): void => { + push(verts, + nx + su * ux + sv * vx, ny + su * uy + sv * vy, nz + su * uz + sv * vz, + nx, ny, nz, ux, uy, uz, 1, tu, tv); + }; + corner(-1, -1, 0, 0); + corner(1, -1, 1, 0); + corner(1, 1, 1, 1); + corner(-1, 1, 0, 1); + indices.push(base, base + 1, base + 2, base, base + 2, base + 3); + base += 4; + } + return { vertices: new Float32Array(verts), indices: new Uint16Array(indices) }; +} diff --git a/packages/data-gpu/src/graphics/scene/model/shape/upload-shape-mesh.ts b/packages/data-gpu/src/graphics/scene/model/shape/upload-shape-mesh.ts new file mode 100644 index 00000000..f9a0a7a7 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/shape/upload-shape-mesh.ts @@ -0,0 +1,14 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { ShapeMesh } from "./shape-mesh.js"; + +/** Upload a procedural shape mesh into GPU vertex + index buffers (one-time, + * when the device is ready). Shared by `shapeGeometry` (sphere/cube) and the + * physics bridge (per-dimension capsules). */ +export function uploadShapeMesh(device: GPUDevice, mesh: ShapeMesh): { vb: GPUBuffer; ib: GPUBuffer; count: number } { + const vb = device.createBuffer({ size: mesh.vertices.byteLength, usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST }); + device.queue.writeBuffer(vb, 0, mesh.vertices); + const ib = device.createBuffer({ size: mesh.indices.byteLength, usage: GPUBufferUsage.INDEX | GPUBufferUsage.COPY_DST }); + device.queue.writeBuffer(ib, 0, mesh.indices); + return { vb, ib, count: mesh.indices.length }; +} diff --git a/packages/data-gpu/src/graphics/scene/model/world-bounds-plugin.ts b/packages/data-gpu/src/graphics/scene/model/world-bounds-plugin.ts new file mode 100644 index 00000000..bedc5900 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/model/world-bounds-plugin.ts @@ -0,0 +1,88 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Aabb, Mat4x4, type Vec3 } from "@adobe/data/math"; +import { transform } from "../node/transform-plugin.js"; +import { modelLoader } from "./model-loader-plugin.js"; + +/** + * worldBoundsCreate + * query: Model-_worldBounds + * write: _worldBounds (archetype migration; unit placeholder) + * + * Adds the `_worldBounds` column to any Model that doesn't have it yet, so + * `worldBoundsSystem` can write directly without further migration. + * + * worldBoundsSystem + * query: Model+_worldMatrix+_worldBounds + * read: geometry → _bounds, _worldMatrix + * write: _worldBounds + * + * Transforms the asset-space AABB on each Model's `geometry` by the Model's + * `_worldMatrix` (all 8 corners, reduce to min/max) and writes the result. + * Geometries still loading have no `_bounds` yet — those Models are skipped + * until the asset finishes; their `_worldBounds` stays at its placeholder. + * + * Cost: one matrix-vec multiply × 8 corners × N visible Models, per frame. + * At N=1000 that's ~8K vec3 ops; well under 0.1ms on modern CPUs. + */ +export const worldBounds = Database.Plugin.create({ + extends: Database.Plugin.combine(modelLoader, transform), + systems: { + worldBoundsCreate: { + schedule: { after: ["transformCreateWorldMatrix"] }, + create: db => () => { + for (const arch of db.store.queryArchetypes( + ["geometry", "_worldMatrix"], + { exclude: ["_worldBounds"] }, + )) { + const ids = arch.columns.id; + // Iterate tail→head: every row migrates out of this archetype. + for (let i = arch.rowCount - 1; i >= 0; i--) { + db.store.update(ids.get(i), { _worldBounds: Aabb.unit }); + } + } + }, + }, + worldBoundsSystem: { + schedule: { after: ["transformSystem", "worldBoundsCreate"] }, + create: db => () => { + for (const arch of db.store.queryArchetypes([ + "geometry", "_worldMatrix", "_worldBounds", + ])) { + const geos = arch.columns.geometry; + const worldMats = arch.columns._worldMatrix; + const worldBoundsCol = arch.columns._worldBounds; + for (let i = 0; i < arch.rowCount; i++) { + const localBounds = db.store.get(geos.get(i), "_bounds"); + if (!localBounds) continue; + worldBoundsCol.set(i, transformAabb(localBounds, worldMats.get(i))); + } + } + }, + }, + }, +}); + +const transformAabb = (local: Aabb, m: Mat4x4): Aabb => { + const { min, max } = local; + let minX = Infinity, minY = Infinity, minZ = Infinity; + let maxX = -Infinity, maxY = -Infinity, maxZ = -Infinity; + // The 8 box corners. + for (let cx = 0; cx < 2; cx++) { + for (let cy = 0; cy < 2; cy++) { + for (let cz = 0; cz < 2; cz++) { + const corner: Vec3 = [ + cx === 0 ? min[0] : max[0], + cy === 0 ? min[1] : max[1], + cz === 0 ? min[2] : max[2], + ]; + const w = Mat4x4.multiplyVec3(m, corner); + if (w[0] < minX) minX = w[0]; if (w[0] > maxX) maxX = w[0]; + if (w[1] < minY) minY = w[1]; if (w[1] > maxY) maxY = w[1]; + if (w[2] < minZ) minZ = w[2]; if (w[2] > maxZ) maxZ = w[2]; + } + } + } + return { min: [minX, minY, minZ], max: [maxX, maxY, maxZ] }; +}; diff --git a/packages/data-gpu/src/graphics/scene/node/node-data-plugin.ts b/packages/data-gpu/src/graphics/scene/node/node-data-plugin.ts new file mode 100644 index 00000000..5e429137 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/node/node-data-plugin.ts @@ -0,0 +1,28 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, Entity } from "@adobe/data/ecs"; +import { Aabb, Mat4x4, Quat, Vec3 } from "@adobe/data/math"; +import { True } from "@adobe/data/schema"; + +/** + * The transform-hierarchy core model. Every entity with a position in the + * world is a `Node` — see `node.ts` for the field bundle. + * + * `_worldMatrix` and `_worldBounds` are declared as components but NOT listed + * on the authored Node archetype — systems write them via `db.store.update`, + * migrating each entity once into a wider archetype. + */ +export const nodeData = Database.Plugin.create({ + components: { + visible: True.schema, + position: Vec3.schema, + rotation: Quat.schema, + scale: Vec3.schema, + parent: Entity.schema, + _worldMatrix: Mat4x4.schema, + _worldBounds: Aabb.schema, + }, + archetypes: { + Node: ["position", "rotation", "scale", "parent", "visible"], + }, +}); diff --git a/packages/data-gpu/src/graphics/scene/node/node-plugin.ts b/packages/data-gpu/src/graphics/scene/node/node-plugin.ts new file mode 100644 index 00000000..2c46d69d --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/node/node-plugin.ts @@ -0,0 +1,13 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { nodeData } from "./node-data-plugin.js"; +import { transform } from "./transform-plugin.js"; + +/** + * Full node plugin — spatial hierarchy data plus the `transform` system that + * derives `_worldMatrix` each frame. Use `Node.plugin` (= this) to get both. + * Use `nodeData` directly if you only need the declared components and archetype + * without the per-frame TRS computation. + */ +export const plugin = Database.Plugin.combine(nodeData, transform); diff --git a/packages/data-gpu/src/graphics/scene/node/node.ts b/packages/data-gpu/src/graphics/scene/node/node.ts new file mode 100644 index 00000000..96fd165d --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/node/node.ts @@ -0,0 +1,23 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Entity } from "@adobe/data/ecs"; +import type { Quat, Vec3 } from "@adobe/data/math"; + +/** + * One entity in the Node archetype — a transform-hierarchy slot. Each field + * is its own ECS component (typed-buffer column), but the Node shape names + * the bundle so consumers can declare typed locals like + * `const node: Node = arch.read(i)`. + * + * `_worldMatrix` is *not* on the authored Node — the `transform` system + * adds it later, migrating each entity into a wider archetype. + */ +export interface Node { + position: Vec3; + rotation: Quat; + scale: Vec3; + parent: Entity; + visible: boolean; +} + +export * as Node from "./public.js"; diff --git a/packages/data-gpu/src/graphics/scene/node/public.ts b/packages/data-gpu/src/graphics/scene/node/public.ts new file mode 100644 index 00000000..ec801e79 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/node/public.ts @@ -0,0 +1 @@ +export { plugin } from "./node-plugin.js"; diff --git a/packages/data-gpu/src/graphics/scene/node/transform-plugin.ts b/packages/data-gpu/src/graphics/scene/node/transform-plugin.ts new file mode 100644 index 00000000..6cab4cc2 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/node/transform-plugin.ts @@ -0,0 +1,72 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { Mat4x4, Quat } from "@adobe/data/math"; +import { nodeData } from "./node-data-plugin.js"; + +/** + * transformCreateWorldMatrix + * query: Node-_worldMatrix + * write: _worldMatrix (archetype migration; identity) + * + * Adds the `_worldMatrix` column to any Node that doesn't have it yet, so + * `transformSystem` can assume the column exists and write to it directly. + * + * transformSystem + * query: Node+_worldMatrix + * read: position, rotation, scale, parent + * write: _worldMatrix + * + * Computes TRS for every Node and writes the result into the existing + * `_worldMatrix` column. Parents are typically inserted before children + * (glTF order, sample setup), so by the time we reach a child the parent's + * matrix is already populated this frame. + */ +export const transform = Database.Plugin.create({ + extends: nodeData, + systems: { + transformCreateWorldMatrix: { + create: db => () => { + for (const arch of db.store.queryArchetypes( + ["position", "rotation", "scale", "parent"], + { exclude: ["_worldMatrix"] }, + )) { + const ids = arch.columns.id; + // Iterate tail→head: every row migrates out of this archetype, + // and removing the last row first means no hole-fill shift. + for (let i = arch.rowCount - 1; i >= 0; i--) { + db.store.update(ids.get(i), { _worldMatrix: Mat4x4.identity }); + } + } + }, + }, + transformSystem: { + schedule: { after: ["transformCreateWorldMatrix"] }, + create: db => () => { + for (const arch of db.store.queryArchetypes([ + "position", "rotation", "scale", "parent", "_worldMatrix", + ])) { + const positions = arch.columns.position; + const rotations = arch.columns.rotation; + const scales = arch.columns.scale; + const parents = arch.columns.parent; + const worldMats = arch.columns._worldMatrix; + for (let i = 0; i < arch.rowCount; i++) { + const pos = positions.get(i); + const rot = rotations.get(i); + const scl = scales.get(i); + const parentId = parents.get(i); + const local = Mat4x4.multiply( + Mat4x4.translation(pos[0], pos[1], pos[2]), + Mat4x4.multiply(Quat.toMat4(rot), Mat4x4.scaling(scl[0], scl[1], scl[2])), + ); + const parentWorld = parentId === 0 + ? Mat4x4.identity + : db.store.get(parentId, "_worldMatrix") ?? Mat4x4.identity; + worldMats.set(i, parentId === 0 ? local : Mat4x4.multiply(parentWorld, local)); + } + } + }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/scene/scene-plugin.ts b/packages/data-gpu/src/graphics/scene/scene-plugin.ts new file mode 100644 index 00000000..5f8e0f58 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/scene-plugin.ts @@ -0,0 +1,20 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { plugin as nodePlugin } from "./node/node-plugin.js"; +import { model } from "./model/model-plugin.js"; +import { SceneUniforms } from "./scene-uniforms/scene-uniforms.js"; + +/** + * The complete authored scene — spatial hierarchy, models, camera, lighting, + * and the GPU uniform buffer that ties them together for rendering. + * + * Combines: + * - `Node.plugin` (node data + transform system) + * - `model` (Geometry + Model archetypes) + * - `SceneUniforms.plugin` (camera resource + light resource + GPU uniform packing) + * + * Add a camera controller (e.g. `Orbit.plugin`), animation, and a renderer + * (`pbrIblRender`) to get a working interactive scene. + */ +export const scene = Database.Plugin.combine(nodePlugin, model, SceneUniforms.plugin); diff --git a/packages/data-gpu/src/graphics/scene/scene-uniforms/create-bind-group-layout.ts b/packages/data-gpu/src/graphics/scene/scene-uniforms/create-bind-group-layout.ts new file mode 100644 index 00000000..055da458 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/scene-uniforms/create-bind-group-layout.ts @@ -0,0 +1,19 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +/** + * Bind group layout for the SceneUniforms uniform buffer. Stable across the + * renderer pipeline (consumer) and `_sceneUniforms` system (producer). + * WebGPU compares layouts structurally, so creating two layout objects from + * this descriptor is safe. + */ +export function createBindGroupLayout(device: GPUDevice): GPUBindGroupLayout { + return device.createBindGroupLayout({ + entries: [ + { + binding: 0, + visibility: GPUShaderStage.VERTEX | GPUShaderStage.FRAGMENT, + buffer: { type: "uniform" }, + }, + ], + }); +} diff --git a/packages/data-gpu/src/graphics/scene/scene-uniforms/public.ts b/packages/data-gpu/src/graphics/scene/scene-uniforms/public.ts new file mode 100644 index 00000000..7e6e4584 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/scene-uniforms/public.ts @@ -0,0 +1,5 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export * from "./schema.js"; +export { createBindGroupLayout } from "./create-bind-group-layout.js"; +export { plugin } from "./scene-uniforms-plugin.js"; diff --git a/packages/data-gpu/src/graphics/scene/scene-uniforms/scene-uniforms-plugin.ts b/packages/data-gpu/src/graphics/scene/scene-uniforms/scene-uniforms-plugin.ts new file mode 100644 index 00000000..22bdca8b --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/scene-uniforms/scene-uniforms-plugin.ts @@ -0,0 +1,64 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { createStructBuffer, copyToGPUBuffer, getStructLayout, type TypedBuffer } from "@adobe/data/typed-buffer"; +import { Light } from "../light/light.js"; +import { Camera } from "../../camera/camera.js"; +import { SceneUniforms } from "./scene-uniforms.js"; +// Import the schema from its own module, not via the SceneUniforms namespace: +// this runs at module load and the namespace barrel cycles back here, leaving +// `SceneUniforms.schema` undefined under Node's load order (browsers tolerate it +// via live bindings). The direct import has no cycle. +import { schema as sceneUniformsSchema } from "./schema.js"; + +const sceneUniformsStructLayout = getStructLayout(sceneUniformsSchema); + +/** + * sceneUniforms + * query: — + * read: + * camera + * light + * write: + * _sceneUniformsBuffer: GPUBuffer + */ +export const plugin = Database.Plugin.create({ + extends: Database.Plugin.combine(Camera.plugin, Light.plugin), + resources: { + _sceneUniformsBuffer: { default: null as GPUBuffer | null, transient: true }, + }, + systems: { + sceneUniformsSystem: { + create: db => { + let structBuffer: TypedBuffer | null = null; + return () => { + const { device, light } = db.store.resources; + const cam = db.store.resources.camera; + if (!device || !cam) return; + + structBuffer ??= createStructBuffer(SceneUniforms.schema, sceneUniformsStructLayout.size); + + let gpuBuffer = db.store.resources._sceneUniformsBuffer; + if (!gpuBuffer) { + gpuBuffer = device.createBuffer({ + size: sceneUniformsStructLayout.size, + usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST, + }); + db.store.resources._sceneUniformsBuffer = gpuBuffer; + } + + structBuffer.set(0, { + viewProjectionMatrix: Camera.toViewProjection(cam), + lightDirection: light.direction, + ambientStrength: light.ambientStrength, + lightColor: light.color, + cameraPosition: cam.position, + }); + + db.store.resources._sceneUniformsBuffer = copyToGPUBuffer(structBuffer, device, gpuBuffer); + }; + }, + schedule: { during: ["preRender"] }, + }, + }, +}); diff --git a/packages/data-gpu/src/graphics/scene/scene-uniforms/scene-uniforms.ts b/packages/data-gpu/src/graphics/scene/scene-uniforms/scene-uniforms.ts new file mode 100644 index 00000000..d339b349 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/scene-uniforms/scene-uniforms.ts @@ -0,0 +1,15 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; +import { schema } from "./schema.js"; + +/** + * Default scene uniform struct for a single-viewport, single-directional-light setup. + * Use `_sceneUniforms` plugin to have this written to a GPU buffer each frame. + * + * This is a starting point, not a required contract. Consumers that need different + * fields (multiple lights, fog, time, etc.) should define their own schema + plugin. + */ +export type SceneUniforms = Schema.ToType; + +export * as SceneUniforms from "./public.js"; diff --git a/packages/data-gpu/src/graphics/scene/scene-uniforms/schema.ts b/packages/data-gpu/src/graphics/scene/scene-uniforms/schema.ts new file mode 100644 index 00000000..00970d73 --- /dev/null +++ b/packages/data-gpu/src/graphics/scene/scene-uniforms/schema.ts @@ -0,0 +1,17 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { F32, Mat4x4, Vec3 } from "@adobe/data/math"; +import { Schema } from "@adobe/data/schema"; + +export const schema = { + type: "object", + properties: { + viewProjectionMatrix: Mat4x4.schema, + lightDirection: Vec3.schema, + ambientStrength: F32.schema, + lightColor: Vec3.schema, + cameraPosition: Vec3.schema, + }, + required: ["viewProjectionMatrix", "lightDirection", "ambientStrength", "lightColor", "cameraPosition"], + additionalProperties: false, +} as const satisfies Schema; diff --git a/packages/data-gpu/src/index.ts b/packages/data-gpu/src/index.ts new file mode 100644 index 00000000..74b002f7 --- /dev/null +++ b/packages/data-gpu/src/index.ts @@ -0,0 +1,72 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +// --- Scene model (declarative authored data, no camera, no systems) ---------- +export { scene } from "./graphics/scene/scene-plugin.js"; + +// --- Authoring abstractions -------------------------------------------------- +export { animation } from "./graphics/animation/animation-plugin.js"; +export { AnimationTrack } from "./graphics/animation/animation-track/animation-track.js"; +export { InterpolationMode } from "./graphics/animation/interpolation-mode/interpolation-mode.js"; + +// --- Infrastructure ---------------------------------------------------------- +export { core } from "./core/core-plugin.js"; +export { FrameTime } from "./core/frame-time/frame-time.js"; +export { graphics } from "./graphics/graphics-plugin.js"; + +// --- Physics: shared rigid-body data model + pluggable solver seam ----------- +export { physicsData } from "./physics/physics-data-plugin.js"; +export { physicsClock } from "./physics/physics-clock-plugin.js"; +export type { PhysicsClock } from "./physics/physics-clock-plugin.js"; +export { jointData } from "./physics/joint/joint-plugin.js"; +export type { Joint } from "./physics/joint/joint.js"; +export { JointType } from "./physics/joint/joint-type/joint-type.js"; +export type { RigidBody } from "./physics/body/rigid-body.js"; +export type { StaticCollider } from "./physics/body/static-collider.js"; +export { BodyType } from "./physics/body/body-type/body-type.js"; +export { ColliderShape } from "./physics/body/collider-shape/collider-shape.js"; +export { rapierSolver } from "./physics/solvers/rapier-solver-plugin.js"; +export { joltSolver } from "./physics/solvers/jolt-solver-plugin.js"; +export { runSolverBenchmark } from "./physics/solvers/solver-benchmark.js"; +export type { SolverBenchmarkOptions, SolverBenchmarkResult } from "./physics/solvers/solver-benchmark.js"; + +// --- Material registry (authored entities: physical + visible PBR props) ------ +export { Material } from "./material/material.js"; + +// --- System plugins (consumed via aggregators) ------------------------------- +export { transform } from "./graphics/scene/node/transform-plugin.js"; +export { pbrCore } from "./graphics/rendering/pbr-core-plugin.js"; +export { modelLoader } from "./graphics/scene/model/model-loader-plugin.js"; +export { pbrSkinning } from "./graphics/rendering/skinning/skinning-plugin.js"; +export { picking } from "./graphics/picking/picking-plugin.js"; +export type { PickHit } from "./graphics/picking/pick-hit.js"; + +// --- Rendering --------------------------------------------------------------- +export { rendering } from "./graphics/rendering/rendering-plugin.js"; +export { pbrIblRender } from "./graphics/rendering/ibl-render/ibl-render-plugin.js"; +export { materialGpu } from "./graphics/rendering/material-gpu/material-gpu-plugin.js"; +export { pbrRender } from "./graphics/rendering/pbr-render/pbr-render-plugin.js"; +export { physicsRenderBridge } from "./graphics/rendering/pbr-render/physics-bridge-plugin.js"; +export { displayTransform } from "./graphics/rendering/display-transform-plugin.js"; +export { interpolation } from "./graphics/rendering/interpolation-plugin.js"; +export { modelCollider } from "./graphics/rendering/model-collider-plugin.js"; +export { boneColliders } from "./graphics/rendering/bone-collider-plugin.js"; +export { ragdollTrigger } from "./graphics/rendering/ragdoll-trigger-plugin.js"; +export { joltRagdoll } from "./graphics/rendering/jolt-ragdoll-plugin.js"; +export { fitBoneCapsules } from "./physics/ragdoll/fit-bone-capsules.js"; +export type { BoneCapsule } from "./physics/ragdoll/fit-bone-capsules.js"; +export { shapeGeometry } from "./graphics/scene/model/shape/shape-geometry-plugin.js"; + +// --- Types (type + namespace, access .plugin for the ECS plugin) ------------- +export { Camera } from "./graphics/camera/camera.js"; +export { Light } from "./graphics/scene/light/light.js"; +export { Orbit } from "./graphics/camera/orbit/orbit.js"; +export { Node } from "./graphics/scene/node/node.js"; +export { Model } from "./graphics/scene/model/model.js"; +export { Geometry } from "./graphics/scene/model/geometry/geometry.js"; +export { SceneUniforms } from "./graphics/scene/scene-uniforms/scene-uniforms.js"; +export { StandardVertex } from "./graphics/rendering/standard-vertex/standard-vertex.js"; +export { VisibleMaterial } from "./graphics/rendering/visible-material/visible-material.js"; + +// --- Utilities --------------------------------------------------------------- +export { attachOrbitDrag } from "./graphics/camera/orbit/attach-orbit-drag.js"; +export type { OrbitDragService } from "./graphics/camera/orbit/attach-orbit-drag.js"; diff --git a/packages/data-gpu/src/material/material-plugin.ts b/packages/data-gpu/src/material/material-plugin.ts new file mode 100644 index 00000000..59aea4b3 --- /dev/null +++ b/packages/data-gpu/src/material/material-plugin.ts @@ -0,0 +1,61 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, Entity } from "@adobe/data/ecs"; +import { F32, Vec3, Vec4 } from "@adobe/data/math"; +import { standardMaterials } from "./standard-materials.js"; + +/** + * The material registry — authored materials as ECS entities (an open set; + * added rarely, edited never). Each `Material` row carries physical props (read + * by any solver) and visible PBR factors + texture-source URLs (read by the + * renderer). A body / model references a material by `Entity` id; the + * render-side `materialGpu` plugin derives GPU textures + a palette from these + * rows and caches them, rebuilding only when a new material appears. + * + * `seedStandardMaterials` inserts the standard library and records a + * `name → Entity` lookup in the `materials` resource. + */ +export const plugin = Database.Plugin.create({ + components: { + /** Reference (on a body, prop, or model) to a Material registry entity. */ + material: Entity.schema, + name: { type: "string" }, + density: F32.schema, + restitution: F32.schema, + friction: F32.schema, + compliance: F32.schema, + heatCapacity: F32.schema, + baseColorFactor: Vec4.schema, + emissiveFactor: Vec3.schema, + metallicFactor: F32.schema, + roughnessFactor: F32.schema, + normalScale: F32.schema, + occlusionStrength: F32.schema, + baseColorUrl: { type: "string" }, + metallicRoughnessUrl: { type: "string" }, + normalUrl: { type: "string" }, + occlusionUrl: { type: "string" }, + emissiveUrl: { type: "string" }, + }, + resources: { + /** name → Material entity, populated by `seedStandardMaterials`. */ + materials: { default: {} as Record }, + }, + archetypes: { + Material: [ + "name", + "density", "restitution", "friction", "compliance", "heatCapacity", + "baseColorFactor", "emissiveFactor", "metallicFactor", "roughnessFactor", "normalScale", "occlusionStrength", + "baseColorUrl", "metallicRoughnessUrl", "normalUrl", "occlusionUrl", "emissiveUrl", + ], + }, + transactions: { + seedStandardMaterials(t) { + const map: Record = {}; + for (const m of standardMaterials) { + map[m.name] = t.archetypes.Material.insert(m); + } + t.resources.materials = map; + }, + }, +}); diff --git a/packages/data-gpu/src/material/material.ts b/packages/data-gpu/src/material/material.ts new file mode 100644 index 00000000..332b6215 --- /dev/null +++ b/packages/data-gpu/src/material/material.ts @@ -0,0 +1,38 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Vec3, Vec4 } from "@adobe/data/math"; + +/** + * One row of the `Material` archetype — an authored material carrying both its + * physical properties (read by any physics solver) and its visible PBR + * properties (read by the renderer / material-array builder). Materials are an + * open, data-driven registry, not a closed enum: add a row, never edit code. + * + * Texture maps are *sources* (URLs) fetched at runtime; `""` means "use the + * neutral fallback layer". `metallicRoughnessUrl` and `occlusionUrl` may point + * at the same ARM image (AO = R, Roughness = G, Metalness = B). + */ +export interface Material { + name: string; + // physical (read by any solver) + density: number; + restitution: number; + friction: number; + compliance: number; + heatCapacity: number; + // visible PBR factors (multipliers over the sampled maps) + baseColorFactor: Vec4; + emissiveFactor: Vec3; + metallicFactor: number; + roughnessFactor: number; + normalScale: number; + occlusionStrength: number; + // texture sources, fetched at runtime ("" = neutral fallback layer) + baseColorUrl: string; + metallicRoughnessUrl: string; + normalUrl: string; + occlusionUrl: string; + emissiveUrl: string; +} + +export * as Material from "./public.js"; diff --git a/packages/data-gpu/src/material/public.ts b/packages/data-gpu/src/material/public.ts new file mode 100644 index 00000000..36a4c45e --- /dev/null +++ b/packages/data-gpu/src/material/public.ts @@ -0,0 +1,4 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export { plugin } from "./material-plugin.js"; +export { standardMaterials } from "./standard-materials.js"; diff --git a/packages/data-gpu/src/material/standard-materials.ts b/packages/data-gpu/src/material/standard-materials.ts new file mode 100644 index 00000000..aab113a5 --- /dev/null +++ b/packages/data-gpu/src/material/standard-materials.ts @@ -0,0 +1,43 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. +// +// PBR texture maps © Poly Haven, CC0 (https://polyhaven.com). Fetched at +// runtime by URL — never committed (see data-gpu-samples/CLAUDE.md). Poly +// Haven's `arm` map packs AO(R) / Roughness(G) / Metalness(B), matching glTF's +// occlusion + metallicRoughness channel conventions. + +import type { Material } from "./material.js"; + +const tex = (slug: string, map: string): string => + `https://dl.polyhaven.org/file/ph-assets/Textures/jpg/1k/${slug}/${slug}_${map}_1k.jpg`; + +/** + * A standard material: physical constants (the old material table) plus neutral + * visible factors — the textures supply colour / roughness / metalness, so the + * factors are left at 1 and baseColor white. + */ +function standard( + name: string, slug: string, + density: number, restitution: number, friction: number, compliance: number, heatCapacity: number, +): Material { + return { + name, + density, restitution, friction, compliance, heatCapacity, + baseColorFactor: [1, 1, 1, 1], + emissiveFactor: [0, 0, 0], + metallicFactor: 1, roughnessFactor: 1, normalScale: 1, occlusionStrength: 1, + baseColorUrl: tex(slug, "diff"), + metallicRoughnessUrl: tex(slug, "arm"), + normalUrl: tex(slug, "nor_gl"), + occlusionUrl: tex(slug, "arm"), + emissiveUrl: "", + }; +} + +/** The standard material library, seeded by `seedStandardMaterials`. */ +export const standardMaterials: readonly Material[] = [ + standard("rubber", "rubber_tiles", 1.1, 0.80, 0.90, 1e-5, 2.0), + standard("wood", "wood_table_001", 0.6, 0.35, 0.70, 5e-7, 1.7), + standard("stone", "cobblestone_floor_08", 2.6, 0.20, 0.85, 1e-8, 0.8), + standard("steel", "metal_plate", 7.8, 0.45, 0.50, 1e-9, 0.5), + standard("ice", "snow_02", 0.92, 0.25, 0.05, 5e-9, 2.1), +]; diff --git a/packages/data-gpu/src/physics/README.md b/packages/data-gpu/src/physics/README.md new file mode 100644 index 00000000..8107dba8 --- /dev/null +++ b/packages/data-gpu/src/physics/README.md @@ -0,0 +1,71 @@ +# Physics — feature roadmap + +A general-purpose, solver-agnostic physics layer for `@adobe/data-gpu`, suitable +for games, visualisation, and film/animation. Bodies are authored as ECS data +against a shared seam (`physicsData`); a pluggable **solver** (Jolt or Rapier) +advances them each frame. See `solvers/README.md` for choosing a solver. + +This file is the running plan — check the box when a feature lands so we don't +lose track. Keep it honest about approximations and limitations. + +## Done + +- [x] **Pluggable solver seam** — `physicsData` + interchangeable `joltSolver` / + `rapierSolver`; the same authored scene runs on either. +- [x] **Body types** — `static`, `dynamic`, `kinematic` (kinematic driven to its + authored pose each step, pushing dynamics). +- [x] **Collider shapes** — `sphere`, `box`, `capsule`, convex `hull` (authored + point cloud), static `mesh` (authored triangle soup). +- [x] **Fixed-timestep clock** (`physicsClock`, default 60 Hz, configurable) — + sim rate decoupled from render rate, spiral-of-death capped. +- [x] **Render-rate interpolation** — a pre-render pass blends prev→current into + the display pose (flat instances *and* the model `_worldMatrix` path). +- [x] **Auto-generated colliders from render geometry** — `ModelBody` / + `StaticModelCollider` with `colliderShape: hull | mesh` and no collision data; + generated from `collisionGeometry ?? geometry` (scale baked, cached). Manual + colliders (authored `convexPoints` / `colliderMesh`) still work. +- [x] **Efficient mirroring** — tag+exclude sync (O(new)), `getTypedArray` + write-back, reverse iteration on migrating loops. + +- [x] **Joints / constraints** — `fixed`, `point` (ball), `hinge` (revolute + + angle limits), `cone` (swing-twist: swing cone + twist range — anatomical + ragdoll limits). Demo: a hanging chain + a cone-limited arm. *`cone` limits are + full on Jolt; the Rapier compat binding has no cone constraint, so it + approximates `cone` as a free `point` — use `joltSolver` for ragdoll limits.* + +## Next + +- [ ] **Convex decomposition** (e.g. V-HACD) for concave *dynamic* colliders. + *Today a single auto-`hull` is a **convex approximation** — it fills + concavities (a chair's legs merge). Static concave geometry should use + `mesh`; concave dynamic bodies need decomposition into multiple hulls.* +- [x] **Per-bone colliders for skinned meshes** — `fitBoneCapsules` fits one + capsule per bone from the skin (skinned meshes are excluded from the rigid + auto-collider on purpose; they deform); `boneColliders` spawns a kinematic + capsule per bone and tracks the animated skeleton (`jointWorldMatrix · offset`). + Demo: the `ragdoll` sample (CesiumMan walk). Flipping to dynamic is next. +- [x] **Ragdoll controller + humanoid sample** — two backends behind the shared + `ragdollTrigger`, shown side by side in the `ragdoll` sample (CesiumMan walks, + then collapses onto the floor): + - **`joltRagdoll`** (Jolt-native) — Jolt's `Skeleton`/`RagdollSettings`/`Ragdoll` + with swing-twist limits + `DisableParentChildCollisions`; `DriveToPoseUsing- + Kinematics` while alive, falls + `GetPose` readback when limp. Built into the + solver's world via `_joltContext`. *Active ragdoll (`DriveToPoseUsingMotors`) + + velocity-seeding from the last animated motion are easy follow-ups.* + - **`boneColliders`** (generic) — our per-bone capsules + cone (Jolt) / free-ball + (Rapier) joints; `kinematic→dynamic` flip + `reconcileRagdoll` (world↔local) + so the skin flops. Runs on any solver. *Per-joint limit tuning is a follow-up.* +- [ ] **Collision events + groups/masks + sensors** — contact callbacks drained + to ECS; per-body layer masks; overlap-only sensor colliders. +- [ ] **Spatial queries** — raycast / shape-cast / overlap against the broadphase + (picking, line-of-sight, ground checks). +- [ ] **`uint32`-indexed primitives** — flat-shaded collider/render meshes + currently emit `uint16` indices (fine for authored ramps/props, not dense + terrain). +- [ ] **Deterministic headless joint tests** — the manual frame-stepping harness + (used by the solver benchmark) mis-times joint formation against async WASM + init, so joints are currently verified in-browser only. A proper harness + (await solver-ready before inserting bodies/joints) would let us assert e.g. + the cone clamp numerically. +- [ ] **Active ragdoll** — motorised joints driving toward the animation pose + while dynamic (struck-but-recovers). diff --git a/packages/data-gpu/src/physics/body/body-type/body-type.ts b/packages/data-gpu/src/physics/body/body-type/body-type.ts new file mode 100644 index 00000000..efb7a2e5 --- /dev/null +++ b/packages/data-gpu/src/physics/body/body-type/body-type.ts @@ -0,0 +1,14 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; +import { schema } from "./schema.js"; + +/** + * How the solver treats a body: + * static — never integrated; an immovable collider (the bulk of a scene). + * dynamic — fully simulated. + * kinematic — moved by external code; pushes dynamics but isn't pushed back. + */ +export type BodyType = Schema.ToType; + +export * as BodyType from "./public.js"; diff --git a/packages/data-gpu/src/physics/body/body-type/is-dynamic.ts b/packages/data-gpu/src/physics/body/body-type/is-dynamic.ts new file mode 100644 index 00000000..963b97d7 --- /dev/null +++ b/packages/data-gpu/src/physics/body/body-type/is-dynamic.ts @@ -0,0 +1,8 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { BodyType } from "./body-type.js"; + +/** True for bodies the solver integrates and writes back (vs static / kinematic). */ +export function isDynamic(t: BodyType): boolean { + return t === "dynamic"; +} diff --git a/packages/data-gpu/src/physics/body/body-type/public.ts b/packages/data-gpu/src/physics/body/body-type/public.ts new file mode 100644 index 00000000..02ecf2db --- /dev/null +++ b/packages/data-gpu/src/physics/body/body-type/public.ts @@ -0,0 +1,4 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export { schema } from "./schema.js"; +export { isDynamic } from "./is-dynamic.js"; diff --git a/packages/data-gpu/src/physics/body/body-type/schema.ts b/packages/data-gpu/src/physics/body/body-type/schema.ts new file mode 100644 index 00000000..bc90cc56 --- /dev/null +++ b/packages/data-gpu/src/physics/body/body-type/schema.ts @@ -0,0 +1,5 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; + +export const schema = { type: "string", enum: ["static", "dynamic", "kinematic"] } as const satisfies Schema; diff --git a/packages/data-gpu/src/physics/body/collider-mesh.ts b/packages/data-gpu/src/physics/body/collider-mesh.ts new file mode 100644 index 00000000..c192e4be --- /dev/null +++ b/packages/data-gpu/src/physics/body/collider-mesh.ts @@ -0,0 +1,13 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +/** + * Authored triangle mesh for a **static** `mesh` collider (terrain, level + * geometry): flat vertex positions (xyz triples) + triangle indices. Static only + * — a triangle soup has no interior, so it can't be a moving body. Held as a + * runtime object component (`colliderMesh`); the solver reads it once to build + * the engine trimesh, the bridge once to build the render mesh. + */ +export interface ColliderMesh { + positions: Float32Array; // xyz triples + indices: Uint32Array; // triangle list (3 indices per face) +} diff --git a/packages/data-gpu/src/physics/body/collider-shape/collider-shape.ts b/packages/data-gpu/src/physics/body/collider-shape/collider-shape.ts new file mode 100644 index 00000000..7234f1ed --- /dev/null +++ b/packages/data-gpu/src/physics/body/collider-shape/collider-shape.ts @@ -0,0 +1,19 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; +import { schema } from "./schema.js"; + +/** + * Collider geometry. `halfExtents` carries the dimensions: + * box — all three axes (half-extents). + * sphere — `.x` is the radius. + * capsule — `.x` is the radius, `.y` the cylinder's half-height; Y-aligned + * (total height 2·(y + x)). `.z` unused. + * hull — convex hull of an authored `convexPoints` cloud; `halfExtents` + * is unused (the geometry comes from the points). + * mesh — authored `colliderMesh` triangle soup; **static only** (no + * interior). `halfExtents` unused. Terrain / level geometry. + */ +export type ColliderShape = Schema.ToType; + +export * as ColliderShape from "./public.js"; diff --git a/packages/data-gpu/src/physics/body/collider-shape/list.ts b/packages/data-gpu/src/physics/body/collider-shape/list.ts new file mode 100644 index 00000000..d231c59a --- /dev/null +++ b/packages/data-gpu/src/physics/body/collider-shape/list.ts @@ -0,0 +1,7 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { ColliderShape } from "./collider-shape.js"; +import { schema } from "./schema.js"; + +/** Stable shape order — the index is the solver's numeric shape id. */ +export const list: readonly ColliderShape[] = schema.enum; diff --git a/packages/data-gpu/src/physics/body/collider-shape/mass-properties.ts b/packages/data-gpu/src/physics/body/collider-shape/mass-properties.ts new file mode 100644 index 00000000..25fc43db --- /dev/null +++ b/packages/data-gpu/src/physics/body/collider-shape/mass-properties.ts @@ -0,0 +1,47 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { ColliderShape } from "./collider-shape.js"; + +/** + * Mass + diagonal inverse inertia for a shape of the given half-extents and + * density. The per-shape formulas live here (with the type) rather than leaking + * the shape members into the solver. Allocation-free: the diagonal inverse + * inertia is written into `outInvInertia[o..o+2]` and the inverse mass returned + * (this runs per dynamic body per frame in the solver gather). Density comes + * from the body's material. + */ +export function massProperties( + shape: ColliderShape, hx: number, hy: number, hz: number, density: number, + outInvInertia: Float32Array, o: number, +): number { + if (shape === "box") { + const mass = density * 8 * hx * hy * hz; + const ix = (mass / 3) * (hy * hy + hz * hz); + const iy = (mass / 3) * (hx * hx + hz * hz); + const iz = (mass / 3) * (hx * hx + hy * hy); + outInvInertia[o] = 1 / ix; outInvInertia[o + 1] = 1 / iy; outInvInertia[o + 2] = 1 / iz; + return 1 / mass; + } + if (shape === "capsule") { + // Y-aligned: radius r = hx, cylinder half-height = hy (length L = 2·hy). + // A cylinder plus two hemispheres (together one sphere of radius r). + const r = hx, L = 2 * hy; + const mc = density * Math.PI * r * r * L; // cylinder + const ms = density * (4 / 3) * Math.PI * r * r * r; // two hemispheres = a sphere + const mass = mc + ms; + // axial (Y): cylinder ½mc r² + sphere ⅖ms r² + const iy = 0.5 * mc * r * r + 0.4 * ms * r * r; + // transverse (X = Z): cylinder about centre, plus each hemisphere's own + // inertia (83/320·mh r²) shifted by parallel axis to the capsule centre + // (COM offset L/2 + 3r/8). See standard capsule-inertia derivation. + const d = L / 2 + 3 * r / 8; + const ix = mc * (L * L / 12 + r * r / 4) + ms * ((83 / 320) * r * r + d * d); + outInvInertia[o] = 1 / ix; outInvInertia[o + 1] = 1 / iy; outInvInertia[o + 2] = 1 / ix; + return 1 / mass; + } + // sphere — radius in hx + const mass = density * (4 / 3) * Math.PI * hx * hx * hx; + const inv = 1 / (0.4 * mass * hx * hx); + outInvInertia[o] = inv; outInvInertia[o + 1] = inv; outInvInertia[o + 2] = inv; + return 1 / mass; +} diff --git a/packages/data-gpu/src/physics/body/collider-shape/public.ts b/packages/data-gpu/src/physics/body/collider-shape/public.ts new file mode 100644 index 00000000..86e243d5 --- /dev/null +++ b/packages/data-gpu/src/physics/body/collider-shape/public.ts @@ -0,0 +1,6 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export { schema } from "./schema.js"; +export { list } from "./list.js"; +export { toIndex } from "./to-index.js"; +export { massProperties } from "./mass-properties.js"; diff --git a/packages/data-gpu/src/physics/body/collider-shape/schema.ts b/packages/data-gpu/src/physics/body/collider-shape/schema.ts new file mode 100644 index 00000000..73cbf1c6 --- /dev/null +++ b/packages/data-gpu/src/physics/body/collider-shape/schema.ts @@ -0,0 +1,5 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; + +export const schema = { type: "string", enum: ["sphere", "box", "capsule", "hull", "mesh"] } as const satisfies Schema; diff --git a/packages/data-gpu/src/physics/body/collider-shape/to-index.ts b/packages/data-gpu/src/physics/body/collider-shape/to-index.ts new file mode 100644 index 00000000..641745ec --- /dev/null +++ b/packages/data-gpu/src/physics/body/collider-shape/to-index.ts @@ -0,0 +1,8 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { ColliderShape } from "./collider-shape.js"; + +/** The solver's numeric shape id (sphere 0, box 1, capsule 2, hull 3, mesh 4). */ +export function toIndex(shape: ColliderShape): number { + return shape === "box" ? 1 : shape === "capsule" ? 2 : shape === "hull" ? 3 : shape === "mesh" ? 4 : 0; +} diff --git a/packages/data-gpu/src/physics/body/rigid-body.ts b/packages/data-gpu/src/physics/body/rigid-body.ts new file mode 100644 index 00000000..e017006e --- /dev/null +++ b/packages/data-gpu/src/physics/body/rigid-body.ts @@ -0,0 +1,26 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Entity } from "@adobe/data/ecs"; +import type { Vec3, Quat } from "@adobe/data/math"; +import type { BodyType } from "./body-type/body-type.js"; +import type { ColliderShape } from "./collider-shape/collider-shape.js"; + +/** + * One row of the `RigidBody` archetype — the authored surface every physics + * solver reads, and whose `position`/`rotation`/velocity it writes back each + * frame. Mass and inertia are *derived* from `colliderShape` + `halfExtents` + + * `material` (see ColliderShape.massProperties), so they aren't stored here. + */ +export interface RigidBody { + bodyType: BodyType; + colliderShape: ColliderShape; + /** Box extents on all axes; sphere radius in `.x`. */ + halfExtents: Vec3; + /** Reference to a Material registry entity (physical + visible props). */ + material: Entity; + position: Vec3; + /** Unified with Node.rotation so a body is directly renderable. */ + rotation: Quat; + linearVelocity: Vec3; + angularVelocity: Vec3; +} diff --git a/packages/data-gpu/src/physics/body/static-collider.ts b/packages/data-gpu/src/physics/body/static-collider.ts new file mode 100644 index 00000000..aaecbf6f --- /dev/null +++ b/packages/data-gpu/src/physics/body/static-collider.ts @@ -0,0 +1,28 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Entity } from "@adobe/data/ecs"; +import type { Vec3, Quat } from "@adobe/data/math"; +import type { ColliderShape } from "./collider-shape/collider-shape.js"; + +/** + * One row of the `StaticCollider` archetype — an immovable collider (floor, + * wall, ramp, arbitrary scenery). Dynamic bodies collide with it, but the + * solver never integrates it, so it carries no velocity columns: a lean + * representation for the bulk-static workload (many static, few dynamic). + * + * It is the same authored surface as a `RigidBody` minus `bodyType` (the + * archetype *is* the "static" classification) and the velocity components. + * Friction / restitution / compliance come from its `material`, exactly as for + * a dynamic body. Like a RigidBody, its `position`/`rotation` double as the + * renderable transform (the render bridge gives it geometry). + */ +export interface StaticCollider { + colliderShape: ColliderShape; + /** Box extents on all axes; sphere radius in `.x`. */ + halfExtents: Vec3; + /** Reference to a Material registry entity (physical + visible props). */ + material: Entity; + position: Vec3; + /** Unified with Node.rotation so the collider is directly renderable. */ + rotation: Quat; +} diff --git a/packages/data-gpu/src/physics/joint/joint-plugin.ts b/packages/data-gpu/src/physics/joint/joint-plugin.ts new file mode 100644 index 00000000..f57ca2f5 --- /dev/null +++ b/packages/data-gpu/src/physics/joint/joint-plugin.ts @@ -0,0 +1,33 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database, Entity } from "@adobe/data/ecs"; +import { F32 } from "@adobe/data/schema"; +import { Vec3 } from "@adobe/data/math"; +import { JointType } from "./joint-type/joint-type.js"; + +/** + * jointData — the solver-agnostic constraint data model. A `Joint` connects two + * bodies (`jointBodyA`/`jointBodyB`, any solver's bodies) with anchors in each + * body's local frame; the active solver mirrors each joint into its engine once + * both bodies exist (tag + exclude, like body sync). Authored only — no systems. + * + * Independent of `physicsData` (it only references bodies by `Entity`), so a + * solver `combine`s both. See `joint.ts` for the row type and `README.md` for + * the roadmap (joints are the ragdoll prerequisite). + */ +export const jointData = Database.Plugin.create({ + components: { + jointType: JointType.schema, + jointBodyA: Entity.schema, + jointBodyB: Entity.schema, + jointAnchorA: Vec3.schema, // anchor on body A, A-local + jointAnchorB: Vec3.schema, // anchor on body B, B-local + jointAxis: Vec3.schema, // hinge/cone reference axis (A-local); unused for fixed/point + jointMinLimit: F32.schema, // hinge angle / cone twist lower bound (rad); min >= max ⇒ no limit + jointMaxLimit: F32.schema, // hinge angle / cone twist upper bound (rad) + jointSwingLimit: F32.schema, // cone swing half-angle (rad); cone only + }, + archetypes: { + Joint: ["jointType", "jointBodyA", "jointBodyB", "jointAnchorA", "jointAnchorB", "jointAxis", "jointMinLimit", "jointMaxLimit", "jointSwingLimit"], + }, +}); diff --git a/packages/data-gpu/src/physics/joint/joint-type/joint-type.ts b/packages/data-gpu/src/physics/joint/joint-type/joint-type.ts new file mode 100644 index 00000000..bd4f544a --- /dev/null +++ b/packages/data-gpu/src/physics/joint/joint-type/joint-type.ts @@ -0,0 +1,19 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; +import { schema } from "./schema.js"; + +/** + * How a joint constrains the two bodies it connects: + * fixed — rigidly locks them together (no relative motion). + * point — a ball/spherical joint: anchors coincide, free rotation (chains, ragdoll joints). + * hinge — a revolute joint: 1 rotational DOF about an axis, with optional angle limits (doors, elbows). + * cone — a swing-twist joint: the bone axis is bound to a cone (half-angle + * `jointSwingLimit`) around the reference axis, with a twist range + * (`jointMinLimit`/`jointMaxLimit`) about it — anatomical shoulder/hip + * limits for ragdolls. Full on Jolt (SwingTwist); Rapier's compat + * binding has no cone limit, so it approximates `cone` as a free `point`. + */ +export type JointType = Schema.ToType; + +export * as JointType from "./public.js"; diff --git a/packages/data-gpu/src/physics/joint/joint-type/public.ts b/packages/data-gpu/src/physics/joint/joint-type/public.ts new file mode 100644 index 00000000..fa4419e7 --- /dev/null +++ b/packages/data-gpu/src/physics/joint/joint-type/public.ts @@ -0,0 +1,3 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +export { schema } from "./schema.js"; diff --git a/packages/data-gpu/src/physics/joint/joint-type/schema.ts b/packages/data-gpu/src/physics/joint/joint-type/schema.ts new file mode 100644 index 00000000..de145043 --- /dev/null +++ b/packages/data-gpu/src/physics/joint/joint-type/schema.ts @@ -0,0 +1,5 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Schema } from "@adobe/data/schema"; + +export const schema = { type: "string", enum: ["fixed", "point", "hinge", "cone"] } as const satisfies Schema; diff --git a/packages/data-gpu/src/physics/joint/joint.ts b/packages/data-gpu/src/physics/joint/joint.ts new file mode 100644 index 00000000..65ee8f85 --- /dev/null +++ b/packages/data-gpu/src/physics/joint/joint.ts @@ -0,0 +1,27 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Entity } from "@adobe/data/ecs"; +import type { Vec3 } from "@adobe/data/math"; +import type { JointType } from "./joint-type/joint-type.js"; + +/** + * One row of the `Joint` archetype — a constraint between two bodies, solved by + * whichever physics solver is active. Anchors are in each body's local frame; + * at rest the two anchor points coincide in world space (the joint location). + * + * `jointAxis` is the reference axis (body-A local) for `hinge` and `cone`, + * ignored otherwise. `jointMinLimit`/`jointMaxLimit` bound the hinge angle, or + * the `cone` twist angle, in radians (`min >= max` ⇒ free); `jointSwingLimit` is + * the `cone` swing half-angle. + */ +export interface Joint { + jointType: JointType; + jointBodyA: Entity; + jointBodyB: Entity; + jointAnchorA: Vec3; + jointAnchorB: Vec3; + jointAxis: Vec3; + jointMinLimit: number; + jointMaxLimit: number; + jointSwingLimit: number; +} diff --git a/packages/data-gpu/src/physics/physics-clock-plugin.ts b/packages/data-gpu/src/physics/physics-clock-plugin.ts new file mode 100644 index 00000000..b605ff14 --- /dev/null +++ b/packages/data-gpu/src/physics/physics-clock-plugin.ts @@ -0,0 +1,61 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { core } from "../core/core-plugin.js"; + +/** + * Fixed-timestep physics clock — decouples the simulation rate from the render + * rate (Fiedler "fix your timestep"). Each render frame the accumulator absorbs + * the variable `frameTime.dt`; the solver then runs a whole number of fixed + * `fixedDt` steps (0, 1, or several), and `alpha` is the leftover fraction used + * to interpolate the rendered pose between the last two simulated states. + * + * Default 60 Hz; set another rate at init with `setFixedTimestep(hz)`. + */ +export interface PhysicsClock { + /** Fixed simulation step length, seconds (default 1/60). */ + readonly fixedDt: number; + /** Unsimulated time carried to the next frame, seconds (< fixedDt). */ + readonly accumulator: number; + /** accumulator / fixedDt ∈ [0,1): render interpolation fraction prev→current. */ + readonly alpha: number; + /** Whole steps the solver should run this frame (0..maxSubSteps). */ + readonly steps: number; + /** Cap on steps/frame — prevents the "spiral of death" after a long stall. */ + readonly maxSubSteps: number; +} + +const DEFAULT: PhysicsClock = { fixedDt: 1 / 60, accumulator: 0, alpha: 0, steps: 0, maxSubSteps: 8 }; + +/** + * Provides the `physicsClock` resource and advances it once per render frame. + * Solver plugins extend this and run `physicsClock.steps` fixed steps; the + * interpolation system reads `physicsClock.alpha`. The advance runs first in the + * `physics` phase so solvers (scheduled `after: ["advancePhysicsClock"]`) see it. + */ +export const physicsClock = Database.Plugin.create({ + extends: core, + resources: { + physicsClock: { default: DEFAULT as PhysicsClock, transient: true }, + }, + transactions: { + /** Set the simulation rate (Hz). Call once at init; default is 60. */ + setFixedTimestep(t, hz: number) { + t.resources.physicsClock = { ...t.resources.physicsClock, fixedDt: 1 / hz }; + }, + }, + systems: { + advancePhysicsClock: { + schedule: { during: ["physics"] }, + create: db => () => { + const c = db.store.resources.physicsClock; + const h = c.fixedDt; + // clamp the absorbed time to the step budget so a long stall (tab + // backgrounded, GC) can't queue hundreds of steps (spiral of death) + const acc = c.accumulator + Math.min(db.store.resources.frameTime.dt, h * c.maxSubSteps); + const steps = Math.min(Math.floor(acc / h), c.maxSubSteps); + db.store.resources.physicsClock = { ...c, accumulator: acc - steps * h, alpha: (acc - steps * h) / h, steps }; + }, + }, + }, +}); diff --git a/packages/data-gpu/src/physics/physics-clock.test.ts b/packages/data-gpu/src/physics/physics-clock.test.ts new file mode 100644 index 00000000..e730f404 --- /dev/null +++ b/packages/data-gpu/src/physics/physics-clock.test.ts @@ -0,0 +1,55 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { describe, it, expect } from "vitest"; +import { Database } from "@adobe/data/ecs"; +import { physicsClock, type PhysicsClock } from "./physics-clock-plugin.js"; + +/** The accumulator turns a variable render dt into a whole number of fixed steps + * plus a leftover interpolation `alpha`, capped against the spiral of death. */ +describe("physicsClock", () => { + // A created database carries a writable `store` (runtime invariant; not on the + // public type — same loose access the solver benchmark uses to drive frames). + interface ClockStore { resources: { physicsClock: PhysicsClock; frameTime: { now: number; dt: number; elapsed: number } } } + + const make = () => { + const db = Database.create(physicsClock); // 60 Hz default + const store = (db as unknown as { store: ClockStore }).store; + const advance = (dt: number): PhysicsClock => { + store.resources.frameTime = { now: 0, dt, elapsed: 0 }; + db.system.functions.advancePhysicsClock?.(); + return store.resources.physicsClock; + }; + return { db, advance }; + }; + + it("matched render rate → exactly one step per frame, alpha ≈ 0", () => { + const c = make().advance(1 / 60); + expect(c.steps).toBe(1); + expect(c.alpha).toBeCloseTo(0); + }); + + it("render slower than sim → multiple steps per frame", () => { + expect(make().advance(1 / 30).steps).toBe(2); // two 1/60 steps fit one 1/30 frame + }); + + it("render faster than sim → 0-step frames that accumulate, alpha rising then resetting", () => { + const { advance } = make(); + const a = advance(1 / 120); // half a step accrued + expect(a.steps).toBe(0); + expect(a.alpha).toBeCloseTo(0.5); + const b = advance(1 / 120); // now a full step is due + expect(b.steps).toBe(1); + expect(b.alpha).toBeCloseTo(0); + }); + + it("a long stall is capped at maxSubSteps (no spiral of death)", () => { + const c = make().advance(10); // 600 steps' worth of time + expect(c.steps).toBe(c.maxSubSteps); + }); + + it("setFixedTimestep changes the rate", () => { + const { db, advance } = make(); + db.transactions.setFixedTimestep(30); + expect(advance(1 / 30).steps).toBe(1); // one 1/30 step per 1/30 frame + }); +}); diff --git a/packages/data-gpu/src/physics/physics-data-plugin.ts b/packages/data-gpu/src/physics/physics-data-plugin.ts new file mode 100644 index 00000000..33ce4ca5 --- /dev/null +++ b/packages/data-gpu/src/physics/physics-data-plugin.ts @@ -0,0 +1,73 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; +import { F32 } from "@adobe/data/schema"; +import { Vec3, Quat } from "@adobe/data/math"; +import { BodyType } from "./body/body-type/body-type.js"; +import { ColliderShape } from "./body/collider-shape/collider-shape.js"; +import type { ColliderMesh } from "./body/collider-mesh.js"; +import { Material } from "../material/material.js"; + +/** + * The shared, solver-agnostic rigid-body data model — the seam every physics + * solver plugs into. It declares *only* authored/canonical state (no systems): + * + * bodyType, colliderShape, halfExtents, material — authored shape + role + * position, rotation, linearVelocity, angularVelocity — canonical live state + * + * A **solver plugin** `extends` this, may keep its own private `_`-prefixed + * internal state (broadphase, caches, GPU buffers), and each frame reads the + * authored state and writes back `position`/`rotation`/velocity for dynamic + * bodies. A renderer reads the canonical transforms, decoupled from which + * solver produced them — so solvers (`joltSolver`, `rapierSolver`; the GPU XPBD + * solver is shelved) are interchangeable over identical authored scenes. + * + * Two archetypes share these components: + * - **RigidBody** — `dynamic` or `kinematic` bodies that move; carries + * velocities and a `bodyType` discriminator. + * - **StaticCollider** — immovable colliders (floor, wall, ramp, scenery). + * No velocities (never integrated) and no `bodyType` (the archetype *is* + * the static classification) — a lean row for the bulk-static workload. + * + * `_prevPosition`/`_prevRotation` are derived (`_`-prefixed): the solver snapshots + * the pose *before* its final fixed step into them, so a render-rate interpolation + * pass can smoothly blend the previous → current simulated pose (see + * `physics-clock-plugin` + `interpolation-plugin`). Like `_worldMatrix`, they are + * *not* in the authored archetype — the solver migrates them onto each dynamic + * body the first time it mirrors it, so authors never supply derived state. + * + * Mass + inertia are derived per solver from shape + material (not stored here). + */ +export const physicsData = Database.Plugin.create({ + extends: Material.plugin, // brings the `material` reference component + components: { + bodyType: BodyType.schema, + colliderShape: ColliderShape.schema, + halfExtents: Vec3.schema, // box extents; sphere radius in .x + position: Vec3.schema, + rotation: Quat.schema, // unified with Node.rotation so a body renders directly + linearVelocity: Vec3.schema, + angularVelocity: Vec3.schema, + _prevPosition: Vec3.schema, // derived: pose before the last fixed step (render interpolation) + _prevRotation: Quat.schema, + // Bodies sharing the same non-zero collisionGroup do not collide with each + // other (they still collide with group 0 / the world) — e.g. a ragdoll's + // bones. 0 = default (collide with everything). Honored by both solvers + // (Rapier per-collider groups; Jolt a no-self-collide object layer). Note: + // it's currently binary (group 0 vs "some non-zero group"), not per-id masks. + collisionGroup: F32.schema, + // Authored collision geometry for shapes `halfExtents` can't describe. + // Runtime objects (variable length, no schema): the solver reads them once + // when it mirrors the body, the bridge once to build the render mesh. + convexPoints: { default: null as Float32Array | null }, // colliderShape "hull": point cloud → convex hull + colliderMesh: { default: null as ColliderMesh | null }, // colliderShape "mesh": static triangle soup + }, + archetypes: { + RigidBody: ["bodyType", "colliderShape", "halfExtents", "material", "position", "rotation", "linearVelocity", "angularVelocity"], + StaticCollider: ["colliderShape", "halfExtents", "material", "position", "rotation"], + // A convex-hull body (dynamic / kinematic): authored as a point cloud. + ConvexBody: ["bodyType", "colliderShape", "halfExtents", "material", "position", "rotation", "linearVelocity", "angularVelocity", "convexPoints"], + // A static triangle-mesh collider (terrain / level geometry). + MeshCollider: ["colliderShape", "halfExtents", "material", "position", "rotation", "colliderMesh"], + }, +}); diff --git a/packages/data-gpu/src/physics/ragdoll/fit-bone-capsules.test.ts b/packages/data-gpu/src/physics/ragdoll/fit-bone-capsules.test.ts new file mode 100644 index 00000000..5f5dc4af --- /dev/null +++ b/packages/data-gpu/src/physics/ragdoll/fit-bone-capsules.test.ts @@ -0,0 +1,66 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { describe, it, expect } from "vitest"; +import { fitBoneCapsules, type SkinVertices } from "./fit-bone-capsules.js"; + +const IDENTITY_IBM = (n: number) => { + const m = new Float32Array(n * 16); + for (let j = 0; j < n; j++) { m[j * 16] = 1; m[j * 16 + 5] = 1; m[j * 16 + 10] = 1; m[j * 16 + 15] = 1; } + return m; +}; + +/** 8 corners of a box, all dominantly weighted to `joint`. */ +function boxCorners(cx: number, cy: number, cz: number, hx: number, hy: number, hz: number, joint: number) { + const pos: number[] = [], jn: number[] = [], wt: number[] = []; + for (const sx of [-1, 1]) for (const sy of [-1, 1]) for (const sz of [-1, 1]) { + pos.push(cx + sx * hx, cy + sy * hy, cz + sz * hz); + jn.push(joint, 0, 0, 0); wt.push(1, 0, 0, 0); + } + return { pos, jn, wt }; +} + +const skinFrom = (...parts: { pos: number[]; jn: number[]; wt: number[] }[]): SkinVertices => ({ + positions: new Float32Array(parts.flatMap(p => p.pos)), + joints: new Uint32Array(parts.flatMap(p => p.jn)), + weights: new Float32Array(parts.flatMap(p => p.wt)), +}); + +// rotate (0,1,0) by a quaternion +const rotateUp = (q: readonly number[]): [number, number, number] => { + const [x, y, z, w] = q; + return [2 * (x * y - w * z), 1 - 2 * (x * x + z * z), 2 * (y * z + w * x)]; +}; + +describe("fitBoneCapsules", () => { + it("fits an X-elongated bone to an X-axis capsule of the right size", () => { + // box x∈[-2,2], y∈[-0.5,0.5], z∈[-0.4,0.4] → axis X, radius ½·max(1,0.8)=0.5, halfHeight 2−0.5=1.5 + const caps = fitBoneCapsules({ jointCount: 1, inverseBindMatrices: IDENTITY_IBM(1), skin: skinFrom(boxCorners(0, 0, 0, 2, 0.5, 0.4, 0)) }); + expect(caps).toHaveLength(1); + const c = caps[0]; + expect(c.jointIndex).toBe(0); + expect(c.radius).toBeCloseTo(0.5); + expect(c.halfHeight).toBeCloseTo(1.5); + expect(c.offsetPosition[0]).toBeCloseTo(0); + // the capsule's local +Y, rotated by the offset, points along world X (the fit axis) + const up = rotateUp(c.offsetRotation); + expect(Math.abs(up[0])).toBeCloseTo(1); + expect(Math.abs(up[1])).toBeCloseTo(0); + }); + + it("fits one capsule per bone, grouping vertices by dominant weight", () => { + const caps = fitBoneCapsules({ + jointCount: 2, inverseBindMatrices: IDENTITY_IBM(2), + skin: skinFrom(boxCorners(0, 0, 0, 1, 0.3, 0.3, 0), boxCorners(0, 5, 0, 0.3, 1, 0.3, 1)), + }); + expect(caps.map(c => c.jointIndex).sort()).toEqual([0, 1]); + // bone 1 is Y-elongated → its capsule axis is Y (offset rotation ≈ identity) + const c1 = caps.find(c => c.jointIndex === 1)!; + expect(rotateUp(c1.offsetRotation)[1]).toBeCloseTo(1); + expect(c1.offsetPosition[1]).toBeCloseTo(5); + }); + + it("skips bones with too few assigned vertices", () => { + const sparse = { pos: [0, 0, 0, 1, 0, 0, 2, 0, 0], jn: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], wt: [1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0] }; + expect(fitBoneCapsules({ jointCount: 1, inverseBindMatrices: IDENTITY_IBM(1), skin: skinFrom(sparse) })).toHaveLength(0); + }); +}); diff --git a/packages/data-gpu/src/physics/ragdoll/fit-bone-capsules.ts b/packages/data-gpu/src/physics/ragdoll/fit-bone-capsules.ts new file mode 100644 index 00000000..aadabed7 --- /dev/null +++ b/packages/data-gpu/src/physics/ragdoll/fit-bone-capsules.ts @@ -0,0 +1,106 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Vec3, Quat } from "@adobe/data/math"; + +/** + * Fits one capsule collider to each significant bone of a skinned mesh — the + * per-bone colliders a ragdoll needs (you can't trimesh a deforming skin). Each + * vertex is assigned to the bone it's *most* weighted to; that bone's vertices + * are pushed into the bone's bind-local frame (via the joint's inverse-bind + * matrix) and a capsule is fitted to them: the longest local axis becomes the + * capsule axis, the perpendicular spread its radius. + * + * The result is expressed as a bone-local offset (`offsetPosition` / + * `offsetRotation`, which orients the Y-aligned capsule onto the fitted axis) + * plus dimensions — so the capsule's world pose each frame is just + * `jointWorldMatrix · offset`, tracking the animated skeleton. + */ + +/** Skinned vertices in mesh (bind) space: `positions` (xyz triples), `joints` + * (4 joint indices/vertex), `weights` (4 weights/vertex). */ +export interface SkinVertices { + positions: Float32Array; + joints: Uint32Array; + weights: Float32Array; +} + +export interface FitBoneCapsulesInput { + jointCount: number; + /** glTF inverse-bind matrices, column-major, `jointCount × 16`. */ + inverseBindMatrices: Float32Array; + skin: SkinVertices; + /** A vertex joins a bone only if its dominant weight ≥ this (default 0.5). */ + minWeight?: number; + /** Bones with fewer assigned vertices than this get no capsule (default 6). */ + minVertices?: number; +} + +export interface BoneCapsule { + jointIndex: number; + offsetPosition: Vec3; // capsule centre, bone-bind-local + offsetRotation: Quat; // orients the Y-aligned capsule onto the fitted axis, bone-bind-local + radius: number; + halfHeight: number; // half-length of the cylindrical section +} + +/** Transform a point by a column-major 4×4 matrix (slice `o..o+16`), into `out`. */ +function transformPoint(m: Float32Array, o: number, x: number, y: number, z: number, out: [number, number, number]): void { + out[0] = m[o] * x + m[o + 4] * y + m[o + 8] * z + m[o + 12]; + out[1] = m[o + 1] * x + m[o + 5] * y + m[o + 9] * z + m[o + 13]; + out[2] = m[o + 2] * x + m[o + 6] * y + m[o + 10] * z + m[o + 14]; +} + +/** Shortest-arc quaternion rotating +Y onto unit vector `b`. */ +function quatFromY(bx: number, by: number, bz: number): Quat { + if (by > 0.99999) return [0, 0, 0, 1]; // already +Y + if (by < -0.99999) return [0, 0, 1, 0]; // 180° (about Z) onto −Y + // half-vector method: h = normalize((0,1,0) + b); q = ((0,1,0)×h, (0,1,0)·h) + const hx = bx, hy = 1 + by, hz = bz; + const hl = Math.hypot(hx, hy, hz) || 1; + const nx = hx / hl, ny = hy / hl, nz = hz / hl; + // (0,1,0) × (nx,ny,nz) = (nz·? ) → cross = (1*nz - 0*ny, 0*nx - 0*nz, 0*ny - 1*nx) = (nz, 0, -nx) + return [nz, 0, -nx, ny]; +} + +export function fitBoneCapsules(input: FitBoneCapsulesInput): BoneCapsule[] { + const { jointCount, inverseBindMatrices: ibm, skin } = input; + const minWeight = input.minWeight ?? 0.5; + const minVertices = input.minVertices ?? 6; + const vertexCount = skin.positions.length / 3; + + // bone-local AABB per joint (min/max), accumulated over its dominant vertices + const min = new Float32Array(jointCount * 3).fill(Infinity); + const max = new Float32Array(jointCount * 3).fill(-Infinity); + const count = new Uint32Array(jointCount); + const p: [number, number, number] = [0, 0, 0]; + + for (let v = 0; v < vertexCount; v++) { + // dominant bone = the vertex's max-weight joint + let best = 0, bestW = -1; + for (let k = 0; k < 4; k++) { const w = skin.weights[v * 4 + k]; if (w > bestW) { bestW = w; best = skin.joints[v * 4 + k]; } } + if (bestW < minWeight || best >= jointCount) continue; + transformPoint(ibm, best * 16, skin.positions[v * 3], skin.positions[v * 3 + 1], skin.positions[v * 3 + 2], p); + const b = best * 3; + if (p[0] < min[b]) min[b] = p[0]; if (p[0] > max[b]) max[b] = p[0]; + if (p[1] < min[b + 1]) min[b + 1] = p[1]; if (p[1] > max[b + 1]) max[b + 1] = p[1]; + if (p[2] < min[b + 2]) min[b + 2] = p[2]; if (p[2] > max[b + 2]) max[b + 2] = p[2]; + count[best]++; + } + + const out: BoneCapsule[] = []; + for (let j = 0; j < jointCount; j++) { + if (count[j] < minVertices) continue; + const b = j * 3; + const ex = max[b] - min[b], ey = max[b + 1] - min[b + 1], ez = max[b + 2] - min[b + 2]; + // longest extent = capsule axis; the other two give the (bounding) radius + let axis: 0 | 1 | 2 = 0, axisExtent = ex, r1 = ey, r2 = ez; + if (ey >= ex && ey >= ez) { axis = 1; axisExtent = ey; r1 = ex; r2 = ez; } + else if (ez >= ex && ez >= ey) { axis = 2; axisExtent = ez; r1 = ex; r2 = ey; } + const radius = Math.max(r1, r2) / 2 || 1e-3; + const halfHeight = Math.max(0, axisExtent / 2 - radius); + const offsetPosition: Vec3 = [(min[b] + max[b]) / 2, (min[b + 1] + max[b + 1]) / 2, (min[b + 2] + max[b + 2]) / 2]; + const offsetRotation = quatFromY(axis === 0 ? 1 : 0, axis === 1 ? 1 : 0, axis === 2 ? 1 : 0); + out.push({ jointIndex: j, offsetPosition, offsetRotation, radius, halfHeight }); + } + return out; +} diff --git a/packages/data-gpu/src/physics/solvers/README.md b/packages/data-gpu/src/physics/solvers/README.md new file mode 100644 index 00000000..29c104da --- /dev/null +++ b/packages/data-gpu/src/physics/solvers/README.md @@ -0,0 +1,63 @@ +# Physics solvers + +Rigid-body solvers that plug into the shared `physicsData` seam. A solver reads +the authored `RigidBody` + `StaticCollider` components and writes back +`position`/`rotation`/velocity for dynamic bodies each frame, so the **same +scene runs unchanged on any solver** (`Database.Plugin.combine(scene, solver)`). + +Two production-grade solvers ship, both compiled to WASM and wired identically: + +| solver | engine | license | +| --- | --- | --- | +| `joltSolver` | Jolt Physics (Jorrit Rouwe) | MIT | +| `rapierSolver` | Rapier (dimforge) | Apache-2.0 | + +Both are **opt-in**: the engine packages are regular dependencies, but with +`sideEffects: false` a bundler tree-shakes the solver (and its WASM) out unless +you actually import it. (The from-scratch CPU-XPBD solver was removed — it was +~10–15× slower and unstable on dense piles; these engines supersede it.) + +## Which to use + +**Default to `joltSolver`.** It's the most stable, it sleeps inert bodies (so +"many static, few dynamic" scenes cost almost nothing), and it handles deep/dense +stacks cleanly. + +**Choose `rapierSolver` only when** the scene is dominated by *many +simultaneously-active dynamic bodies* most of the time **and** tight stability +isn't critical — that's the one regime where Rapier's per-step dynamic +throughput wins. + +## Measured baseline + +Headless harness (`runSolverBenchmark`), fixed 1/60 s timestep. Stability shown +as end-state mean height / peak speed (a tight settle is low/low). + +**256 mixed dynamic bodies dropped into a walled bin (300 frames):** + +| solver | ms/frame | sim-fps | stability (avgY · maxV) | +| --- | --- | --- | --- | +| `rapierSolver` | **0.90** | 1117 | −1.3 · 57 | +| `joltSolver` | 1.57 | 638 | **−0.4 · 47** (tighter) | + +→ Rapier ~1.7× faster on a busy dynamic pile; Jolt settles tighter. + +**8000 static + 64 dynamic (the "many static, few dynamic" target):** + +| solver | ms/frame | +| --- | --- | +| `joltSolver` | **0.196** | +| `rapierSolver` | 0.344 | + +→ Jolt ~1.75× faster — it sleeps the inert statics to near-zero step cost. + +Reproduce / track regressions: `npx vitest --run +src/physics/solvers/rapier-jolt.benchmark.test.ts`. Numbers are post the +sync tag-exclude optimization (see this package's `CLAUDE.md`). + +## Adding another solver + +Implement the seam: extend `combine(physicsData, core)`, mirror new bodies into +the engine in the `physics` phase (tag + `exclude` so the sync is O(new) — see +the existing plugins and `CLAUDE.md`), step, and write the dynamic transforms +back via `getTypedArray()`. Benchmark it through `runSolverBenchmark` to compare. diff --git a/packages/data-gpu/src/physics/solvers/jolt-solver-plugin.ts b/packages/data-gpu/src/physics/solvers/jolt-solver-plugin.ts new file mode 100644 index 00000000..c2c0b344 --- /dev/null +++ b/packages/data-gpu/src/physics/solvers/jolt-solver-plugin.ts @@ -0,0 +1,382 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import initJolt from "jolt-physics"; +import { Database, type Entity } from "@adobe/data/ecs"; +import { True } from "@adobe/data/schema"; +import { physicsClock } from "../physics-clock-plugin.js"; +import { physicsData } from "../physics-data-plugin.js"; +import { jointData } from "../joint/joint-plugin.js"; +import { BodyType } from "../body/body-type/body-type.js"; +import { ColliderShape } from "../body/collider-shape/collider-shape.js"; +import type { ColliderMesh } from "../body/collider-mesh.js"; + +/** + * A third rigid-body solver behind the same `physicsData` seam — Jolt Physics + * (Jorrit Rouwe, C++→WASM, the engine from *Horizon Forbidden West*). Like the + * Rapier plugin it reads the identical authored components (`RigidBody` + + * `StaticCollider`) and writes back `position`/`rotation`/velocity for dynamic + * bodies, so the same scene runs unchanged on rapierSolver / joltSolver. + * + * Jolt's WASM initialises asynchronously; per the repo's no-top-level-await rule + * the init promise lives in the system closure and the system no-ops until the + * world exists, then runs every frame. All interop with the Emscripten binding + * is contained in this one module. + */ + +const GRAVITY = 18; // matches rapierSolver so the two solvers are directly comparable + +// Object layers (Jolt's collision-filtering requirement): statics don't collide +// with each other; dynamics collide with everything; the RAGDOLL layer collides +// with the world (static + dynamic) but NOT itself — so a collisionGroup>0 body +// (a ragdoll's bones) never self-collides. +const L_STATIC = 0, L_DYNAMIC = 1, L_RAGDOLL = 2, NUM_OBJECT_LAYERS = 3; +const BP_STATIC = 0, BP_DYNAMIC = 1, NUM_BP_LAYERS = 2; + +const RIGID = ["bodyType", "colliderShape", "halfExtents", "material", "position", "rotation", "linearVelocity", "angularVelocity"] as const; +const STATIC = ["colliderShape", "halfExtents", "material", "position", "rotation"] as const; +// Dynamic bodies carry the derived prev-pose snapshot (added on first sync) used +// for render interpolation; this query reads/writes it alongside the live pose. +const SNAPSHOT = ["bodyType", "position", "rotation", "_prevPosition", "_prevRotation"] as const; +// Sync only over bodies not yet mirrored into Jolt (excluded by the `_joltBody` +// tag added on creation) → steady state iterates zero rows instead of +// re-scanning every body each frame. +const NEW_RIGID = { exclude: ["_joltBody"] } as const; +const NEW_STATIC = { exclude: ["linearVelocity", "_joltBody"] } as const; +// Kinematic bodies, by archetype shape (no per-row value test): they have +// `bodyType` (so not a StaticCollider) and are mirrored (`_joltBody`) but never +// gained `_prevPosition` (only dynamics do) — so excluding it isolates kinematics. +const KINEMATIC = ["bodyType", "position", "rotation", "_joltBody"] as const; +const KINEMATIC_ONLY = { exclude: ["_prevPosition"] } as const; +const JOINT = ["jointType", "jointBodyA", "jointBodyB", "jointAnchorA", "jointAnchorB", "jointAxis", "jointMinLimit", "jointMaxLimit", "jointSwingLimit"] as const; +const NEW_JOINT = { exclude: ["_joltJoint"] } as const; + +type JoltModule = Awaited>; +type JBody = InstanceType; +type JBodyInterface = InstanceType; +type JJoltInterface = InstanceType; +type JRVec3 = InstanceType; +type JQuat = InstanceType; +type JVec3 = InstanceType; +type JShape = InstanceType; +type JPhysicsSystem = InstanceType; + +interface MatProps { restitution: number; friction: number } + +/** The live Jolt world, published once WASM init completes, so Jolt-native + * extensions (e.g. `joltRagdoll`) can add their own bodies/constraints into the + * same physics system the solver steps. The RAGDOLL object layer (no + * self-collision) is exposed for ragdoll parts. */ +export interface JoltContext { + jolt: JoltModule; + physicsSystem: JPhysicsSystem; + bodyInterface: JBodyInterface; + ragdollLayer: number; +} + +export const joltSolver = Database.Plugin.create({ + extends: Database.Plugin.combine(physicsData, jointData, physicsClock), + components: { + _joltBody: True.schema, // tag: this body has been mirrored into the Jolt world + _joltJoint: True.schema, // tag: this joint has been mirrored into the Jolt world + }, + resources: { + _joltContext: { default: null as JoltContext | null, transient: true }, + }, + systems: { + joltStep: { + schedule: { during: ["physics"], after: ["advancePhysicsClock"] }, + create: db => { + let J: JoltModule | null = null; + let initStarted = false; + let joltInterface: JJoltInterface | null = null; + let bodyInterface: JBodyInterface | null = null; + let physicsSystem: JPhysicsSystem | null = null; + let kPos: JRVec3 | null = null, kRot: JQuat | null = null; // reused kinematic-drive temporaries + const bodies = new Map(); // entity → Jolt body + + const matPropsOf = (id: Entity): MatProps => { + const m = db.store.read(id) as { restitution?: number; friction?: number } | null; + return { restitution: m?.restitution ?? 0.2, friction: m?.friction ?? 0.5 }; + }; + + // hull/mesh colliders may be auto-generated from a model that's still + // loading — defer mirroring such a body until its collision data exists. + const colliderReady = (id: Entity, shape: ColliderShape): boolean => { + if (shape !== "hull" && shape !== "mesh") return true; + const r = db.store.read(id) as { convexPoints?: unknown; colliderMesh?: unknown } | null; + return !!(r?.convexPoints || r?.colliderMesh); + }; + + // Jolt joints anchor in world space; map each body-local anchor (and the + // hinge axis) to world using the body's spawn pose. Reused scratch triples. + const rotateInto = (q: ArrayLike, vx: number, vy: number, vz: number, out: [number, number, number]): void => { + const qx = q[0], qy = q[1], qz = q[2], qw = q[3]; + const tx = 2 * (qy * vz - qz * vy), ty = 2 * (qz * vx - qx * vz), tz = 2 * (qx * vy - qy * vx); + out[0] = vx + qw * tx + (qy * tz - qz * ty); + out[1] = vy + qw * ty + (qz * tx - qx * tz); + out[2] = vz + qw * tz + (qx * ty - qy * tx); + }; + const perpInto = (v: ArrayLike, out: [number, number, number]): void => { + if (Math.abs(v[0]) <= Math.abs(v[1]) && Math.abs(v[0]) <= Math.abs(v[2])) { out[0] = 0; out[1] = v[2]; out[2] = -v[1]; } + else { out[0] = -v[2]; out[1] = 0; out[2] = v[0]; } + const l = Math.hypot(out[0], out[1], out[2]) || 1; out[0] /= l; out[1] /= l; out[2] /= l; + }; + const wa: [number, number, number] = [0, 0, 0], wb: [number, number, number] = [0, 0, 0]; + const wAxis: [number, number, number] = [0, 0, 0], wNorm: [number, number, number] = [0, 0, 0]; + + const setup = (jolt: JoltModule): void => { + const objectFilter = new jolt.ObjectLayerPairFilterTable(NUM_OBJECT_LAYERS); + objectFilter.EnableCollision(L_STATIC, L_DYNAMIC); + objectFilter.EnableCollision(L_DYNAMIC, L_DYNAMIC); + objectFilter.EnableCollision(L_STATIC, L_RAGDOLL); + objectFilter.EnableCollision(L_DYNAMIC, L_RAGDOLL); // ragdoll collides with the world, not itself + const bp = new jolt.BroadPhaseLayerInterfaceTable(NUM_OBJECT_LAYERS, NUM_BP_LAYERS); + bp.MapObjectToBroadPhaseLayer(L_STATIC, new jolt.BroadPhaseLayer(BP_STATIC)); + bp.MapObjectToBroadPhaseLayer(L_DYNAMIC, new jolt.BroadPhaseLayer(BP_DYNAMIC)); + bp.MapObjectToBroadPhaseLayer(L_RAGDOLL, new jolt.BroadPhaseLayer(BP_DYNAMIC)); + const settings = new jolt.JoltSettings(); + settings.mObjectLayerPairFilter = objectFilter; + settings.mBroadPhaseLayerInterface = bp; + settings.mObjectVsBroadPhaseLayerFilter = new jolt.ObjectVsBroadPhaseLayerFilterTable(bp, NUM_BP_LAYERS, objectFilter, NUM_OBJECT_LAYERS); + joltInterface = new jolt.JoltInterface(settings); + jolt.destroy(settings); + physicsSystem = joltInterface.GetPhysicsSystem(); + const g = new jolt.Vec3(0, -GRAVITY, 0); + physicsSystem.SetGravity(g); + jolt.destroy(g); + bodyInterface = physicsSystem.GetBodyInterface(); + kPos = new jolt.RVec3(0, 0, 0); kRot = new jolt.Quat(0, 0, 0, 1); + }; + + const ensureBody = (jolt: JoltModule, bi: JBodyInterface, id: Entity, motion: BodyType, shape: ColliderShape, hx: number, hy: number, hz: number, mat: Entity, px: number, py: number, pz: number, q: ArrayLike, vx: number, vy: number, vz: number, wx: number, wy: number, wz: number): void => { + if (bodies.has(id)) return; + const m = matPropsOf(mat); + // box needs a Vec3 half-extent temporary; sphere/capsule are scalar; + // hull is built from the authored point cloud (read once here). + // capsule: Y-aligned, halfHeight = cylinder half (hy), radius = hx. + let half: JVec3 | null = null; + let shp: JShape; + if (shape === "sphere") shp = new jolt.SphereShape(hx); + else if (shape === "capsule") shp = new jolt.CapsuleShape(hy, hx); + else if (shape === "hull") { + const pts = (db.store.read(id) as { convexPoints?: Float32Array | null }).convexPoints; + const hs = new jolt.ConvexHullShapeSettings(); + if (pts) for (let i = 0; i < pts.length; i += 3) { + const v = new jolt.Vec3(pts[i], pts[i + 1], pts[i + 2]); + hs.mPoints.push_back(v); jolt.destroy(v); // push_back copies the value + } + const res = hs.Create(); + shp = res.IsValid() ? res.Get() : new jolt.SphereShape(Math.max(hx, 0.1)); // degenerate fallback + jolt.destroy(hs); + } else if (shape === "mesh") { + const cm = (db.store.read(id) as { colliderMesh?: ColliderMesh | null }).colliderMesh; + const verts = new jolt.VertexList(), tris = new jolt.IndexedTriangleList(), mats = new jolt.PhysicsMaterialList(); + if (cm) { + for (let i = 0; i < cm.positions.length; i += 3) { + const f = new jolt.Float3(cm.positions[i], cm.positions[i + 1], cm.positions[i + 2]); + verts.push_back(f); jolt.destroy(f); // copied by value + } + for (let i = 0; i < cm.indices.length; i += 3) { + const t = new jolt.IndexedTriangle(cm.indices[i], cm.indices[i + 1], cm.indices[i + 2], 0); + tris.push_back(t); jolt.destroy(t); + } + } + const ms = new jolt.MeshShapeSettings(verts, tris, mats); + const res = ms.Create(); + shp = res.IsValid() ? res.Get() : new jolt.SphereShape(0.1); + jolt.destroy(ms); jolt.destroy(verts); jolt.destroy(tris); jolt.destroy(mats); + } else { half = new jolt.Vec3(hx, hy, hz); shp = new jolt.BoxShape(half); } + const pos = new jolt.RVec3(px, py, pz), rot = new jolt.Quat(q[0], q[1], q[2], q[3]); + // dynamic = simulated; kinematic = position-driven (pushes dynamics, in the + // dynamic layer so it collides with them); static = immovable collider. + const motionType = motion === "dynamic" ? jolt.EMotionType_Dynamic + : motion === "kinematic" ? jolt.EMotionType_Kinematic : jolt.EMotionType_Static; + // collisionGroup>0 ⇒ the no-self-collide RAGDOLL layer. + const grp = (db.store.read(id) as { collisionGroup?: number } | null)?.collisionGroup ?? 0; + const layer = motion === "static" ? L_STATIC : grp > 0 ? L_RAGDOLL : L_DYNAMIC; + const settings = new jolt.BodyCreationSettings(shp, pos, rot, motionType, layer); + settings.mRestitution = m.restitution; + settings.mFriction = m.friction; + const body = bi.CreateBody(settings); + bi.AddBody(body.GetID(), motion === "static" ? jolt.EActivation_DontActivate : jolt.EActivation_Activate); + if (motion === "dynamic" && (vx || vy || vz || wx || wy || wz)) { + const lv = new jolt.Vec3(vx, vy, vz), av = new jolt.Vec3(wx, wy, wz); + bi.SetLinearAndAngularVelocity(body.GetID(), lv, av); + jolt.destroy(lv); jolt.destroy(av); + } + bodies.set(id, body); + // free the construction temporaries (the body keeps a ref to the shape) + jolt.destroy(settings); jolt.destroy(pos); jolt.destroy(rot); + if (half) jolt.destroy(half); + }; + + return () => { + if (!J || !bodyInterface || !joltInterface) { + if (!initStarted) { + initStarted = true; + initJolt().then((j: JoltModule) => { + J = j; setup(j); + db.store.resources._joltContext = { jolt: j, physicsSystem: physicsSystem!, bodyInterface: bodyInterface!, ragdollLayer: L_RAGDOLL }; + }); + } + return; // WASM not ready yet + } + const jolt = J, bi = bodyInterface; + + // sync: mirror only bodies not yet in Jolt (excluded by tag), + // then tag them. Tail→head since every row migrates out on + // tagging; reads via column accessors (robust to migration, and + // this loop only touches genuinely-new bodies, so it's cold). + for (const arch of db.store.queryArchetypes(RIGID, NEW_RIGID)) { + const ids = arch.columns.id, bt = arch.columns.bodyType, cs = arch.columns.colliderShape, mat = arch.columns.material; + const he = arch.columns.halfExtents, pos = arch.columns.position, ori = arch.columns.rotation, lv = arch.columns.linearVelocity, av = arch.columns.angularVelocity; + for (let r = arch.rowCount - 1; r >= 0; r--) { + const id = ids.get(r), bodyType = bt.get(r), shape = cs.get(r), h = he.get(r), p = pos.get(r), o = ori.get(r), v = lv.get(r), w = av.get(r); + if (!colliderReady(id, shape)) continue; // auto-collider not generated yet + ensureBody(jolt, bi, id, bodyType, shape, h[0], h[1], h[2], mat.get(r), p[0], p[1], p[2], o, v[0], v[1], v[2], w[0], w[1], w[2]); + // Tag as mirrored. Dynamics also migrate onto the derived prev-pose + // snapshot (the interpolator reads it); kinematic bodies are authored + // each frame, so they render at the live pose with no snapshot — the + // _prevPosition absence is exactly what the kinematic-drive query keys on. + db.store.update(id, BodyType.isDynamic(bodyType) ? { _joltBody: true, _prevPosition: p, _prevRotation: o } : { _joltBody: true }); + } + } + for (const arch of db.store.queryArchetypes(STATIC, NEW_STATIC)) { + const ids = arch.columns.id, cs = arch.columns.colliderShape, mat = arch.columns.material; + const he = arch.columns.halfExtents, pos = arch.columns.position, ori = arch.columns.rotation; + for (let r = arch.rowCount - 1; r >= 0; r--) { + const id = ids.get(r), shape = cs.get(r), h = he.get(r), p = pos.get(r); + if (!colliderReady(id, shape)) continue; // auto-collider not generated yet + ensureBody(jolt, bi, id, "static", shape, h[0], h[1], h[2], mat.get(r), p[0], p[1], p[2], ori.get(r), 0, 0, 0, 0, 0, 0); + db.store.update(id, { _joltBody: true }); + } + } + + // Mirror new joints once both bodies exist (tag + exclude; tail→head). + // Anchors/axes are body-local; map to world (Jolt WorldSpace) via spawn pose. + for (const arch of db.store.queryArchetypes(JOINT, NEW_JOINT)) { + const ids = arch.columns.id, jt = arch.columns.jointType, ba = arch.columns.jointBodyA, bbc = arch.columns.jointBodyB; + const aa = arch.columns.jointAnchorA, ab = arch.columns.jointAnchorB, axc = arch.columns.jointAxis, lo = arch.columns.jointMinLimit, hi = arch.columns.jointMaxLimit, sw = arch.columns.jointSwingLimit; + for (let r = arch.rowCount - 1; r >= 0; r--) { + const a = bodies.get(ba.get(r)), b = bodies.get(bbc.get(r)); + if (!a || !b || !physicsSystem) continue; // a body isn't mirrored yet — retry next frame + const recA = db.store.read(ba.get(r)) as { position: ArrayLike; rotation: ArrayLike }; + const recB = db.store.read(bbc.get(r)) as { position: ArrayLike; rotation: ArrayLike }; + const an = aa.get(r), bn = ab.get(r); + rotateInto(recA.rotation, an[0], an[1], an[2], wa); wa[0] += recA.position[0]; wa[1] += recA.position[1]; wa[2] += recA.position[2]; + rotateInto(recB.rotation, bn[0], bn[1], bn[2], wb); wb[0] += recB.position[0]; wb[1] += recB.position[1]; wb[2] += recB.position[2]; + const p1 = new jolt.RVec3(wa[0], wa[1], wa[2]), p2 = new jolt.RVec3(wb[0], wb[1], wb[2]); + const temps: { __brand?: never }[] = [p1, p2]; + const type = jt.get(r); + let settings: InstanceType; + if (type === "hinge") { + const ax = axc.get(r); + rotateInto(recA.rotation, ax[0], ax[1], ax[2], wAxis); perpInto(wAxis, wNorm); + const h1 = new jolt.Vec3(wAxis[0], wAxis[1], wAxis[2]), h2 = new jolt.Vec3(wAxis[0], wAxis[1], wAxis[2]); + const n1 = new jolt.Vec3(wNorm[0], wNorm[1], wNorm[2]), n2 = new jolt.Vec3(wNorm[0], wNorm[1], wNorm[2]); + const hs = new jolt.HingeConstraintSettings(); + hs.mSpace = jolt.EConstraintSpace_WorldSpace; hs.mPoint1 = p1; hs.mPoint2 = p2; + hs.mHingeAxis1 = h1; hs.mHingeAxis2 = h2; hs.mNormalAxis1 = n1; hs.mNormalAxis2 = n2; + const min = lo.get(r), max = hi.get(r); if (min < max) { hs.mLimitsMin = min; hs.mLimitsMax = max; } + settings = hs; temps.push(h1, h2, n1, n2, hs); + } else if (type === "cone") { + // swing-twist: bone axis bound to a (symmetric) cone around the + // reference axis, plus a twist range about it — anatomical limits. + const ax = axc.get(r); + rotateInto(recA.rotation, ax[0], ax[1], ax[2], wAxis); perpInto(wAxis, wNorm); + const t1 = new jolt.Vec3(wAxis[0], wAxis[1], wAxis[2]), t2 = new jolt.Vec3(wAxis[0], wAxis[1], wAxis[2]); + const pa1 = new jolt.Vec3(wNorm[0], wNorm[1], wNorm[2]), pa2 = new jolt.Vec3(wNorm[0], wNorm[1], wNorm[2]); + const cs = new jolt.SwingTwistConstraintSettings(); + cs.mSpace = jolt.EConstraintSpace_WorldSpace; cs.mPosition1 = p1; cs.mPosition2 = p2; + cs.mTwistAxis1 = t1; cs.mTwistAxis2 = t2; cs.mPlaneAxis1 = pa1; cs.mPlaneAxis2 = pa2; + cs.mSwingType = jolt.ESwingType_Cone; + const swing = sw.get(r); cs.mNormalHalfConeAngle = swing; cs.mPlaneHalfConeAngle = swing; + const min = lo.get(r), max = hi.get(r); + cs.mTwistMinAngle = min < max ? min : -Math.PI; cs.mTwistMaxAngle = min < max ? max : Math.PI; + settings = cs; temps.push(t1, t2, pa1, pa2, cs); + } else if (type === "fixed") { + const fs = new jolt.FixedConstraintSettings(); fs.mSpace = jolt.EConstraintSpace_WorldSpace; fs.mPoint1 = p1; fs.mPoint2 = p2; + settings = fs; temps.push(fs); + } else { + const ps = new jolt.PointConstraintSettings(); ps.mSpace = jolt.EConstraintSpace_WorldSpace; ps.mPoint1 = p1; ps.mPoint2 = p2; + settings = ps; temps.push(ps); + } + physicsSystem.AddConstraint(settings.Create(a, b)); + for (const t of temps) jolt.destroy(t); + db.store.update(ids.get(r), { _joltJoint: true }); + } + } + + // Step on the fixed clock: 0..N steps of `fixedDt` this frame + // (decoupled from the render rate). On a stepping frame, snapshot + // the pose entering the final step into `_prevPosition`/`_prevRotation` + // first, so the pre-render interpolator can blend prev→current. + const clock = db.store.resources.physicsClock; + const steps = clock.steps; + if (steps === 0) return; // no sim time accrued — leave pose + snapshot intact + + // Drive kinematic bodies to their authored pose: MoveKinematic derives + // the velocity to reach the target over the step, so the body pushes + // dynamics. (Author the pose however you like; the solver just follows.) + const dt = clock.fixedDt * steps; + const toFlip: Entity[] = []; // kinematic bodies whose bodyType became "dynamic" (ragdoll) + for (const arch of db.store.queryArchetypes(KINEMATIC, KINEMATIC_ONLY)) { + const ids = arch.columns.id, bt = arch.columns.bodyType; + const pos = arch.columns.position.getTypedArray(), ori = arch.columns.rotation.getTypedArray(); + for (let r = 0; r < arch.rowCount; r++) { + const body = bodies.get(ids.get(r)); + if (!body || !kPos || !kRot) continue; + if (bt.get(r) === "dynamic") { toFlip.push(ids.get(r)); continue; } + const r3 = r * 3, r4 = r * 4; + kPos.Set(pos[r3], pos[r3 + 1], pos[r3 + 2]); + kRot.Set(ori[r4], ori[r4 + 1], ori[r4 + 2], ori[r4 + 3]); + bi.MoveKinematic(body.GetID(), kPos, kRot, dt); + } + } + // Flip kinematic→dynamic (after the loop — migrating the row moves the typed + // arrays): become dynamic + gain the prev-pose snapshot, so it's simulated. + for (const id of toFlip) { + const body = bodies.get(id); + if (!body) continue; + bi.SetMotionType(body.GetID(), jolt.EMotionType_Dynamic, jolt.EActivation_Activate); + const rec = db.store.read(id) as { position: ArrayLike; rotation: ArrayLike }; + db.store.update(id, { + _prevPosition: [rec.position[0], rec.position[1], rec.position[2]], + _prevRotation: [rec.rotation[0], rec.rotation[1], rec.rotation[2], rec.rotation[3]], + }); + } + for (const arch of db.store.queryArchetypes(SNAPSHOT)) { + const bt = arch.columns.bodyType; + const pos = arch.columns.position.getTypedArray(), ori = arch.columns.rotation.getTypedArray(); + const pp = arch.columns._prevPosition.getTypedArray(), pr = arch.columns._prevRotation.getTypedArray(); + for (let r = 0; r < arch.rowCount; r++) { + if (!BodyType.isDynamic(bt.get(r))) continue; + const r3 = r * 3, r4 = r * 4; + pp[r3] = pos[r3]; pp[r3 + 1] = pos[r3 + 1]; pp[r3 + 2] = pos[r3 + 2]; + pr[r4] = ori[r4]; pr[r4 + 1] = ori[r4 + 1]; pr[r4 + 2] = ori[r4 + 2]; pr[r4 + 3] = ori[r4 + 3]; + } + } + for (let s = 0; s < steps; s++) joltInterface.Step(clock.fixedDt, 1); + + // write the dynamic bodies' new pose + velocity back onto the canonical columns + for (const arch of db.store.queryArchetypes(RIGID)) { + const ids = arch.columns.id, bt = arch.columns.bodyType; + const pos = arch.columns.position.getTypedArray(), ori = arch.columns.rotation.getTypedArray(); + const lv = arch.columns.linearVelocity.getTypedArray(), av = arch.columns.angularVelocity.getTypedArray(); + for (let r = 0; r < arch.rowCount; r++) { + if (!BodyType.isDynamic(bt.get(r))) continue; + const body = bodies.get(ids.get(r)); + if (!body) continue; + const t = body.GetPosition(), rot = body.GetRotation(), v = body.GetLinearVelocity(), w = body.GetAngularVelocity(); + const r3 = r * 3, r4 = r * 4; + pos[r3] = t.GetX(); pos[r3 + 1] = t.GetY(); pos[r3 + 2] = t.GetZ(); + ori[r4] = rot.GetX(); ori[r4 + 1] = rot.GetY(); ori[r4 + 2] = rot.GetZ(); ori[r4 + 3] = rot.GetW(); + lv[r3] = v.GetX(); lv[r3 + 1] = v.GetY(); lv[r3 + 2] = v.GetZ(); + av[r3] = w.GetX(); av[r3 + 1] = w.GetY(); av[r3 + 2] = w.GetZ(); + } + } + }; + }, + }, + }, +}); diff --git a/packages/data-gpu/src/physics/solvers/rapier-jolt.benchmark.test.ts b/packages/data-gpu/src/physics/solvers/rapier-jolt.benchmark.test.ts new file mode 100644 index 00000000..166cd520 --- /dev/null +++ b/packages/data-gpu/src/physics/solvers/rapier-jolt.benchmark.test.ts @@ -0,0 +1,39 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { describe, it, expect } from "vitest"; +import { runSolverBenchmark } from "./solver-benchmark.js"; +import { rapierSolver } from "./rapier-solver-plugin.js"; +import { joltSolver } from "./jolt-solver-plugin.js"; + +// Side-by-side performance + stability baseline of the reference solvers through +// the shared harness. See ./README.md for the Jolt-vs-Rapier guidance. +// npx vitest --run src/physics/solvers/rapier-jolt.benchmark.test.ts +describe("solver benchmarks", () => { + const OPTS = { bodies: 256, frames: 300 }; + + it("rapierSolver vs joltSolver — same dynamic pile", async () => { + const rap = await runSolverBenchmark(rapierSolver, OPTS); + const jol = await runSolverBenchmark(joltSolver, OPTS); + const line = (name: string, r: typeof rap) => + `${name.padEnd(14)} ${r.msPerFrame.toFixed(3).padStart(8)} ms/frame · ${r.simFps.toFixed(0).padStart(5)} sim-fps · max ${r.maxFrameMs.toFixed(2).padStart(7)} ms · avgY ${r.avgY.toFixed(2).padStart(7)} · maxV ${r.maxSpeed.toFixed(1).padStart(7)} · sunk ${String(r.belowFloor).padStart(3)}`; + // eslint-disable-next-line no-console + console.log(`\n${OPTS.bodies} bodies × ${OPTS.frames} frames\n${line("rapierSolver", rap)}\n${line("joltSolver", jol)}\n`); + expect(rap.frames).toBe(300); + expect(jol.frames).toBe(300); + }, 90_000); + + // The "many static, few dynamic" target: thousands of resting static bodies + // plus a handful of dynamics. The solver's per-frame sync must NOT re-scan + // every body — bodies already mirrored into the engine are tagged and + // excluded, so steady-state iteration is ~zero. (Measured win at 20k static: + // jolt 0.53→0.07 ms/frame, rapier 1.51→1.13.) + it("scales with many static bodies (sync excludes already-mirrored)", async () => { + const OPTS_S = { bodies: 64, staticBodies: 8000, frames: 120, warmupFrames: 60 }; + const rap = await runSolverBenchmark(rapierSolver, OPTS_S); + const jol = await runSolverBenchmark(joltSolver, OPTS_S); + // eslint-disable-next-line no-console + console.log(`\n${OPTS_S.staticBodies} static + ${OPTS_S.bodies} dynamic\nrapierSolver ${rap.msPerFrame.toFixed(3)} ms/frame\njoltSolver ${jol.msPerFrame.toFixed(3)} ms/frame\n`); + expect(rap.frames).toBe(120); + expect(jol.frames).toBe(120); + }, 90_000); +}); diff --git a/packages/data-gpu/src/physics/solvers/rapier-solver-plugin.ts b/packages/data-gpu/src/physics/solvers/rapier-solver-plugin.ts new file mode 100644 index 00000000..01ce009c --- /dev/null +++ b/packages/data-gpu/src/physics/solvers/rapier-solver-plugin.ts @@ -0,0 +1,251 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import RAPIER from "@dimforge/rapier3d-compat"; +import { Database, type Entity } from "@adobe/data/ecs"; +import { True } from "@adobe/data/schema"; +import { physicsClock } from "../physics-clock-plugin.js"; +import { physicsData } from "../physics-data-plugin.js"; +import { jointData } from "../joint/joint-plugin.js"; +import { BodyType } from "../body/body-type/body-type.js"; +import { ColliderShape } from "../body/collider-shape/collider-shape.js"; + +/** + * A second rigid-body solver behind the same `physicsData` seam — the + * battle-tested Rapier engine (dimforge, Rust→WASM). It reads the identical + * authored components (`RigidBody` + `StaticCollider`) and writes back + * `position`/`rotation`/velocity for dynamic bodies, exactly like `joltSolver`, + * so the same scene runs unchanged on either solver. + * + * Rapier's WASM must be initialised before use; per the repo's no-top-level-await + * rule, the init promise lives in the system closure and is awaited lazily — the + * system no-ops until the world exists, then runs every frame. + */ + +const GRAVITY = 18; // matches joltSolver so the two solvers are directly comparable + +const RIGID = ["bodyType", "colliderShape", "halfExtents", "material", "position", "rotation", "linearVelocity", "angularVelocity"] as const; +const STATIC = ["colliderShape", "halfExtents", "material", "position", "rotation"] as const; +// Dynamic bodies carry the derived prev-pose snapshot (added on first sync) used +// for render interpolation; this query reads/writes it alongside the live pose. +const SNAPSHOT = ["bodyType", "position", "rotation", "_prevPosition", "_prevRotation"] as const; +// Sync only over bodies not yet mirrored into Rapier: a private `_rapierBody` +// tag is added once a body is created, and excluded here, so steady state +// iterates zero rows (archetype-level) instead of re-scanning every body each +// frame — the win for the "many static, few dynamic" target. +const NEW_RIGID = { exclude: ["_rapierBody"] } as const; +const NEW_STATIC = { exclude: ["linearVelocity", "_rapierBody"] } as const; +// Kinematic bodies, by archetype shape (no per-row value test): they have +// `bodyType` (so not a StaticCollider) and are mirrored (`_rapierBody`) but never +// gained `_prevPosition` (only dynamics do) — so excluding it isolates kinematics. +const KINEMATIC = ["bodyType", "position", "rotation", "_rapierBody"] as const; +const KINEMATIC_ONLY = { exclude: ["_prevPosition"] } as const; +const JOINT = ["jointType", "jointBodyA", "jointBodyB", "jointAnchorA", "jointAnchorB", "jointAxis", "jointMinLimit", "jointMaxLimit", "jointSwingLimit"] as const; +const NEW_JOINT = { exclude: ["_rapierJoint"] } as const; + +interface MatProps { density: number; restitution: number; friction: number } + +export const rapierSolver = Database.Plugin.create({ + extends: Database.Plugin.combine(physicsData, jointData, physicsClock), + components: { + _rapierBody: True.schema, // tag: this body has been mirrored into the Rapier world + _rapierJoint: True.schema, // tag: this joint has been mirrored into the Rapier world + }, + systems: { + rapierStep: { + schedule: { during: ["physics"], after: ["advancePhysicsClock"] }, + create: db => { + let world: RAPIER.World | null = null; + let initStarted = false; + const bodies = new Map(); // entity → Rapier body + + const matPropsOf = (id: Entity): MatProps => { + // Material entities carry density/restitution/friction (read once at body creation). + const m = db.store.read(id) as { density?: number; restitution?: number; friction?: number } | null; + return { density: m?.density ?? 1, restitution: m?.restitution ?? 0.2, friction: m?.friction ?? 0.5 }; + }; + + // hull/mesh colliders may be auto-generated from a model that's still + // loading — defer mirroring such a body until its collision data exists. + const colliderReady = (id: Entity, shape: ColliderShape): boolean => { + if (shape !== "hull" && shape !== "mesh") return true; + const r = db.store.read(id) as { convexPoints?: unknown; colliderMesh?: unknown } | null; + return !!(r?.convexPoints || r?.colliderMesh); + }; + + const ensureBody = (id: Entity, motion: BodyType, shape: ColliderShape, hx: number, hy: number, hz: number, mat: Entity, px: number, py: number, pz: number, q: ArrayLike, vx: number, vy: number, vz: number, wx: number, wy: number, wz: number): void => { + if (!world || bodies.has(id)) return; + const m = matPropsOf(mat); + // dynamic = simulated; kinematic = position-driven (pushes dynamics, + // isn't pushed back); fixed = immovable collider. + const desc = motion === "dynamic" ? RAPIER.RigidBodyDesc.dynamic() + : motion === "kinematic" ? RAPIER.RigidBodyDesc.kinematicPositionBased() + : RAPIER.RigidBodyDesc.fixed(); + desc.setTranslation(px, py, pz).setRotation({ x: q[0], y: q[1], z: q[2], w: q[3] }); + if (motion === "dynamic") { desc.setLinvel(vx, vy, vz); desc.setAngvel({ x: wx, y: wy, z: wz }); } + const body = world.createRigidBody(desc); + // capsule: Y-aligned, halfHeight = cylinder half (hy), radius = hx. + // hull: convex hull of the authored point cloud (read once here). + let col: RAPIER.ColliderDesc | null; + if (shape === "sphere") col = RAPIER.ColliderDesc.ball(hx); + else if (shape === "capsule") col = RAPIER.ColliderDesc.capsule(hy, hx); + else if (shape === "hull") { + const pts = (db.store.read(id) as { convexPoints?: Float32Array | null }).convexPoints; + col = pts ? RAPIER.ColliderDesc.convexHull(pts) : null; + if (!col) col = RAPIER.ColliderDesc.ball(Math.max(hx, 0.1)); // degenerate cloud fallback + } else if (shape === "mesh") { + const cm = (db.store.read(id) as { colliderMesh?: { positions: Float32Array; indices: Uint32Array } | null }).colliderMesh; + col = cm ? RAPIER.ColliderDesc.trimesh(cm.positions, cm.indices) : RAPIER.ColliderDesc.cuboid(0.1, 0.1, 0.1); + } else col = RAPIER.ColliderDesc.cuboid(hx, hy, hz); + col.setRestitution(m.restitution).setFriction(m.friction).setDensity(m.density); + // collisionGroup g (>0): membership bit g, collide with everything + // except bit g — so same-group bodies (e.g. a ragdoll's bones) skip + // each other but still hit the world (group 0). + const g = (db.store.read(id) as { collisionGroup?: number } | null)?.collisionGroup ?? 0; + if (g > 0) { const bit = 1 << (g & 15); col.setCollisionGroups(((bit << 16) | (0xffff & ~bit)) >>> 0); } + world.createCollider(col, body); + bodies.set(id, body); + }; + + return () => { + if (!world) { + if (!initStarted) { + initStarted = true; + RAPIER.init().then(() => { world = new RAPIER.World({ x: 0, y: -GRAVITY, z: 0 }); }); + } + return; // WASM not ready yet + } + + // sync: mirror only bodies not yet in Rapier (excluded by tag), + // then tag them. Iterate tail→head since every row migrates out + // on tagging. Reads use column accessors (robust to the migration; + // this loop only touches genuinely-new bodies, so it's cold). + for (const arch of db.store.queryArchetypes(RIGID, NEW_RIGID)) { + const ids = arch.columns.id, bt = arch.columns.bodyType, cs = arch.columns.colliderShape, mat = arch.columns.material; + const he = arch.columns.halfExtents, pos = arch.columns.position, ori = arch.columns.rotation, lv = arch.columns.linearVelocity, av = arch.columns.angularVelocity; + for (let r = arch.rowCount - 1; r >= 0; r--) { + const id = ids.get(r), bodyType = bt.get(r), shape = cs.get(r), h = he.get(r), p = pos.get(r), o = ori.get(r), v = lv.get(r), w = av.get(r); + if (!colliderReady(id, shape)) continue; // auto-collider not generated yet + ensureBody(id, bodyType, shape, h[0], h[1], h[2], mat.get(r), p[0], p[1], p[2], o, v[0], v[1], v[2], w[0], w[1], w[2]); + // Tag as mirrored. Dynamics also migrate onto the derived prev-pose + // snapshot (the interpolator reads it); kinematic bodies are authored + // each frame, so they render at the live pose with no snapshot — the + // _prevPosition absence is exactly what the kinematic-drive query keys on. + db.store.update(id, BodyType.isDynamic(bodyType) ? { _rapierBody: true, _prevPosition: p, _prevRotation: o } : { _rapierBody: true }); + } + } + for (const arch of db.store.queryArchetypes(STATIC, NEW_STATIC)) { + const ids = arch.columns.id, cs = arch.columns.colliderShape, mat = arch.columns.material; + const he = arch.columns.halfExtents, pos = arch.columns.position, ori = arch.columns.rotation; + for (let r = arch.rowCount - 1; r >= 0; r--) { + const id = ids.get(r), shape = cs.get(r), h = he.get(r), p = pos.get(r); + if (!colliderReady(id, shape)) continue; // auto-collider not generated yet + ensureBody(id, "static", shape, h[0], h[1], h[2], mat.get(r), p[0], p[1], p[2], ori.get(r), 0, 0, 0, 0, 0, 0); + db.store.update(id, { _rapierBody: true }); + } + } + + // Mirror new joints once both their bodies exist (tag + exclude; + // tail→head since tagging migrates the row). Anchors are body-local. + for (const arch of db.store.queryArchetypes(JOINT, NEW_JOINT)) { + const ids = arch.columns.id, jt = arch.columns.jointType; + const ba = arch.columns.jointBodyA, bb = arch.columns.jointBodyB; + const aa = arch.columns.jointAnchorA, ab = arch.columns.jointAnchorB, axc = arch.columns.jointAxis; + const lo = arch.columns.jointMinLimit, hi = arch.columns.jointMaxLimit; + for (let r = arch.rowCount - 1; r >= 0; r--) { + const a = bodies.get(ba.get(r)), b = bodies.get(bb.get(r)); + if (!a || !b) continue; // a body isn't mirrored yet — retry next frame + const type = jt.get(r), pa = aa.get(r), pb = ab.get(r), ax = axc.get(r), min = lo.get(r), max = hi.get(r); + const A = { x: pa[0], y: pa[1], z: pa[2] }, B = { x: pb[0], y: pb[1], z: pb[2] }; + let jd: RAPIER.JointData; + if (type === "fixed") jd = RAPIER.JointData.fixed(A, { x: 0, y: 0, z: 0, w: 1 }, B, { x: 0, y: 0, z: 0, w: 1 }); + else if (type === "hinge") { jd = RAPIER.JointData.revolute(A, B, { x: ax[0], y: ax[1], z: ax[2] }); if (min < max) { jd.limitsEnabled = true; jd.limits = [min, max]; } } + // point — and cone: the rapier3d-compat binding has no cone/swing-twist + // limit, so a `cone` joint is approximated as a free spherical (use joltSolver + // for anatomical ragdoll limits; see physics/README.md). + else jd = RAPIER.JointData.spherical(A, B); + world.createImpulseJoint(jd, a, b, true); + db.store.update(ids.get(r), { _rapierJoint: true }); + } + } + + // Step on the fixed clock: 0..N steps of `fixedDt` this frame + // (decoupled from the render rate). On a stepping frame, snapshot + // the pose entering the final step into `_prevPosition`/`_prevRotation` + // first, so the pre-render interpolator can blend prev→current. + const clock = db.store.resources.physicsClock; + const steps = clock.steps; + if (steps === 0) return; // no sim time accrued — leave pose + snapshot intact + + // Drive kinematic bodies to their authored pose: Rapier moves a + // position-based kinematic toward the target over the step, deriving + // the velocity that pushes dynamics. (Author the pose however you like — + // a system, animation, input — the solver just follows it.) + const toFlip: Entity[] = []; // kinematic bodies whose bodyType became "dynamic" (ragdoll) + for (const arch of db.store.queryArchetypes(KINEMATIC, KINEMATIC_ONLY)) { + const ids = arch.columns.id, bt = arch.columns.bodyType; + const pos = arch.columns.position.getTypedArray(), ori = arch.columns.rotation.getTypedArray(); + for (let r = 0; r < arch.rowCount; r++) { + const body = bodies.get(ids.get(r)); + if (!body) continue; + if (bt.get(r) === "dynamic") { toFlip.push(ids.get(r)); continue; } + const r3 = r * 3, r4 = r * 4; + body.setNextKinematicTranslation({ x: pos[r3], y: pos[r3 + 1], z: pos[r3 + 2] }); + body.setNextKinematicRotation({ x: ori[r4], y: ori[r4 + 1], z: ori[r4 + 2], w: ori[r4 + 3] }); + } + } + // Flip kinematic→dynamic (after the loop, since migrating the row moves the + // typed arrays): the engine body becomes dynamic and it gains the prev-pose + // snapshot, so from now on it's simulated + written back like any dynamic body. + for (const id of toFlip) { + const body = bodies.get(id); + if (!body) continue; + body.setBodyType(RAPIER.RigidBodyType.Dynamic, true); + const rec = db.store.read(id) as { position: ArrayLike; rotation: ArrayLike }; + db.store.update(id, { + _prevPosition: [rec.position[0], rec.position[1], rec.position[2]], + _prevRotation: [rec.rotation[0], rec.rotation[1], rec.rotation[2], rec.rotation[3]], + }); + } + for (const arch of db.store.queryArchetypes(SNAPSHOT)) { + const bt = arch.columns.bodyType; + const pos = arch.columns.position.getTypedArray(), ori = arch.columns.rotation.getTypedArray(); + const pp = arch.columns._prevPosition.getTypedArray(), pr = arch.columns._prevRotation.getTypedArray(); + for (let r = 0; r < arch.rowCount; r++) { + if (!BodyType.isDynamic(bt.get(r))) continue; + const r3 = r * 3, r4 = r * 4; + pp[r3] = pos[r3]; pp[r3 + 1] = pos[r3 + 1]; pp[r3 + 2] = pos[r3 + 2]; + pr[r4] = ori[r4]; pr[r4 + 1] = ori[r4 + 1]; pr[r4 + 2] = ori[r4 + 2]; pr[r4 + 3] = ori[r4 + 3]; + } + } + world.timestep = clock.fixedDt; + for (let s = 0; s < steps; s++) world.step(); + + // write the dynamic bodies' new pose + velocity back onto the canonical columns + for (const arch of db.store.queryArchetypes(RIGID)) { + const ids = arch.columns.id, bt = arch.columns.bodyType; + const pos = arch.columns.position.getTypedArray(), ori = arch.columns.rotation.getTypedArray(); + const lv = arch.columns.linearVelocity.getTypedArray(), av = arch.columns.angularVelocity.getTypedArray(); + for (let r = 0; r < arch.rowCount; r++) { + if (!BodyType.isDynamic(bt.get(r))) continue; + const body = bodies.get(ids.get(r)); + if (!body) continue; + // PERF residual: rapier3d-compat returns a fresh {x,y,z}/{x,y,z,w} + // object from each of these getters → ~4 small allocations per + // dynamic body per frame (GC pressure). It scales with the *dynamic* + // count only (small for our target), and the compat binding offers + // no out-param read, so it's left as-is. If dynamic counts ever get + // large, drop to rapier's raw bindings (rawBodies / a flat buffer) + // to read straight into our typed arrays with zero allocation. + const t = body.translation(), rot = body.rotation(), v = body.linvel(), w = body.angvel(); + const r3 = r * 3, r4 = r * 4; + pos[r3] = t.x; pos[r3 + 1] = t.y; pos[r3 + 2] = t.z; + ori[r4] = rot.x; ori[r4 + 1] = rot.y; ori[r4 + 2] = rot.z; ori[r4 + 3] = rot.w; + lv[r3] = v.x; lv[r3 + 1] = v.y; lv[r3 + 2] = v.z; + av[r3] = w.x; av[r3 + 1] = w.y; av[r3 + 2] = w.z; + } + } + }; + }, + }, + }, +}); diff --git a/packages/data-gpu/src/physics/solvers/solver-benchmark.ts b/packages/data-gpu/src/physics/solvers/solver-benchmark.ts new file mode 100644 index 00000000..0b5e157b --- /dev/null +++ b/packages/data-gpu/src/physics/solvers/solver-benchmark.ts @@ -0,0 +1,172 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { Database } from "@adobe/data/ecs"; + +/** + * Headless benchmark harness for a physics **solver plugin** (anything built on + * the `physicsData` seam — `rapierSolver`, `joltSolver`, …). It builds a fixed, + * deterministic scene (a floor + a grid of bodies dropped to pile up), then + * advances the simulation at a fixed timestep as fast as the CPU allows and + * reports the wall-clock cost per frame. No rendering, no rAF, no GPU — so the + * numbers reflect solver compute only and are directly comparable across plugins. + * + * Async because some solvers (Rapier) initialise WASM lazily; the warm-up phase + * yields to the event loop so that init + scene settling complete before timing. + */ + +export interface SolverBenchmarkOptions { + /** Dynamic bodies dropped into the scene (≈ workload size). Default 256. */ + bodies?: number; + /** Timed frames. Default 300. */ + frames?: number; + /** Warm-up frames before timing (lets async init finish + the pile settle). Default 90. */ + warmupFrames?: number; + /** Fixed timestep per frame (seconds). Default 1/60. */ + dt?: number; + /** Fraction of `bodies` that are spheres (rest are boxes). Default 0.5. */ + sphereFraction?: number; + /** Extra *static* collider boxes laid out as resting scenery. They barely + * cost the engine (nothing moves) but they ARE mirrored once and then scanned + * by a naive per-frame sync — so this isolates the gather/sync overhead, which + * is the realistic "many static, few dynamic" target workload. Default 0. */ + staticBodies?: number; +} + +export interface SolverBenchmarkResult { + bodies: number; + frames: number; + /** Total wall-clock time of the timed frames (ms). */ + totalMs: number; + /** Mean cost of one frame (ms) — the headline number. */ + msPerFrame: number; + /** Simulation frames per real second (1000 / msPerFrame). */ + simFps: number; + /** Slowest single frame (ms) — surfaces hitches. */ + maxFrameMs: number; + // --- coarse end-state, for cross-solver parity / sanity (not perf) --- + avgY: number; // mean height of dynamic bodies (piled ≈ low, exploded ≈ wild) + maxSpeed: number; // fastest dynamic body (a stable pile is near 0) + belowFloor: number; // dynamic bodies that sank below the floor top (should be 0) +} + +interface LooseStore { + archetypes: Record; + queryArchetypes(include: readonly string[], opts?: { exclude?: readonly string[] }): Iterable<{ + rowCount: number; + columns: Record; + }>; + resources: Record; +} +interface LooseDb { store: LooseStore; system: { functions: Record unknown>; order?: string[][] }; } + +const yieldToEventLoop = () => new Promise(r => setTimeout(r, 0)); + +export async function runSolverBenchmark(solver: Database.Plugin, opts: SolverBenchmarkOptions = {}): Promise { + const bodies = opts.bodies ?? 256; + const frames = opts.frames ?? 300; + const warmupFrames = opts.warmupFrames ?? 90; + const dt = opts.dt ?? 1 / 60; + const sphereFraction = opts.sphereFraction ?? 0.5; + const staticBodies = opts.staticBodies ?? 0; + + // The solver plugin extends `physicsData`, so the RigidBody / StaticCollider + // archetypes and the `frameTime` resource exist on the created database. The + // benchmark drives the sim dynamically, so the store is used through a loose + // shape (runtime invariant: these members exist on any physicsData solver). + const db = Database.create(solver) as unknown as LooseDb; + + buildScene(db, bodies, sphereFraction, staticBodies); + + // Drive one frame: set a fixed dt, then run every system except the rAF/clock + // ones (we supply dt ourselves so timing is deterministic and high-rate). + const order = db.system.order; + const names = order ? order.flat() : Object.keys(db.system.functions); + const runNames = names.filter(n => n !== "_frameTime" && n !== "schedulerSystem"); + const tick = (f: number): void => { + db.store.resources.frameTime = { now: f * dt * 1000, dt, elapsed: f * dt }; + for (const n of runNames) { const fn = db.system.functions[n]; if (fn) fn(); } + }; + + // Warm-up: yield to the event loop so lazy WASM init (Rapier) completes and + // the pile settles into a steady-state contact count before we measure. + for (let f = 0; f < warmupFrames; f++) { tick(f); await yieldToEventLoop(); } + + // Timed run: tight synchronous loop — pure solver compute. + let maxFrameMs = 0; + const t0 = performance.now(); + for (let f = 0; f < frames; f++) { + const a = performance.now(); + tick(warmupFrames + f); + const ms = performance.now() - a; + if (ms > maxFrameMs) maxFrameMs = ms; + } + const totalMs = performance.now() - t0; + + const { avgY, maxSpeed, belowFloor } = sampleState(db); + return { + bodies, frames, + totalMs, msPerFrame: totalMs / frames, simFps: 1000 / (totalMs / frames), maxFrameMs, + avgY, maxSpeed, belowFloor, + }; +} + +/** A thick static floor + `bodies` dynamic bodies in a compact cube grid that + * drops a short distance and piles up — a steady-state contact workload. The + * floor is deliberately thick (a finite box has a far side) and the drop is + * short, so a correct solver never tunnels: end-state avgY/maxV then cleanly + * separate a stable solver from an exploding one. Optional `staticBodies` adds + * resting static scenery to exercise the sync/gather path at scale. */ +function buildScene(db: LooseDb, bodies: number, sphereFraction: number, staticBodies: number): void { + db.store.archetypes.StaticCollider.insert({ + colliderShape: "box", halfExtents: [12, 2, 12], material: 0, + position: [0, -2, 0], rotation: [0, 0, 0, 1], // top face at y = 0 + }); + // resting static scenery far from the action (negligible engine cost, but it + // is mirrored once and then scanned every frame by a naive per-frame sync) + if (staticBodies > 0) { + const side = Math.ceil(Math.cbrt(staticBodies)); + let n = 0; + for (let x = 0; x < side && n < staticBodies; x++) + for (let y = 0; y < side && n < staticBodies; y++) + for (let z = 0; z < side && n < staticBodies; z++, n++) + db.store.archetypes.StaticCollider.insert({ + colliderShape: "box", halfExtents: [0.4, 0.4, 0.4], material: 0, + position: [100 + x, y, z], rotation: [0, 0, 0, 1], + }); + } + const side = Math.ceil(Math.cbrt(bodies)); + const gap = 1.5, base = -((side - 1) / 2) * gap; // spaced so the initial drop is non-overlapping + let n = 0; + for (let y = 0; y < side && n < bodies; y++) { + for (let x = 0; x < side && n < bodies; x++) { + for (let z = 0; z < side && n < bodies; z++, n++) { + const sphere = ((n * 0x9e3779b1) >>> 0) / 4294967296 < sphereFraction; + db.store.archetypes.RigidBody.insert({ + bodyType: "dynamic", + colliderShape: sphere ? "sphere" : "box", + halfExtents: sphere ? [0.45, 0, 0] : [0.45, 0.45, 0.45], + material: 0, + position: [base + x * gap, 0.6 + y * gap, base + z * gap], + rotation: [0, 0, 0, 1], + linearVelocity: [0, 0, 0], + angularVelocity: [0, 0, 0], + }); + } + } + } +} + +function sampleState(db: LooseDb): { avgY: number; maxSpeed: number; belowFloor: number } { + let count = 0, sumY = 0, maxSpeed = 0, belowFloor = 0; + for (const arch of db.store.queryArchetypes(["linearVelocity", "position"])) { + const pos = arch.columns.position.getTypedArray(), lv = arch.columns.linearVelocity.getTypedArray(); + for (let r = 0; r < arch.rowCount; r++, count++) { + const r3 = r * 3, y = pos[r3 + 1]; + sumY += y; + if (y < -0.5) belowFloor++; // floor top is y = 0; a resting body sits above it + const sp = Math.hypot(lv[r3], lv[r3 + 1], lv[r3 + 2]); + if (sp > maxSpeed) maxSpeed = sp; + } + } + return { avgY: count ? sumY / count : 0, maxSpeed, belowFloor }; +} diff --git a/packages/data-gpu/tsconfig.json b/packages/data-gpu/tsconfig.json new file mode 100644 index 00000000..c6a26cf6 --- /dev/null +++ b/packages/data-gpu/tsconfig.json @@ -0,0 +1,25 @@ +{ + "compilerOptions": { + "composite": true, + "target": "ESNext", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "lib": ["ES5", "ES2015", "ES2016", "ES2017", "ES2018", "ES2019", "ES2020", "ES2021", "ES2022", "ES2023", "ESNext", "DOM", "DOM.Iterable"], + "types": ["@webgpu/types"], + "experimentalDecorators": true, + "useDefineForClassFields": false, + "skipLibCheck": true, + "rootDir": "src", + "outDir": "dist", + "declaration": true, + "resolveJsonModule": true, + "isolatedModules": true, + "strict": true, + "noFallthroughCasesInSwitch": true, + "stripInternal": true, + "sourceMap": true + }, + "include": ["src/**/*"], + "exclude": ["node_modules", "dist"], + "references": [] +} diff --git a/packages/data-lit-tictactoe/package.json b/packages/data-lit-tictactoe/package.json index 24159d79..51bb046b 100644 --- a/packages/data-lit-tictactoe/package.json +++ b/packages/data-lit-tictactoe/package.json @@ -1,6 +1,6 @@ { "name": "data-lit-tictactoe", - "version": "0.9.68", + "version": "0.9.69", "description": "Tic-Tac-Toe sample - Lit web components with @adobe/data-lit and AgenticService", "type": "module", "private": true, diff --git a/packages/data-lit-tictactoe/src/elements/tictactoe-app/tictactoe-app.ts b/packages/data-lit-tictactoe/src/elements/tictactoe-app/tictactoe-app.ts index fab5d366..6008d46f 100644 --- a/packages/data-lit-tictactoe/src/elements/tictactoe-app/tictactoe-app.ts +++ b/packages/data-lit-tictactoe/src/elements/tictactoe-app/tictactoe-app.ts @@ -14,5 +14,5 @@ type TictactoeService = Database.Plugin.ToDatabase; */ export const Tictactoe = (args: { service: S }): TemplateResult => { void import("./tictactoe-app-element.js"); - return html``; + return html``; }; diff --git a/packages/data-lit-todo/package.json b/packages/data-lit-todo/package.json index bd45bbd4..4ed39ecf 100644 --- a/packages/data-lit-todo/package.json +++ b/packages/data-lit-todo/package.json @@ -1,6 +1,6 @@ { "name": "data-lit-todo", - "version": "0.9.68", + "version": "0.9.69", "description": "Todo sample app demonstrating @adobe/data with Lit", "type": "module", "private": true, diff --git a/packages/data-lit/package.json b/packages/data-lit/package.json index b8cbd5ee..4c8b9a39 100644 --- a/packages/data-lit/package.json +++ b/packages/data-lit/package.json @@ -1,6 +1,6 @@ { "name": "@adobe/data-lit", - "version": "0.9.68", + "version": "0.9.69", "description": "Adobe data Lit bindings - hooks, elements, decorators", "type": "module", "private": false, diff --git a/packages/data-lit/src/elements/database-element.ts b/packages/data-lit/src/elements/database-element.ts index 0bf9ca6c..62cdbf6f 100644 --- a/packages/data-lit/src/elements/database-element.ts +++ b/packages/data-lit/src/elements/database-element.ts @@ -9,8 +9,23 @@ import { attachDecorator, withHooks } from '../index.js'; export abstract class DatabaseElement

extends LitElement { + /** + * The live database, fully typed. Set by an ancestor via DI (`.database=…`) + * or created from `plugin` on connect. Bootstrap containers — those that own + * a controller or drive a streaming (async-generator) transaction — read + * this directly; pure widgets use the restricted `service` view below. + */ @property({ type: Object, reflect: false }) - service!: UIService.FromService>; + database!: Database.Plugin.ToDatabase

; + + /** + * UI-restricted view of {@link database} for pure-widget rendering: every + * transaction / mutator is rewritten to fire-and-forget `void` so a widget + * can never await on or read back a mutation; reads go through `observe`. + */ + get service(): UIService.FromService> { + return UIService.restrict(this.database); + } constructor() { super(); @@ -20,18 +35,18 @@ export abstract class DatabaseElement

extends LitElem abstract get plugin(): P; connectedCallback(): void { - if (!this.service) { - const service = this.findAncestorDatabase(); - this.service = (service?.extend(this.plugin) ?? Database.create(this.plugin)) as unknown as UIService.FromService>; + if (!this.database) { + const ancestor = this.findAncestorDatabase(); + this.database = ancestor?.extend(this.plugin) ?? Database.create(this.plugin); } super.connectedCallback(); } protected findAncestorDatabase(): Database | void { for (const element of iterateSelfAndAncestors(this)) { - const { service } = element as Partial>; - if (Database.is(service)) { - return service; + const { database } = element as Partial>; + if (Database.is(database)) { + return database; } } } diff --git a/packages/data-p2p-tictactoe/package.json b/packages/data-p2p-tictactoe/package.json index 9c2bb851..c0c890f8 100644 --- a/packages/data-p2p-tictactoe/package.json +++ b/packages/data-p2p-tictactoe/package.json @@ -1,6 +1,6 @@ { "name": "data-p2p-tictactoe", - "version": "0.9.68", + "version": "0.9.69", "description": "Serverless P2P tic-tac-toe — WebRTC DataChannel + @adobe/data-sync", "type": "module", "private": true, @@ -17,6 +17,7 @@ }, "devDependencies": { "typescript": "^5.8.3", - "vite": "^5.1.1" + "vite": "^5.1.1", + "vite-plugin-checker": "^0.12.0" } } diff --git a/packages/data-p2p-tictactoe/src/elements/p2p-negotiation/p2p-negotiation-element.ts b/packages/data-p2p-tictactoe/src/elements/p2p-negotiation/p2p-negotiation-element.ts index ee5ab7d2..1ca7b01b 100644 --- a/packages/data-p2p-tictactoe/src/elements/p2p-negotiation/p2p-negotiation-element.ts +++ b/packages/data-p2p-tictactoe/src/elements/p2p-negotiation/p2p-negotiation-element.ts @@ -65,19 +65,24 @@ export class P2pNegotiationElement extends DatabaseElement ({ - phase: this.service.observe.resources.phase, - connection: this.service.observe.resources.connection, - offerCode: this.service.observe.resources.offerCode, - answerCode: this.service.observe.resources.answerCode, - bannerText: this.service.observe.resources.bannerText, - bannerError: this.service.observe.resources.bannerError, - hostAnswerInput: this.service.observe.resources.hostAnswerInput, - joinerOfferInput: this.service.observe.resources.joinerOfferInput, - gameDb: this.service.observe.resources.gameDb, + phase: service.observe.resources.phase, + connection: service.observe.resources.connection, + offerCode: service.observe.resources.offerCode, + answerCode: service.observe.resources.answerCode, + bannerText: service.observe.resources.bannerText, + bannerError: service.observe.resources.bannerError, + hostAnswerInput: service.observe.resources.hostAnswerInput, + joinerOfferInput: service.observe.resources.joinerOfferInput, + gameDb: service.observe.resources.gameDb, }), []); - const controller = useNegotiationController(this.service, this.gamePlugin, this.assignUserId); + const controller = useNegotiationController(service, this.gamePlugin, this.assignUserId); if (!values) return undefined; @@ -89,8 +94,8 @@ export class P2pNegotiationElement extends DatabaseElement controller.startJoin(), submitAnswer: () => controller.submitAnswer(), generateAnswer: () => controller.generateAnswer(), - setHostAnswerInput: (value) => this.service.transactions.setHostAnswerInput({ value }), - setJoinerOfferInput: (value) => this.service.transactions.setJoinerOfferInput({ value }), + setHostAnswerInput: (value) => service.transactions.setHostAnswerInput({ value }), + setJoinerOfferInput: (value) => service.transactions.setJoinerOfferInput({ value }), copyText: (text) => controller.copyText(text), reconnect: () => controller.reconnect(), }); diff --git a/packages/data-p2p-tictactoe/src/elements/p2p-presence-overlay/p2p-presence-overlay-element.ts b/packages/data-p2p-tictactoe/src/elements/p2p-presence-overlay/p2p-presence-overlay-element.ts index 5f6a7415..226166d9 100644 --- a/packages/data-p2p-tictactoe/src/elements/p2p-presence-overlay/p2p-presence-overlay-element.ts +++ b/packages/data-p2p-tictactoe/src/elements/p2p-presence-overlay/p2p-presence-overlay-element.ts @@ -37,7 +37,10 @@ export class P2pPresenceOverlayElement extends DatabaseElement { const positions = Observe.toAsyncGenerator(pointerPos, () => false); @@ -50,14 +53,16 @@ export class P2pPresenceOverlayElement extends DatabaseElement undefined); + database.transactions.movePresence(presenceArgs).catch(() => undefined); return () => { void positions.throw(new Error("disposed")).catch(() => undefined); }; - }, [pointerPos, element, service]); + }, [pointerPos, element, database]); - const localMark = this.service.sync?.userId; + // The peer identity is the rebase-replay concurrency userId (set to the + // player's mark by the negotiation controller); there is no `.sync` view. + const localMark = database.concurrency.userId; return presentation.render({ cursors: values?.cursors, localMark }); } diff --git a/packages/data-p2p-tictactoe/src/elements/p2p-presence-overlay/p2p-presence-overlay.ts b/packages/data-p2p-tictactoe/src/elements/p2p-presence-overlay/p2p-presence-overlay.ts index 036a4f07..e5474bad 100644 --- a/packages/data-p2p-tictactoe/src/elements/p2p-presence-overlay/p2p-presence-overlay.ts +++ b/packages/data-p2p-tictactoe/src/elements/p2p-presence-overlay/p2p-presence-overlay.ts @@ -18,5 +18,5 @@ export const PresenceOverlay = (args: { children: TemplateResult; }): TemplateResult => { void import("./p2p-presence-overlay-element.js"); - return html`${args.children}`; + return html`${args.children}`; }; diff --git a/packages/data-p2p-tictactoe/vite.config.ts b/packages/data-p2p-tictactoe/vite.config.ts index 48fe3b9d..aa270bbf 100644 --- a/packages/data-p2p-tictactoe/vite.config.ts +++ b/packages/data-p2p-tictactoe/vite.config.ts @@ -1,6 +1,8 @@ import { defineConfig } from "vite"; +import checker from "vite-plugin-checker"; export default defineConfig({ + plugins: [checker({ typescript: true })], optimizeDeps: { esbuildOptions: { tsconfigRaw: { diff --git a/packages/data-persistence/package.json b/packages/data-persistence/package.json index 6abc0b4e..aa859ccf 100644 --- a/packages/data-persistence/package.json +++ b/packages/data-persistence/package.json @@ -1,6 +1,6 @@ { "name": "@adobe/data-persistence", - "version": "0.9.68", + "version": "0.9.69", "description": "Worker-based incremental persistence layer for @adobe/data ECS over OPFS (browser) and node:fs (server).", "type": "module", "sideEffects": false, diff --git a/packages/data-react-hello/package.json b/packages/data-react-hello/package.json index ca078f8c..8da0b0fa 100644 --- a/packages/data-react-hello/package.json +++ b/packages/data-react-hello/package.json @@ -1,6 +1,6 @@ { "name": "data-react-hello", - "version": "0.9.68", + "version": "0.9.69", "description": "Hello World sample - click counter using @adobe/data-react", "type": "module", "private": true, @@ -18,8 +18,9 @@ "devDependencies": { "@types/react": "^19", "@types/react-dom": "^19", + "@vitejs/plugin-react": "^4.3.0", "typescript": "^5.8.3", "vite": "^5.1.1", - "@vitejs/plugin-react": "^4.3.0" + "vite-plugin-checker": "^0.12.0" } } diff --git a/packages/data-react-hello/vite.config.ts b/packages/data-react-hello/vite.config.ts index d3b4b453..6feb768e 100644 --- a/packages/data-react-hello/vite.config.ts +++ b/packages/data-react-hello/vite.config.ts @@ -1,8 +1,9 @@ import { defineConfig } from "vite"; import react from "@vitejs/plugin-react"; +import checker from "vite-plugin-checker"; export default defineConfig({ - plugins: [react()], + plugins: [react(), checker({ typescript: true })], root: ".", build: { outDir: "dist" }, server: { port: 3001, open: false }, diff --git a/packages/data-react-pixie/package.json b/packages/data-react-pixie/package.json index 1ad70eb1..cef08f53 100644 --- a/packages/data-react-pixie/package.json +++ b/packages/data-react-pixie/package.json @@ -1,6 +1,6 @@ { "name": "data-react-pixie", - "version": "0.9.68", + "version": "0.9.69", "description": "PixiJS React sample - ECS sprites (bunny, fox) with @adobe/data-react", "type": "module", "private": true, diff --git a/packages/data-react/package.json b/packages/data-react/package.json index f171f515..181e4872 100644 --- a/packages/data-react/package.json +++ b/packages/data-react/package.json @@ -1,6 +1,6 @@ { "name": "@adobe/data-react", - "version": "0.9.68", + "version": "0.9.69", "description": "Adobe data React bindings — hooks and context for ECS database", "type": "module", "private": false, diff --git a/packages/data-solid-dashboard/package.json b/packages/data-solid-dashboard/package.json index 1201ea34..2d294252 100644 --- a/packages/data-solid-dashboard/package.json +++ b/packages/data-solid-dashboard/package.json @@ -1,6 +1,6 @@ { "name": "data-solid-dashboard", - "version": "0.9.68", + "version": "0.9.69", "description": "Mini dashboard sample — multiple components sharing one @adobe/data ECS database with SolidJS", "type": "module", "private": true, @@ -17,6 +17,7 @@ "devDependencies": { "typescript": "^5.8.3", "vite": "^5.1.1", + "vite-plugin-checker": "^0.12.0", "vite-plugin-solid": "^2.11.0" } } diff --git a/packages/data-solid-dashboard/vite.config.ts b/packages/data-solid-dashboard/vite.config.ts index a8c183b1..f550f4d3 100644 --- a/packages/data-solid-dashboard/vite.config.ts +++ b/packages/data-solid-dashboard/vite.config.ts @@ -1,8 +1,9 @@ import { defineConfig } from "vite"; import solid from "vite-plugin-solid"; +import checker from "vite-plugin-checker"; export default defineConfig({ - plugins: [solid()], + plugins: [solid(), checker({ typescript: true })], root: ".", build: { outDir: "dist" }, server: { port: 3004, open: false }, diff --git a/packages/data-solid/package.json b/packages/data-solid/package.json index a15cc628..5298254f 100644 --- a/packages/data-solid/package.json +++ b/packages/data-solid/package.json @@ -1,6 +1,6 @@ { "name": "@adobe/data-solid", - "version": "0.9.68", + "version": "0.9.69", "description": "Adobe data SolidJS bindings — context and provider for ECS database", "type": "module", "private": false, diff --git a/packages/data-sync/package.json b/packages/data-sync/package.json index 6936bcbb..743e0d50 100644 --- a/packages/data-sync/package.json +++ b/packages/data-sync/package.json @@ -1,6 +1,6 @@ { "name": "@adobe/data-sync", - "version": "0.9.68", + "version": "0.9.69", "description": "Multi-user real-time synchronisation for @adobe/data ECS — server, client, and in-process loopback.", "type": "module", "sideEffects": false, diff --git a/packages/data/package.json b/packages/data/package.json index 01a20383..e3b093c4 100644 --- a/packages/data/package.json +++ b/packages/data/package.json @@ -1,6 +1,6 @@ { "name": "@adobe/data", - "version": "0.9.68", + "version": "0.9.69", "description": "Adobe data oriented programming library", "type": "module", "sideEffects": false, diff --git a/packages/data/src/ecs/plugins/scheduler/scheduler.ts b/packages/data/src/ecs/plugins/scheduler/scheduler.ts index 4716e8b2..c8e64353 100644 --- a/packages/data/src/ecs/plugins/scheduler/scheduler.ts +++ b/packages/data/src/ecs/plugins/scheduler/scheduler.ts @@ -11,6 +11,14 @@ export const scheduler = createPlugin({ systems: { schedulerSystem: { create: (db) => { + // The frame loop runs on requestAnimationFrame in the browser. In a + // headless host (Node — tests, server-side simulation) there is no + // rAF; fall back to a no-op so the database still constructs and the + // host can drive frames itself by invoking `db.system.functions` in + // `db.system.order`. This keeps a simulation fully runnable with no + // rendering attached. + const raf: (cb: () => void) => unknown = + typeof requestAnimationFrame !== "undefined" ? requestAnimationFrame : () => 0; // Execute one frame const executeFrame = async () => { if (db.resources.schedulerState === "running") { @@ -39,12 +47,12 @@ export const scheduler = createPlugin({ } if (db.resources.schedulerState !== "disposed") { - requestAnimationFrame(executeFrame); + raf(executeFrame); } }; // Defer execution until after all systems are created and db.system.functions is populated - requestAnimationFrame(executeFrame); + raf(executeFrame); // Return a no-op system function (the real work happens in the RAF loop) return () => { diff --git a/packages/data/src/math/mat4x4/functions.ts b/packages/data/src/math/mat4x4/functions.ts index e10545af..61388816 100644 --- a/packages/data/src/math/mat4x4/functions.ts +++ b/packages/data/src/math/mat4x4/functions.ts @@ -208,12 +208,15 @@ export const perspective = (fovy: number, aspect: number, near: number, far: num if (far <= near) throw new Error('Far plane must be greater than near plane'); const f = 1.0 / Math.tan(fovy / 2); - const nf = near / (near - far); + // WebGPU clip-space convention: z_ndc ∈ [0, 1]. + // m[2][2] = far / (near - far) maps z_view = -far → z_ndc = 1 + // m[2][3] = (near*far) / (near-far) and m[3][2] = -1 maps z_view = -near → z_ndc = 0 + const fn = far / (near - far); return [ f / aspect, 0, 0, 0, 0, f, 0, 0, - 0, 0, nf, -1, - 0, 0, near * nf, 0 + 0, 0, fn, -1, + 0, 0, near * fn, 0 ]; }; diff --git a/packages/data/src/math/quat/index.ts b/packages/data/src/math/quat/index.ts index 5e0367c1..a97ce7a4 100644 --- a/packages/data/src/math/quat/index.ts +++ b/packages/data/src/math/quat/index.ts @@ -1,8 +1,8 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. -import { Schema } from "../../schema/index.js"; -import { schema } from "./schema.js"; - -export type Quat = Schema.ToType; +// Defined directly rather than via Schema.ToType so the schema +// can declare interpolators that reference functions whose own signatures +// use Quat — otherwise the type alias would be circular. +export type Quat = readonly [number, number, number, number]; export * as Quat from "./public.js"; diff --git a/packages/data/src/math/quat/schema.ts b/packages/data/src/math/quat/schema.ts index d1ee2a4c..63354770 100644 --- a/packages/data/src/math/quat/schema.ts +++ b/packages/data/src/math/quat/schema.ts @@ -2,6 +2,7 @@ import { F32 } from "../f32/index.js"; import { Schema } from "../../schema/index.js"; +import { slerp } from "./functions.js"; export const schema = { type: 'array', @@ -9,5 +10,8 @@ export const schema = { minItems: 4, maxItems: 4, default: [0, 0, 0, 1], // identity quaternion + interpolators: { + linear: slerp, + }, } as const satisfies Schema; diff --git a/packages/data/src/schema/index.ts b/packages/data/src/schema/index.ts index 2443fd75..f33dd1aa 100644 --- a/packages/data/src/schema/index.ts +++ b/packages/data/src/schema/index.ts @@ -16,5 +16,7 @@ export * from "./boolean/index.js"; // these are both math types and basic schema types. export { F32, I32, U32, F64 } from "../math/index.js"; export * from "./time/index.js"; +export { toVertexBufferLayout, toVertexBufferLayoutForType } from "./to-vertex-buffer-layout.js"; +export type { GPUVertexBufferLayout, GPUVertexAttributeDescriptor, GPUVertexFormat } from "./to-vertex-buffer-layout.js"; export * from "./fractional-index/fractional-index.js"; export * from "./guid/index.js"; diff --git a/packages/data/src/schema/schema.ts b/packages/data/src/schema/schema.ts index f0e1f62e..0967587f 100644 --- a/packages/data/src/schema/schema.ts +++ b/packages/data/src/schema/schema.ts @@ -50,4 +50,13 @@ export interface Schema { const?: any; enum?: readonly any[]; layout?: Layout; // Memory layout for typed buffers (std140 or packed) + // Per-type interpolation overrides used by the animation system. Schemas omit + // this when the componentwise lerp / step default is correct (Vec3, scalar, …). + // Quat declares { linear: slerp } so quaternion tracks are interpolated on the + // 4-sphere instead of component-wise. + interpolators?: { + readonly linear?: (prev: any, next: any, t: number) => any; + readonly step?: (prev: any, next: any, t: number) => any; + readonly cubicSpline?: (prev: any, next: any, t: number) => any; + }; } diff --git a/packages/data/src/service/ui-service/public.ts b/packages/data/src/service/ui-service/public.ts index 0fd15e8d..2fde3c12 100644 --- a/packages/data/src/service/ui-service/public.ts +++ b/packages/data/src/service/ui-service/public.ts @@ -2,3 +2,4 @@ export * from "./is-valid.js"; export * from "./from-service.js"; +export * from "./restrict.js"; diff --git a/packages/data/src/service/ui-service/restrict.ts b/packages/data/src/service/ui-service/restrict.ts new file mode 100644 index 00000000..42d1f3bd --- /dev/null +++ b/packages/data/src/service/ui-service/restrict.ts @@ -0,0 +1,23 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import type { Service } from "../service.js"; +import type { FromService } from "./from-service.js"; + +/** + * Narrows a full service/database to its UI-restricted view (see + * {@link FromService}): transactions and other mutators become fire-and-forget + * `void`, while `observe` surfaces pass through. The restriction is **purely + * type-level** — the value returned is the very same instance — so this is the + * single sanctioned boundary between the full transactional surface (used by + * controllers / bootstrap containers) and the surface UI widgets consume. + * + * `T` is always assignable to `FromService` by construction, but TypeScript + * cannot prove it for a generic `T` (the `IsValid` conditional inside + * `FromService` stays deferred). Rather than reach for a cast, we declare the + * precise mapped return as an overload and broaden the *implementation* + * signature to `Service` — the identity body then type-checks with no cast. + */ +export function restrict(service: T): FromService; +export function restrict(service: Service): Service { + return service; +} diff --git a/packages/data/src/typed-buffer/structs/index.ts b/packages/data/src/typed-buffer/structs/index.ts index 6be34c89..61ff71b3 100644 --- a/packages/data/src/typed-buffer/structs/index.ts +++ b/packages/data/src/typed-buffer/structs/index.ts @@ -1,3 +1,4 @@ // © 2026 Adobe. MIT License. See /LICENSE for details. export { getStructLayout } from "./get-struct-layout.js"; export * from "./struct-layout.js"; +export { wgslStructFields } from "./wgsl-struct-fields.js"; diff --git a/packages/data/src/typed-buffer/structs/wgsl-struct-fields.ts b/packages/data/src/typed-buffer/structs/wgsl-struct-fields.ts new file mode 100644 index 00000000..6cd0145d --- /dev/null +++ b/packages/data/src/typed-buffer/structs/wgsl-struct-fields.ts @@ -0,0 +1,57 @@ +// © 2026 Adobe. MIT License. See /LICENSE for details. + +import { type Schema } from "../../schema/index.js"; + +const primitiveWgslType = (schema: Schema): string | null => { + if (schema.type === "number") return "f32"; + if (schema.type === "integer") { + return schema.minimum !== undefined && schema.minimum >= 0 ? "u32" : "i32"; + } + return null; +}; + +const fieldWgslType = (schema: Schema): string | null => { + const prim = primitiveWgslType(schema); + if (prim) return prim; + if ( + schema.type === "array" && + schema.items !== undefined && + !Array.isArray(schema.items) && + schema.minItems === schema.maxItems && + schema.minItems !== undefined + ) { + const elemType = primitiveWgslType(schema.items); + if (!elemType) return null; + const n = schema.minItems; + const suffix = elemType === "f32" ? "f" : elemType === "i32" ? "i" : "u"; + if (n === 16 && suffix === "f") return "mat4x4f"; + if (n === 2 || n === 3 || n === 4) return `vec${n}${suffix}`; + } + return null; +}; + +/** + * Generates WGSL struct field declarations from a JSON Schema object type. + * + * Maps each property to the appropriate WGSL type (f32, vec3f, mat4x4f, etc.) + * using the same schema that drives host-side TypedBuffer layout — so the host + * struct and the WGSL struct are guaranteed to agree on field order and types. + * + * Usage: + * ```ts + * const source = ` + * struct MyUniforms { + * ${wgslStructFields(MyUniforms.schema)} + * }`; + * ``` + */ +export const wgslStructFields = (schema: Schema): string => { + if (schema.type !== "object" || !schema.properties) return ""; + return Object.entries(schema.properties) + .map(([name, fieldSchema]) => { + const type = fieldWgslType(fieldSchema); + if (!type) throw new Error(`Cannot map schema field "${name}" to a WGSL type`); + return ` ${name}: ${type},`; + }) + .join("\n"); +}; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index f71adc5d..54069795 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -133,6 +133,56 @@ importers: specifier: ^9.0.9 version: 9.0.9 + packages/data-gpu: + dependencies: + '@adobe/data': + specifier: workspace:* + version: link:../data + '@dimforge/rapier3d-compat': + specifier: ^0.19.3 + version: 0.19.3 + jolt-physics: + specifier: ^1.0.0 + version: 1.0.0 + devDependencies: + '@webgpu/types': + specifier: ^0.1.61 + version: 0.1.61 + typescript: + specifier: ^5.8.3 + version: 5.8.3 + vitest: + specifier: ^1.6.0 + version: 1.6.0(@types/node@25.6.0)(@vitest/browser@1.6.0)(jsdom@24.1.0) + + packages/data-gpu-samples: + dependencies: + '@adobe/data': + specifier: workspace:* + version: link:../data + '@adobe/data-gpu': + specifier: workspace:* + version: link:../data-gpu + '@adobe/data-lit': + specifier: workspace:* + version: link:../data-lit + lit: + specifier: ^3.3.1 + version: 3.3.1 + devDependencies: + '@webgpu/types': + specifier: ^0.1.61 + version: 0.1.61 + typescript: + specifier: ^5.8.3 + version: 5.8.3 + vite: + specifier: ^5.1.1 + version: 5.1.1(@types/node@25.6.0) + vite-plugin-checker: + specifier: ^0.12.0 + version: 0.12.0(typescript@5.8.3)(vite@5.1.1) + packages/data-lit: dependencies: '@adobe/data': @@ -241,6 +291,9 @@ importers: vite: specifier: ^5.1.1 version: 5.1.1(@types/node@25.6.0) + vite-plugin-checker: + specifier: ^0.12.0 + version: 0.12.0(typescript@5.8.3)(vite@5.1.1) packages/data-persistence: dependencies: @@ -313,6 +366,9 @@ importers: vite: specifier: ^5.1.1 version: 5.1.1(@types/node@25.6.0) + vite-plugin-checker: + specifier: ^0.12.0 + version: 0.12.0(typescript@5.8.3)(vite@5.1.1) packages/data-react-pixie: dependencies: @@ -388,6 +444,9 @@ importers: vite: specifier: ^5.1.1 version: 5.1.1(@types/node@25.6.0) + vite-plugin-checker: + specifier: ^0.12.0 + version: 0.12.0(typescript@5.8.3)(vite@5.1.1) vite-plugin-solid: specifier: ^2.11.0 version: 2.11.0(solid-js@1.9.12)(vite@5.1.1) @@ -759,6 +818,10 @@ packages: postcss-selector-parser: 6.1.2 dev: true + /@dimforge/rapier3d-compat@0.19.3: + resolution: {integrity: sha512-mMVdSj1PRTT108s9Swbu2GQOmHbn8kbJANRV5xfczL3s0T4vkgZAuoMRgvBzQcHanpKusbC0ZJj6z3mC3aj3vg==} + dev: false + /@esbuild/aix-ppc64@0.19.12: resolution: {integrity: sha512-bmoCYyWdEL3wDQIVbcyzRyeKLgk2WtWLTWz1ZIAZF/EGbNOwSA6ew3PftJ1PqMiOOGu0OyFMzG53L0zqIpPeNA==} engines: {node: '>=12'} @@ -5058,6 +5121,10 @@ packages: resolution: {integrity: sha512-nO6jcEfZWQXDhOiBtG2KvKyEptz7RVbpGP4vTD2hLBdmNQSsCiicO2Ioinv6UI4y9ukqnBpy+XZ9H6uLNgJTlw==} dev: true + /jolt-physics@1.0.0: + resolution: {integrity: sha512-rA7Mcb3CDqsDzr0J15P2DDftMR4d15/B6hfvvVh88Se3KFCYXGbPKGK2sJFGOpzUksRpyQhgbfLHgHL4SA5UzQ==} + dev: false + /js-tokens@4.0.0: resolution: {integrity: sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==} dev: true diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 7fd38ecc..981a3121 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -11,3 +11,5 @@ packages: - 'packages/data-p2p-tictactoe' - 'packages/data-solid' - 'packages/data-solid-dashboard' + - 'packages/data-gpu' + - 'packages/data-gpu-samples' diff --git a/solar-post-refactor.png b/solar-post-refactor.png new file mode 100644 index 00000000..72728c0c Binary files /dev/null and b/solar-post-refactor.png differ diff --git a/solar-system-2.png b/solar-system-2.png new file mode 100644 index 00000000..f353ec5e Binary files /dev/null and b/solar-system-2.png differ diff --git a/solar-system-3.png b/solar-system-3.png new file mode 100644 index 00000000..f353ec5e Binary files /dev/null and b/solar-system-3.png differ diff --git a/solar-system-4.png b/solar-system-4.png new file mode 100644 index 00000000..7353cdd0 Binary files /dev/null and b/solar-system-4.png differ diff --git a/solar-system-5.png b/solar-system-5.png new file mode 100644 index 00000000..b3f6b75e Binary files /dev/null and b/solar-system-5.png differ diff --git a/solar-system-initial.png b/solar-system-initial.png new file mode 100644 index 00000000..f353ec5e Binary files /dev/null and b/solar-system-initial.png differ