Design exploration: Scalar schema for lens-parameterized data

[ntucker]

Motivation

A primary use case is displaying vast amounts of information in a grid/spreadsheet. Each row is represented as an entity. However, some columns need to show relational data parameterized by a user-selected "lens" entity -- e.g., "what % of Company X is in Portfolio Y?" The user selects which portfolio (lens) to view, which changes the column data.

Current schema primitives don't handle this well:

• Plain Entity references get overwritten on lens switch (shallow merge clobbers the pk reference)
• Query + All scans are O(n) per cell -- unacceptable for large grids
• No mechanism to cache multiple lens values for the same entity and switch between them instantly

Solution

Explored 4 distinct architectures for a new Scalar schema type, each analyzed for normalization/denormalization flow, store layout, merge behavior, type safety, MemoCache interaction, and performance:

1. Entity subclass with compound PK + ScalarRef wrapper (recommended) -- Each (row, lens) combination is a standard entity with compound pk. A ScalarRef wrapper handles denormalize indirection. Zero changes to normalizr/MemoCache.

2. Collection-like non-Entity with lens map -- Follows Collection pattern, stores lens-keyed value map. Requires MemoCache args-awareness.

3. Entity with lens map field + fieldMerge protocol -- Stores all lens values as a map on the parent entity. Introduces a general fieldMerge extension point on Entity.merge.

4. Query-like wrapper with ScalarNormalizedValue -- Stores {pk, stableFields} on parent, recomputes desired pk during denormalize. Requires MemoCache modifications.

Recommendation: Approach 1, because it requires zero changes to normalizr internals, zero MemoCache modifications, supports per-lens invalidation/expiry, and leverages the full Entity lifecycle.

Open questions

• Multi-lens dimensions (e.g., Portfolio AND TimePeriod) -- Approach 1 generalizes via compound pk
• Should fieldMerge be added as a standalone feature regardless of Scalar approach?
• Naming: Scalar vs Intersection vs LensEntity
• Should Entity.denormalize be extended to pass parent context to nested schemas?
• What separator to use for compound pks ('__' vs '\x00' vs multi-character)

  

[changeset-bot]

⚠️ No Changeset found

Latest commit: b608e50

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs-site	Ignored		Apr 7, 2026 1:55am

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.11%. Comparing base (0ccd6c6) to head (b608e50).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3885   +/-   ##
=======================================
  Coverage   98.11%   98.11%           
=======================================
  Files         153      153           
  Lines        2913     2913           
  Branches      565      565           
=======================================
  Hits         2858     2858           
  Misses         11       11           
  Partials       44       44           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.