feat(endpoint): add Scalar schema for lens-dependent entity fields by ntucker · Pull Request #3887 · reactive/data-client

ntucker · 2026-04-08T03:56:57Z

Motivation

A primary use case is displaying vast amounts of information in a grid (like a spreadsheet). Each row is an entity, but some columns display relational data that depends on a user selection — for example, which portfolio's equity percentages to show for each company. We call this selection a "lens."

The goal is to:

Cache different lens selections independently so switching back doesn't require a refetch
Support column-only fetches that update just the lens data without touching entity fields
Allow multiple components to render the same entity through different lenses simultaneously

Solution

Introduces a new Scalar schema type following the Union pattern (wrapper objects in entity fields, custom denormalize routing):

Two classes:

Scalar (SchemaSimple, NOT entity-like) — the schema used in Entity.schema. Routes normalize/denormalize, handles lens logic.
ScalarCell (entity-like, internal) — stores grouped cell data in the normalized entity table.

Normalize: Entity fields like pct_equity: 0.5 are replaced with lens-independent wrappers { id: '1', field: 'pct_equity' }. The actual scalar data is stored in ScalarCell entries keyed by entityKey|entityPk|lensValue.

Denormalize: EntityMixin.denormalize is completely unchanged. The standard unvisit(schema, input[key]) loop calls Scalar.denormalize, which reads the wrapper, adds the lens from current endpoint args, looks up the correct ScalarCell, and returns just the field value.

EntityMixin.normalize change: One if/else in the schema field loop to pass the whole entity object (not the primitive field value) to visit for Scalar fields. This avoids the primitive short-circuit in getVisit.ts.

No changes to packages/normalizr/ — getVisit, unvisit, normalize all unchanged.

const PortfolioScalar = new Scalar({
  lens: args => args[0]?.portfolio,
  key: 'portfolio',
});

class Company extends Entity {
  id = '';
  price = 0;
  pct_equity = 0;
  shares = 0;
  static schema = {
    pct_equity: PortfolioScalar,
    shares: PortfolioScalar,
  };
}

// Column-only endpoint (dictionary keyed by company pk)
const getPortfolioColumns = new Endpoint(
  ({ portfolio }) => fetch(`/companies/columns?portfolio=${portfolio}`),
  { schema: new schema.Values(PortfolioScalar) },
);

Open questions

Should Scalar support a process or validate hook for cell data?
Should compound pk delimiter (|) be configurable for entity keys/pks that contain |?
MemoCache caching behavior with different args for the same entity — currently requires separate memo instances per lens. Integration with useSuspense/useQuery hook args should handle this naturally.

changeset-bot · 2026-04-08T03:57:02Z

⚠️ No Changeset found

Latest commit: f989a17

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

vercel · 2026-04-08T03:57:05Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs-site	Ignored	Preview	Apr 13, 2026 1:04pm

github-actions · 2026-04-08T03:59:06Z

Size Change: +32 B (+0.04%)

Total Size: 80.7 kB

📦 View Changed

Filename	Size	Change
`examples/test-bundlesize/dist/rdcEndpoint.js`	8.03 kB	+32 B (+0.4%)

ℹ️ View Unchanged

Filename	Size
`examples/test-bundlesize/dist/App.js`	1.46 kB
`examples/test-bundlesize/dist/polyfill.js`	307 B
`examples/test-bundlesize/dist/rdcClient.js`	10.5 kB
`examples/test-bundlesize/dist/react.js`	59.7 kB
`examples/test-bundlesize/dist/webpack-runtime.js`	726 B

_{compressed-size-action}

codecov · 2026-04-08T04:00:10Z

Codecov Report

❌ Patch coverage is 80.35714% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.77%. Comparing base (57512fb) to head (f989a17).

Files with missing lines	Patch %	Lines
packages/endpoint/src/schemas/Scalar.ts	78.43%	5 Missing and 6 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3887      +/-   ##
==========================================
- Coverage   98.11%   97.77%   -0.34%     
==========================================
  Files         153      154       +1     
  Lines        2916     2971      +55     
  Branches      566      576      +10     
==========================================
+ Hits         2861     2905      +44     
- Misses         11       16       +5     
- Partials       44       50       +6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions

Benchmark

Details

Benchmark suite	Current: `f989a17`	Previous: `e66cc98`	Ratio
`normalizeLong`	`440` ops/sec (`±1.89%`)	`458` ops/sec (`±1.77%`)	`1.04`
`normalizeLong Values`	`403` ops/sec (`±1.28%`)	`403` ops/sec (`±1.01%`)	`1`
`denormalizeLong`	`396` ops/sec (`±1.55%`)	`279` ops/sec (`±5.83%`)	`0.70`
`denormalizeLong Values`	`333` ops/sec (`±1.26%`)	`326` ops/sec (`±0.48%`)	`0.98`
`denormalizeLong donotcache`	`1009` ops/sec (`±0.20%`)	`1087` ops/sec (`±0.61%`)	`1.08`
`denormalizeLong Values donotcache`	`741` ops/sec (`±0.25%`)	`764` ops/sec (`±0.17%`)	`1.03`
`denormalizeShort donotcache 500x`	`1576` ops/sec (`±0.36%`)	`1520` ops/sec (`±0.09%`)	`0.96`
`denormalizeShort 500x`	`1053` ops/sec (`±2.06%`)	`988` ops/sec (`±1.85%`)	`0.94`
`denormalizeShort 500x withCache`	`7525` ops/sec (`±0.15%`)	`7951` ops/sec (`±0.13%`)	`1.06`
`queryShort 500x withCache`	`2894` ops/sec (`±0.09%`)	`3134` ops/sec (`±0.50%`)	`1.08`
`buildQueryKey All`	`54638` ops/sec (`±0.67%`)	`49814` ops/sec (`±0.40%`)	`0.91`
`query All withCache`	`5658` ops/sec (`±0.27%`)	`5680` ops/sec (`±0.49%`)	`1.00`
`denormalizeLong with mixin Entity`	`354` ops/sec (`±3.19%`)	`350` ops/sec (`±2.40%`)	`0.99`
`denormalizeLong withCache`	`7705` ops/sec (`±0.57%`)	`7652` ops/sec (`±0.56%`)	`0.99`
`denormalizeLong Values withCache`	`5049` ops/sec (`±0.19%`)	`6209` ops/sec (`±1.48%`)	`1.23`
`denormalizeLong All withCache`	`5426` ops/sec (`±0.18%`)	`5440` ops/sec (`±0.56%`)	`1.00`
`denormalizeLong Query-sorted withCache`	`5656` ops/sec (`±0.80%`)	`5743` ops/sec (`±0.13%`)	`1.02`
`denormalizeLongAndShort withEntityCacheOnly`	`1793` ops/sec (`±0.71%`)	`1667` ops/sec (`±0.18%`)	`0.93`
`denormalize bidirectional 50`	`6988` ops/sec (`±0.26%`)	`6655` ops/sec (`±0.23%`)	`0.95`
`denormalize bidirectional 50 donotcache`	`40383` ops/sec (`±1.65%`)	`44707` ops/sec (`±1.21%`)	`1.11`
`getResponse`	`4546` ops/sec (`±0.66%`)	`5600` ops/sec (`±1.29%`)	`1.23`
`getResponse (null)`	`10555769` ops/sec (`±1.09%`)	`10824484` ops/sec (`±0.56%`)	`1.03`
`getResponse (clear cache)`	`338` ops/sec (`±0.34%`)	`335` ops/sec (`±1.92%`)	`0.99`
`getSmallResponse`	`3510` ops/sec (`±0.22%`)	`4075` ops/sec (`±0.24%`)	`1.16`
`getSmallInferredResponse`	`2615` ops/sec (`±0.52%`)	`2906` ops/sec (`±0.05%`)	`1.11`
`getResponse Collection`	`4478` ops/sec (`±0.52%`)	`5497` ops/sec (`±1.19%`)	`1.23`
`get Collection`	`4488` ops/sec (`±0.31%`)	`5463` ops/sec (`±0.68%`)	`1.22`
`get Query-sorted`	`4547` ops/sec (`±1.19%`)	`5542` ops/sec (`±0.27%`)	`1.22`
`setLong`	`447` ops/sec (`±0.63%`)	`461` ops/sec (`±0.18%`)	`1.03`
`setLongWithMerge`	`255` ops/sec (`±0.37%`)	`253` ops/sec (`±0.97%`)	`0.99`
`setLongWithSimpleMerge`	`273` ops/sec (`±0.35%`)	`271` ops/sec (`±0.20%`)	`0.99`
`setSmallResponse 500x`	`901` ops/sec (`±1.05%`)	`888` ops/sec (`±0.95%`)	`0.99`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

Benchmark React

Details

Benchmark suite	Current: `f989a17`	Previous: `e66cc98`	Ratio
`data-client: getlist-100`	`138.89` ops/s (`± 5.4%`)	`136.06` ops/s (`± 7.2%`)	`0.98`
`data-client: getlist-500`	`40` ops/s (`± 5.0%`)	`40.73` ops/s (`± 6.2%`)	`1.02`
`data-client: update-entity`	`357.14` ops/s (`± 10.7%`)	`294.12` ops/s (`± 7.4%`)	`0.82`
`data-client: update-user`	`317.54` ops/s (`± 9.5%`)	`333.33` ops/s (`± 8.3%`)	`1.05`
`data-client: getlist-500-sorted`	`43.57` ops/s (`± 6.4%`)	`41.76` ops/s (`± 7.1%`)	`0.96`
`data-client: update-entity-sorted`	`285.71` ops/s (`± 5.6%`)	`294.12` ops/s (`± 5.7%`)	`1.03`
`data-client: update-entity-multi-view`	`333.33` ops/s (`± 1.9%`)	`322.58` ops/s (`± 7.1%`)	`0.97`
`data-client: list-detail-switch-10`	`6.39` ops/s (`± 4.6%`)	`6.46` ops/s (`± 5.6%`)	`1.01`
`data-client: update-user-10000`	`72.47` ops/s (`± 12.8%`)	`78.12` ops/s (`± 1.5%`)	`1.08`
`data-client: invalidate-and-resolve`	`36.36` ops/s (`± 3.9%`)	`33.9` ops/s (`± 5.5%`)	`0.93`
`data-client: unshift-item`	`222.22` ops/s (`± 3.9%`)	`212.77` ops/s (`± 3.7%`)	`0.96`
`data-client: delete-item`	`274.02` ops/s (`± 4.8%`)	`270.27` ops/s (`± 5.2%`)	`0.99`
`data-client: move-item`	`172.41` ops/s (`± 5.6%`)	`172.41` ops/s (`± 6.2%`)	`1`

This comment was automatically generated by workflow using github-action-benchmark.

cursor

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit dbf275e. Configure here.}

cursor · 2026-04-08T05:48:13Z

packages/endpoint/src/schemas/Scalar.ts

+    this._entityKey = undefined;
+    this._lastProcessed = undefined;
+    this._lastCpk = '';
+  }


Mutable _entityKey state breaks multi-entity and Values-first usage

High Severity

_entityKey is stored as mutable instance state on the Scalar object, set only during the entity normalize path and read during both the Values normalize path and the entity denormalize path. If two entity types (e.g., Company and Fund) share the same Scalar instance, the last one to normalize overwrites _entityKey, causing denormalization of the other entity type to build incorrect compound pks (e.g., Fund|1|portfolioA instead of Company|1|portfolioA). Additionally, a Values-only (column) fetch before any entity fetch leaves _entityKey as undefined, producing compound pks like undefined|1|portfolioB. The wrapper { id, field } doesn't include the entity key, so denormalize has no independent way to recover it.

Additional Locations (2)

packages/endpoint/src/schemas/Scalar.ts#L152-L159

packages/endpoint/src/schemas/Scalar.ts#L169-L179

^{Reviewed by Cursor Bugbot for commit dbf275e. Configure here.}

packages/endpoint/src/schemas/Scalar.ts

cursor · 2026-04-08T05:48:13Z

packages/endpoint/src/schemas/Scalar.ts

+      const entityPart = keyStr.slice(0, lastPipe);
+      const secondLastPipe = entityPart.lastIndexOf('|');
+      const entityPk = entityPart.slice(secondLastPipe + 1);
+      const entityKey = entityPart.slice(0, secondLastPipe);


Pipe delimiter in entity PK silently corrupts key parsing

Medium Severity

The composite key entityKey|entityPk|fieldName is parsed using two lastIndexOf('|') calls, but entity PKs (and entity keys) may legitimately contain | — for example, composite primary keys like "type|id". When they do, the parsing silently extracts incorrect entityPk and entityKey values. This corrupts the wrapper id field and the cell compound pk, causing the entity path and Values path to produce inconsistent keys, so column-only fetches can never be joined with entity data during denormalize.

Additional Locations (1)

packages/endpoint/src/schemas/EntityMixin.ts#L327-L328

^{Reviewed by Cursor Bugbot for commit dbf275e. Configure here.}

Introduces Scalar + ScalarCell classes following the Union pattern: - Scalar (SchemaSimple, not entity-like) routes normalize/denormalize - ScalarCell (entity-like, internal) stores grouped cell data - EntityMixin.normalize: if/else to pass whole entity to Scalar - EntityMixin.denormalize: completely unchanged (Union-like wrapper) - Entity stores lens-independent {id,field} wrappers - Denormalize joins correct cell based on endpoint args Co-authored-by: natmaster <natmaster@gmail.com>

Tests cover: - normalize stores wrapper refs in entity, cell data in ScalarCell - multiple entities, different lenses produce separate cells - denormalize joins correct cell based on lens args - different lens args produce different results from same entity - missing lens returns undefined for scalar fields - column-only fetch via Values stores cells without modifying entities - column fetch cells joinable via denormalize with Company schema - merge accumulation updates existing cells - Scalar constructor and queryKey Co-authored-by: natmaster <natmaster@gmail.com>

Covers usage (full entity + column-only endpoints), constructor options, normalize/denormalize flow, normalized storage model, and related APIs. Co-authored-by: natmaster <natmaster@gmail.com>

_lastCpk was declared and initialized but never read or written elsewhere in the codebase. Co-authored-by: Nathaniel Tucker <me@ntucker.me>

github-actions bot reviewed Apr 8, 2026

View reviewed changes

ntucker marked this pull request as ready for review April 8, 2026 05:40

cursor bot reviewed Apr 8, 2026

View reviewed changes

cursoragent and others added 4 commits April 13, 2026 09:03

docs(rest): add Scalar schema API documentation

57786f9

Covers usage (full entity + column-only endpoints), constructor options, normalize/denormalize flow, normalized storage model, and related APIs. Co-authored-by: natmaster <natmaster@gmail.com>

Remove dead _lastCpk field from Scalar class

f989a17

_lastCpk was declared and initialized but never read or written elsewhere in the codebase. Co-authored-by: Nathaniel Tucker <me@ntucker.me>

ntucker force-pushed the cursor/scalar-schema-design-for-lenses-d6e3 branch from 352ef6f to f989a17 Compare April 13, 2026 13:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(endpoint): add Scalar schema for lens-dependent entity fields#3887

feat(endpoint): add Scalar schema for lens-dependent entity fields#3887
ntucker wants to merge 4 commits intomasterfrom
cursor/scalar-schema-design-for-lenses-d6e3

ntucker commented Apr 8, 2026

Uh oh!

changeset-bot bot commented Apr 8, 2026 •

edited

Loading

Uh oh!

vercel bot commented Apr 8, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 8, 2026 •

edited

Loading

Uh oh!

codecov bot commented Apr 8, 2026 •

edited

Loading

Uh oh!

github-actions bot left a comment •

edited

Loading

Uh oh!

github-actions bot left a comment •

edited

Loading

Uh oh!

cursor bot left a comment

Uh oh!

cursor bot Apr 8, 2026

Uh oh!

Uh oh!

cursor bot Apr 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

ntucker commented Apr 8, 2026

Motivation

Solution

Open questions

Uh oh!

changeset-bot bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

vercel bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Benchmark

Uh oh!

github-actions bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Benchmark React

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Apr 8, 2026

Choose a reason for hiding this comment

Mutable _entityKey state breaks multi-entity and Values-first usage

Uh oh!

Uh oh!

cursor bot Apr 8, 2026

Choose a reason for hiding this comment

Pipe delimiter in entity PK silently corrupts key parsing

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

changeset-bot bot commented Apr 8, 2026 •

edited

Loading

vercel bot commented Apr 8, 2026 •

edited

Loading

github-actions bot commented Apr 8, 2026 •

edited

Loading

codecov bot commented Apr 8, 2026 •

edited

Loading

github-actions bot left a comment •

edited

Loading

github-actions bot left a comment •

edited

Loading

Mutable `_entityKey` state breaks multi-entity and Values-first usage