Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .changeset/content-property.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
'@data-client/rest': patch
---

Add `content` property to RestEndpoint for typed response parsing

Set `content` to control how the response body is parsed, with automatic return type inference:

```ts
const downloadFile = new RestEndpoint({
path: '/files/:id/download',
content: 'blob',
dataExpiryLength: 0,
});
const blob: Blob = await ctrl.fetch(downloadFile, { id: '123' });
```

Accepted values: `'json'`, `'blob'`, `'text'`, `'arrayBuffer'`, `'stream'`.

Non-JSON content types (`'blob'`, `'text'`, `'arrayBuffer'`, `'stream'`) constrain `schema` to
`undefined` at the type level, with a runtime check that throws if a normalizable schema is set.

When `content` is not set, auto-detection now handles binary Content-Types (`image/*`,
`application/octet-stream`, `application/pdf`, etc.) by returning `response.blob()` instead of
corrupting data via `.text()`.
7 changes: 7 additions & 0 deletions .changeset/deepclone-own-properties.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@data-client/normalizr': patch
---

Fix deepClone to only copy own properties

`deepClone` in the immutable store path now uses `Object.keys()` instead of `for...in`, preventing inherited properties from being copied into cloned state.
15 changes: 15 additions & 0 deletions .cursor/rules/benchmarking.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,21 @@ Use this mapping when deciding which React benchmark scenarios are relevant to a

Regressions >5% on stable scenarios or >15% on volatile scenarios are worth investigating.

### Profiling / tracing (opt + deopt investigation)

The React benchmark supports the same V8 opt/deopt investigation as the Node benchmark, via Chromium's `--js-flags`:

- **`bench:trace`** (`BENCH_V8_TRACE=true`): launches Chromium with `--trace-opt --trace-deopt`; browser process output piped to `v8-trace.log`. Equivalent to `examples/benchmark`'s `start:trace`.
- **`bench:deopt`** (`BENCH_V8_DEOPT=true`): launches Chromium with `--prof`; V8 writes per-process logs to `v8-logs/v8-<pid>.log`. Process the renderer log (largest file) with `node --prof-process`.

Both default to `--lib data-client --size small` for focused runs. Override with additional flags:

```bash
yarn workspace example-benchmark-react bench:trace
yarn workspace example-benchmark-react bench:deopt
BENCH_V8_TRACE=true yarn workspace example-benchmark-react bench --scenario update-entity
```

### When to use Node vs React benchmark

- **Core/normalizr/endpoint changes only** (no rendering impact): Run `examples/benchmark` (Node). Faster iteration, no browser needed.
Expand Down
14 changes: 12 additions & 2 deletions .cursor/skills/data-client-schema/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,14 +93,24 @@ export const EventResource = resource({

---

## 5. Best Practices & Notes
## 5. Supplementary Endpoints (enrich existing entities)

When an endpoint returns partial or differently-shaped data for an entity already in cache
(e.g., a metadata endpoint, a stats endpoint, a lazy-load expansion endpoint),
use the **same Entity** as the schema — don't create a wrapper entity.

See [partial-entities](references/partial-entities.md) for patterns and examples.

---

## 6. Best Practices & Notes

- Always set up `schema` on every resource/entity/collection for normalization
- Normalize deeply nested or relational data by defining proper schemas
- Use `Entity.schema` for client-side joins
- Use `Denormalize<>` type from rest/endpoint/graphql instead of InstanceType<>. This will handle all schemas like Unions, not just Entity.

## 6. Common Mistakes to Avoid
## 7. Common Mistakes to Avoid

- Don't forget to use `fromJS()` or assign default properties for class fields
- Manually merging or 'enriching' data; instead use `Entity.schema` for client-side joins
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Logs
logs
*.log
v8-logs/
npm-debug.log*
yarn-debug.log*
yarn-error.log*
Expand Down
54 changes: 37 additions & 17 deletions docs/rest/api/RestEndpoint.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ interface RestGenerics {
readonly body?: any;
readonly searchParams?: any;
readonly paginationField?: string;
readonly content?: 'json' | 'blob' | 'text' | 'arrayBuffer' | 'stream';
process?(value: any, ...args: any): any;
}

Expand All @@ -55,6 +56,7 @@ export class RestEndpoint<O extends RestGenerics = any> extends Endpoint {
readonly requestInit: RequestInit;
readonly method: string;
readonly paginationField?: string;
readonly content?: 'json' | 'blob' | 'text' | 'arrayBuffer' | 'stream';
readonly signal: AbortSignal | undefined;
url(...args: Parameters<F>): string;
searchToString(searchParams: Record<string, any>): string;
Expand Down Expand Up @@ -776,41 +778,45 @@ Performs the [fetch(input, init)](https://developer.mozilla.org/en-US/docs/Web/A
[response.ok](https://developer.mozilla.org/en-US/docs/Web/API/Response/ok) is not `true` (like 404),
will throw a NetworkError.

### parseResponse(response): Promise {#parseResponse}
### content {#content}

Takes the [Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) and parses via [.text()](https://developer.mozilla.org/en-US/docs/Web/API/Response/text) or [.json()](https://developer.mozilla.org/en-US/docs/Web/API/Response/json) depending
on ['content-type' header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type) having 'json' (e.g., `application/json`).
Controls how the [Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) body is parsed.
When set, the return type is inferred automatically, and `schema` is constrained to `undefined`
for non-JSON content types.

If `status` is 204, resolves as `null`.
| Value | Parses via | Return type |
|---|---|---|
| `'json'` | [response.json()](https://developer.mozilla.org/en-US/docs/Web/API/Response/json) | `any` |
| `'blob'` | [response.blob()](https://developer.mozilla.org/en-US/docs/Web/API/Response/blob) | `Blob` |
| `'text'` | [response.text()](https://developer.mozilla.org/en-US/docs/Web/API/Response/text) | `string` |
| `'arrayBuffer'` | [response.arrayBuffer()](https://developer.mozilla.org/en-US/docs/Web/API/Response/arrayBuffer) | `ArrayBuffer` |
| `'stream'` | `response.body` | `ReadableStream<Uint8Array>` |
| *unset* | Auto-detect from Content-Type header | `any` |

Override this to handle other response types like [blob](https://developer.mozilla.org/en-US/docs/Web/API/Response/blob) or [arrayBuffer](https://developer.mozilla.org/en-US/docs/Web/API/Response/arrayBuffer).
When `content` is not set, `parseResponse` auto-detects the response type from the
`Content-Type` header: JSON types call `.json()`, binary types (images, `application/octet-stream`,
PDFs, etc.) call `.blob()`, and text-like types call `.text()`.

#### File downloads {#file-download}

For binary responses like file downloads, override `parseResponse` to use `response.blob()`.
Set `schema: undefined` since binary data is not normalizable. Use `dataExpiryLength: 0` to
avoid caching large blobs in memory.
For file downloads, set `content: 'blob'`. The return type is `Blob` and `schema` must be
`undefined` (binary data cannot be normalized). Use `dataExpiryLength: 0` to avoid caching
large blobs in memory.

```ts
const downloadFile = new RestEndpoint({
path: '/files/:id/download',
schema: undefined,
content: 'blob',
dataExpiryLength: 0,
parseResponse(response) {
return response.blob();
},
process(blob): { blob: Blob; filename: string } {
return { blob, filename: 'download' };
},
});
```

To extract the filename from the `Content-Disposition` header, override both `parseResponse` and `process`:
To extract the filename from the `Content-Disposition` header, override `parseResponse`:

```ts
const downloadFile = new RestEndpoint({
path: '/files/:id/download',
schema: undefined,
content: 'blob',
dataExpiryLength: 0,
async parseResponse(response) {
const blob = await response.blob();
Expand All @@ -827,6 +833,20 @@ const downloadFile = new RestEndpoint({

See [file download guide](../guides/network-transform.md#file-download) for complete usage with browser download trigger.

### parseResponse(response): Promise {#parseResponse}

Takes the [Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) and parses the body.

When [`content`](#content) is set, it controls parsing directly. Otherwise, auto-detection runs
based on the [`Content-Type` header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type):
JSON types call [.json()](https://developer.mozilla.org/en-US/docs/Web/API/Response/json), binary
types call [.blob()](https://developer.mozilla.org/en-US/docs/Web/API/Response/blob), and text-like
types call [.text()](https://developer.mozilla.org/en-US/docs/Web/API/Response/text).

If `status` is 204, resolves as `null`.

Override this for advanced cases like extracting headers alongside the body.

### process(value, ...args): any {#process}

Perform any transforms with the parsed result. Defaults to identity function (do nothing).
Expand Down
50 changes: 31 additions & 19 deletions docs/rest/guides/network-transform.md
Original file line number Diff line number Diff line change
Expand Up @@ -296,29 +296,18 @@ class GithubEndpoint<

## File download {#file-download}

For endpoints that return binary data (files, images, PDFs), override
[parseResponse](../api/RestEndpoint.md#parseResponse) to call
[response.blob()](https://developer.mozilla.org/en-US/docs/Web/API/Response/blob) instead of the
default JSON/text parsing. Set `schema: undefined` since binary data isn't normalizable, and
`dataExpiryLength: 0` to avoid caching large blobs in memory.
For endpoints that return binary data (files, images, PDFs), set
[`content: 'blob'`](../api/RestEndpoint.md#content). The return type is `Blob` and
`schema` defaults to `undefined` (binary data isn't normalizable). Use `dataExpiryLength: 0`
to avoid caching large blobs in memory.

```typescript title="downloadFile.ts"
import { RestEndpoint } from '@data-client/rest';

const downloadFile = new RestEndpoint({
path: '/files/:id/download',
schema: undefined,
content: 'blob',
dataExpiryLength: 0,
async parseResponse(response) {
const blob = await response.blob();
const disposition = response.headers.get('Content-Disposition');
const filename =
disposition?.match(/filename="?(.+?)"?$/)?.[1] ?? 'download';
return { blob, filename };
},
process(value): { blob: Blob; filename: string } {
return value;
},
});
```

Expand All @@ -330,11 +319,11 @@ function DownloadButton({ id }: { id: string }) {
const ctrl = useController();

const handleDownload = async () => {
const { blob, filename } = await ctrl.fetch(downloadFile, { id });
const blob: Blob = await ctrl.fetch(downloadFile, { id });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
a.download = 'download';
a.click();
URL.revokeObjectURL(url);
};
Expand All @@ -343,8 +332,31 @@ function DownloadButton({ id }: { id: string }) {
}
```

To extract the filename from the `Content-Disposition` header, override
[parseResponse](../api/RestEndpoint.md#parseResponse):

```typescript title="downloadFile.ts"
import { RestEndpoint } from '@data-client/rest';

const downloadFile = new RestEndpoint({
path: '/files/:id/download',
content: 'blob',
dataExpiryLength: 0,
async parseResponse(response) {
const blob = await response.blob();
const disposition = response.headers.get('Content-Disposition');
const filename =
disposition?.match(/filename="?(.+?)"?$/)?.[1] ?? 'download';
return { blob, filename };
},
process(value): { blob: Blob; filename: string } {
return value;
},
});
```

For `ArrayBuffer` responses (useful for processing binary data in-memory), use
`response.arrayBuffer()` the same way.
`content: 'arrayBuffer'` the same way.

## Name calling

Expand Down
2 changes: 2 additions & 0 deletions examples/benchmark-react/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,3 +73,5 @@ Filtering: `yarn bench --lib data-client --size small --action update`
| `BENCH_LABEL=<tag>` | Appends `[<tag>]` to result names |
| `BENCH_PORT` | Preview port (default 5173) |
| `BENCH_TRACE=true` | Chrome tracing for duration scenarios |
| `BENCH_V8_TRACE=true` | Launch Chromium with `--trace-opt --trace-deopt`; output to `v8-trace.log` |
| `BENCH_V8_DEOPT=true` | Launch Chromium with `--prof`; V8 logs to `v8-logs/` |
8 changes: 8 additions & 0 deletions examples/benchmark-react/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# example-benchmark-react

## 0.1.6

### Patch Changes

- Updated dependencies [[`16f5d92`](https://github.com/reactive/data-client/commit/16f5d92598de05e92b88af98a9d63eecf27ab819)]:
- @data-client/rest@0.16.5
- @data-client/react@0.16.0

## 0.1.5

### Patch Changes
Expand Down
30 changes: 29 additions & 1 deletion examples/benchmark-react/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,34 @@ The runner prints a JSON array in `customBiggerIsBetter` format (name, unit, val

To view results locally, open `bench/report-viewer.html` in a browser and paste the JSON (or upload `react-bench-output.json`) to see a comparison table and bar chart.

## Optional: Chrome trace
## Profiling

### Chrome trace (timeline duration)

Set `BENCH_TRACE=true` when running the bench to enable Chrome tracing for duration scenarios. Trace files are written to disk; parsing and reporting trace duration is best-effort and may require additional tooling for the trace zip format.

### V8 opt/deopt investigation

For the same granularity of V8 optimization investigation available in `examples/benchmark` (Node), two modes pass `--js-flags` to Chromium via Playwright:

```bash
yarn bench:trace # --trace-opt --trace-deopt → v8-trace.log
yarn bench:deopt # --prof → v8-logs/v8-<pid>.log
```

Both default to `--lib data-client --size small` for focused, fast investigation. Add other flags as needed (e.g. `--scenario update-entity`).

**`bench:trace`** (`BENCH_V8_TRACE=true`) launches Chromium with `--js-flags="--trace-opt --trace-deopt"`. The runner uses Playwright `launchServer` (not `launch`) so the root browser process stdout/stderr can be piped to `v8-trace.log` — `Browser` from `launch()` does not expose `process()`. This is the browser equivalent of `examples/benchmark`'s `start:trace` — look for optimization and deoptimization lines for functions of interest.

**`bench:deopt`** (`BENCH_V8_DEOPT=true`) launches Chromium with `--js-flags="--prof"`. V8 writes per-process profiling logs to `v8-logs/v8-<pid>.log`. Chromium is multi-process, so several files are created; the renderer log (typically the largest) contains the benchmark's hot path. Process it with:

```bash
node --prof-process v8-logs/v8-<pid>.log > processed.txt
```

Both env vars can be combined for simultaneous trace output and profiling logs. The convenience scripts can be overridden:

```bash
BENCH_V8_TRACE=true yarn bench --lib data-client --scenario update-entity
BENCH_V8_DEOPT=true yarn bench --lib data-client --size large
```
Loading
Loading