Skip to content

httpapi: WithHeaders response header schemas - #6934

Open
tim-smart wants to merge 11 commits into
mainfrom
feat/httpapi-with-headers
Open

httpapi: WithHeaders response header schemas#6934
tim-smart wants to merge 11 commits into
mainfrom
feat/httpapi-with-headers

Conversation

@tim-smart

@tim-smart tim-smart commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add HttpApiSchema.WithHeaders and branded withHeaders response values for typed success headers across handlers and generated clients.
  • Add HttpApiSchema.encodeToWithHeaders for folding response headers into domain types such as error classes.
  • Encode and decode buffered and streaming response headers in HttpApiBuilder, HttpApiClient, and HttpApiTest, including mixed-success dispatch and user-managed codecs.
  • Render declared response headers in OpenAPI and document the public API, stream behavior, and overlapping-union constraint.
  • Make explicit content-type and content-length values applied through HttpServerResponse.setHeader or setHeaders override body-derived values.

Closes EFF-369

Validation

  • pnpm check
  • pnpm lint
  • pnpm vitest --run --project effect test/unstable/httpapi/ test/unstable/http/HttpServerResponse.test.ts (148 passed)
  • pnpm doctest --run packages/effect/src/unstable/httpapi/HttpApiSchema.ts (3 passed)
  • Root pnpm docgen reaches only the pre-existing HttpStaticServer example error for the missing HttpPlatform.compression field; the new examples typecheck and execute successfully.

Adds the WithHeaders branded declaration schema for wrapping success
schemas with a response headers schema, the branded WithHeaders.Value
pair constructed via withHeaders, and the encodeToWithHeaders transform
for folding response headers into domain types such as error classes.
HttpApiEndpoint gains a WithHeaders case in its success types and
unwraps the brand when validating per-status combinations.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-project-automation github-project-automation Bot moved this to Discussion Ongoing in PR Backlog Aug 4, 2026
@changeset-bot

changeset-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b581fb2

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages
Name Type
effect Patch
@effect/opentelemetry Patch
@effect/platform-browser Patch
@effect/platform-bun Patch
@effect/platform-deno Patch
@effect/platform-node-shared Patch
@effect/platform-node Patch
@effect/vitest Patch
@effect/ai-anthropic Patch
@effect/ai-openai-compat Patch
@effect/ai-openai Patch
@effect/ai-openrouter Patch
@effect/atom-react Patch
@effect/atom-solid Patch
@effect/atom-vue Patch
@effect/sql-clickhouse Patch
@effect/sql-d1 Patch
@effect/sql-libsql Patch
@effect/sql-mssql Patch
@effect/sql-mysql2 Patch
@effect/sql-pg Patch
@effect/sql-pglite Patch
@effect/sql-sqlite-bun Patch
@effect/sql-sqlite-do Patch
@effect/sql-sqlite-node Patch
@effect/sql-sqlite-react-native Patch
@effect/sql-sqlite-wasm Patch
@effect/docgen Patch
@effect/doctest Patch
@effect/openapi-generator Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@effect-slopcop effect-slopcop Bot added 4.0 enhancement New feature or request labels Aug 4, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — schema-level response-header support for unstable httpapi, plus endpoint validation/codec wiring to keep WithHeaders wrappers intact through construction.

  • Added HttpApiSchema.WithHeaders, withHeaders, isWithHeaders, and the WithHeadersTypeId / WithHeadersValueTypeId brands.
  • Added HttpApiSchema.encodeToWithHeaders for folding response headers into opaque schemas such as tagged errors.
  • Updated HttpApiEndpoint success-type extraction and endpoint construction so wrapper annotations resolve first, inner schemas are transformed by the appropriate body codec, and headers become Schema.toCodecStringTree.
  • Rejected WithHeaders in endpoint error position with a clear pointer toward encodeToWithHeaders.

The validation checks all pass locally, including pnpm lint, pnpm check, the targeted runtime tests, and the HttpApiEndpoint / HttpApiSchema type tests.

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name Current Size Previous Size Difference
basic.ts 7.06 KB 7.06 KB 0.00 KB (0.00%)
batching.ts 9.86 KB 9.86 KB 0.00 KB (0.00%)
brand.ts 6.34 KB 6.34 KB 0.00 KB (0.00%)
cache.ts 10.62 KB 10.62 KB 0.00 KB (0.00%)
config.ts 20.60 KB 20.60 KB 0.00 KB (0.00%)
differ.ts 20.20 KB 20.20 KB 0.00 KB (0.00%)
http-client.ts 21.49 KB 21.49 KB 0.00 KB (0.00%)
logger.ts 10.76 KB 10.76 KB 0.00 KB (0.00%)
metric.ts 8.98 KB 8.98 KB 0.00 KB (0.00%)
optic.ts 7.18 KB 7.18 KB 0.00 KB (0.00%)
pubsub.ts 14.90 KB 14.90 KB 0.00 KB (0.00%)
queue.ts 11.58 KB 11.58 KB 0.00 KB (0.00%)
schedule.ts 10.74 KB 10.74 KB 0.00 KB (0.00%)
schema-class.ts 19.14 KB 19.14 KB 0.00 KB (0.00%)
schema-fromJsonSchemaDocument.ts 28.96 KB 28.96 KB 0.00 KB (0.00%)
schema-representation-roundtrip.ts 25.29 KB 25.29 KB 0.00 KB (0.00%)
schema-string-transformation.ts 13.30 KB 13.30 KB 0.00 KB (0.00%)
schema-string.ts 10.94 KB 10.94 KB 0.00 KB (0.00%)
schema-template-literal.ts 15.17 KB 15.17 KB 0.00 KB (0.00%)
schema-toArbitraryLazy.ts 21.94 KB 21.94 KB 0.00 KB (0.00%)
schema-toCodeDocument.ts 24.34 KB 24.34 KB 0.00 KB (0.00%)
schema-toCodecJson.ts 19.18 KB 19.18 KB 0.00 KB (0.00%)
schema-toEquivalence.ts 19.01 KB 19.01 KB 0.00 KB (0.00%)
schema-toFormatter.ts 18.87 KB 18.87 KB 0.00 KB (0.00%)
schema-toJsonSchemaDocument.ts 22.60 KB 22.60 KB 0.00 KB (0.00%)
schema-toRepresentation.ts 19.52 KB 19.52 KB 0.00 KB (0.00%)
schema.ts 18.41 KB 18.41 KB 0.00 KB (0.00%)
stm.ts 12.54 KB 12.54 KB 0.00 KB (0.00%)
stream.ts 9.80 KB 9.80 KB 0.00 KB (0.00%)

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — minor suggestions inline.

Reviewed changes — builder-level integration for the response-header schemas added in the previous commit, plus encoding support for HttpApiSchema.encodeToWithHeaders error schemas.

  • HttpApiBuilder now detects branded WithHeaders.Value handler results and encodes their headers separately through Schema.toCodecStringTree, merging them onto the encoded body/stream response; buffered and streaming body paths were updated to allow header injection.
  • HttpApiBuilder.toResponseSchema honors the ~httpApiWithHeaders annotation produced by encodeToWithHeaders, encoding the annotated body schema and applying the resulting headers directly to the response.
  • HttpServerResponse.setHeaders / makeResponse now prefer explicitly supplied content-type and content-length headers over body-derived values, enabling wrapper headers to override the body's default content type.
  • HttpApiEndpoint success-type extraction, validation, codec transformation, and error rejection were updated to unwrap WithHeaders consistently.
  • HttpApiSchemaError gained a ResponseHeaders kind, and runtime/type tests were added covering buffered headers, content-type override, mixed success unions, error headers, and header-encoding failures.

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | Fix all ➔Fix 👍s ➔View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

Comment thread packages/effect/src/unstable/http/HttpServerResponse.ts
Comment thread packages/effect/src/unstable/httpapi/HttpApiBuilder.ts Outdated

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — minor suggestions inline.

Reviewed changes — client-side decoding for response-header schemas added since the prior review.

  • Added client-side decoding for buffered WithHeaders success schemas: success alternatives are re-bucketed by wrapper-first status, the body and response headers are decoded together as { body, headers }, and the result is wrapped back into the branded WithHeaders.Value used by handlers.
  • Extended the client decoder to handle encodeToWithHeaders error schemas by splitting body/headers before applying the user fold.
  • Refactored toCodecArrayBuffer into fromArrayBuffer so both buffered and WithHeaders codecs share the same body-from-ArrayBuffer logic.
  • Added runtime tests covering buffered-header decode failures and round-tripping branded values through a client call.
  • Retired two open Pullfrog review threads: the HttpServerResponse.setHeaders behavioural change is now documented in the PR description, and encodeToWithHeaders error schemas now carry headersCodec in their annotation so headers are validated and string-tree encoded.

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | Fix all ➔Fix 👍s ➔View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

Comment thread packages/effect/src/unstable/httpapi/HttpApiClient.ts

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — test-only commit adding two builder/client integration tests for buffered WithHeaders success decoding.

  • Added a test that a transformed buffered success body (Schema.Date decoded from the JSON payload) and its declared response headers round-trip through the generated client.
  • Added a test that a plain success alternative inside a mixed WithHeaders / non-WithHeaders union decodes as the plain value.
  • Retired the open Pullfrog review thread at HttpApiClient.ts:748: client-side streaming for wrapped headers is acknowledged as out of scope for this stage and deferred to the finalize stage.

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — commit 79f3022e completed the deferred client-side streaming integration for WithHeaders response schemas.

  • Added client-side decoding for WithHeaders<StreamSse | StreamUint8Array, Headers> success schemas: wrapped streams are routed through the existing streaming path, response headers are decoded first, and the result is rebuilt as a WithHeaders.Value<body, headers> whose body is a live stream.
  • Updated HttpApiClient.SuccessType to recurse through WithHeaders before dispatching to stream types, preserving the stream error channel (SSE retry/SSE error/transport error) inside the branded wrapper.
  • Updated HttpApiClient.makeClient to bucket buffered WithHeaders schemas by wrapper-first status and to collect wrapped streams via getStreamSuccessSchemas, including the mixed-union cases covered by tests.
  • Refactored toCodecArrayBuffer into fromArrayBuffer so buffered and stream-with-headers codecs share the same body-from-ArrayBuffer logic.
  • Added runtime tests covering wrapped StreamUint8Array and StreamSse decode/encode, invalid stream headers failing before body consumption, and mixed buffered/stream alternatives; added type tests verifying the widened stream error channels for wrapped streams.

The previously open Pullfrog thread at HttpApiClient.ts:759 (client-side streaming for wrapped headers) is now addressed by this commit.

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — the final commit completes OpenAPI response-header rendering for WithHeaders success schemas and encodeToWithHeaders error schemas.

  • Updated HttpApiSchema.OpenApi response-body extraction to unwrap WithHeaders wrappers and ~httpApiWithHeaders annotations, collecting header schemas alongside content and stream definitions.
  • Emitted response headers in generated OpenAPI specs: "content-type" is intentionally excluded to avoid colliding with the response media type; other headers are written as lower-case names with JSON schemas and required flags.
  • Added the OpenAPISpecHeader public type and surfaced it on OpenApiSpecResponse.headers.
  • Added runtime coverage for success, error, and stream response headers in OpenApi.test.ts.

pnpm vitest --run --project effect packages/effect/test/unstable/httpapi/OpenApi.test.ts, pnpm test-types for the affected httpapi typetests, pnpm lint, and pnpm check all pass.

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — final polish commit ca8ba123 tightened HttpApiBuilder.toResponseSchema header encoding and expanded documentation/tests.

  • Simplified the encodeHeaders helper in toResponseSchema so the headersCodec path and the disableCodecs fallback both consistently report schema errors via error.issue.
  • Added runnable JSDoc examples for WithHeaders, isWithHeaders, and encodeToWithHeaders.
  • Documented the overlapping-union constraint for multiple WithHeaders success alternatives.
  • Added test coverage for user-managed header codecs (disableCodecs: true) and the HttpServerResponse.setHeader precedence change.

All prior Pullfrog review threads remain resolved, and the incremental diff introduces no new behavioral concerns.

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

Comment thread packages/effect/src/unstable/httpapi/HttpApiBuilder.ts Outdated
Comment thread packages/effect/src/unstable/httpapi/HttpApiEndpoint.ts Outdated

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — since the prior pullfrog review (ca8ba123), commit ebc76321 added direct WithHeaders support for non-streaming error responses and tightened the builder narrowing.

  • Exported HttpApiSchema.isWithHeadersValue as an internal type guard and replaced the Predicate.hasProperty cast in HttpApiBuilder.handlerToHttpEffect so wrapper branches narrow correctly.
  • Removed the blanket rejection of WithHeaders in endpoint error position: HttpApiEndpoint.getErrorResponse now unwraps the wrapper only to reject streamed bodies, then routes WithHeaders errors through a new transformError that applies body response encoding and string-tree header codecs and preserves the wrapper in endpoint.error.
  • Added HttpApiSchema.getStatusErrorSchema for wrapper-first error-status lookup and updated HttpApi / OpenApi response extraction to use the wrapper-aware helpers.
  • Added HttpApiBuilder.toResponseErrorSchema, which encodes a branded { body, headers } error value into an HttpServerResponse with status, body, and encoded headers.
  • Expanded tests and type tests to cover direct WithHeaders error schemas, encodeToWithHeaders errors with and without codecs, and OpenAPI error-header rendering.

All prior Pullfrog review threads remain resolved, and the incremental diff introduces no new behavioral concerns.

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

- share one WithHeaders response transformation between the error path and
  the encodeToWithHeaders annotation path in HttpApiBuilder
- pass schema-level status getters into toResponseSchema, removing the
  wrapper ternaries and restoring caching for WithHeaders schemas
- skip WithHeaders-wrapped streams in HttpApi response reflection so the
  client no longer re-groups success schemas by status
- fold HttpApiEndpoint transformError into transformResponseSchema
- drop the single-member union in toCodecArrayBufferWithHeaders
- hoist header codec parsers out of the per-response encode closure
- apply response header schemas once in OpenApi extractResponseBodies

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — since the prior pullfrog review (ebc7632), commit 90125f70 simplifies the WithHeaders integration across the unstable httpapi stack without changing behavior.

  • Unified HttpApiEndpoint.transformSuccess and transformError into a single transformResponseSchema that handles both buffered and stream bodies nested inside WithHeaders.
  • Simplified HttpApiBuilder.toResponseSchema by extracting withHeadersTransformation and caching response schemas by WithHeaders instance or AST.
  • Removed redundant client-side success re-bucketing in HttpApiClient.makeClient; successes are already keyed by wrapper-first status from HttpApi.extractResponseContent.
  • Simplified HttpApiSchema.getResponseEncodingSchema and OpenApi.extractResponseBodies so header collection is centralized and the multipart guard remains in getResponseEncoding.

Prior Pullfrog review threads remain resolved, and the targeted validation passes: pnpm check, pnpm lint, runtime tests (vitest --run --project effect test/unstable/httpapi/ test/unstable/http/HttpServerResponse.test.ts — 150 passed), pnpm doctest --run packages/effect/src/unstable/httpapi/HttpApiSchema.ts (3 passed), and the affected type tests (HttpApiSchema.tst.ts, HttpApiEndpoint.tst.ts, HttpApiClient.tst.ts — 226 passed / 494 assertions across TS 5.9 and 6.0).

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — the final commit 4a0a24483 ensures WithHeaders response headers are stringified even when endpoint codecs are disabled.

  • Updated HttpApiBuilder.makeWithHeadersEncoder to encode the union of declared success header schemas through Schema.toCodecStringTree, so disableCodecs: true still turns non-string header values into strings before they reach the HTTP response.
  • Updated HttpApiBuilder.toResponseErrorSchema to apply the same string-tree encoding when encoding direct WithHeaders error responses.
  • Added a builder test covering both success and error WithHeaders responses with disableCodecs: true, asserting the encoded header values are strings.

All prior Pullfrog review threads remain resolved, and the incremental diff introduces no new behavioral concerns.

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

Expand the Setting Response Headers section with WithHeaders and
encodeToWithHeaders usage, and keep the pre-response handler as the
untyped escape hatch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4.0 enhancement New feature or request

Projects

Status: Discussion Ongoing

Development

Successfully merging this pull request may close these issues.

1 participant