Add HttpApiSchema.WithHeaders for typed response headers - #6880
Open
wmaurer wants to merge 17 commits into
Open
Conversation
🦋 Changeset detectedLatest commit: b7dec01 The changes in this PR will be included in the next version bump. This PR includes changesets to release 30 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — 15 files adding HttpApiSchema.WithHeaders for typed HTTP response headers across HttpApiSchema, HttpApiEndpoint, HttpApiBuilder, HttpApiClient, and OpenApi.
- Schema definition:
WithHeadersmarker schema stores.headersand.bodysub-schemas with a structural declaration predicate ({ headers, body }), matching the existing stream-marker pattern. - AST annotation lifting: Status and encoding annotations are lifted from the body onto the wrapper’s AST at construction time; annotations applied to the wrapper itself win. Each wrapper gets its own AST instance so per-instance caches stay sound.
- Endpoint validation:
validateResponseExclusivityrejects aWithHeadersmember sharing a(status, content-type)pair with any other member.unwrapResponseSchemathreads throughisStreamSchemaandisNoContentchecks so the existing validation logic works unchanged. - Server encoding: Buffered encroachment passes through
getWithHeadersTransformation— encode body, thenResponse.setHeaders. The stream encoder inmakeStreamEncoderextracts headers from the handler response and forwards them toResponse.stream.disableCodecs: trueis respected. - Client decoding:
withHeadersToResponsedecodes headers viaSchema.decodeUnknownEffect, then decodes the body (buffered or stream). The single-encoding-override overloads onschemasToResponse/toCodecArrayBufferkeep the wrapper AST authoritative for content-type. - OpenAPI:
dropUndefinedMemberstrips theundefinedmember thatSchema.optionalinjects, so optional headers render as plain{ type: "string" }rather thananyOfwith a null variant. Non-object-shaped headers schemas silently contribute no entries (mirroringprocessParameters). - Tests: 139 runtime tests across 5 test files plus an integration test over
NodeHttpServer.layerTest, covering all body encodings, streams, errors, both exclusivity-error paths, wrapper-annotation precedence,disableCodecs,responseModevariants, and the OpenAPI output. 4 type-test files cover handler return shape (with negative.not.toBeCallableWithassertions), client types across all response modes, and the error channel.
DeepSeek Pro (free via Pullfrog for OSS) (Kimi K2 not used — the program covers this model; add its provider key to run your pick) ｜ 𝕏
Contributor
Bundle Size AnalysisGenerated from PR build output; treat the content below as untrusted.
|
Contributor
|
I'm not too sure about the experience of defining error headers yet. I'll have a look into it shortly. |
Adds the WithHeaders marker schema (and isWithHeaders guard) for wrapping a response body with a typed headers schema, following the StreamSse / StreamUint8Array declare-schema idiom. Lifts the body's httpApiStatus and ~httpApiEncoding annotations onto the wrapper so existing status/encoding getters keep working unchanged, while an explicit annotation on the wrapper itself still wins. Also adds internal classification helpers (unwrapResponseSchema, isNoContentSchema, getResponseContentType, replaceWithHeaders) for later endpoint/server/client/OpenAPI integration.
Endpoint construction now recognizes HttpApiSchema.WithHeaders success and error responses: codecs are applied to the wrapper's headers/body sub-schemas inside transformResponse (keeping disableCodecs: true honest), and a new validateResponseExclusivity check rejects a WithHeaders response sharing a (status, content-type) pair with any other response in the same success/error set. Streaming bodies remain forbidden inside WithHeaders errors and are supported inside WithHeaders successes.
Split HttpApiBuilder's getResponseTransformation to detect HttpApiSchema.WithHeaders success/error schemas and encode headers and body separately, applying the encoded headers to the buffered response via Response.setHeaders. getResponseEncode itself is untouched, so all four buffered encodings (including the empty-body path) gain headers uniformly.
Extends makeStreamEncoder to encode declared headers on streaming success responses (StreamUint8Array and StreamSse) wrapped in HttpApiSchema.WithHeaders, mirroring the buffered path from Task 3.
Client-generated response decoders now unwrap HttpApiSchema.WithHeaders,
decoding response headers alongside the buffered or streamed body and
returning { headers, body }. Restores the five HttpApiBuilder tests that
Tasks 3/4 had to weaken to responseMode: "response-only" pending this.
Extend OpenApi.ts to describe responses[status].headers for endpoints whose success/error schemas use HttpApiSchema.WithHeaders. A header is required only when its property signature is non-optional and every response member at that status declares it.
The existing type test for WithHeaders handler results only checked
that a full {headers, body} return value compiles, which is vacuous
under hole<T>() assignability checking. Add companion
.type.not.toBeCallableWith assertions proving omission of headers or
body is rejected, matching the file's existing negative-assertion
idiom.
…apper annotations Three defects in the WithHeaders response-header feature: - `WithHeaders` reused a module-level declaration whenever the body carried neither a status nor an encoding annotation, so every default 200-JSON success (and default 500-JSON error) wrapper shared one AST object. The response-schema cache in `HttpApiBuilder` is keyed on that AST and the `WithHeaders` transformation closes over instance-specific `headers` / `body` sub-schemas, so a second endpoint encoded its value through the first endpoint's schemas. Annotate unconditionally (which allocates a fresh AST), and skip the cache for `WithHeaders` since the opaque AST cannot describe the sub-schemas it closes over. - A status annotation applied to a wrapper around a stream diverged from the status recomputed off the stream body: OpenAPI threw a `TypeError` and the server responded with the body's status while the client decoded on the wrapper's. Thread the effective status through `addStreamContent` and `getStreamSuccessSchema`, and stop asserting the status map entry exists. - `~httpApiEncoding` applied to a wrapper was honoured only by the server encoder; the endpoint codecs, content-type registration, OpenAPI content and client decoder all read the body's AST. The wrapper's AST is now authoritative everywhere, as the constructor's JSDoc promises.
validateSuccessResponse still read body.ast instead of schema.ast when checking a buffered success against a concurrent streaming response, causing a false rejection for WithHeaders responses whose wrapper overrides the body's content type (e.g. asText() over a String body). Bring it in line with the established rule that the WithHeaders wrapper's AST is authoritative for encoding everywhere. Also constrain toCodecArrayBuffer/schemasToResponse's encodingOverride parameter with overloads so it can only be passed alongside a single-element schema array, matching its one actual override-passing caller (withHeadersToResponse) instead of relying on convention.
- Un-export WithHeadersTypeId, matching StreamSchemaTypeId, the marker
WithHeaders mirrors. MultipartTypeId is public only because it appears
in the public asMultipart<S> brand type; WithHeadersTypeId does not.
Users discriminate via isWithHeaders.
- Document that only object-shaped headers schemas reach the OpenAPI
document, and pin the behaviour with tests on both sides: no
responses[status].headers is emitted, but the schema still decodes.
This matches processParameters, which bails the same way for request
params/query/headers.
- Prefer a description annotation on the wrapper over one on the body,
consistently for no-content and content responses, per the rule that
the wrapper's AST is authoritative.
- Mark the WithHeaders JSDoc example as a doctest; it had never been
typechecked and did not compile.
- Pin the encode-union dispatch for a plain Schema.Struct({ headers, body })
declared alongside a WithHeaders member: declaration order decides.
- Normalize the new internal helpers to the file's bare @internal idiom.
`HttpApiTest` runs in memory, so nothing in the branch exercised the socket path. Adds one case to the platform-node integration suite covering the empty-body, JSON, and streaming responses, asserting both the raw wire headers and the generated client's decoded shape.
The existing example only declares the endpoint. Adds a second,
executable example covering the handler returning `{ headers, body }`
and the generated client decoding it back.
`deno check .` (the Types on Deno CI job) runs real tsc, which rejected
three things the repo's tsgo-based `pnpm check` accepted:
- `asText()` applied to a `WithHeaders` wrapper. Its constraint is
`Encoded: string`, and a wrapper's encoded side is `{ headers, body }`.
Encoding annotations belong on the body, which the wrapper lifts; the
wrapper-precedence case now uses `asJson`, whose signature admits it,
and pins precedence against a body annotated `asText()`.
- `respond` in the ambiguous-dispatch test inferred `unknown` rather than
`any` from its `any`-typed inputs, so every use of its result errored
and the surrounding `it.effect` lost its requirements. Annotated its
return type.
Every other changeset in .changeset/ is unwrapped. Changeset prose also lands in GitHub release notes, which render a single newline as a line break, so hard-wrapped paragraphs come out ragged there.
Hand-written source in packages/effect/src does not use em dashes; the only occurrences under httpapi/ are in the vendored Scalar and Swagger bundles.
wmaurer
force-pushed
the
httpapi-response-headers
branch
from
August 4, 2026 04:23
4295267 to
b7dec01
Compare
Contributor
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — the latest commits on the httpapi-response-headers branch. The prior review approved the feature; this run focused on the follow-up commits, which harden the implementation and add integration coverage.
- Added a changeset for the
WithHeadersresponse-header feature. - Hardened AST handling: every
WithHeaderswrapper now allocates its own declaration AST, andHttpApiBuilder.toResponseSchemaskips AST-keyed caching for wrappers so per-instance sub-schemas cannot collide. - Fixed encoding resolution: status and content-type are now read from the wrapper AST everywhere, so annotations applied directly to the wrapper consistently win over the lifted body annotations.
- Added Node server integration coverage:
packages/platform-node/test/HttpApi.test.tsexercises the empty-body, JSON-body, and streaming paths over a real socket. - Improved documentation: added runnable JSDoc examples showing how handlers produce and clients consume the
{ headers, body }shape.
Validation run for this review: effect httpapi runtime tests (139 tests), httpapi type tests (142 assertions), platform-node HttpApi.test.ts, pnpm lint, and pnpm doctest --run packages/effect/src/unstable/httpapi/HttpApiSchema.ts all pass. pnpm check and pnpm docgen failed only on unrelated pre-existing files (packages/ai/openai/test/OpenAiLanguageModel.test.ts and an HttpStaticServer example).
Kimi K2 (free via Pullfrog for OSS) ｜ 𝕏
Contributor
I think this might be a good approach: https://github.com/Effect-TS/effect/pull/6934/changes#diff-dd36f5096e4fc3b43f5cba36ed8c34e8c236dc9967912db8e6e71d883c5a8511R1648 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type
Description
HttpApiSchema.WithHeaders, a marker schema pairing a headers schema with a body schema, usable for successes and errorsHttpApiEndpoint,HttpApiBuilder,HttpApiClient, andOpenApi, the same way those modules already intercept the stream markers{ headers, body }, generated clients decode the same shape, and the OpenAPI document gainsresponses[status].headersWithHeadersmember that shares a(status, content-type)pair with another member of the same endpoint, since the client discriminates on exactly those two signalsWithHeadersresolves insideunstable/httpapiinto an ordinaryHttpServerResponseMotivation
HttpApican declare a schema for request headers, but the success and error sides can describe only a status code and a body. There is no typed way to declare, produce, or consume a response header.The motivating case is a
POSTthat creates an entity and returns201 Createdwith an empty body and aLocationheader, a bog-standard REST pattern. TodayHttpApiClientgives no typed access to thatLocation.Existing escape hatches all work but are untyped: reading
HttpClientResponseviaresponseMode, returning anHttpServerResponsefrom a handler (which requires a cast and teaches neither the client types nor the OpenAPI document anything), or setting blanket headers fromHttpApiMiddleware.This follows the API shape @tim-smart called for in the Discord thread on 15 Oct 2025:
#5633 was closed as stale in Jul 2026 asking for a fresh proposal against the current API shape. The API has moved since that thread:
effect-smolmerged, HttpApi lives inpackages/effect/src/unstable/httpapi/, the builder-styleaddSuccess/addErrorare gone, and the client'swithResponse: trueis nowresponseMode. This is written against the current shape.API
Handlers return
{ headers, body }; generated clients decode the same shape.Strictness follows the schema, matching request headers:
Schema.Stringis required and a missing header is a decode failure surfaced asSchema.SchemaError, the same failure type the client already produces for response body decode failures.Schema.optional(...)yieldsundefined. No implicit leniency, so the declared type never lies.Status and content-type are not re-derived. The constructor lifts
httpApiStatusand~httpApiEncodingoffbody.astonto the wrapper's AST, soHttpApiSchema.Created,asText(), andHttpApiSchema.status(...)compose unchanged. The wrapper's AST is authoritative everywhere; an annotation applied to the wrapper itself wins over the body's.responseModeinteraction:decoded-only(default){ headers, body }decoded-and-response[{ headers, body }, HttpClientResponse]response-onlyHttpClientResponse, with no header decodingresponse-onlyshort-circuits before any schema work, as it does today, so a declared-but-missing header cannot fail aresponse-onlycall.Implementation
WithHeadersis a marker schema carrying.headersand.bodysub-schemas, which the other httpapi modules intercept. That is structurally identical to howHttpApiSchema.StreamSse/StreamUint8Arrayare built and detected today viaisStreamSchema. The interception mechanism already exists, is already threaded through all five modules, and is already tested.HttpApiSchema.ts: the schema,isWithHeaders, and internal unwrap helpers.HttpApiEndpoint.ts:transformResponseapplies codecs (headers viaSchema.toCodecStringTree, matching how request headers are built), sodisableCodecs: truestays honest.validateSuccessResponseandgetErrorResponseunwrap before classifying.HttpApiBuilder.ts: the buffered path encodes the body through the untouchedgetResponseEncode, then applies a single trailingResponse.setHeaders, so all four encodings gain headers uniformly including the empty-body path. The streaming path passes encoded headers toResponse.stream.HttpApiClient.ts: decodesresponse.headersthrough the headers codec alongside the body.OpenApi.ts: emitsresponses[status].headers. The OpenAPIResponse.headersobject was previously never emitted, so this is purely additive.Errors use the same wrapper and the same machinery:
makeErrorSchemaalready routes throughtoResponseSchema. The accepted consequence is that the value a handler fails with becomes{ headers, body: MyError }, and the client's error channel matches. That is noisier at thefail/catchsite than for successes, but a second, divergent mechanism for errors would be worse and would not handle empty-body error responses cleanly.Deliberate decisions worth a reviewer's attention
Exclusivity rule. A
WithHeadersmember may not share a(status, content-type)pair with any other member of the same endpoint; violations throw at endpoint construction. The client's decode map is keyed by status, and within a statusgroupSchemasByContentTypeunions each content-type bucket into one codec over the response bytes, so status and content-type are the only runtime signals. If a plain schema and aWithHeadersschema shared a bucket, the caller's return shape would vary at runtime depending on which headers the server happened to send. This is a client-side ambiguity only; the server discriminates on the input value's shape.Its main cost falls on errors, which share a status more often: two
500JSON errors where only one carriesRetry-Afterare rejected and need distinct statuses. A looser rule, permitting a shared bucket when every member isWithHeaderswith a structurally equal headers schema, was considered and deferred, since loosening later is backwards-compatible while tightening is not.Nesting throws in the schema constructor, not at endpoint construction. That is stricter and earlier than strictly necessary, and it gives a better error site.
The declaration predicate is shape-based, matching any
{ headers, body }value. A plainSchema.Struct({ headers, body })declared alongside aWithHeadersmember at a different status is therefore indistinguishable in the server's encode union, and the first declared member wins. This is pinned by a test (HttpApiBuilder.test.ts, "dispatches an ambiguous{ headers, body }value to the first declared member") rather than prevented, and it is not new in kind:streamSchema = Schema.declare(Stream.isStream)is looser still and backs both stream markers. The difference is that endpoint validation already guarantees at most one stream candidate per status, whereas these two members coexist legally. Happy to close it with an extra construction-time check if you would prefer that.Only object-shaped headers schemas reach the OpenAPI document. OpenAPI models
Response.headersas a fixed name to schema map, so a schema with dynamic keys (Schema.Record) has no representation there. It still encodes and decodes normally; it simply contributes no entries. This mirrorsprocessParameters, which bails identically for non-object-shaped requestparams/query/headersonmain. Documented in theWithHeadersJSDoc and pinned on both sides.Two bits of machinery that are not obvious from the diff:
dropUndefinedMember(OpenApi.ts) strips theundefinedunion member thatSchema.optionalinjects, so an optional header renders as{ type: "string" }rather than ananyOf. The same defect is latent inprocessParametersfor request parameters; that is pre-existing and left for a follow-up rather than mixed into this changeset.schemasToResponse/toCodecArrayBuffer(HttpApiClient.ts) exist so the wrapper's AST stays authoritative for content-type. The overload is type-level, so a multi-member array cannot be passed together with an override.Testing
Runtime (
packages/effect/test/unstable/httpapi/): the motivatingLocation-on-201-with-empty-body round trip, headers on a JSON body, a missing required header failing asSchema.SchemaError, an absent optional header yieldingundefined, error responses with headers, SSE andUint8Arraystreams with headers,response-onlynot decoding headers, both construction-time validation errors, wrapper-annotation precedence for status/encoding/description, twoWithHeadersendpoints staying independent, and the OpenAPIresponses[status].headersoutput.Integration (
packages/platform-node/test/HttpApi.test.ts):HttpApiTestruns in memory, so one case was added to the platform-node suite, which serves over a real socket viaNodeHttpServer.layerTest. It covers the empty-body path, where the headers are the only thing written, and the streaming path, where they must be flushed before the first chunk. It asserts both the raw wire headers and the generated client's decoded shape.No platform package needed a source change:
WithHeadersresolves entirely insideunstable/httpapiinto an ordinaryHttpServerResponse, and bothNodeHttpServerandBunHttpServeralready applyresponse.headersuniformly across every body variant.Type-level (
packages/effect/typetest/unstable/httpapi/): handler return type, client return type across all threeresponseModevalues, and the error channel for aWithHeaderserror. The handler-return test carries negative.not.toBeCallableWithassertions so it cannot pass vacuously.Validation
Every job in
.github/workflows/check.ymlwas run locally:pnpm lint: passpnpm checkandpnpm test-types: pass, 1992 tests and 4986 assertionspnpm build: pass, withstripInternal: trueas CI sets itdeno check .: pass, samestripInternalpatchpnpm test: 8717 passed across 340 files. Two unrelated flakes, both passing on their own and neither touching httpapi:@effect/platform-browserRpcWorker.test.ts > Streamasserts on a 2-second sleep, and@effect/platform-nodeNodeHttpClient.test.ts > jsonplaceholder schemaBodyJsonmakes a live network request.pnpm doctest: pass, 3376 testspnpm docgen: pass, 3319 examples typecheckedpnpm ai-docgen: pass,LLMS.mdunchangedpnpm circular: passpnpm bundle-compare 0a0f2f0fb: 0.00 KB / 0.00% on every fixture. Note there is no httpapi bundle fixture, so this confirms no regression elsewhere rather than measuring this feature's own cost.Narrower runs worth naming: the httpapi suite is 139/139, and
@effect/platform-nodeHttpApi.test.tsis 45/45.A changeset is included (
patch, matching the repo's pre-release convention).Related