Skip to content

fix: keep the HPACK decoder usable after a header fails to parse - #1252

Closed
pjfanning wants to merge 1 commit into
apache:mainfrom
pjfanning:hpack-parse-error-keeps-connection-usable
Closed

pjfanning wants to merge 1 commit into
apache:mainfrom
pjfanning:hpack-parse-error-keeps-connection-usable

Conversation

@pjfanning

Copy link
Copy Markdown
Member

Noticed while working on #1251. A single malformed header currently makes every later HEADERS frame on that connection undecodable.

The bug

HeaderDecompression's HeaderListener threw ParsingException straight through the HPACK decoder. Three things follow from that, and they compound:

  1. Decoder.insertHeader calls the listener before dynamicTable.add(...), so the entry for the offending header never reached the table.
  2. decode() unwound at that point, so every representation after it in the block was never read, and never added to the table either.
  3. endHeaderBlock() sat inside the try, so it was skipped — leaving the decoder holding the state and headerSize of the abandoned block.

(1) and (2) desynchronise the decoder's dynamic table from the peer's encoder's, which is not something HPACK can recover from: from then on the peer's indexed references resolve to the wrong entries. (3) resumes the next block part way through a representation.

This is reachable, not theoretical. HeaderDecompression deliberately answers a parse failure with a bad request and keeps the connection open, so an unknown method — already covered by "return bad request response when header parsing fails" — is enough to trigger it.

The fix

Catch ParsingException in the listener, remember the first ErrorInfo, and return null so decoding runs to the end of the block and the dynamic table keeps tracking the peer's. Report the remembered failure once the block is fully decoded, which produces the same bad request response as before.

null rather than the raw value on purpose: the decoder caches what the listener returns against the table entry, and caching an unparsed String where a ContentType is expected would hand a wrong type to a later indexed reference. null means it is parsed again — and fails again, consistently.

endHeaderBlock() also moves into a finally, so anything else that unwinds still resets the decoder — a malformed :path raises Http2ProtocolException, which is not a ParsingException. Running it twice on the success path is a no-op (headerSize is already 0). The outer ParsingException handler stays as a fallback for anything thrown outside the listener.

Tests

New "keep the connection usable after a header parsing failure" in Http2ClientServerSpec: send a request with an unknown method, expect the bad request, then send a valid request on the same connection.

Verified it actually catches the bug — with the fix stashed so HeaderDecompression matched main exactly:

- should keep the connection usable after a header parsing failure *** FAILED ***
  java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsgClass
  waiting for class ...Http2ClientServerSpec$ServerRequest

The follow-up request does not merely come back wrong — it never reaches the handler at all, which is the desync showing up end to end.

  • http2-tests/testOnly ...Http2ClientServerSpec — 8 passed
  • http-core/testOnly org.apache.pekko.http.impl.engine.http2.* — 13 passed
  • http2-tests/test — 353 passed, 25 ignored, 26 pending
  • scalafmtCheckAll, headerCheck, http-core/mimaReportBinaryIssues — clean

Note on #1251

This is based on main so it can be reviewed and backported on its own. It touches the same try block that #1251 rewrites, so the two conflict textually; I will rebase whichever lands second.

🤖 Generated with Claude Code

Motivation:
HeaderDecompression's HeaderListener threw ParsingException straight through the
HPACK decoder, which left the connection unable to decode any later HEADERS
frame:

- Decoder.insertHeader calls the listener before adding the entry to the dynamic
  table, so the entry for the offending header was never added
- decode() unwound at that point, so every representation after it in the block
  was never read and never added either
- endHeaderBlock() was called inside the try, so it was skipped and the decoder
  kept the state and headerSize of the abandoned block

The first two desynchronise the decoder's dynamic table from the peer's
encoder's, which HPACK cannot recover from; the third resumes the next block
part way through a representation. HeaderDecompression answers a parse failure
with a bad request and keeps the connection open, so this is reachable with a
single malformed header - an unknown method is enough.

Modification:
Catch ParsingException in the listener, remember the first ErrorInfo and return
null so that decoding runs to the end of the block and the dynamic table keeps
tracking the peer's. Report the remembered failure once the block is decoded.
Call endHeaderBlock() in a finally as well, so anything else that unwinds - a
malformed pseudo header raises Http2ProtocolException - still resets the
decoder. The outer ParsingException handler stays as a fallback.

Result:
A request with an unparseable header still gets a bad request response, and
subsequent requests on the same connection are decoded correctly.

Tests:
- New "keep the connection usable after a header parsing failure" in
  Http2ClientServerSpec sends a request with an unknown method, expects the bad
  request, then sends a valid request on the same connection. Without the fix
  the second request never reaches the handler at all - the spec times out
  waiting for it - and with the fix it is served normally
- sbt "http2-tests/testOnly ...Http2ClientServerSpec" - 8 passed
- sbt "http-core/testOnly org.apache.pekko.http.impl.engine.http2.*" - 13 passed
- sbt http2-tests/test - 353 passed, 25 ignored, 26 pending
- scalafmtCheckAll, headerCheck, http-core/mimaReportBinaryIssues - clean

References:
Noticed while working on apache#1251

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@pjfanning pjfanning added this to the 2.0.0-M2 milestone Sep 6, 2026
@pjfanning

Copy link
Copy Markdown
Member Author

Closing: this change is already on main.

#1251 was stacked on this branch, and when it merged it carried this PR's commit with it — 4e07cbc98 fix: keep the HPACK decoder usable after a header fails to parse is on main, test included (the Http2ClientServerSpec change there is byte-identical to this PR's). Rebasing this branch onto main leaves an empty diff: the only conflict was against #1297's later CR/LF/NUL check in the same addHeader block, and once that resolves to main's side there is nothing left to apply.

No follow-up needed; the fix is in and covered by its test.

@pjfanning pjfanning closed this Sep 11, 2026
@pjfanning
pjfanning deleted the hpack-parse-error-keeps-connection-usable branch September 11, 2026 14:56
pjfanning added a commit to pjfanning/incubator-pekko-http that referenced this pull request Sep 11, 2026
Motivation:
Nineteen commits landed on main after the model was pinned to `6740cbd`,
ten of which touch a claim it makes. One changes a triage line: apache#1194
documents `idle-timeout` as a bidirectional inactivity timeout that a
client sending bytes inside every window keeps alive by design, where §9
called such a connection an evasion and therefore in scope. Two others
defend a property the model never stated: that a malformed request on
one HTTP/2 stream is answered on that stream and leaves the connection,
its HPACK state and its other streams alone (apache#1252, apache#1297).

Checking the error path for apache#1246 also surfaced a shipped default the
model never mentioned: `error-logging-verbosity = full` writes the
failing request target into the log at warning level, independent of the
client-facing `verbose-error-messages = off` that P3 rests on.

Modification:
Re-pin to `40b07a2`. Add P9, HTTP/2 stream isolation, cited to
`RequestErrorFlow`, with a note recording the two paths around it that
review found and closed: a header that failed to parse unwound out of
the HPACK decoder before the dynamic table was updated, desynchronising
every later HEADERS frame on the connection (apache#1252, merged through
apache#1251); and a field the HTTP/1.1 header parser rejects was reported with
an exception type nothing on the HTTP/2 side caught, failing the
connection (apache#1297, which also closed the RFC 9113 8.2.1 gap of a CR LF
value being accepted silently truncated). Restate P5 and the §9
slow-loris bullet around inactivity rather than evasion, citing the new
`reference.conf` and `timeouts.md` wording, and record apache#1284's leak of
one scheduled task per open request as the in-scope shape. Add apache#1257 to
the §9 smuggling bullet beside apache#1267, and apache#1281 to the P1 note beside
apache#1259 as the third discard path. Add `max-part-count` (apache#1266) to §5a and
a multipart row to §6, which had no multipart input at all, noting
apache#1279's header-state bleed between parts. Add `error-logging-verbosity`
to §5a, a §9 false-friend entry explaining what `verbose-error-messages`
does not govern, and §10.9 recommending `simple` where logs are shipped
or alerted on. Add two §11 misuse patterns: building `Raw-Request-URI`
from request input, and echoing `IllegalRequestContext.rawRequestTarget`
unescaped. Turn §5b.4's two concrete examples into a running tally of
the defects review has found of that shape. Extend §15 to match.

Result:
Every claim is verified against `40b07a2`. Provenance is 23 documented /
33 maintainer / 0 inferred. The P1 frame-size gap stays open; apache#1264 has
not merged.

Tests:
Not run - docs only

References:
Refs apache#1194, apache#1246, apache#1251, apache#1252, apache#1257, apache#1266, apache#1279, apache#1280, apache#1281, apache#1284, apache#1297
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant