Skip to content

0.10.0 Release - #86

Open
liquidsec wants to merge 48 commits into
stablefrom
dev
Open

0.10.0 Release#86
liquidsec wants to merge 48 commits into
stablefrom
dev

Conversation

@liquidsec

@liquidsec liquidsec commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Cookies across redirects (apply cookies from redirect hops to the hops that follow #75): a cookie set by one hop is sent on the hops that follow it within the same request, which is what lets a login or bot-check page resolve instead of looping. What a chain collects lives for that one request, so nothing carries between requests and a batch of 500 URLs stays 500 independent results. Selection follows RFC 6265 for Domain, Path and Secure, plus a Public Suffix List check so Domain=com or Domain=github.io can't carry a cookie onto an unrelated host, and a cookie may only widen within one registrable domain. A cookie set in the request's own Cookie header always wins over one the chain sets under the same name. What a chain will hold is capped at 4096 bytes per cookie, 50 cookies and 8KB total. redirect_cookies=False, or --no-redirect-cookies, reverts to the old behavior.
  • Responses lost to decompression (Fix responses lost to decompression, add alpn_protocols to request() #78): a response declaring Content-Encoding on an empty body was thrown away entirely, which hit every bodyless redirect, HEAD, and 304, since we ask for gzip, deflate, br on every request. A body that won't decode now comes back undecoded with Response.decode_error saying why, rather than costing the whole response. Also: zlib-wrapped deflate (RFC 1950) is accepted, which is what the header is specified as and what IIS and several CDN fronts send; every Content-Encoding line is undone rather than just the first; and a stack that only partly comes off keeps the deepest result instead of reverting to the bytes that arrived.
  • max_body_size now bounds both directions: it stops the read instead of buffering the whole body and slicing it, so a target can't answer with an unbounded body regardless of the cap, and it bounds decompressed output, so a small response can't inflate past it.
  • alpn_protocols on request(): pick the ALPN offer per request. Requests using resolve_ip now speak HTTP/2 when ALPN negotiates it, instead of negotiating h2 and then sending HTTP/1.1 over it. Defaults are unchanged, since the offer is part of the client's TLS fingerprint: h2 then http/1.1 on the pooled path, http/1.1 alone on the resolve_ip / request_target path.
  • New dependency: psl for the Public Suffix List, compiled in, so no network and no runtime lookup. About 823KB of .so, 297KB in a wheel.
  • CI: Rust toolchain pinned in rust-toolchain.toml (1.97.1). A new stable's clippy had been failing every open PR at once, including PRs that touched nothing but a Python dev dependency, because the workflow lints with -D warnings. Pinning also means local cargo clippy matches CI.
  • Dependency bumps: hyper 1.11.0, http 1.5.0, http-body-util 0.1.5, tokio 1.53.1, futures 0.3.34, serde 1.0.229, serde_json 1.0.151, clap 4.6.6, thiserror 2.0.20, bytes 1.12.1, ruff 0.16.2, actions/setup-python.
  • Version: 0.9.0 -> 0.10.0 (Cargo.toml, pyproject.toml, Cargo.lock).
  • Tests: 209 Rust unit tests, 41 Rust integration tests, 161 Python tests.

The cookie entries in CHANGELOG.md were filed under ## Unreleased while #75 was in flight. #87 folds them into the 0.10.0 section, and wants merging before this does.

liquidsec and others added 30 commits July 27, 2026 16:49
When following redirects, a cookie set by one hop is now sent on the
later hops of the same request, which is what a browser does. It's what
lets a login or bot-check page resolve: those hand you a cookie along
with the redirect, and the cookie has to be on the next request to count
for anything. Without it you land back on the same page or loop.

The jar is request-scoped. It's created in send_inner and dropped when
the request returns, so nothing carries into the next request and two
concurrent requests can never see each other's cookies. A batch of 500
URLs runs 500 independent jars, which keeps every result reproducible on
its own. Every HTTP path shares send_inner, so request(), the batch and
streaming-batch paths, and download() all get this.

Cookie selection follows RFC 6265, which matters beyond correctness: a
cookie with no Domain is host-only, a Domain that doesn't cover the host
that set it is rejected, Path has to match on a segment boundary, and
Secure cookies never go over plain HTTP. That's what stops a redirect
from being used to walk a session cookie onto an unrelated host.

Chain cookies are merged into a caller-supplied Cookie header rather
than sent as a second one.

Opt out with redirect_cookies=False or --no-redirect-cookies.
Every open PR has been failing Rust Tests at the clippy step, dependabot's
and ours alike, including PRs that touch nothing but a Python dev
dependency. The bumps aren't the cause.

CI installs whatever the latest stable is, with no pin. Stable moved to
1.97.1, whose clippy extended manual_filter to catch this in mock.rs:

    files_obj.and_then(|f| if f.is_none() { None } else { Some(f) })

That code is unchanged and has been on dev for a while. It only started
failing because the lint is new and the workflow runs -D warnings, so a
fresh lint turns into a hard error everywhere at once. Locally we were on
1.95, which is why nobody saw it coming.

Rewritten as filter(|f| !f.is_none()), which is what clippy suggests and
is behavior-identical.

The pin is the actual fix for the class of problem. rust-toolchain.toml
rather than a workflow input, so local cargo resolves to the same version
CI uses and this gets caught before pushing instead of after. Keeping
-D warnings is fine once the version is pinned, since new lints then
arrive only when someone bumps it deliberately.

Verified with 1.97.1: fmt, clippy --all-targets --all-features --locked
-D warnings, and cargo test --locked all pass.
It was shipping in the packaged crate, overriding the toolchain for
anyone building from source. Also fixes the comment typo.
fix clippy on rust 1.97 and pin the toolchain
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.52.3 to 1.53.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](tokio-rs/tokio@tokio-1.52.3...tokio-1.53.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.53.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.150 to 1.0.151.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](serde-rs/json@v1.0.150...v1.0.151)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-version: 1.0.151
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
…/tokio-1.53.1

chore(deps): bump tokio from 1.52.3 to 1.53.1
…/serde_json-1.0.151

chore(deps): bump serde_json from 1.0.150 to 1.0.151
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.228 to 1.0.229.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](serde-rs/serde@v1.0.228...v1.0.229)

---
updated-dependencies:
- dependency-name: serde
  dependency-version: 1.0.229
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.12.0 to 1.12.1.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](tokio-rs/bytes@v1.12.0...v1.12.1)

---
updated-dependencies:
- dependency-name: bytes
  dependency-version: 1.12.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.15.20 to 0.16.0.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](astral-sh/ruff@0.15.20...0.16.0)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.16.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…/serde-1.0.229

chore(deps): bump serde from 1.0.228 to 1.0.229
…/bytes-1.12.1

chore(deps): bump bytes from 1.12.0 to 1.12.1
Bumps [clap](https://github.com/clap-rs/clap) from 4.6.1 to 4.6.4.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](clap-rs/clap@clap_complete-v4.6.1...clap_complete-v4.6.4)

---
updated-dependencies:
- dependency-name: clap
  dependency-version: 4.6.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 2.0.18 to 2.0.19.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](dtolnay/thiserror@2.0.18...2.0.19)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-version: 2.0.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
…uff-0.16.0

chore(deps-dev): bump ruff from 0.15.20 to 0.16.0
…/clap-4.6.4

chore(deps): bump clap from 4.6.1 to 4.6.4
…/thiserror-2.0.19

chore(deps): bump thiserror from 2.0.18 to 2.0.19
Two problems found while building against real targets.

A response that declared Content-Encoding but carried no body was thrown
away entirely. gzip and brotli correctly report there's no stream to
read, and the `?` turned that into a transport error, so the caller
couldn't tell the host apart from an unreachable one. Since we ask for
`gzip, deflate, br` on every request, any bodyless response carrying a
Content-Encoding hit this: bodyless redirects, HEAD responses (which echo
the entity headers of the GET they mirror), and 304s (which carry the
headers a 200 would). Those last two are correct server behavior, not
edge cases. `request_batch` and `request_batch_stream` were affected too,
since they share `parse_response`.

`max_body_size` made it worse in two ways. It truncated the compressed
bytes mid-stream, and the resulting decode failure discarded the whole
response, so the cap couldn't be used with compressed bodies at all. And
it only ever bounded the bytes read off the wire, never the inflated
result, so a 291KB response could expand to 300MB resident, well past
the 10MiB default.

So `decompress` now takes the cap and reads incrementally: an empty body
is empty whatever the header claims, output is bounded at every layer,
and a stream that breaks partway keeps what inflated instead of failing.

More broadly, a body that won't decode no longer costs you the response.
Whatever the reason, the status line and headers arrived cleanly and the
raw bytes are what the server actually sent, so they're handed back
undecoded with a note in the debug log. Discarding the response instead
looks identical to an unreachable host from the caller's side, which
throws away far more than a body we can't read. That matters most for the
tool this is for: unexpected bytes aren't grounds for dropping evidence.
`read_body` already bounds those bytes, so returning them can't exceed
the cap.

Content-Encoding is an ordered list, but only an exact match on the whole
header value was recognized, so `gzip, br` and the `x-gzip` alias fell
through and handed back a still-compressed body as if it were content.
Those are now parsed as a list and undone in reverse. A coding we can't
undo returns the body untouched, since decoding the layers beneath it
would only produce nonsense.

Note this narrows what `max_body_size` returns. A caller setting a small
cap on a large compressed body used to receive the whole thing inflated
and now gets it truncated to the cap, which is what the parameter says it
does.

Separately, `RequestConfig.alpn_protocols` already existed and the
direct-connection path already honored it, but the pooled path hardcoded
an h2-first offer and never looked at the config, and the pyo3 signature
for `request()` didn't accept the parameter at all. So from Python there
was no way to keep a request off HTTP/2. That matters when a server puts
a connection-specific header in an HTTP/2 response: RFC 9113 8.2.2
forbids it, hyper kills the stream with PROTOCOL_ERROR, and the response
is lost even though the same server answers cleanly over HTTP/1.1. The
only workaround was passing `resolve_ip` to divert onto the direct path,
which is a DNS-pinning parameter doing protocol selection. The h2-first
default is unchanged, and different offers already can't share a pooled
connection since `TlsKey` includes `alpn_protocols`.

Also gitignores local agent settings and compiled extension modules. A
build left at the repo root shadows the installed package, because pytest
puts the rootdir on sys.path, so a stale one silently gets tested instead
of what you built.
Bumps the github-actions group with 1 update in the / directory: [actions/setup-python](https://github.com/actions/setup-python).


Updates `actions/setup-python` from 6 to 7
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](actions/setup-python@v6...v7)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [http-body-util](https://github.com/hyperium/http-body) from 0.1.3 to 0.1.5.
- [Release notes](https://github.com/hyperium/http-body/releases)
- [Commits](hyperium/http-body@http-body-util-v0.1.3...http-body-util-v0.1.5)

---
updated-dependencies:
- dependency-name: http-body-util
  dependency-version: 0.1.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 2.0.19 to 2.0.20.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](dtolnay/thiserror@2.0.19...2.0.20)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-version: 2.0.20
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [clap](https://github.com/clap-rs/clap) from 4.6.4 to 4.6.6.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](clap-rs/clap@clap_complete-v4.6.4...clap_complete-v4.6.6)

---
updated-dependencies:
- dependency-name: clap
  dependency-version: 4.6.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.16.0 to 0.16.2.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](astral-sh/ruff@0.16.0...0.16.2)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.16.2
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.32 to 0.3.34.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/main/CHANGELOG.md)
- [Commits](rust-lang/futures-rs@0.3.32...0.3.34)

---
updated-dependencies:
- dependency-name: futures
  dependency-version: 0.3.34
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
liquidsec and others added 18 commits August 17, 2026 07:14
…/futures-0.3.34

chore(deps): bump futures from 0.3.32 to 0.3.34
Bumps [http](https://github.com/hyperium/http) from 1.4.2 to 1.5.0.
- [Release notes](https://github.com/hyperium/http/releases)
- [Changelog](https://github.com/hyperium/http/blob/master/CHANGELOG.md)
- [Commits](hyperium/http@v1.4.2...v1.5.0)

---
updated-dependencies:
- dependency-name: http
  dependency-version: 1.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…/http-body-util-0.1.5

chore(deps): bump http-body-util from 0.1.3 to 0.1.5
…/clap-4.6.6

chore(deps): bump clap from 4.6.4 to 4.6.6
…/http-1.5.0

chore(deps): bump http from 1.4.2 to 1.5.0
…/thiserror-2.0.20

chore(deps): bump thiserror from 2.0.19 to 2.0.20
…uff-0.16.2

chore(deps-dev): bump ruff from 0.16.0 to 0.16.2
…tions/dev/github-actions-696b649f44

chore(deps): bump actions/setup-python from 6 to 7 in the github-actions group across 1 directory
Bumps [hyper](https://github.com/hyperium/hyper) from 1.10.1 to 1.11.0.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](hyperium/hyper@v1.10.1...v1.11.0)

---
updated-dependencies:
- dependency-name: hyper
  dependency-version: 1.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Four problems the review caught, plus one flag that makes the third fix
safe to rely on.

`Content-Encoding: deflate` is specified as a zlib stream (RFC 9110
8.4.1.2 points at RFC 1950) and only bare deflate (RFC 1951) was handled.
IIS and several CDN fronts send the wrapped form, and we ask for deflate
on every request, so a real body came back as compressed bytes with
nothing to distinguish it from content. `decode_deflate` now picks a
decoder from the two-byte zlib header and falls back to the other flavor
when the first inflates nothing, since a raw stream can pass that check
by coincidence. Which shape arrived goes in the debug log, because it
says something about what is in front of the server.

Repeated `Content-Encoding` lines mean the same thing as one comma-joined
line in arrival order (RFC 9110 5.3), and an edge that compresses an
already-compressed body adds its own line rather than editing the one
below it. Only the first line was read, so one layer came off and the
still-compressed remainder was returned as the body. Joining `get_all`
feeds the list parser. The lines stay untouched in `headers`, so the
front-end/back-end disagreement is still visible. Empty list elements are
ignored now as well (RFC 9110 5.6.1.2), which the join can produce from a
header line that carried no value.

`dispatch_direct` discarded the negotiated protocol and always spoke
HTTP/1.1, so `resolve_ip` with an h2 offer negotiated h2 and then sent
HTTP/1.1 over it, which a server can only answer by hanging up. It now
dispatches over whatever ALPN agreed on. `request_target` with an h2
offer is refused, with an error saying where to go instead: h2 carries
the target in `:path`, built from the URI, so there is no request-line to
override. The http/1.1-only default on that path stays, on stronger
grounds than compatibility. The ALPN offer is part of the client's TLS
fingerprint, so changing it would change how every request host_header
and the SSRF paths have ever sent looks on the wire.

`read_body` collected the whole body and then truncated it, so the new
comment claiming it stopped at the cap was wrong and `max_body_size`
never bounded the read at all: a 40MB response against a 1KB cap was
buffered in full. It now stops asking for frames at the cap and reports
whether it cut the body, which also replaces inferring truncation from
`len() >= max_body`, a test that can't tell a cut body from one landing
exactly on the cap. Stopping early abandons the response so the
connection can't be reused, which is the better trade when the
alternative is unbounded.

Handing back an undecodable body instead of dropping the response is
right, but it turned what used to be a loud error into a quiet wrong
answer: `content` held compressed bytes with nothing saying so, and the
debug log only prints at verbosity >= 1. `Response.decode_error` now
carries the reason, so anything that hashes, matches or diffs bodies can
tell encoded bytes from content before it treats them as evidence.
Reading every `Content-Encoding` line rather than just the first fixed the
genuinely double-compressed body, and broke the other shape of the same
header. A proxy that re-adds `Content-Encoding: gzip` in front of a
backend that already set it, without compressing again, leaves two lines
over a singly-compressed body. Joined, that reads `gzip,gzip`: the first
peel gives the body, the second fails because plain HTML is not gzip, and
`decompress` was reverting to the bytes that arrived. So a response that
used to hand back a readable body handed back compressed bytes and a
`decode_error` instead.

A layer that won't come off underneath one that did now keeps the deepest
result. The alternative reading, a body really encoded that many times,
leaves a layer on and is indistinguishable from this one, so the response
is still flagged, and the reason says how far decoding got:

    1 of 2 content-encoding layers came off: gzip decompression failed:
    invalid gzip header

Nothing came off at all is unchanged and still returns exactly what
arrived, which is what `gzip, br` over a gzip-only body does, since
brotli is the outermost coding there.

`decode_error` on `Response` therefore no longer means "these are the
bytes as they arrived" on its own. It means the body is not what
`Content-Encoding` declared, and the reason distinguishes untouched bytes
from a stack only partly undone. Docs and the README say so.
Merging chain cookies into a caller-supplied `Cookie` header appended
without looking at names, so a site that resets a cookie the caller had
pinned produced `Cookie: session=OLD; session=NEW` on the next hop. Which
value the target reads is then up to its stack: Express keeps the first,
PHP and Django keep the last. Same request, different session depending on
what the site is written in.

A header the caller wrote is a header we send, so their value wins, and it
goes out once. The jar records their cookie names when the request starts
and refuses to store a `Set-Cookie` naming one of them. Doing it at store
time is what makes the rule hold everywhere in one place: the chain can't
replace the value, an expiry can't delete it, and the two can't go out
together as a duplicate pair. Names compare case-sensitively (RFC 6265
4.1.1), and every `Cookie` header the caller supplied counts, since the
server sees all of them.

Cookies the chain sets under other names are unaffected and still merge in
behind the caller's.

`store` now hands back the names it refused so the chain logs them. A site
trying to overwrite a cookie you pinned is worth seeing rather than
silently dropping.

Documented on the `redirect_cookies` field, on
`should_forward_redirect_cookies`, in the `cookies` module docs, in
`request()`'s docstring (which said nothing about `redirect_cookies`
before), and in the README section, whose old note only described the
ordering.

Also narrowed the README's claim that the RFC 6265 scoping stops a cookie
being walked onto another host. That holds for cookies the chain picked up.
Headers the caller supplies are sent as given on every hop, cross-host
redirects included, which is the point of setting one.
A response may carry as many `Set-Cookie` headers as it likes, and every hop
after it carried all of them. 90 cookies of 2KB in one redirect, which is a
legal response and well under hyper's header limit, made the next request
send a 184,848 byte `Cookie` header, with each further hop free to add more.
Against hosts we don't trust, which is the normal case, that let the target
decide how much we hold and how much we send.

Three ceilings, with numbers taken from what clients and servers already do
rather than picked: 4096 bytes per cookie and 50 cookies, which RFC 6265 6.1
asks a client to support and which is roughly where browsers sit, so a chain
needing more is not a login flow; and 8KB across the chain, counting the
separators, because past that the next server rejects the header line anyway
(nginx's `large_client_header_buffers`, Apache's `LimitRequestFieldSize`), so
growing further would only mean sending traffic that can't be answered.

Whatever gets there first keeps the room. A chain's early cookies are the ones
a login flow needs, so what gets dropped is whatever a later hop piles on top.
A reset frees what the old value held, so a site updating one cookie every hop
never fills the budget. The same flood now leaves the second hop carrying 6171
bytes, and what was dropped goes in the debug log rather than being swallowed,
since an invisible cap reads as full coverage. The caller's own cookies aren't
subject to any of this: their header is theirs.

`CookieJar` is now `ChainCookies`. Everywhere else that word means storage
attached to a session that outlives a request, expires cookies on a clock, and
is shared. This is a scratch buffer for one redirect chain that gets thrown
away when the request returns, and the old name had the docs explaining that it
wasn't the thing it was named after. Renaming now costs nothing since the type
has never shipped.
…d-alpn

Fix responses lost to decompression, add alpn_protocols to request()
#78 landed on dev, so the decompression and ALPN work now overlaps this
branch in three places.

CHANGELOG: both sides added a section. The cookie entries stay under
Unreleased, above the 0.10.0 section dev shipped.

`request()`'s docstring: both sides documented a new parameter in the same
block. Both paragraphs stay, `redirect_cookies` first to match the order the
parameters appear in the signature.

`dispatch_direct`: dev made it dispatch over HTTP/2 when ALPN negotiates it,
this branch gave `build_request` a chain-cookie argument. Both, with `None`
passed on either protocol branch, since this path doesn't follow redirects
and so never has a chain to take cookies from.

Verified together: 206 lib tests, 10 redirect-cookie tests, 161 Python
tests, clippy and fmt clean, plus a redirect chain that sets a cookie and
carries a compressed body on both hops, which exercises the two features in
one request.
`domain_matches` is a suffix check with a label-boundary test, and on its own
that accepts `Domain=com`: it does cover the host that set it, so RFC 6265
5.3 step 6 passes, and then it covers every other `.com` the chain visits. So
a hop on a host an attacker owns could scope a cookie to a whole suffix and
have the next hop carry it:

    hop 1  GET https://attacker.com/x
           302 Location: https://victim.com/
           Set-Cookie: session=forced; Domain=com; Path=/
    hop 2  GET https://victim.com/  ->  Cookie: session=forced

Same for `co.uk`, `github.io` and `s3.amazonaws.com`, and those last two are
worse in practice: a sibling under them costs an attacker nothing. It runs the
other way too, a real site setting an over-broad Domain (harmless in browsers,
because browsers reject it, which is why servers ship it) having its session
handed to whatever the chain redirects to under the same suffix. Exactly the
property the module docs and the README claimed to guarantee.

Step 5 of the same section is the missing piece and it needs the list, so
`psl` is now a dependency. The list is compiled in, so no network and no
runtime lookup, and dependabot already watches cargo deps here, which keeps
the list current without anyone remembering to. Costs about 823KB of `.so`,
297KB in a wheel.

Three cases now, after the existing shape check:

An IP host has no hierarchy to widen into, so it stays host-only.

A `Domain` that is itself a public suffix is refused, unless it is the host,
where the RFC makes it host-only because the attribute says nothing extra.
That exception is load-bearing for us: an unlisted TLD counts as a suffix, so
without it `Domain=localhost` would be dropped and local targets would quietly
stop working.

Otherwise the `Domain` has to land on the same registrable domain as the host
that set it. That is stricter than step 5, which only refuses a Domain that is
itself listed, and still lets a host reach past its own registrable domain
when the wider name isn't: `attacker.s3.amazonaws.com` setting
`Domain=amazonaws.com` reaches `victim.s3.amazonaws.com`, since only
`s3.amazonaws.com` is in the list. Every real widening still works, an auth
host handing a cookie to an app host under the same domain included.

Docs say what the rule is now rather than claiming the shape check covers it.
Also fixes a doc line the earlier rename mangled, which read "deliberately not
a cookie chain" where the point was that it isn't a jar.
…/hyper-1.11.0

chore(deps): bump hyper from 1.10.1 to 1.11.0
apply cookies from redirect hops to the hops that follow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant