Skip to content

fix(l7): detect HTTP requests whose request line is longer than 64 bytes - #397

Merged
mayankpande88 merged 3 commits into
mainfrom
fix/http-long-request-line
Oct 12, 2026
Merged

mayankpande88 merged 3 commits into
mainfrom
fix/http-long-request-line

Conversation

@mayankpande88

@mayankpande88 mayankpande88 commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Many outbound HTTP/1.x requests were never captured. When a request's method and URL (path plus query string) took up more than about 56 characters, the agent didn't recognize it as HTTP. If that request was the first on its connection, it was lost. Clients that open a new connection per request (wget, curl, many language HTTP libraries, short-lived jobs) lost every request with a long URL. Keep-alive connections that happened to start with a short request were detected and captured normally, so the gap was easy to miss.

This PR detects HTTP requests whatever their URL length.

Engineering detail

Cause. is_http_request (eBPF) reads the first 64 bytes of a write. It checks the method, then requires HTTP/ to start within them. The version check was added to keep Redis inline commands and binary data from being taken for HTTP. A request line whose method and target fill those bytes fails it. Protocol detection runs on every write until a connection's protocol is known, so only a connection's first requests are affected, but for connection-per-request clients that is every request.

Fix.

  • After the method and its space, a request target in origin form (starting with /) or absolute form (http) identifies an HTTP request without the version.
  • Other targets (CONNECT host:port, OPTIONS *) still need HTTP/ within the bytes read, as before. GET somekey and similar non-HTTP text still don't pass.
  • Each method branch copies the target's first bytes from fixed offsets, and a single version scan follows. That keeps verifier-visible reads constant and the program size close to before: the write-path programs grow from 7,887 to 7,978 instructions.
  • Review follow-up (d64f632): the buffer is zeroed before bpf_probe_read_str, which leaves the bytes past the copied string as the stack had them. Without zeroing, a previous request's HTTP/ there could pass the scan. Bounding the scan by the returned length instead cost the verifier 53% more work on writev on 4.18, against about 3% for the zeroing.
  • ebpftracer/ebpf.go regenerated with make build (all 10 variants).

Oldest supported kernel. TestProgramsLoad (the ebpftracer load test) passes on Rocky 8.10, kernel 4.18.0-553 (variant 4.16), for both main and this branch (d64f632). Instructions the verifier processed for sys_enter_write/sys_enter_sendto: 134,613 → 136,096; for sys_enter_writev: 190,337 → 196,030.

CI-equivalent checks. In a Linux container with Go 1.26.5: gofmt, go vet ./..., golangci-lint v2.13.2 (0 issues), and go test ./... (including /containers) all pass.

Local e2e: in Docker Desktop (kernel 6.10, arm64), I ran the published 0.1.10-rc.5 image and a build of this branch as the agent, each with three clients against a server that answers 200 for any path:

  • 20 wgets of a short path: 19 captured on rc.5, 20 on this branch.
  • 20 wgets of a 132-character path with a query string, each on a new connection: rc.5 captured 0, this branch 20.
  • 20 inline GET somekey lines over TCP: 0 HTTP on both builds.

Total node_agent_l7_events_total{protocol="HTTP"}: 19 on rc.5, 40 on this branch (d64f632; 39 on 983e189). No panics.

is_http_request read the first 64 bytes of a write and required " HTTP/"
in them, to keep Redis inline commands and binary data from passing as
HTTP. A request whose method and target took more than about 56 of those
bytes was not detected. Detection runs until a connection's protocol is
known, so such a request was lost whenever it was the first on its
connection: clients that open a connection per request lost every request
with a long path or query string. Keep-alive connections that began with a
short request were detected and kept capturing.

A request target in origin form (a path) or absolute form (http://) now
identifies an HTTP request by itself. Other targets (CONNECT host:port,
OPTIONS *) still need the version within the bytes read. The checks read
fixed offsets per method and keep a single scan, so the programs grow by
about 1%.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request optimizes HTTP request detection in is_http_request by checking the start of the request target (e.g., / or http) directly after the method, which prevents dropping requests with long targets. A fallback loop still scans the buffer for the " HTTP/" signature. Feedback recommends using the return value of bpf_probe_read_str to limit this fallback loop's boundary, avoiding the evaluation of stale or out-of-bounds bytes without the overhead of zero-initializing the scratch buffer.

Comment thread ebpftracer/ebpf/l7/http.c Outdated
bpf_probe_read_str leaves the bytes past the string it copies as the stack
had them, so after a short write the scan for " HTTP/" could match a
previous request's bytes. The 64-byte buffer is now zeroed first: eight
stores, where stopping the scan at the length read added a branch per step
and half again to the verifier's work on the writev program.
@mayankpande88
mayankpande88 merged commit 46ac4c6 into main Oct 12, 2026
7 checks passed
@mayankpande88
mayankpande88 deleted the fix/http-long-request-line branch October 12, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants