Skip to content

Support large WebKit Inspector messages (heap snapshots)#433

Open
nnemirovsky wants to merge 1 commit intogoogle:masterfrom
nnemirovsky:fix/large-message-support
Open

Support large WebKit Inspector messages (heap snapshots)#433
nnemirovsky wants to merge 1 commit intogoogle:masterfrom
nnemirovsky:fix/large-message-support

Conversation

@nnemirovsky
Copy link

Problem

The WebKit Inspector Protocol's Heap.startTracking command returns a trackingStart event containing a full heap snapshot. On real-world pages, this snapshot can be 20-200+ MB. Currently, iwdp fails to relay these messages due to three issues:

  1. MAX_BODY_LENGTH is 64 MB — messages larger than this are silently rejected as "Invalid packet header", breaking the entire event pipeline for that inspector session
  2. Single WebSocket frame — the full payload is buffered into one WebSocket frame, causing peak memory ~2x the payload size on the outgoing side
  3. UTF-8 validation — byte-by-byte validation of 50-200 MB payloads is expensive and unnecessary since the data is always valid JSON from WebKit

Changes

1. Raise MAX_BODY_LENGTH from 64 MB to 256 MB (webinspector.c)

Also adds parentheses to the #define to fix a latent operator precedence issue (1<<26 without parens is ambiguous in compound expressions).

2. WebSocket continuation frames for large payloads (ios_webkit_debug_proxy.c)

Messages larger than 1 MB are split into 1 MB chunks sent as RFC 6455 continuation frames. This reduces peak memory on the outgoing side since ws_send_frame only buffers one chunk at a time.

Small messages (≤1 MB) take the existing single-frame fast path — no behavior change for the vast majority of messages.

3. Skip UTF-8 validation for large payloads (websocket.c)

Payloads larger than 1 MB skip the byte-by-byte UTF-8 validation loop. The data is JSON from WebKit Inspector and is always valid UTF-8.

Also fixes ws_send_frame to clear continued_opcode on FIN frames, matching the receive-side behavior at line 740. Without this fix, sending continuation frames leaves stale state that corrupts subsequent single-frame sends.

Testing

Tested with a connected iPhone running Safari on a Google Shopping page:

  • Heap.enableHeap.gcHeap.startTrackingHeap.gcHeap.stopTracking
  • 33 MB heap snapshot received successfully via continuation frames in ~2 seconds
  • GC events dispatched correctly after the large snapshot
  • No errors in iwdp output
  • Builds clean with -Wall -Werror

Three changes to handle messages that can reach 50-200+ MB:

1. Raise MAX_BODY_LENGTH from 64 MB to 256 MB (webinspector.c)
   - The previous 64 MB limit silently dropped large messages like
     Heap.trackingStart snapshots, breaking the event pipeline
   - Also adds parentheses to fix operator precedence bug (1<<26
     without parens is ambiguous in some expressions)

2. Send large payloads as WebSocket continuation frames (ios_webkit_debug_proxy.c)
   - Splits messages >1 MB into 1 MB chunks using RFC 6455
     continuation frames
   - Reduces peak memory on the outgoing side since the WebSocket
     layer no longer needs to buffer the entire payload at once

3. Skip UTF-8 validation for large payloads (websocket.c)
   - Payloads >1 MB skip the byte-by-byte UTF-8 validation
   - The data is JSON from WebKit Inspector and is always valid UTF-8
   - Also fixes ws_send_frame to clear continued_opcode on FIN,
     matching the receive-side behavior
@muratpekff
Copy link

Great improvement. This was definitely needed. Thanks for tackling it

@Mizzery47
Copy link

Literally good decision. Respect

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants