Support large WebKit Inspector messages (heap snapshots)#433
Open
nnemirovsky wants to merge 1 commit intogoogle:masterfrom
Open
Support large WebKit Inspector messages (heap snapshots)#433nnemirovsky wants to merge 1 commit intogoogle:masterfrom
nnemirovsky wants to merge 1 commit intogoogle:masterfrom
Conversation
Three changes to handle messages that can reach 50-200+ MB:
1. Raise MAX_BODY_LENGTH from 64 MB to 256 MB (webinspector.c)
- The previous 64 MB limit silently dropped large messages like
Heap.trackingStart snapshots, breaking the event pipeline
- Also adds parentheses to fix operator precedence bug (1<<26
without parens is ambiguous in some expressions)
2. Send large payloads as WebSocket continuation frames (ios_webkit_debug_proxy.c)
- Splits messages >1 MB into 1 MB chunks using RFC 6455
continuation frames
- Reduces peak memory on the outgoing side since the WebSocket
layer no longer needs to buffer the entire payload at once
3. Skip UTF-8 validation for large payloads (websocket.c)
- Payloads >1 MB skip the byte-by-byte UTF-8 validation
- The data is JSON from WebKit Inspector and is always valid UTF-8
- Also fixes ws_send_frame to clear continued_opcode on FIN,
matching the receive-side behavior
|
Great improvement. This was definitely needed. Thanks for tackling it |
|
Literally good decision. Respect |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The WebKit Inspector Protocol's
Heap.startTrackingcommand returns atrackingStartevent containing a full heap snapshot. On real-world pages, this snapshot can be 20-200+ MB. Currently, iwdp fails to relay these messages due to three issues:MAX_BODY_LENGTHis 64 MB — messages larger than this are silently rejected as "Invalid packet header", breaking the entire event pipeline for that inspector sessionChanges
1. Raise
MAX_BODY_LENGTHfrom 64 MB to 256 MB (webinspector.c)Also adds parentheses to the
#defineto fix a latent operator precedence issue (1<<26without parens is ambiguous in compound expressions).2. WebSocket continuation frames for large payloads (
ios_webkit_debug_proxy.c)Messages larger than 1 MB are split into 1 MB chunks sent as RFC 6455 continuation frames. This reduces peak memory on the outgoing side since
ws_send_frameonly buffers one chunk at a time.Small messages (≤1 MB) take the existing single-frame fast path — no behavior change for the vast majority of messages.
3. Skip UTF-8 validation for large payloads (
websocket.c)Payloads larger than 1 MB skip the byte-by-byte UTF-8 validation loop. The data is JSON from WebKit Inspector and is always valid UTF-8.
Also fixes
ws_send_frameto clearcontinued_opcodeon FIN frames, matching the receive-side behavior at line 740. Without this fix, sending continuation frames leaves stale state that corrupts subsequent single-frame sends.Testing
Tested with a connected iPhone running Safari on a Google Shopping page:
Heap.enable→Heap.gc→Heap.startTracking→Heap.gc→Heap.stopTracking-Wall -Werror