Skip to content

Make CompositorScreenshot markers schema-based - #6261

Open
fatadel wants to merge 6 commits into
firefox-devtools:mainfrom
fatadel:issue-5303
Open

fatadel wants to merge 6 commits into
firefox-devtools:mainfrom
fatadel:issue-5303

Conversation

@fatadel

@fatadel fatadel commented Aug 14, 2026 •

Copy link
Copy Markdown
Contributor

Main | Deploy preview

Screenshot markers had no schema and were handled by custom code in tooltip rendering, string-table lookup, marker derivation, and screenshot track creation.

They now use a marker schema. The windowWidth and windowHeight fields collapse into windowSize: { width, height }, described by the new screenshot-size format. The new screenshot-data-url format renders a string-table image URL using the aspect ratio of the sibling windowSize field. The schema also uses the new timeline-screenshots display location to drive screenshot track creation.

Screenshot markers are stored as start and end pairs named CompositorScreenshot <windowID>, allowing generic name-based pairing. CompositorScreenshotWindowDestroyed is converted into the end marker of the last screenshot for its window.

Notes:

  • The final screenshot of a window that is never destroyed is marked incomplete. Its time range is unchanged, but its duration displays as unknown in tooltips and the Marker Table, affecting duration sorting.
  • The Marker Chart shows one row per window, with no separate CompositorScreenshotWindowDestroyed row.
  • Track ordering is unchanged: computeGlobalTracks still adds windows in first-screenshot order.

Closes #5303


Profile

@fatadel
fatadel requested review from canova and mstange August 14, 2026 12:08
@codecov

codecov Bot commented Aug 14, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.83333% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.15%. Comparing base (1132ef9) to head (c5ea847).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/profile-logic/process-profile.ts 89.28% 3 Missing ⚠️
src/profile-logic/marker-schema.ts 83.33% 2 Missing ⚠️
src/profile-logic/process-screenshot-markers.ts 94.59% 2 Missing ⚠️
src/profile-logic/processed-profile-versioning.ts 96.07% 2 Missing ⚠️
src/components/timeline/TrackScreenshots.tsx 95.23% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6261      +/-   ##
==========================================
+ Coverage   84.11%   84.15%   +0.03%     
==========================================
  Files         356      357       +1     
  Lines       38444    38588     +144     
  Branches    10887    10814      -73     
==========================================
+ Hits        32339    32473     +134     
- Misses       5676     5686      +10     
  Partials      429      429              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@canova
canova removed their request for review September 3, 2026 14:33
Comment thread src/profile-logic/marker-schema.ts Outdated
Comment thread src/profile-logic/marker-data.ts Outdated
Comment thread src/profile-logic/marker-data.ts Outdated
@fatadel
fatadel force-pushed the issue-5303 branch 2 times, most recently from 281d211 to 900ffff Compare September 23, 2026 12:02
@fatadel
fatadel requested a review from mstange September 23, 2026 12:17
@fatadel

fatadel commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for your feedback, @mstange! I've addressed the issues and made some other improvements, please re-review when you have time.

@mstange mstange left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I dug into the patch in a bit more detail and found more things; it's a big patch and I'm not sure if I've found everything yet, so apologies in advance if this needs another round or two.

One thing that came to mind is whether it makes sense to take care of #4162 at the same time. Rather than sanitizing out all screenshot fields during sanitization, we could add a privacy category for screenshots and annotate the screenshot URL fields with that. Not sure if that would be an improvement but it seems somewhat consistent; it would let us move towards a world where sanitization only needs to look at the privacy categories.

Comment thread src/components/timeline/TrackScreenshots.tsx Outdated
Comment thread src/components/tooltip/Marker.tsx
Comment thread src/profile-query/formatters/marker-info.ts Outdated
Comment on lines +448 to +449
"windowWidth": 1280,
"windowHeight": 1000

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, what am I looking at here? Wouldn't we expect a windowSize property here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this is an upgrader input fixture for processed profile version 24. So the two fields are expected. We also have the upgraded output snapshot that verifies their conversion to windowSize. Am I not getting right smth?

Comment thread src/profile-logic/marker-schema.ts Outdated
Comment thread docs-developer/CHANGELOG-formats.md Outdated
Comment thread src/profile-logic/marker-schema.ts Outdated
@fatadel

fatadel commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

One thing that came to mind is whether it makes sense to take care of #4162 at the same time. Rather than sanitizing out all screenshot fields during sanitization, we could add a privacy category for screenshots and annotate the screenshot URL fields with that. Not sure if that would be an improvement but it seems somewhat consistent; it would let us move towards a world where sanitization only needs to look at the privacy categories.

The direction makes sense to me too, but I'd prefer to keep #4162 separate to avoid expanding this patch further.

Combine schemas from all input profiles instead of keeping only the
first profile's schemas. Schema-driven screenshots need their field
definitions to translate image URL indexes and create screenshot tracks,
including when only the second profile contains screenshots.
Add schema formats for image string indexes and window dimensions so
marker rendering can describe screenshots without knowing their payload
field names. Resolve the image aspect ratio through a sibling size field
and keep image data out of text formatting and marker searches.
Store window dimensions as one windowSize value so a schema field can
describe them together. Normalize window IDs to strings at the profile
boundary so consumers can use one representation regardless of whether
Gecko supplies a numeric ID or a hexadecimal string.

Apply the same payload shape in Gecko processing, Chrome import, and
the processed profile upgrader.
Persist screenshot field definitions with profiles so tooltip rendering,
string-index translation, and query output can use the schema instead
of separate CompositorScreenshot cases. Identify image URLs by their
field format to keep query results compact.
Encode screenshot lifetimes as start and end marker pairs and include
the window ID in their names. Generic interval derivation can then match
screenshots without its own per-window bookkeeping.

Window destruction closes the last screenshot. An unclosed window leaves
an incomplete interval extending to the end of the thread.
Select screenshot markers through the timeline-screenshots display
location and group tracks by marker name. Read image fields from their
schema so tracks do not depend on CompositorScreenshot payload names.

Use the same classification during sanitization and comparison, and
keep first-screenshot ordering so shared track indexes remain stable.
@fatadel

fatadel commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

Thanks a lot for your thorough feedback, @mstange! I've addressed the issues now. Also, to make it easier to review I've split the changes further into more commits - hope that helps!

@fatadel
fatadel requested a review from mstange September 24, 2026 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make CompositorScreenshot markers schema-based

2 participants