Devtools tracemode#222
Conversation
…d core helper; Restrict screencast feature only for the live mode
There was a problem hiding this comment.
Holistic review against the parallel tracing work on feature/elements-snapshot. Both branches aim for the same end goal — producing a trace from a test — with different architectural choices. These comments flag discrepancies and suggest improvements drawn from the PoC.
packages/shared/src/types.ts
DevToolsMode = 'live' | 'trace'
suggestion: The
modetoggle couples two independent concerns — disabling the debug UI and enabling trace capture. The PoC uses separate flags (captureElements,disableDebugger) so users can record traces with the debugger running during development. Either document why these are coupled here, or split them.
ActionSnapshot interface
praise: Clean typed contract in
shared. One gap worth noting:commandstores the raw adapter command name (url,setValuefor WDIO;get,navigateTofor Selenium/Nightwatch). Consider normalizing to the trace action vocabulary (Page.navigate,Element.fill) at capture time so the downstream exporter stays adapter-agnostic.
ActionSnapshot.screenshot?: string
nitpick: Missing JSDoc — should note this is base64-encoded and which format to expect. The
trace-exporter.tsassumes base64; an adapter author passing a raw Buffer would get a silent corruption.
TraceLog.actionSnapshots?
suggestion: Snapshots are stored in a separate array joined to
commands[]by temporal proximity. The PoC attaches snapshots inline on eachCommandLog— no join needed. If this separate array stays, consider adding acallIdfield toActionSnapshotso it can be deterministically matched to the command that triggered it.
packages/core/src/trace-exporter.ts
buildContextOptions(): no mobile support
issue: Context-options always uses
browserNamefrom capabilities with a generic viewport. For mobile (Android/iOS), the MCP reference trace usesbrowserName: "chromium", platform-specific viewports (412×915 Android, 390×844 iOS), and a device-aware title ("android - emulator-5554 (17)"). The capabilities carryappium:platformName,appium:deviceName, andappium:platformVersion— suggest reading them and adjusting the context accordingly.
buildActionEvents(): params are positional
suggestion: Params use positional keys (
{"0": "value"}). The MCP reference trace uses named keys matching the tool vocabulary:{selector: "#id"},{value: "text"},{url: "https://..."}. TheformatActionTitlefunction already handles selector extraction — it just needs the right param keys. Named params make trace viewers and transcripts more readable.
buildActionEvents(): no selector in element commands
issue: WDIO element commands (
click,setValue) receive the value but not the selector in their args — the selector lives on the element from the prior$()call. Without tracking, trace events showElement.click()instead ofElement.click("#submit"). The PoC tracks the last$/findElementselector and injects it into subsequent element commands. Suggest adding a light last-selector tracker in the capturer (a few lines).
ScreencastFrameEvent: no snapshot field
suggestion: The event type has
elements?but nosnapshot?for the LLM-readable text rendering. MCP tools consume the snapshot for page-state reasoning. Consider addingsnapshot?: stringto both the event type and the builder, referencing a-snapshot.txtfile. The PoC attaches it assnapshot: "page@id-ts-snapshot.txt"on the screencast-frame event.
exportTraceZip(): event ordering
issue: All
screencast-frameevents are grouped before allbefore/afterevents. The MCP reference trace interleaves them:context-options → sf(timestamp=0) → before → after → sf → before → after → sf …. Trace viewers use the screencast-frame's position in the stream to select the current screenshot for each action. With all frames first, viewers show only the last screenshot for the entire session. Suggest interleaving per the Playwright/MCP convention.
exportTraceZip(): no initial screencast-frame
issue: The reference trace emits a
screencast-framewithtimestamp: 0representing the page state before the first action. Without it, trace viewers show a blank screen until the first post-action frame renders. Suggest emitting the first snapshot as this initial frame — a one-line addition.
Resource naming convention
thought: Resources use
{pageId}-{timestamp}.jpeg/elements-{pageId}-{timestamp}.json. The PoC uses suffix-based naming:{pageId}-{wallTimestamp}.png/{pageId}-{wallTimestamp}-elements.json/{pageId}-{wallTimestamp}-snapshot.txt. A parser discovers the three correlated files for a given timestamp by appending well-known suffixes rather than by prepending a regex-smashable prefix. Either convention works; worth converging on one.
Zip vs directory output
suggestion: Writing a
.zipis portable, but for agentic workflows (LLMs reading trace files directly) a plain directory is simpler — no unzip step, instant file access. The PoC writes atrace-{sessionId}/directory with NDJSON files and aresources/folder. Consider making zip vs directory a format option (traceFormat: 'dir' | 'zip') so agentic consumers can opt into direct filesystem access without the yazl dependency.
packages/service/src/action-snapshot.ts
Duplicated across three adapters
issue: This file is duplicated in
service/src/,nightwatch-devtools/src/, andselenium-devtools/src/with near-identical logic. The framework-agnostic parts (getElements,getBrowserAccessibilityTree,serialize*Snapshot) already live in@wdio/elements/@wdio/devtools-core. Suggest extracting the shared capture logic into core and keeping only the adapter-specific glue (browser.takeScreenshot(),browser.getUrl()) in each adapter.
Fire-and-forget capture pattern
issue: Snapshots are captured via fire-and-forget (
this.#snapshotCaptures.push(…)) withPromise.allSettled()at session end. This means the screenshot for command N can arrive AFTER command N+1 has already changed the page — the frame no longer represents the post-N state. The PoCawaits the snapshot insideafterCommand()for correctness. Consider either documenting this eventual-consistency trade-off, or offering anawaitmode for accuracy-critical scenarios.
Missing timeout guard
issue: No timeout on
getElements()orgetBrowserAccessibilityTree(). A hung page (or a slow Appium round-trip on mobile) could stall capture indefinitely. The PoC wraps these calls withPromise.race()+ 3–5s timeout. Suggest adding a timeout so a slow page doesn't accumulate unresolved promises.
packages/service/src/index.ts
mode: 'trace' coupling
thought:
mode === 'trace'gates three behaviors: screencast suppression, snapshot capture, and zip writing. The PoC uses independent flags (captureElements,disableDebugger,traceFormat) which allows, e.g., recording element snapshots while keeping the debugger for development. Ifmodestays as the single toggle, consider documenting the coupling explicitly in theServiceOptionsJSDoc.
Double disk write (JSON + zip)
issue: In
after(), both the existingwdio-trace-{id}.jsonAND the new.trace.zipare written unconditionally whenmode === 'trace'. This doubles disk I/O for trace-mode sessions. If both serve different consumers (dashboard needs JSON, MCP needs zip), that's fine — but consider documenting the dual-output rationale.
Selector not tracked
issue: Before-event params use raw positional args. For element commands like
click, WDIO passes the value/selector on the element chain, not in the command args — so params come through as{}. The PoC tracks the last$/findElementselector in the session capturer and injects it into subsequent element commands, producing trace events likeElement.click("submit")instead ofElement.click(). Suggest adding a light last-selector tracker in the capturer.
packages/service/src/launcher.ts
onPrepare() missing mode guard
issue: The launcher's
onPrepare()still launches the backend server and Chrome window regardless ofmode. Ifmode: 'trace'is meant to skip the debug UI, a guard is needed here. The PoC doesif (this.#options.disableDebugger) returnat the top ofonPrepare()— a one-line addition that prevents the backend server + Chrome window for trace-only runs.
packages/nightwatch-devtools/src/action-snapshot.ts / packages/selenium-devtools/src/action-snapshot.ts
Duplicated across adapters
suggestion: These files are near-identical to
service/src/action-snapshot.ts— only thebrowseraccess pattern differs. Since@wdio/elementsand@wdio/devtools-corealready provide the shared logic, these three files could collapse into one helper incorethat takes abrowser.takeScreenshot()factory. Each adapter would pass its flavor ofbrowser, keeping ~5 lines of glue instead of ~55 lines of duplicated capture logic.
packages/core/src/index.ts
Missing barrel exports
nitpick:
action-mapping.tsexports (mapCommandToAction,formatActionTitle,FILL_METHODS) should be re-exported from the package index. Currently consumers importmapCommandToActiondirectly from the package root, which works via the./*export map, but the barrel export makes it discoverable in the public API surface.
|
The PR imports directly from @wdio/elements, not from core: Why this matters: The core extraction in our branch separates the two — pure formatting lives in core (no WDIO dep), browser wrappers stay in Concrete issue: action-snapshot.ts in the nightwatch and selenium adapters imports from @wdio/elements, dragging a WDIO peer dependency into non-WDIO adapters. If those adapters don't declare webdriverio, their TypeScript compilation or runtime resolution may break. Our branch fixes this by having core own the serializers and locators. |
No description provided.