Add the ability to apply source maps from the CLI - #6229
Conversation
Added a SourceMapRunner type and an optional `run` argument to doSourceMapSymbolication through applySourceMapFile, to make the source map symbolication runner injectable. The default behavior stays as the Web Worker runner, but let's the non-browser callers inject a runner that calls the core logic directly, since the profiler-cli Node daemon has no Workers.
`ApplySourceMapFileResult` only carried the resolved bundle source's filename, which is all the web UI needs. Add the source table index alongside it, so callers that identify sources by index rather than by name can report which source the map landed on. A follow-up uses this to add "sourcemap" commands to profiler-cli, which refers to sources by "src-N" handles.
Lists the bundle sources that carry a `sourceMapURL` and are therefore eligible to have a `.map` applied to them -- the same set the web app's "Apply source map…" picker offers. Each gets a `src-N` handle, which is a direct index into `profile.shared.sources` and so is stable across sessions for the same profile, like `f-N`. Firefox stores an inline map's entire `data:` URL in the source table, so a "URL" can be megabytes of base64. Those are reported by media type and size instead, which keeps the payload out of both the text and `--json` output. The integration fixtures are pre-generated and committed because the profile builders are DOM-coupled and can't run in the node-env test process, so sourcemap-generator.ts regenerates them via a browser-env test. A follow-up adds "sourcemap apply", which consumes these handles.
Brings the web app's "Apply source map…" feature to profiler-cli, reusing the shared `applySourceMapFile` thunk. Since the CLI is one-shot, the web picker becomes a two-step flow: "sourcemap sources" lists the candidates, then "sourcemap apply <path> [--to src-N]" reads the .map on the daemon, auto-matches it to a source (or applies to the source given by --to) and re-symbolicates in place. Ambiguous matches and errors exit non-zero so scripts can branch on them. The daemon has no Web Worker, so it injects a runner that calls the symbolication core directly on the current thread. That core's `source-map` dependency reads its WASM parser from `path.join(__dirname, 'mappings.wasm')` at runtime, and esbuild does not bundle that file, so the build copies it next to the bundle and the publish check now fails if it is missing -- without it, "sourcemap apply" silently applies nothing. `--to` only accepts sources that carry a `sourceMapURL`, the same set the web picker offers. Applying a map to any other source would skip the auto-match step's `no-eligible-sources` guard and then de-minify nothing, which reads as a successful apply.
Adds a SOURCE MAPS section covering the sources -> apply flow and the --to disambiguation step, and lists src-N in the handle reference tables alongside t-N / f-N / c-N / ts-N.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6229 +/- ##
==========================================
- Coverage 83.65% 83.52% -0.14%
==========================================
Files 348 350 +2
Lines 37339 37399 +60
Branches 10376 10394 +18
==========================================
+ Hits 31237 31238 +1
- Misses 5674 5733 +59
Partials 428 428 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
fatadel
left a comment
There was a problem hiding this comment.
Thanks for the work and sorry for the long review! I've left some comments but I think they are mostly minor ones.
| case 'sourcemap-ambiguous': { | ||
| const lines = result.candidates.map(formatSourceEntry); | ||
| return [ | ||
| 'The source map matches more than one source. Re-run with --to <src-N> to pick one:', |
There was a problem hiding this comment.
This is returned for both "matches several" and "matches nothing" leading to:
➜ profiler git:(sourcemap-cli) pq sourcemap apply ~/Downloads/mystery.js.map
The source map matches more than one source. Re-run with --to <src-N> to pick one:
when in reality it does not match any.
| (result.type === 'sourcemap-ambiguous' || | ||
| result.type === 'sourcemap-error') | ||
| ) { | ||
| process.exit(1); |
There was a problem hiding this comment.
console.log(formatOutput... then process.exit(1) can truncate stdout on a pipe, because when stdout is a pipe, Node's writes are async; process.exit drops anything still queued.
Fix: process.exitCode = 1; instead - the process still exits 1, but after the write drains.
| } | ||
| return { | ||
| kind: 'inline', | ||
| mediaType: sourceMapURL.slice('data:'.length).split(',', 1)[0], |
There was a problem hiding this comment.
Maybe worth checking for presence of comma before splitting on it? Otherwise, the whole (potentially) megabytes of base64 would be returned in case of malformed source map url.
| return { | ||
| kind: 'inline', | ||
| mediaType: sourceMapURL.slice('data:'.length).split(',', 1)[0], | ||
| byteLength: sourceMapURL.length, |
There was a problem hiding this comment.
Should this probably exclude the prefix length?
| if (!existsSync(wasmUrl)) { | ||
| console.error( | ||
| `profiler-cli source map parser not found at ${fileURLToPath(wasmUrl)}.\n` + | ||
| `Without it, 'sourcemap apply' silently applies nothing.\n` + |
There was a problem hiding this comment.
Without it, 'sourcemap apply' fails with a symbolication error.\n would be more correct, because in that case it actually fails.
|
|
||
| import { getThreadsKey } from 'firefox-profiler/profile-logic/profile-data'; | ||
| import type { Store } from '../types/store'; | ||
| import * as fs from 'fs'; |
There was a problem hiding this comment.
Shouldn't this be at the very top of the import block? 🤔
|
|
||
| const sourceIndex = Number(match[1]); | ||
| if ( | ||
| !Number.isInteger(sourceIndex) || |
There was a problem hiding this comment.
Having this regex, we always have digits. So, Number() can't produce a non-integer or a negative. Thus, we can simplify the condition:
const sourceIndex = Number(match[1]);
if (sourceIndex >= sourceCount) {
throw new Error(`Unknown source ${sourceHandle}`);
}
Main | Deploy preview
This fixes #6196.
This PR adds 2 commands to the CLI so we can apply source maps from local files later:
pq sourcemap sourcespq sourcemap applysourcemap sourcesis used for determining which sources we can apply the source maps onto. andsourcemap applyis to apply that source map that we provide as an argument.For example the output of
sourcemap sources:Then
pq sourcemap apply <sourcemap-path>could optionally take--to <src-N>argument.If it's not provided, we try to match the source map given to an existing source, if it matches, great, it's done. If it doesn't match, we show an error saying that the user has to provide the
--to <src-N>by looking at the output ofsourcemap sources. If that argument is provided, we automatically apply to that only.For example:
And then:
Example profile to test:
https://share.firefox.dev/4fIRaUh
And this is the source map for the bundle: index-XVVABR7J.js.map.zip
(note you have to unzip it before using, apparently github doesn't allow uploading .map file extensions, so I had to zip it)