Universal PowerSync Diagnostics (POC) - #1094
khawarizmus wants to merge 30 commits into
Conversation
…lient implementations
…ge with components and playground
…d indexes; add SchemaTab to diagnostics UI
…line module imports and setup
…ding columns, indexes, and table options
…ream, UI components, and schema inspection features
…for a cleaner separation
…ion alongside currentOptions
…t and enhance connection handling in extension
…ostMessageTransport, enhance package metadata, and add Chrome types
…nents with icons and status bar
…play and integrate virtual scrolling for upload queue
…t detail component, and improved data handling features
…sizable columns, and theme synchronization
🦋 Changeset detectedLatest commit: 9c6336a The changes in this PR will be included in the next version bump. This PR includes changesets to release 13 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
simolus3
left a comment
There was a problem hiding this comment.
To share more context here (I've also shared that offline with more context @khawarizmus). I think the idea is awesome, my initial architectural complaints are:
- It's not clear whether we need a browser extension, so remove that from this PR which is already very big (you can open a follow-up PR for that separately).
- I think a JSON-based protocol over "any two way channel" is the wrong abstraction here. It e.g. forces us to re-invent an RPC protocol when some implementations we'll use (say the Dart VM service protocol) are already RPC based.
- None of this should require any changes to an existing SDK package (aside from forwarding core diagnostics events, but even that should likely be a separate PR to validate that the basic integration can work without SDK changes). We can likely inject an agent by writing a vite devtools package instead. Or initially, we could also expose it as a seperate package a user would depend on to expose the channel (e.g. a
enablePowerSyncDiagnostics(port: MessagePort, db: BasePowerSyncDatabase)). - Related to 2 and 3, the interface the diagnostics tool expects an SDK to provide should be an actual TypeScript interface with async methods and event listeners. It is the responsibility of each integration (vite,
postMessageiframe, Dart) to figure out how to implement that interface by doing the serialization itself.
Add dependency-free `shapes.ts` and `integration.ts` as the tool-owned protocol; move PROTOCOL.md into diagnostics-core and rewrite it to interface + shapes.
simolus3
left a comment
There was a problem hiding this comment.
I didn't take a look at the UI yet, but the architecture of how devtools are served and integrated looks pretty good at this point. I'll also try to scaffold a Dart integration over the vm service protocol.
| */ | ||
|
|
||
| /** The shape exposed over comlink: `observeEvents` takes a proxied callback instead of a function. */ | ||
| type Remote = Omit<SdkIntegration, 'observeEvents'> & { |
There was a problem hiding this comment.
Let's just make observeEvents return a promise on SdkIntegration, or a Promise<void> | void.
There was a problem hiding this comment.
This was addressed
| // The JavaScript integration (runs in the app page) and the structural database it reads. | ||
| export * from './live-database.js'; | ||
| export * from './agent.js'; | ||
| export * from './state.js'; | ||
|
|
||
| // Moving an integration across an iframe boundary, and deriving UI state from its events. | ||
| export * from './bridge.js'; | ||
| export * from './store.js'; |
There was a problem hiding this comment.
Relying on structural typing to avoid pulling in the JS SDK here is clever, but also feels somewhat fragile.
Given that these files are still only useful when providing diagnostics for JavaScript SDKs, maybe they shouldn't be part of the default export. I think putting them into a separate package is overkill, but maybe they should be exported from a separate entrypoint (say @powersync/diagnostics-core-js-web).
(and then if they're separate exports, I wouldn't mind this package having an optional dependency on @powersync/common and defining the JS agent on BasePowerSyncDatabase).
There was a problem hiding this comment.
The JS-specific code now lives in @powersync/diagnostics-core/js. I kept the structural type instead of adding an optional @powersync/common dependency. The Vite client can now pass WebPowerSyncDatabase directly, and only the Playground UI mock needs to implement that small interface.
|
|
||
| The tool never carries a wire format of its own. Request/response correlation, connection and reconnection are the implementation's concern. | ||
|
|
||
| ## 2. The interface |
There was a problem hiding this comment.
Having the overview here is helpful, but it could also be part of the readme. Sections 2-5 of this document should be tsdoc comments instead.
There was a problem hiding this comment.
This was addressed. The relevant parts are now in the README, and the rest has been ported to tsdocs comments
|
|
||
| /** | ||
| * Emitted by the core extension when diagnostics are enabled on the sync stream (see | ||
| * {@link https://github.com/powersync-ja/powersync-sqlite-core diagnostics}). Reports detailed |
There was a problem hiding this comment.
Should probably link to https://github.com/powersync-ja/powersync-sqlite-core/blob/main/crates/core/src/sync/diagnostics.rs directly?
…ors, support several app tabs
… 7 and build behaviour in the README
…the Nuxt changeset breaking
…rd, fast refusal; drop the stdio mcp command
This PR implements a live-attach diagnostics tool built once and reused across different hosts.
New packages
@powersync/diagnostics-ui— the shared UI (contains six tabs: Sync Status, Data, Buckets, Streams, Config, Logs).@powersync/diagnostics-core— the host-side client and thePostMessageTransport.tools/diagnostics-extension— a Chrome DevTools extension (using WXT).@powersync/commondiagnosticssubpath: the protocol, the agent, and the state mappers.Schema.serialize()andfromSerialized(), with tests.connectorandconnectionOptionsto the public database interface.diagnosticssync option.@powersync/webBroadcastChanneltransport and event source.enableDiagnostics(db).shared-internals
connectorgetter to persist for the connection lifetime.HandleDiagnostics) into the JS SDK.@powersync/nuxtNotes
Claude code was used to steer the discovery and development. The code was reviewed and modified manually.