Enhance widget support with live channels, state management, and documentation - #99
Merged
Conversation
When embedding ipywidgets, include the widget manager state that retains class defaults the page needs while stripping null defaults that were never assigned. Pass a reconciled state to ipywidgets.embed.embed_snippet, implemented via new _embedded_state and _without_unset_values helpers. Add Python unit tests and a C# integration test to cover the behavior, and run the real-ipywidgets widget tests in an isolated venv on CI (Python 3.13) so the integration can be validated without affecting the main test environment. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Introduce HostProtocol.IsExecutionScoped to distinguish messages owned by a running cell from session-scoped messages. Add EventReceived forwarding on PythonHostProcess and update PythonHostConnection docs to note replies abandoned before arrival. In PythonHostSession track whether an execution handler is attached, prevent duplicate delivery by routing only execution-scoped messages to the per-execution handler, and add session-scoped EventUnclaimed plus ReportUnclaimed/OnSessionEvent to log and surface messages that no handler claimed. Include tests (SessionEventDeliveryTests) verifying correct delivery and that unclaimed messages are reported. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Introduce two-way, session-scoped output channels to support live communication between a cell output and the view that renders it. Add IOutputChannel and IOutputChannelHost interfaces, and a concrete OutputChannelHost (with transport abstraction) implementing channel lifecycle, queuing, byte/message bounds, reserved/extension type handling, delivery, and diagnostics. Wire the capability through IVersoContext.OutputChannels (default null) and propagate the host: Scaffold now owns an OutputChannelHost (attachable via an IOutputChannelTransport), ExecutionPipeline passes the host into magic/extension contexts, and VersoContext and testing stubs expose the OutputChannels property. Update docs (context reference) and add unit tests covering the default null behavior and comprehensive OutputChannelHost behavior. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Add support for live output channels used by widget outputs. - Introduce OutputChannelProtocol with Version and MajorOf helpers. - Add LiveChannelId to CellOutput and a Widget(document, channelId) overload. - Propagate channel id into WidgetFrame (data-verso-live attribute) and pass it to JS interop on mount. - Extend widget-interop.js to inject channel handshake script, track frames by channel, route messages between views and the host, and expose window.versoOutputChannel for the host to register and post/close channels. - Wire Blazor page: register vers oOutputChannel, add JSInvokable handlers OnOutputChannelReady/OnOutputChannelMessage and dispose registration. - ServerNotebookService: expose Scaffold for tests, handle OutputChannelReady/OutputChannelMessage, decode base64 buffers, add BrowserOutputChannelTransport to post/close channel messages via JS, and attach/detach transport to scaffold lifecycle. - Update OutputChannelHost.HandleReadyAsync to report protocol-major mismatches using OutputChannelProtocol but mount the view anyway. - Add InternalsVisibleTo for shared tests in Verso.Blazor.csproj. - Add unit tests covering routing, widget frame mounting/attributes, and version/host behavior adjustments. These changes enable two-way, routed communication for live widget outputs while preserving backward compatibility for saved/static widgets. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Introduce support for live output channels so views and channel owners can exchange messages. Adds new protocol method names (channel/ready, channel/message, channel/post, channel/closed), DTOs for channel params, an out-of-process OutputChannelHandler, and a SessionOutputChannelTransport that forwards owner messages as session notifications. Wire transport into HostSession and expose channel post/closed handling in RemoteNotebookService (JS interop calls versoOutputChannel.post/closed). Include LiveChannelId on cell outputs and update notebook mapping. Also update the Blazor bridge to subscribe to channel notifications and add unit tests covering the transport, handler, and bridge behavior. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Implement end-to-end widget comm handling between subprocess and views. Add HostProtocol constants for comm types/fields and a Comm capability flag; embed a new Python module (_versohost_comm.py) that implements comm integration for the interpreter. Introduce PythonCommRouter to route comm frames to output channels (ordered delivery, broadcasting to holders, channel-targeted replies) and wire it into PythonHostSession (TrackCommChannel, route incoming comm frames, send-to-subprocess helper). Update versohost.py to install and dispatch comm messages on the main thread and include the new script in the host extractor and project resources. Add unit and integration tests (CommRoutingTests, CommIntegrationTests) and a Python test (tests/versohost/test_comm.py); update CI to run the comm test alongside display tests. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Add support for live widget channels and configurable widget asset source. Introduces new host protocol fields (live_widgets, widget_asset_source, msg_id, comm_ack) and routes widget "live" MIME through the host so the session can open an output channel for interactive views. Injects a manager script into widget documents and improves iframe sizing (initial height and an unsized grace fallback) so widgets render stably while measuring. Implements comm acknowledgements in the Python comm layer so the front-end's single-message throttle is released, and propagates msg_id through the router. Add WidgetAssetSource enum, kernel/env parsing, and plumbing to configure CDN vs bundled assets. Update tests to exercise live channel workflows and adjust comm routing logic and bookkeeping. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Add support for capturing a widget's current page when saving and improve handling of live outputs. Key changes: - Introduce SnapshotProvider on IOutputChannel and RequestSnapshotAsync in OutputChannelHost so callers can ask a channel's owner for a current document. - Wire snapshot requests through the host protocol (widget_snapshot / widget_snapshot_reply) and carry widget_id with live widget payloads. - Extend Python host/display code to emit widget_id, build/lookup widget documents, and answer snapshot requests on the interpreter thread. - Close output channels when cells are re-run, cleared, deleted, or kernels restart; add CloseForCell and CloseChannelsForKernelAsync to silently teardown channels where views go away. - Before serializing/saving a notebook, RefreshLiveOutputsAsync asks live outputs (with a bounded deadline) for fresh content so the saved file reflects what readers see. - Frontend: show an inert notice and visual treatment when a widget is not live (CSS, JS interop, markup), and add localized strings for the notice. - Add and update tests covering snapshot requests, close behavior, and view re-attachment semantics. These changes ensure saves can record the state a reader is looking at, prevent stale live controls after kernel restarts, and keep channel lifecycles consistent across host and client. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Introduce cross-kernel projection for widget traits so a widget trait can be projected into the notebook's shared variable store. Changes include: - HostProtocol: new message and field types (bind, unbind, bind_reply, bind_set, bind_update, related fields and bind capability). - PythonHostSession: projection lifecycle, storing projections, applying updates, sending/receiving projection messages, fingerprinting to avoid echoes, projection timeout handling, and store subscription management. - New VariableProjection and ProjectionOutcome records for exposing projection state. - New Python-side support module _versohost_bind.py to implement binding, unbinding, change publication and applying remote writes; integrated into versohost.py. - Expose reduce_one in _versohost_vars to reduce single values for projection. - New BindMagicCommand providing #!bind command (list/remove/bind) and related parsing/validation logic. - Kernel helper GetBoundSessionAsync to obtain a session bound to the notebook for use by the magic command. - Tests added for bind behavior and CI updated to run the new tests. - Localized resource strings updated for the new magic command messages. This adds end-to-end support for projecting a widget trait under a shared name, keeping the shared store and widget trait in sync across kernels while avoiding infinite echo and limiting size of transmitted values. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Add a new Interactive Widgets guide (docs/guides/interactive-widgets.md) describing live vs static widgets, saved state, network/loading behavior, anywidget support, binding (#!bind) and size/transport limits. Update top-level README, language-kernels guide, migration notes, Python README, and VS Code README to reference the new guide, clarify that ipywidgets and anywidget widgets can be live (controls reach the interpreter), and document widget binding/sharing between kernels. These changes clarify widget behavior, cross-language variable sharing, and export/import implications for readers and users. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Introduce the DAG Notebook showcase: adds a new samples/showcase/dag-notebook package with demo .verso notebooks and README, and implements the layout extension (DagNotebookLayout.cs) plus supporting files (DagNotebookStyles.cs, DependencyAnalyzer.cs, csproj). The layout provides dependency-aware, reactive notebook behavior (badges, stale marking, auto-run cascade, widget-bound controls) and emits CSS/JS assets for the client. Also includes minor updates to host integration points (LayoutHandler.cs, HostSession.cs) and the Python host display helper to support the extension's runtime behavior. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Use an explicit identity (LiveChannelId when present, otherwise the output) for ErrorBoundary and WidgetFrame keys instead of the output record. This prevents tearing down views for live outputs when their serialized content is refreshed (value-equality of the record would force rebuilds), while still treating re-runs (new channel IDs) as new outputs. Updated comments explain the rationale and retained retry/version behavior for forcing re-attempts. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Enable capturing of stdout/stderr and display payloads into ipywidgets.Output areas when running without a kernel. Adds a per-thread capture stack, pending-clear tracking, and helpers (capture_stream, capture_display, capture_clear) plus _record/_append plumbing in _versohost_display. install_output_capture overrides Output.__enter__/__exit__/clear_output to redirect writes and exceptions into the active Output, and is installed on first comm open in _versohost_comm.__init__. StreamWriter.write is updated to offer stream text to the current capture before routing to the host. Defensive error handling and weakrefs are used so widgets still work if capture installation fails. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Assign module.clear_output to the host clear_output implementation so ipywidgets and other helpers clear output areas correctly instead of emitting terminal escape sequences in notebooks. The assignment is wrapped in a try/except to avoid breaking environments where the binding may not be possible. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Add missing "widget" glossary entries (German, Spanish, Japanese) and add localized configuration strings for widgets.assetSource (bundled, cdn, description) in package.nls.* for de, es, ja, qps-ploc and zh-cn. Also adjust pseudo-localized UI text in UI.qps-Ploc.resx and reorder the Magic_Bind_Removed entry in Strings.qps-Ploc.resx to match surrounding entries. These changes surface the new widget asset-source settings in translations and fix/update qps-ploc resource ordering/strings. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
SummarySummary
CoverageVerso - 79.8%
Verso.Abstractions - 83.9%
Verso.Ado - 81.4%
Verso.Blazor - 26.8%
Verso.Blazor.Shared - 61.1%
Verso.Cli - 41.3%
Verso.FSharp - 82.4%
Verso.Host - 56.2%
Verso.Http - 74.7%
Verso.JavaScript - 52.7%
Verso.PowerShell - 64.7%
Verso.Python - 76.9%
Verso.Sample.Diagram - 80.1%
Verso.Sample.Dice - 82.2%
Verso.Sample.Slides - 58.8%
Verso.Sample.Sparkline - 67.3%
Verso.Testing - 80.1%
|
Ensure replies are addressed only when they come from the same comm by tracking comm_id per-thread and adding _answering_channel logic; update dispatch to set/clear comm_id. Improve embed state extraction to use ipywidgets.dependency_state for a view-local closure (with a fallback to session-wide get_manager_state), and fix embed_snippet call to pass the widget instance. Prevent double-counting replayed messages in OutputChannelHost by tracking _replayHeld, change overflow handling to mark closed under lock and report outside it, and make FlushAsync replay semantics safer. Make Scaffold refreshes robust by reading outputs defensively (ReadOutputs) and matching snapshots by channel id; start snapshot deadline lazily. Adjust BindMagicCommand parsing to handle a trailing keyword with no name and refine separator detection. Add/adjust tests for comm behavior, embedding state, and BindMagic parsing. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Fix widget height reporting by including children bottoms so margin-collapsed content and output areas are measured correctly. Introduce a pending queue to hold kernel-initiated comm messages until the state snapshot is applied, and add drainPending to replay or discard held messages after snapshot. Add hasModel helper to avoid exceptions from get_model for unknown ids, use it when checking for missing models and before drawing slots, and adjust the comm message handler to enqueue messages when appropriate. These changes make widget loading more robust against out-of-order messages and model lookup errors. Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
Delete samples/showcase/dag-notebook/dag-notebook-test.verso, removing the DAG Notebook example notebook (verso format). The deleted file included metadata, markdown and C# example cells, and layout configuration (com.verso.showcase.dag-notebook:dag-notebook with autoRun). Signed-off-by: Torrey Betts <torrey.betts@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds comprehensive support and documentation for interactive widgets in Python cells, including both
ipywidgetsandanywidget. It clarifies how widgets behave in live and static states, introduces a new user guide, updates the documentation to cover new features, and improves test coverage for widgets. The most important changes are:Documentation: Interactive Widgets
docs/guides/interactive-widgets.md, covering how widgets work, their live/static states, file exports, network requirements, sharing widget values with other kernels via#!bind, and limitations.README.md,docs/guides/language-kernels.md, anddocs/migration/from-jupyter.mdto clarify that widgets built onipywidgetsandanywidgetare supported, are live when possible, and their values can be shared with other languages. References to the new guide were added. [1] [2] [3] [4] [5]Testing: Widget Integration
.github/workflows/verso-ci.ymlto run widget tests against a realipywidgetsinstallation in a dedicated environment, ensuring both real and stand-in widget behaviors are covered.API/Reference Updates
OutputChannelsproperty in the extension context reference (docs/extensions/context-reference.md), describing two-way message channels for output and view communication.