Skip to content

feat(handoff): create a standalone report from a chat session - #245

Draft
cpsievert wants to merge 43 commits into
mainfrom
feat/artifact-feature
Draft

feat(handoff): create a standalone report from a chat session #245
cpsievert wants to merge 43 commits into
mainfrom
feat/artifact-feature

Conversation

@cpsievert

@cpsievert cpsievert commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

What this adds

querychat is great for exploring data conversationally — but today that exploration is trapped in the chat conversation. This PR lets a user turn the work they did during a querychat session into a standalone, runnable, downloadable handoff: a Quarto dashboard, a Shiny app, a Marimo or Jupyter notebook, or a freeform format they describe in words.

In short: chat your way to the right queries and visualizations, then use the /handoff slash command. You'll be prompted with a dialog to choose relevant findings, an output format (e.g., Quarto, Jupyter, etc), a language (e.g., R and Python), and addition instructions for generating the "handoff" document. This standalone document, a README, as well as the data (if it's small enough), will all be bundled into downloaded zip file.

Why it's useful

  • Bridges exploration → deliverable. Insights found by asking questions become a reproducible asset.
  • Meets people where they work. Pick the output format (dashboard / app / notebook) and language (R or Python) that fits the team.
  • Self-contained. Small datasets are bundled as CSV so the handoff runs as-is; larger / DB-backed sources get a clearly-marked data-setup TODO.
  • Iterative. Revise with AI and download a zip (source + README + data).

See it in action

Upon requesting a new handoff:

01-modal

After generating the handoff:

02-panel

How it works (user's-eye view)

  1. Open the Prepare Handoff modal — type /handoff, or the assistant offers it when it senses you're ready.
  2. The modal shows a gallery of the session's queries & charts, an output-format picker, a language picker, and a directions box. An LLM recommendation pre-selects sensible defaults while it opens.
  3. Hit Generate — a side panel slides open and the handoff source streams into a read-only editor; a pill is added to the chat linking back to it.
  4. From the panel: revise with AI and download the zip.
  5. State survives bookmarking, so a restored session re-shows its handoffs.

Approach & architecture (for reviewers)

Built around a few deliberate constraints:

  • A hard reactivity boundary. HandoffOrchestrator is plain async business logic — reads no reactive.Value, defines no effects, never touches input.*. All reactivity lives in handoff_server(). That's what lets every flow be unit-tested with plain fakes.
  • One owner per concern.
    • HandoffView — the only place server→client output happens (custom messages, modal, chat pill).
    • HandoffChat — the only place chatlas is touched; deep-copies the live chat before each call, so generate/revise/recommend never pollute the visible conversation.
    • HandoffStore — the only place handoffs are held (LRU) and serialized.
    • active_handoff_id — a single reactive value, the sole source of truth for panel visibility.
  • State is pydantic; bookmarks stay small. HandoffState holds the current source plus cumulative turns. Immutable CSV snapshots live in a bounded in-session bundle store, not bookmark payloads; if a restored handoff's snapshot is unavailable, its source remains viewable but download is disabled.
  • Tool names are a shared contract centralized in _tool_names.py, so tool registration and the gallery that mines chat turns can't silently drift.
  • Two entry paths, one modal-opening path. The /handoff command and the querychat_request_handoff tool converge on the same flow (the tool fires mid-stream where no reactive context exists, so it relays through a status-gated effect).
Browser (handoff-core.ts)
   ▲  querychat-handoff-* messages         │ inputs / setInputValue
   │                                        ▼
handoff_server()   ← all reactivity, active_handoff_id
   │ drives
   ▼
HandoffOrchestrator   ← no reactive state
   ├── HandoffChat   (chatlas fork + streaming)
   ├── HandoffView   (server→client, modal, pill)
   ├── HandoffStore  (LRU + bookmark)
   └── pure helpers  (data · prompt · gallery · readme)

Limitations & things to know

  • Depends on unreleased shinychat. Relies on shinychat slash commands and a recent shiny; pyproject.toml currently pins shinychat to the dev/querychat-pr311-history-save git branch. This must move to a released version before any release.
  • The Py↔JS wire contract is hand-duplicated on both sides with no shared schema; a rename in one language silently no-ops in the other. Known maintenance hazard.
  • Generated front-end assets are committed. static/js/handoff.js / static/css/handoff.css are build outputs of js/src/* — edit the source and rebuild, never the generated files.
  • Stored handoffs are LRU-capped (25/session); reopening an evicted handoff's pill no-ops.
  • Data bundling threshold: DataFrame sources ≤ 5 MB are bundled as CSV; larger or DB-backed sources get a data-setup TODO.

Testing

  • Unit tests per module (tests/test_handoff_*.py); the orchestrator suite runs on plain fakes thanks to the no-reactive-state rule.
  • Playwright integration tests (test_13_handoff.py, test_15_handoff_module_scope.py) cover the modal, generation, panel, pill navigation, browser-history restore, and module scoping.
  • Full Python unit suite passes locally; handoff e2e (modal → generate → panel → pill) verified against a live LLM.

Comment thread pyproject.toml Outdated
@cpsievert
cpsievert force-pushed the feat/artifact-feature branch 2 times, most recently from 313666b to 9e64a59 Compare June 15, 2026 16:06
@cpsievert
cpsievert requested a review from Copilot June 15, 2026 16:39

This comment was marked as resolved.

@cpsievert
cpsievert force-pushed the feat/artifact-feature branch from 13266ba to 5284bda Compare June 15, 2026 23:06
@cpsievert cpsievert changed the title feat(artifact): turn a querychat session into a standalone, runnable artifact (Python) feat(handoff): turn a querychat session into a standalone, runnable handoff (Python) Aug 20, 2026
Brings the /handoff feature to the R package at parity with Python:
users can turn completed query and visualization results into a
downloadable Quarto, Marimo, Shiny, Jupyter, or custom-format project,
with AI-assisted revisions and restorable chat history.

- Internal S7 value types, LRU handoff/bundle stores, and validated
  server-to-browser message contracts (handoff_types/store/protocol.R)
- Isolated ellmer chat forking with structured JSON streaming that
  never pollutes live chat history (handoff_chat.R)
- Non-reactive orchestration for recommend/generate/revise/restore/
  download, including atomic rollback and oversized-data correction
  that externalizes dataframes exceeding the bundle budget
  (handoff_orchestrator.R)
- Shiny wiring: slash command, panel/modal UI, downloads, and
  Shiny-bookmark + shinychat-history persistence hooks
  (handoff_server.R, handoff_ui.R, handoff_view.R)
- Data catalog and CSV snapshotting that only ever reads live data
  sources for tables actually referenced by the generated handoff
  (handoff_data.R, handoff_download.R)
- Shared canonical TypeScript/CSS/icons now build both the R and
  Python installed assets from one source
- Deterministic shinytest2 browser coverage exercising two isolated
  QueryChat modules against a scripted ellmer double, plus the
  underlying unit suite for every new module

Two issues surfaced only by the browser-level tests and fixed here:
- `parse_handoff_generate_request()` rejected `selected_ids` when
  Shiny deserializes a single-element browser JSON array as a bare
  list rather than an atomic vector
- `querychat_module.R`'s bookmark/history snapshot read reactive
  values without `isolate()`, which throws when invoked from a
  promise continuation with no active reactive context

Known limitation: the currently pinned
`shinychat@dev/querychat-pr311-history-save` branch no longer exposes
a working `chat_module$history$save()` (confirmed at runtime), so a
handoff commit's history-save step raises a non-blocking notification
even though the handoff itself commits correctly. Chat-history-based
restore-after-reload could not be verified against this branch as a
result. This should be treated as a release blocker until the pinned
branch is fixed or replaced with a released version.
@cpsievert cpsievert changed the title feat(handoff): turn a querychat session into a standalone, runnable handoff (Python) feat(handoff): create a standalone report from a chat session Aug 21, 2026
Ports the /handoff feature (natural-language handoff to Quarto,
Marimo, Shiny, Jupyter, or custom-format projects) to the R package
at parity with the existing Python implementation.
LLM-facing parsing (release blockers):
- parse_handoff_result()/parse_handoff_recommendation() now flatten the
  real payload shapes ellmer produces: jsonlite::parse_json() plain lists
  (ContentJson@parsed streaming path) and convert_from_type() factors for
  type_array(type_enum(...)) fields, via payload_character_vector().
- optional_payload_value() treats NULL as absent, matching ellmer's
  materialization of omitted optional fields (directions, summary, etc.).
- history$save() is guarded by is.function() and wrapped in tryCatch with
  a transient warning notification, so a missing/failing history save can
  never reject the handoff task. The pinned shinychat branch exposes no
  save() method; history persistence on that branch remains a release
  blocker to resolve upstream.

Post-commit error handling:
- generate()/revise() no longer re-throw once committed: post-commit
  cleanup or view failures warn instead of misreporting a saved handoff
  as failed, and revise() no longer reverts the view to the pre-revision
  state after a committed replacement.
- generate() appends the chat pill only after the store commit succeeds,
  so a commit failure cannot leave an orphaned, dead pill.

Parity and robustness:
- validate_handoff_source() restores notebook-JSON structure and
  kernelspec-language validation for notebook-json targets.
- revise() quietly no-ops on NULL/blank instructions, matching Python.
- HandoffBundleStore$evict() guards against an empty order with a stale
  byte total; discard() computes its byte refund before mutating.
- Unexported ellmer APIs (ContentJson, turn_contents_expand) are isolated
  behind wrappers in handoff_ellmer_compat.R.

Tests exercise the real ellmer/shinychat shapes (factor-valued
selected_ids, parse_json list output, save-less history object) rather
than hand-built fixtures.
@cpsievert
cpsievert force-pushed the feat/artifact-feature branch from 5284bda to 52d1d9e Compare August 25, 2026 14:51
Following #276, querychat_query results carry the result data frame as a
JSON string in ContentToolResult@value rather than a raw data frame.
Parse the JSON back to a data frame for gallery previews (tolerating raw
data frames from older sessions), and update test fixtures to use the
new value shape.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants