Skip to content

feat(expert): human-in-the-loop tool permissions (#421)#7639

Open
andypalmi wants to merge 12 commits into
feat/408-expert-plan-modefrom
feat/421-expert-tool-permissions
Open

feat(expert): human-in-the-loop tool permissions (#421)#7639
andypalmi wants to merge 12 commits into
feat/408-expert-plan-modefrom
feat/421-expert-tool-permissions

Conversation

@andypalmi

Copy link
Copy Markdown
Contributor

Human-in-the-loop tool permissions for the Expert

Implements per-tool human-in-the-loop permissions for the Expert's flow-building tools, in the immersive editor, as described in FlowFuse/product#421. The builder (and their team role) controls which flow-building actions the Expert may run, which need approval, and which are off limits, so it never makes a change they would not have allowed.

Stacked on feat/408-expert-plan-mode (base of this PR). Review only the two commits on top; the rest of the diff belongs to #408 / #407.

What it does

  • Inline approval card in chat when a tool's policy is Ask: friendly tool name, action type (Read / Write / Delete) and the concrete call parameters, with Allow / Always allow / Deny. The agent pauses on the round-trip with no session timeout, however long the user takes; the chat stop button cancels it (treated as denied).
  • Settings panel (in the Expert settings dialog, immersive only) with a per-action-type default (Always allow / Ask / Never) and a per-tool override for every flow-building tool.
  • Role inheritance, fail-closed: read-only team members cannot enable or trigger write/delete tools and see why; the agent also fails closed server-side.
  • Version gating: each tool carries a min/max nr-assistant version. Tools render as available, "update required", or deprecated against the instance's nr-assistant version. Versioned variants (e.g. Manage Groups v1/v2) collapse into one row and resolve to the in-range variant, nudging an update to the newest variant's min version when behind.

Architecture

  • The agent decides policy at the toolsNode seam (sibling of the plan-mode gate): role check first, then per-tool policy. allow runs, deny feeds the denial back to the model so it adapts and explains, ask publishes expert:tool-approval and awaits the browser's decision.
  • The tool catalog is served over an HTTP GET endpoint (friendly name + scope + version window only). Every chat response carries a hash of the tool definitions; the browser refetches the catalog only when the hash diverges, so it stays correct across rolling deploys where instances can be on different versions.
  • Catalog and pending approvals live in the existing product-assistant / product-expert Pinia stores.

UI

The settings panel follows existing FlowFuse patterns: FormHeading for section titles and ff-data-table for both the action-type defaults and the tool list, so each tool name lines up with its own permission control across the row border (no bespoke section/group styling, no non-standard scope headers).

Out of scope (follow-ups)

  • Admin-configurable team-wide default policy (DB migration + admin UI + server enforcement).
  • Platform (non-flow-building) MCP tools, once Steve's platform-tool work is merged into the agent. The UI has a TODO marking where they slot in.

Testing

  • Build + color/eslint lint green.
  • Manual: catalog populates on opening the immersive editor; Ask shows the card and pauses with no timeout (Allow applies to canvas, Deny explains gracefully); Always allow persists; Never never executes; read-only role sees write/delete disabled with a reason and cannot trigger them; leaving immersive hides the panel and stops sending permissions; chat stop while a card is open recovers cleanly.

Requires the matching agent-side change in the ai-assistants repo.

Refs FlowFuse/product#421

andypalmi added 2 commits June 30, 2026 15:23
Add per-tool approval for the Expert's flow-building tools in the immersive
editor. The agent gates each tool call at the toolsNode seam by class
(read/write/delete) and per-tool preference; write/delete default to Ask and
surface an inline approval card (Allow / Always allow / Never) that holds the
call open with no session timeout, while read defaults to allow.

- Catalog delivered over HTTP (GET /api/v1/expert/mcp/tools), curated to
  friendly names so raw tool identifiers never reach the browser; a per-response
  hash triggers a background refetch when the catalog drifts.
- HITL state consolidated into the product-assistant store (defaults,
  per-tool preferences, pending-approval map) with SemVer version gating.
- Settings panel groups versioned tool variants into one family and points
  update hints at the newest variant's required version.
- Role inheritance is fail-closed: read-only members cannot enable or trigger
  write/delete tools and are shown why.
Use FormHeading for the section titles and ff-data-table for both the
action-type defaults and the flow-building tool list, replacing the
bespoke section/group styling and the non-standard uppercase scope
headers. Bordered table rows pair each tool with its permission control
across the row rather than leaving them to float across whitespace; tool
scope moves into a Type column.

The approval card no longer sends or renders a tool summary; the tool
name, scope and call parameters describe the action.
@andypalmi andypalmi force-pushed the feat/421-expert-tool-permissions branch from 8a97b8e to bdb36fb Compare June 30, 2026 13:46
@codecov

codecov Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 18.18182% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.44%. Comparing base (b6ab7c3) to head (462da8d).

Files with missing lines Patch % Lines
forge/ee/routes/expert/index.js 18.18% 9 Missing ⚠️
Additional details and impacted files
@@                      Coverage Diff                      @@
##           feat/408-expert-plan-mode    #7639      +/-   ##
=============================================================
- Coverage                      76.47%   76.44%   -0.04%     
=============================================================
  Files                            413      413              
  Lines                          21775    21785      +10     
  Branches                        5736     5741       +5     
=============================================================
+ Hits                           16652    16653       +1     
- Misses                          5123     5132       +9     
Flag Coverage Δ
backend 76.44% <18.18%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…nd platform tools

Fetch the tool catalog when the Expert panel mounts (not only in the
editor) so the permissions settings render wherever the Expert is.

Split the settings into a Flow Building Tools section, with its own
per-action-type default permissions, and a separate FlowFuse Platform
Tools section (a placeholder until those tools ship, with TODOs marking
where they get mapped in). Flow-building tools are listed everywhere but
noted as usable only from an instance editor.
- Show plain Read / Write / Delete scope instead of phrases like Read only
- Stop the Setup Guide badge rendering above the approval card
- Disable the action buttons as soon as a choice is made
andypalmi added 2 commits June 30, 2026 16:35
…rmissions

# Conflicts:
#	frontend/src/components/expert/components/ExpertChatInput.vue
Raise the conversation-history expiry from 28 to 30 minutes (warning at
27), so the human-in-the-loop tool-approval wait, which is bounded by the
session lifetime, has the full 30-minute window the agent now allows.
Revert the 30-minute expiry back to 28 (warning at 25). The agent clears
old transactions/context at 30 minutes, so the chat must expire a moment
earlier to avoid referencing backend history that has already been purged.
The tool-approval wait is bounded by this 28-minute session lifetime.
Replace the flat key/value list on the approval card with a prettified
JSON view of the call payload. Adds a small single-value JsonViewer that
reuses the prettify + word-wrap + horizontal-scroll presentation of the
snapshot comparison diff panel, without its two-sided diff machinery. The
payload is prettified by default; an ff-button Wrap toggle appears for
long lines and reflects its on/off state rather than changing its label.
andypalmi added 3 commits July 1, 2026 00:31
Harden the JSON payload viewer against malformed input and collapse the
payload once a decision is made.

- JsonViewer stringify can no longer throw: circular refs, BigInt and any
  other non-serialisable value fall back to a circular-safe pass, then to a
  plain coercion, so a bad payload never breaks the approval card.
- Add a live collapse toggle to JsonViewer (collapsible + defaultCollapsed).
  The header caret expands/collapses at any time; the parent can seed the
  initial state.
- ToolApprovalCard collapses the payload once the call is allowed, always
  allowed or denied (local decision or round-tripped status), while leaving
  the toggle live so the user can re-expand it.
Drop the circular-safe/BigInt fallback machinery from the payload viewer.
Tool-call params are plain JSON; if they somehow can't be serialised, show
a simple 'Could not display the payload.' message rather than placeholder
markers.
- Replace the unicode caret on the JSON payload collapse toggle with the
  standard rotating ChevronRightIcon (matches ToolCallItem section headers).
- Add a 'bare' prop to MessageBubble that strips the bubble background and
  padding, and use it for tool-approval answers so the approval card renders
  as a standalone card instead of a card nested inside an AI bubble.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant