Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,10 @@ _Ask Scout to open a website, summarize it, save notes, and verify the file. Eve

Ask a Dot to show a draft before saving it. A CopilotKit human-in-the-loop card pauses the conversation for **Approve & save** or **Decline**. Approval creates the page in an authorized Space and returns a link; retries with the same draft recover that saved page. A changed draft needs a new review. The agent continues after your decision.

### Connections

Give a Dot tools from any MCP server, such as email, calendar, GitHub, or your own services. Read-only tools run on their own. Any other tool pauses for an **Approve & run** card in chat, and only your approval runs it. You can turn each tool on or off per Dot. See [Connections](docs/CONNECTIONS.md).

### Text and calls

A continuous conversation keeps the Dot's avatar and status above the messages, with text and call controls close at hand. Work updates, source links, and call receipts appear in the timeline; a side panel shows results or the agent's computer.
Expand Down Expand Up @@ -197,6 +201,7 @@ CopilotKit SDK telemetry collects usage metadata separately from conversation pe
| Background work | Scheduled server-side turns in their original conversation, with pause and retry controls |
| Browser | Separate read-only public-page service with page capture and navigation limits |
| Dot computers | Per-Dot browser profiles, files, shell, takeover, permissions, and action records through OpenBot |
| Connections | Per-Dot MCP servers, per-tool access, and owner approval for non-read-only actions |
| Memory | User-managed preferences that permitted Dots can use |
| Automatic Learning | Per-Dot Learning containers, conversation evidence routing, and published-skill delivery; see [setup](docs/SETUP.md#automatic-learning) |
| Deployment | Local Node setup and separate application/browser containers |
Expand Down
30 changes: 30 additions & 0 deletions docs/CONNECTIONS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Connections

Connections give a Dot tools from remote [MCP](https://modelcontextprotocol.io) servers: email, calendars, issue trackers, notes, or your own services. Each connection belongs to one Dot.

## Add a connection

1. Open a Dot's settings (**Edit specialist**).
2. Under **Connections**, enter a name, the server's Streamable HTTP endpoint (for example `https://example.com/mcp`), and an optional bearer token.
3. Select **Connect**. OpenDots lists the server's tools and saves them.

Use **Refresh** after the server adds or changes tools. Your choices for existing tools are kept.

## Approvals

Every tool starts enabled. A tool the server marks as read-only (`readOnlyHint`) runs on its own. Every other tool starts with **Ask first** on.

When a Dot calls an **Ask first** tool, the tool does not run. The server stores the exact connection, tool and arguments, and the Dot shows an approval card in chat with a summary and those stored arguments. The action runs only when you select **Approve & run**, through an owner-only server route. That route runs the stored request, never arguments sent with the approval, and only if the conversation's Dot still has that exact tool enabled. Requests expire after an hour. Each approval runs at most once. Reopening the conversation shows the saved result, or keeps checking while the action is still running. A saved result is only ever shown for the approval that produced it.

The read-only hint comes from the server, so it is only a hint. Turn on **Ask first** for any tool you do not fully trust, and turn off tools a Dot does not need.

Approval cards appear only in the web app. Through Slack or in scheduled runs, an **Ask first** tool tells the Dot to ask you to continue in the web app.

Changing a Dot's connections or tool settings stops that Dot's active turn.

## Security notes

- Tokens are stored in the server's SQLite database and are never sent to the browser. Protect `DATABASE_PATH` the way you protect `.env`.
- Tool results are passed to the model as untrusted data.
- Endpoints must use `http` or `https` and cannot contain credentials in the URL. Local addresses are allowed, so you can run MCP servers on the same machine. Only add servers you trust.
- OAuth-only servers are not supported yet. Use a server that accepts a bearer token, or put a token-authenticated proxy in front of it.
19 changes: 18 additions & 1 deletion src/client/Chat.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
import { PageReviewCard } from './PageReviewCard';
import { pageReviewSchema, pageReviewTool } from '../shared/page-review';
import {
connectionActionSchema,
connectionActionTool,
} from '../shared/connection-types';
import { ConnectionActionCard } from './ConnectionActionCard';
import { contextualMessage, type PageContext } from './page-context';
import { api } from './api';
import type { Page } from '../server/pages';
Expand Down Expand Up @@ -180,6 +185,17 @@ export function Chat({
},
[thread.id, onSaved],
);
useHumanInTheLoop(
{
name: connectionActionTool.name,
description: connectionActionTool.description,
parameters: connectionActionSchema,
render: (props) => (
<ConnectionActionCard {...props} threadId={thread.id} />
),
},
[thread.id],
);
const computerCalls = agent.messages.flatMap((message) =>
message.role === 'assistant' ? (message.toolCalls ?? []) : [],
);
Expand Down Expand Up @@ -221,7 +237,8 @@ export function Chat({
message.toolCalls?.some(
(call) =>
call.function.name.startsWith('computer_') ||
call.function.name === pageReviewTool.name,
call.function.name === pageReviewTool.name ||
call.function.name === connectionActionTool.name,
))),
);
return (
Expand Down
244 changes: 244 additions & 0 deletions src/client/ConnectionActionCard.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,244 @@
import { useEffect, useRef, useState } from 'react';
import { Check, PlugZap } from 'lucide-react';
import {
connectionActionSchema,
type ConnectionActionResult,
type PendingApproval,
} from '../shared/connection-types';
import { api } from './api';
import { computerToolResult } from './ComputerToolCard';
type Receipt = {
approvalId: string | null;
status: 'running' | 'done';
result: ConnectionActionResult | null;
};
const conversation = (threadId: string) =>
`/conversations/${encodeURIComponent(threadId)}`;
const display = (value: unknown) =>
typeof value === 'string' ? value : JSON.stringify(value, null, 2);
export function ConnectionActionCard({
args,
status,
result,
respond,
threadId,
toolCallId,
}: {
args: unknown;
status: string;
result?: unknown;
respond?: (result: unknown) => Promise<void>;
threadId: string;
toolCallId: string;
}) {
const action = connectionActionSchema.safeParse(args);
const recorded = computerToolResult(result);
const [approval, setApproval] = useState<PendingApproval>();
const [approvalError, setApprovalError] = useState('');
const [receipt, setReceipt] = useState<Receipt | null>();
const [error, setError] = useState('');
const [busy, setBusy] = useState(false);
const [attempt, setAttempt] = useState(0);
const pending = useRef(false);
const finished = status === 'complete';
const approvalId = action.success ? action.data.approvalId : '';
// Show the server's record of what will run, not the model's description.
useEffect(() => {
if (!approvalId) return;
let active = true;
setApprovalError('');
void api<PendingApproval>(
`${conversation(threadId)}/connection-approvals/${encodeURIComponent(approvalId)}`,
)
.then((value) => active && setApproval(value))
.catch(
(cause) =>
active &&
setApprovalError(
cause instanceof Error
? cause.message
: 'Could not load this request.',
),
);
return () => {
active = false;
};
}, [threadId, approvalId]);
// A saved result counts only if it came from this card's approval.
const mismatch = !!receipt && receipt.approvalId !== approvalId;
const own = mismatch ? null : receipt;
// A receipt that is still running is checked until the server finishes.
const running = own?.status === 'running';
useEffect(() => {
let active = true;
const load = () =>
api<Receipt | null>(
`${conversation(threadId)}/connection-actions/${encodeURIComponent(toolCallId)}`,
)
.then((value) => active && setReceipt(value))
.catch((cause) => {
if (active)
setError(
cause instanceof Error
? cause.message
: 'Could not check this action.',
);
});
setError('');
void load();
const timer = running ? setInterval(() => void load(), 2000) : undefined;
return () => {
active = false;
clearInterval(timer);
};
}, [threadId, toolCallId, attempt, running]);
const outcome = own?.result ?? null;
const approved = recorded.approved === true || own?.status === 'done';
const declined = recorded.approved === false;
const ready = receipt !== undefined;
const decide = async (approve: boolean) => {
if (!respond || !action.success || pending.current) return;
pending.current = true;
setBusy(true);
setError('');
try {
if (!approve && !outcome) {
await respond({
approved: false,
message:
'The owner declined this action. Do not perform it or try another way.',
});
return;
}
// A previous approval may have run even if its response never arrived.
const value =
outcome ??
(await api<ConnectionActionResult>(
`${conversation(threadId)}/connection-actions`,
'POST',
{ toolCallId, approvalId: action.data.approvalId },
));
setReceipt({ approvalId, status: 'done', result: value });
await respond({ approved: true, ...value });
} catch (cause) {
setError(
cause instanceof Error ? cause.message : 'Could not run this action.',
);
} finally {
pending.current = false;
setBusy(false);
}
};
const entries = approval ? Object.entries(approval.arguments) : [];
return (
<section
className="page-review-card connection-action-card"
aria-label="Approve connected-service action"
>
<header>
<PlugZap size={17} />
<strong>
{approval
? `${approval.connection} · ${approval.title}`
: 'Connected service'}
</strong>
<span>
{approved
? outcome?.isError
? 'Failed'
: 'Approved'
: declined
? 'Declined'
: running
? 'Running'
: finished
? 'Ended'
: !ready
? 'Checking'
: 'Needs your approval'}
</span>
</header>
<div className="page-review-body">
<h3>
{action.success ? action.data.summary : 'Preparing the action…'}
</h3>
{entries.length > 0 && (
<dl className="connection-action-args">
{entries.map(([key, value]) => (
<div key={key}>
<dt>{key}</dt>
<dd>{display(value)}</dd>
</div>
))}
</dl>
)}
{mismatch && (
<div className="connection-action-result failed">
<strong>Cannot run</strong>
<pre>
This card’s saved result belongs to a different approval request.
Nothing was run for this one.
</pre>
</div>
)}
{approvalError && !outcome && (
<div className="connection-action-result failed">
<strong>Cannot run</strong>
<pre>{approvalError}</pre>
</div>
)}
{outcome && (
<div
className={`connection-action-result ${outcome.isError ? 'failed' : ''}`}
>
<strong>
{outcome.isError ? 'Service error' : 'Service response'}
</strong>
<pre>{outcome.text}</pre>
</div>
)}
</div>
{error && <p role="alert">{error}</p>}
<footer>
{!ready && error && (
<button type="button" onClick={() => setAttempt((n) => n + 1)}>
Retry
</button>
)}
{!finished && respond && ready && !running && !mismatch && (
<>
<button
type="button"
className="review-primary"
disabled={busy || !action.success || (!outcome && !approval)}
onClick={() => void decide(true)}
>
<Check size={15} />
{busy
? 'Running…'
: outcome
? 'Continue conversation'
: 'Approve & run'}
</button>
{!outcome && (
<button
type="button"
disabled={busy}
onClick={() => void decide(false)}
>
Decline
</button>
)}
</>
)}
<small>
{running
? 'This action is still running on the server.'
: approved || declined || finished
? ''
: 'Nothing runs until you approve. These are the exact arguments.'}
</small>
</footer>
</section>
);
}
Loading
Loading