Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/features/tui/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,14 @@ For large or frequently-reused documents, or for getting content to an agent ove

Attached files are also recorded on the session so sub-agents spawned by task transfer can read them. To review what is attached, open `/context`: the dialog lists every attached file (and resolved prompt file) with a per-file token estimate and, when a compaction has occurred, displays the verbatim text of the most recent compaction summary. Use <kbd>↑</kbd>/<kbd>↓</kbd> to select an attached file and press <kbd>d</kbd> (or <kbd>x</kbd>/<kbd>Del</kbd>) to drop it, or run `/drop <path>` directly — press <kbd>Tab</kbd> after `/drop` and a space to complete the path from the currently attached files. Dropping stops sharing the file with sub-agents and skills; content already inlined in earlier messages stays in the conversation until compaction, and the file can always be re-attached with `@` or `/attach`.

### Generated Media

Some models (e.g. Gemini image-output models) can generate binary media — typically an image — as part of their reply. When that happens, docker-agent writes the generated bytes into the session's workspace (the directory the session was started in) as an ordinary, visible file, and the assistant message keeps only a relative reference to that file plus its MIME type, display name, and size — never the raw bytes.

This keeps session JSON/database rows lightweight regardless of how many images a conversation accumulates, and the generated file is a regular workspace deliverable — visible to every tool, and yours to edit, commit, move, or delete — the same way generated code or text lands there.

Generated media is **not** automatically resent to the model on later turns: only the surrounding text is replayed in the outgoing history, the same way a large tool result would be summarized rather than repeated. This avoids silently ballooning the context window with image bytes on every follow-up message. A future step will add TUI rendering for these files (e.g. displaying the generated image inline); today this slice covers the domain, persistence, and safety mechanics only.

### Team Context Budgets and Targeted Compaction

The `/context` dialog also shows a **Live sessions** section: the current session plus every currently running sub-agent session (foreground children spawned by task transfer and long-running `run_background_agent` tasks). Each row shows the agent name, a short session ID (so two concurrent runs of the same agent stay distinguishable), and that session's context budget: used tokens, context limit, and percentage, or an explicit "limit unknown" reading when the model's window cannot be resolved. Live-sessions rows do not repeat the compaction-cap wording themselves — the dialog's header line is the sole authority on which model, if any, caps the effective limit.
Expand Down
1 change: 1 addition & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,7 @@ remote MCP endpoints.
| [`rule_based_routing.yaml`](rule_based_routing.yaml) | Cheap router model dispatches the user message to fast or capable models. |
| [`structured-output.yaml`](structured-output.yaml) | Forces the model to return JSON matching a schema. |
| [`google_search_grounding.yaml`](google_search_grounding.yaml) | Enables Google Search grounding on Gemini models. |
| [`gemini_image_output.yaml`](gemini_image_output.yaml) | Gemini image-output model (generated images are saved into the workspace, not inlined as base64). |
| [`sampling-opts.yaml`](sampling-opts.yaml) | Provider-specific sampling parameters (`top_k`, `repetition_penalty`, …). |
| [`thinking_budget.yaml`](thinking_budget.yaml) | Reasoning/thinking budgets across OpenAI, Anthropic and Google. |
| [`task_budget.yaml`](task_budget.yaml) | Anthropic `task_budget`: cap total tokens spent across a multi-step agentic task. |
Expand Down
49 changes: 49 additions & 0 deletions pkg/chat/document.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,19 @@ package chat
// deprecated but remain supported for backward compatibility.
const MessagePartTypeDocument MessagePartType = "document"

// ArtifactRootKind identifies which root a DocumentSource.ArtifactPath is
// relative to.
type ArtifactRootKind string

// ArtifactRootWorkspace means ArtifactPath is relative to the OWNING
// session's workspace root (the session's effective WorkingDir, resolved
// via session.ResolveWorkingDir) — generated media lands in the user's
// workspace as an ordinary visible file, written by pkg/workspacemedia.
//
// An empty ArtifactRoot marks a reference whose root is unknown; such a
// reference is never resolved and surfaces as unavailable.
const ArtifactRootWorkspace ArtifactRootKind = "workspace"

// DocumentSource holds the actual content of a document. Exactly one of the
// fields should be set.
type DocumentSource struct {
Expand All @@ -18,6 +31,42 @@ type DocumentSource struct {
// InlineData holds binary content (images, PDFs, Office docs, …) that is
// base64-encoded when sent to the provider. Used for StrategyB64 attachments.
InlineData []byte `json:"inline_data,omitempty"`

// ArtifactPath references binary content that was generated by a model
// (not user-attached) and materialized to disk instead of being kept
// inline, so session JSON never carries generated bytes. It is
// interpreted against the root selected by ArtifactRoot:
//
// - ArtifactRootWorkspace: relative, slash-separated — never absolute,
// never containing ".." — resolved against the owning session's
// workspace root, exactly as returned by workspacemedia.Write. The
// path alone is never trusted to read a workspace file back —
// resolution must also verify the (owner session, path) pair against
// the generated-media manifest (session.GeneratedMediaManifest),
// which only materialization writes.
// - empty: the root is unknown — never resolved; the part surfaces
// as unavailable.
ArtifactPath string `json:"artifact_path,omitempty"`

// ArtifactRoot is the root kind ArtifactPath is relative to. See
// ArtifactRootWorkspace; empty means the root is unknown and the
// reference is unresolvable.
ArtifactRoot ArtifactRootKind `json:"artifact_root,omitempty"`

// ArtifactOwnerSessionID is the ID of the session the artifact was
// materialized under — always the session active at generation time,
// which never changes even after the message is copied into a branched
// or forked session. Resolving ArtifactPath under the CURRENT session
// instead of this owner is exactly the bug this field exists to prevent:
// branching/forking clones message structs (see pkg/session/branch.go)
// but never copies the materialized files themselves, so a lookup keyed
// on the current session ID silently misses once the message is viewed
// from anywhere but the original session.
//
// Empty on any non-media document part. Resolution treats an ownerless
// media reference as unavailable rather than guessing a session to look
// under.
ArtifactOwnerSessionID string `json:"artifact_owner_session_id,omitempty"`
}

// Document represents a file attachment in a message part. It carries
Expand Down
15 changes: 15 additions & 0 deletions pkg/compaction/compaction.go
Original file line number Diff line number Diff line change
Expand Up @@ -246,13 +246,28 @@ func promptAndTotalTokens(msg *chat.Message) (prompt, total int64) {
// text), reasoning content and tool-call payloads, plus a flat charge
// per binary attachment and a small per-message overhead for
// role/metadata tokens.
//
// Runtime-generated assistant messages deliberately mirror Content into a
// MultiContent text part with the exact same string (see
// pkg/runtime.recordAssistantMessage and stripGeneratedMediaTransform's
// doc comments) so that providers treating a non-empty MultiContent as
// authoritative (e.g. pkg/model/provider/oaistream) don't silently lose
// the text. Counting both would double the estimate for every such
// message, so the first MultiContent text part that exactly matches
// Content is skipped — it is the same content already counted above, not
// additional text.
func heuristicMessageTokens(msg *chat.Message) int64 {
var chars int
chars += len(msg.Content)
chars += len(msg.ReasoningContent)

var attachments int64
skippedContentMirror := msg.Content == ""
for _, part := range msg.MultiContent {
if !skippedContentMirror && part.Type == chat.MessagePartTypeText && part.Text == msg.Content {
skippedContentMirror = true
continue
}
chars += len(part.Text)
if part.Document != nil {
chars += len(part.Document.Source.InlineText)
Expand Down
43 changes: 43 additions & 0 deletions pkg/compaction/compaction_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,49 @@ func TestEstimateMessageTokens(t *testing.T) {
// 21 total chars → 21/3.5 = 6 + 5 overhead = 11
expected: 11,
},
{
// Regression test for the runtime-generated assistant shape
// (pkg/runtime.recordAssistantMessage /
// stripGeneratedMediaTransform) that mirrors Content into a
// MultiContent text part verbatim, so oaistream-style converters
// treating MultiContent as authoritative don't lose the text.
// The mirrored part must be counted once, not twice.
name: "content mirrored into a multi-content text part is not double-counted",
msg: chat.Message{
Role: chat.MessageRoleAssistant,
Content: "here you go", // 11 chars
MultiContent: []chat.MessagePart{
{Type: chat.MessagePartTypeText, Text: "here you go"}, // mirror of Content, must be skipped
{Type: chat.MessagePartTypeDocument, Document: &chat.Document{
Name: "cat.png", MimeType: "image/png",
Source: chat.DocumentSource{ArtifactPath: "cat.png"},
}},
},
},
// 11 chars (Content, counted once) → 11/3.5 = 3 + 5 overhead = 8.
// ArtifactPath-referenced generated media carries no InlineData, so
// it draws no binary-attachment charge here; what this case checks
// is that the mirrored text part does NOT add another 3 on top
// (which would make it 11).
expected: 8,
},
{
// A MultiContent text part that happens to repeat Content's exact
// text but is NOT the mirror (it comes after another part with
// the same text already skipped) must still be counted: only the
// first match is treated as the mirror, so genuinely repeated
// user-authored text is never silently dropped from the estimate.
name: "only the first multi-content match of Content is treated as the mirror",
msg: chat.Message{
Content: "same", // 4 chars
MultiContent: []chat.MessagePart{
{Type: chat.MessagePartTypeText, Text: "same"}, // skipped as the mirror
{Type: chat.MessagePartTypeText, Text: "same"}, // counted: 4 chars
},
},
// 4 (Content) + 4 (second "same") = 8 chars → 8/3.5 = 2 + 5 overhead = 7
expected: 7,
},
{
name: "message with tool calls",
msg: chat.Message{
Expand Down
Loading
Loading