Attach files to a chat message (logs, screenshots, config), with model-capability gating

[gangtao]

Enhancement

Let users attach files to a chat message — logs, config dumps, stack traces, screenshots, etc. — so the agent can reason over pasted context instead of the user hand-copying snippets. The chat runs on a configurable model (Anthropic or any OpenAI-compatible model via a gateway base_url), and not every model can read images, so the feature must degrade gracefully: unsupported attachment types are either rejected with a clear error or the upload control is disabled up front.

Tech investigation (why capability gating is required)

The chat model is configurable and can be a non-vision model:

• src/tpk/config.py:127 — AgentConfig carries only provider ("anthropic" | "openai") and model. There is no capability/vision field today.
• src/tpk/agent.py:41 — build_chat_model() builds ChatAnthropic or _ReasoningChatOpenAI, both against a configurable base_url gateway. The app routinely runs OpenAI-compatible reasoning models (gpt-oss-120b, DeepSeek, Qwen, Kimi) that cannot accept images.
• src/tpk/server.py:168 — /chat builds messages = [(role, content)...] and feeds the LangChain agent via astream_events.
• src/tpk/server.py:156 — /chat/model already returns {provider, model} to the UI — the right place to also advertise capabilities.

API-capability facts (both providers):

Payload	API mechanism	Supported by
Screenshots (PNG/JPEG/WebP)	multimodal image content block	Anthropic (all vision models) + OpenAI vision models only — not gpt-oss/DeepSeek-text/etc.
Logs / config / stack traces (.log/.txt/.json/…)	inlined as text in the message	Every model — it's just text, not an "attachment" in either API
PDFs (out of scope here)	document/file block	Both, if added later

Takeaway: text-ish attachments always work (they're plain text). Images are the only capability-gated case — and it can't be auto-detected reliably because the model is arbitrary behind a gateway. So we declare the capability explicitly.

Capability model

Add an explicit supports_images capability, derived once from AgentConfig:

• Default heuristic: provider == "anthropic" → True; openai → True only for a small allowlist of known vision models (e.g. gpt-5*, gpt-4o*), else False.
• Override with an env var (e.g. TPK_AGENT_VISION=1|0) so operators running a vision-capable model behind a custom gateway name can force it on/off.
• Expose it by extending /chat/model → {provider, model, supports_images} (src/tpk/server.py:156). Keep it non-sensitive; return supports_images: false when the agent LLM isn't configured.

Frontend (web/src/Chat.tsx + app.css)

• On load, read supports_images from /chat/model (already fetched at Chat.tsx:272).
• Attach affordances: paperclip button (<input type="file" multiple>), drag-and-drop onto the composer, and clipboard paste of an image (onPaste → clipboardData.files).
• When supports_images is false: disable/hide the image path — reject image files/paste with an inline note ("current model can't read images"), while still allowing text files. send() enabled when there is text or ≥1 accepted attachment.
• Attachment chips above the textarea: filename + size (thumbnail for images) + × remove. Read files client-side via FileReader (text → UTF-8 string, image → base64 data URL); enforce size limits before send; clear on successful send.

Transport + backend (src/tpk/server.py)

Extend ChatRequest/ChatTurn with an optional attachments list:

class Attachment(BaseModel):
    kind: Literal["text", "image"]
    name: str
    mime: str
    content: str  # text body, or base64 (no data: prefix) for images

class ChatRequest(BaseModel):
    message: str = Field(min_length=1)
    attachments: list[Attachment] = []
    history: list[ChatTurn] = []

In /chat, build a multimodal content list for the final user turn instead of ("user", req.message):

• text attachments → append a fenced text block: f"Attached file {name}:\n```\n{content}\n```".
• image attachments → a provider-agnostic LangChain image block ({"type": "image", "source_type": "base64", "mime_type": ..., "data": ...}), which LangChain normalizes for both ChatAnthropic and ChatOpenAI ([[llm-provider-preference]]).

Capability enforcement (defense in depth — the client is not trusted): if any attachment is kind == "image" and supports_images is false, reject the request with 415/422 and a clear message ("the configured model <model> cannot read images") rather than passing it to the model and 500ing. This backstops the disabled UI in case a client sends images anyway. History turns with attachments reconstruct the same content shape.

Limits & safety

• Per-file and per-request size caps (~256 KB per text file, ~5 MB per image, ~10 MB total), enforced client- and server-side; reject oversize with 413 + clear error.
• Allowlist mime/extensions; treat all attachment content as untrusted data, never instructions (the agent runs tools).
• Per-message context only — no persistence (persistence would ride on Per-user chat conversation history (persistent, isolated, multiple threads) #10).

Out of scope

• Storing/serving uploaded files (inline context only).
• PDF/office extraction — text + images first.

Tests (tests/test_server.py)

• Text attachment reaches the agent's user message.
• Image attachment → correct multimodal block shape for the configured provider.
• /chat/model reports supports_images correctly for anthropic vs a non-vision openai model.
• Image attachment against a non-vision model is rejected (415/422), not 500.
• Oversized / disallowed-type payloads are rejected (413).