Skip to content

Commit 4e78409

Browse files
feat(function): mount referenced files into the code sandbox (#7251)
* feat(function): mount referenced files into the code sandbox Reference a file's path in Function block code and it is mounted for you: `<block.file.path>` resolves to its location on the sandbox filesystem, so any language can open it. It is the counterpart to `.base64`, which inlines the bytes and only works in JavaScript, leaving Python and Shell with no way to read a referenced file at all. Resolution happens long before a sandbox exists, and mount paths are only assigned once the whole set is planned together, so the resolver leaves a marker that the function runtime swaps for the real path — the same shape as LargeValueRef. Files written to /tmp/sim/outputs come back as platform file objects, so they can be attached or uploaded with no intermediate step. Harvesting is automatic on runs already in the remote sandbox; isolate runs pay nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(execution): cover the sim.* helpers in a real isolate isolated-vm.test.ts mocks the spawn, so nothing proved the sim.* namespace was reachable from user code — only that the process plumbing was called. These run the real worker and assert values cross the boundary both ways. Pins the fast runtime's global surface, which turned out narrower than assumed: plain ECMAScript plus fetch, console and sim.*, with no Buffer, require, process, crypto, TextDecoder, atob or setTimeout. That list is exactly what decides whether a block needs an import and so moves to the remote sandbox, so it is asserted rather than described. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: regenerate tool metadata and integration docs Changing function_execute's params and adding file_write's fileInput made both generated artifacts stale, which check:audits catches. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(function): address review findings on sandbox file I/O - Classify harvested output by content, not by file name. Gating the provenance scan on a filename-derived MIME type let a resolved secret be written as plaintext under a binary-looking extension and skip the only guard before upload. Bytes that round-trip as UTF-8 are scannable whatever they are called. - Format the .path replacement through the shared context-aware helper. Returning a bare identifier inserted __blockRef_N literally in Shell and inside quoted strings instead of the mounted path. - Enforce the mount ceiling on the combined set. Marker-derived mounts bypassed the contract's max, which only bounded the explicit files param. - Reuse one marker per file key, so referencing a path twice mounts once. - Raise a coded error when the code deletes the output directory, so it reports as a 400 with remediation rather than an opaque 500. - Drop declared paths from the discovered set; counting a file in both rejected a single output larger than half the byte ceiling. - defineProperty when rebuilding context values, so an own __proto__ key survives instead of hitting Object.prototype's setter. - Give the directory sentinel a collision-resistant name, and cap file_write's fileInput at the destination's own limit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(function): include files in the FAQ's output answer Adding <function.files> to the outputs table left the FAQ still saying the block returns two outputs, and the reference FAQ never mentioned that a file is read by referencing its path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(function): scan every harvested output for resolved secrets Gating the scan on whether the bytes looked textual was defeatable twice over: name the file .png, or append a single invalid byte, and a plaintext secret skipped the only guard before upload. A lossy UTF-8 decode preserves ASCII runs, so a literal secret is findable in any buffer. The scan is now unconditional. What remains out of reach is a secret carried in transformed form, which no substring scan can see — an inherent limit of scanning rather than a hole in the gate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(function): correct provenance, mount caps and runtime file plumbing Review findings from the sandbox file I/O change, traced to root cause. A file_write carrying fileInput copied its bytes into a new workspace file without the source's secret lineage, so a file the platform had locked as secret-derived became readable again under a new id. It now derives provenance from the source through the same helper archiving uses, which also marks a source with no workspace row unknown rather than empty. The tool half of that feature had shipped without the block half: the File block's write mapper listed four params and dropped fileInput, and no sub-block could supply one, so the documented binary path was unreachable from the canvas and from an agent holding the File tool. Adds the canonical pair, maps it, and drops content's unconditional required now that the two are mutually exclusive sources. The harvest counted secret names the matcher had already discarded as too short to identify anything, so a workflow whose every secret was under the substitutable-literal minimum built no matcher, classified every output unknown, and refused each one while claiming it held a secret. The counter now applies the matcher's own predicate. The execution file index was lazily created on the per-call context clone, so files a tool produced were recorded onto a throwaway and the next call in the run never saw them. Materializing it on the source before the spread makes both objects share one map. A .path reference inside a quoted Python or JavaScript string was JSON-encoded, putting literal quote characters inside the path the code then opened. Mount paths are sanitized to a safe character set, so they splice raw. URL mounts now carry a ceiling that curl enforces on the bytes actually served, rather than trusting a recorded size, and declared sandbox outputs opt out of the harvest explicitly instead of relying on which branch returns first. Also stops rebuilding non-plain context values into stripped objects, and removes an agent tool-description branch that no longer had a sub-block to read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(file): send one write source, and bound mounts by what they were charged The write card sent `content` unconditionally, and the contract counts any defined `content` as "text was provided" — so an untouched Content box, which serializes as an empty string, collided with a selected file and rejected every binary write. The mapper now emits only the source the card carries, leaving the contract to catch filling both or neither. A generated document that references other files needs a principal to resolve them; without one the resolver can only serve an already-published artifact and throws. The write path now passes it. URL mounts are granted exactly the byte count they were charged against the aggregate, rather than being charged a reported size while permitted the global per-file maximum — twenty mounts each claiming a byte could otherwise be allowed 500MB apiece. An honest size fetches normally and an understated one is refused. `--max-filesize` only refuses a transfer up front when the response declares a Content-Length, so a chunked reply slipped past it. The delivered file is now measured and removed if it overran, in the same command rather than a second round trip. Also documents that naming an explicit sandbox output path excludes the harvest directory, in both the block docs and the tool description the model reads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(file): accept the picker shape, and bound a mount while it downloads The write path required an already-complete UserFile, but the file picker stores {name, path, key, size, type} with no id or url — so selecting a file in the new basic field was rejected as not a file object before any bytes moved. It now runs through the same normalizer every other operation in that file applies to its own input, while a block reference or an agent-resolved id still passes through as-is. Writing a deliberately empty text file was also rejected, because an empty string read as "no text". The selected file is what disambiguates now: with one present an empty Content box means unused, and with none, content always goes through. --max-filesize only refuses a transfer that declares a Content-Length, so the previous size check ran after the whole object had already been written to sandbox disk. The fetch now streams through a byte cap, so at most one byte over the limit can ever land, and curl's status travels through a file so a 403 on an expired URL is still distinguishable from an empty download. A non-finite mount size made every comparison false, so the aggregate check passed while the mount was charged the per-file maximum anyway; the size is resolved once now, before either test. Mount resolution failures are the caller's files — unreadable, oversized, or over the aggregate — and now answer 400 with the message naming the file, rather than a 500. Files already uploaded when a later one in the same harvest is refused are removed, since the export is all-or-nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(file): clamp a declared mount cap, and demand identity not full metadata sandboxFiles reaches the sandbox layer from the request body, so a declared maxBytes is a caller's number. It may now lower its own mount's ceiling but never raise it past the one that layer guarantees. The write path required full UserFile metadata, but size is never read before the download and the download reports the real content type — so a reference carrying id, key, url and name was rejected over two fields nothing depends on. It now asks only for identity and fills the rest. Uploads already made are also discarded when a later upload throws, not only when a later file is refused for carrying a secret. Both exits leave the harvest all-or-nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(file): answer null for a key a file input cannot be classified by fileInputToUserFile normalizes caller-supplied file objects and returns null for anything it cannot use — but it classified the storage key with the throwing form, so a key without a recognized context prefix escaped as a 500 from every operation that normalizes a file input, not just write. Adds tryInferContextFromKey beside inferContextFromKey, sharing the one prefix list so a new context cannot be added to half of them. The throwing form stays right where an unclassifiable key means the platform built one wrong; the nullable form is for keys that arrived in a request, where an unrecognized prefix only means this is not a file we can use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 62d0800 commit 4e78409

39 files changed

Lines changed: 3542 additions & 300 deletions

apps/docs/content/docs/integrations/file.mdx

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -87,15 +87,16 @@ Fetch and parse a file from a URL with optional custom headers.
8787

8888
### File Write
8989

90-
Create a new workspace file. If a file with the same name already exists, a numeric suffix is added (e.g., "data (1).csv").
90+
Create a new workspace file, either from text content or from an existing file. If a file with the same name already exists, a numeric suffix is added (e.g., "data (1).csv").
9191

9292
#### Input
9393

9494
| Parameter | Type | Required | Description |
9595
| --------- | ---- | -------- | ----------- |
96-
| `fileName` | string | Yes | File name \(e.g., "data.csv"\). If a file with this name exists, a numeric suffix is added automatically. |
97-
| `content` | string | Yes | The text content to write to the file. |
98-
| `contentType` | string | No | MIME type for new files \(e.g., "text/plain"\). Auto-detected from file extension if omitted. |
96+
| `fileName` | string | No | File name \(e.g., "data.csv"\). Required when writing text; optional when storing a file, which keeps its own name unless this overrides it. If the name already exists, a numeric suffix is added automatically. |
97+
| `content` | string | No | The text content to write to the file. Provide exactly one of content or fileInput. |
98+
| `fileInput` | file | No | An existing file to store in the workspace, such as one produced by an earlier tool. Use this for anything that is not text — PDFs, images, audio, archives. Provide exactly one of content or fileInput. |
99+
| `contentType` | string | No | MIME type for new files \(e.g., "text/plain"\). Auto-detected from the file extension, or taken from the stored file, if omitted. |
99100

100101
#### Output
101102

apps/docs/content/docs/workflows/blocks/function.mdx

Lines changed: 45 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,49 @@ Sim supplies the rendered heredoc privately while preserving the quoted delimite
102102
| --- | --- |
103103
| `<function.result>` | The value your code returns (object, array, string, number, …) |
104104
| `<function.stdout>` | Anything printed with `console.log()` or `print()` |
105+
| `<function.files>` | Files your code wrote to `/tmp/sim/outputs`, ready to attach or upload |
106+
107+
## Files
108+
109+
**Reading.** Reference a file's `path` and it is mounted for you:
110+
111+
```python
112+
import pandas as pd
113+
114+
frame = pd.read_csv(<gmail.attachments[0].path>)
115+
frame.describe().to_csv('/tmp/sim/outputs/summary.csv')
116+
```
117+
118+
`.path` resolves to the file's location on the sandbox filesystem, so any language
119+
can open it — pandas, ffmpeg, a CLI. It is the counterpart to `.base64`, which
120+
inlines the contents instead and works only in JavaScript. Both appear in the
121+
reference dropdown next to `.name` and `.size`.
122+
123+
**Writing.** Anything your code writes to `/tmp/sim/outputs` comes back as
124+
`<function.files>`, a list of file objects any file-accepting block takes directly —
125+
attach them to an email, upload them to storage, or save them to the workspace with
126+
the File block. There is nothing to turn on.
127+
128+
The one exception is a call that names an explicit `outputSandboxPath`. That asks
129+
for particular paths to be exported and answers with that export's own result, so
130+
the output directory is not harvested alongside it — choose one or the other rather
131+
than expecting both in the same run.
132+
133+
<Callout type="info">
134+
Referencing `.path` runs the block in the remote sandbox, since the local
135+
JavaScript VM has no filesystem — expect the slower start of a remote run even for
136+
plain JavaScript. Referencing the file itself (`<gmail.attachments[0]>`, `.name`,
137+
`.url`) does not, and stays local. Up to 20 files come back per run, 50MB total,
138+
nested no more than 11 directories deep; a run that exceeds any of these fails
139+
rather than returning part of what your code wrote.
140+
</Callout>
141+
142+
<Callout type="warn">
143+
Returned files live with the execution rather than in your workspace, and a text
144+
file containing a resolved secret value is refused rather than returned — there is
145+
nowhere on an execution file to record that it carries one. Write such a file to a
146+
workspace path instead, or keep the secret out of the output.
147+
</Callout>
105148

106149
## Language
107150

@@ -401,8 +444,8 @@ The lazy `sim.files` and `sim.values` helpers are available only in JavaScript f
401444
{ question: "What languages does the Function block support?", answer: "JavaScript, Python, and Shell. JavaScript is the default. Python remains a stable saved language choice; Shell and custom Sandbox controls appear when a remote sandbox provider is enabled. Python and Shell execution require that provider." },
402445
{ question: "When does code run locally vs. in a sandbox?", answer: "JavaScript without external imports runs in a local isolated sandbox for speed. JavaScript that uses import or require, Python, and Shell run in the configured remote sandbox." },
403446
{ question: "Does JavaScript still work without E2B or Daytona?", answer: "Yes. JavaScript without import or require runs in Sim's local isolated VM and does not require a remote provider. JavaScript with external imports, Python, Shell, and custom Sandboxes require E2B or Daytona and fail explicitly when it is unavailable." },
404-
{ question: "How do I reference outputs from other blocks inside my code?", answer: "Use angle-bracket syntax directly, like <agent.content> or <api.data>, with no quotes around the tag — Sim replaces it with the real value before execution. For environment variables, use double curly braces: {{API_KEY}}." },
405-
{ question: "What does the Function block return?", answer: "Two outputs: result and stdout. Use return in JavaScript, assign __sim_result__ in Python, or print an __SIM_RESULT__= marker in Shell to set result. Ordinary console, print, and command output goes to stdout." },
447+
{ question: "How do I reference outputs from other blocks inside my code?", answer: "Use angle-bracket syntax directly, like <agent.content> or <api.data>, with no quotes around the tag — Sim replaces it with the real value before execution. For environment variables, use double curly braces: {{API_KEY}}. To read a file, reference its path — <gmail.attachments[0].path> mounts it and resolves to a location any language can open." },
448+
{ question: "What does the Function block return?", answer: "Three outputs: result, stdout, and files. Use return in JavaScript, assign __sim_result__ in Python, or print an __SIM_RESULT__= marker in Shell to set result. Ordinary console, print, and command output goes to stdout. Anything your code writes to /tmp/sim/outputs comes back in files as a file object later blocks can accept directly." },
406449
{ question: "Can I make HTTP requests from a Function block?", answer: "Yes. fetch() is available in JavaScript with async/await. In Python, use requests or httpx. In Shell, use curl or a CLI available on the selected sandbox." },
407450
{ question: "Is there a timeout for Function block execution?", answer: "Yes, a configurable execution timeout. If your code exceeds it, the run is terminated and the block reports an error. Keep this in mind for external calls or heavy processing." },
408451
]} />

apps/sim/blocks/blocks/file.ts

Lines changed: 41 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,9 @@ const APPEND_FILE_FIELD = ['appendFile', 'appendFileName'] as const
8080
const COMPRESS_FILE_FIELD = ['compressFile', 'compressFileId'] as const
8181
const DECOMPRESS_FILE_FIELD = ['decompressFile', 'decompressFileId'] as const
8282
const SHARE_FILE_FIELD = ['shareFile', 'shareFileId'] as const
83+
/* Text and file are mutually exclusive sources, so the clause names whichever
84+
one the card actually carries. */
85+
const WRITE_CONTENT_FIELD = ['content', 'writeFile', 'writeFileId'] as const
8386

8487
export const FileBlock: BlockConfig<FileParserOutput> = {
8588
type: 'file',
@@ -921,7 +924,7 @@ export const FileV5Block: BlockConfig<FileParserV3Output> = {
921924
file_fetch: [{ text: 'Fetch and parse', field: 'fileUrl', core: true }],
922925
file_write: [
923926
{ text: 'Create', field: 'fileName', core: true },
924-
{ text: 'containing', field: 'content' },
927+
{ text: 'containing', field: WRITE_CONTENT_FIELD },
925928
],
926929
file_append: [
927930
{ text: 'Append', field: 'appendContent', core: true },
@@ -1031,7 +1034,25 @@ export const FileV5Block: BlockConfig<FileParserV3Output> = {
10311034
type: 'long-input' as SubBlockType,
10321035
placeholder: 'File content to write...',
10331036
condition: { field: 'operation', value: 'file_write' },
1034-
required: { field: 'operation', value: 'file_write' },
1037+
},
1038+
{
1039+
id: 'writeFile',
1040+
title: 'File',
1041+
type: 'file-upload' as SubBlockType,
1042+
canonicalParamId: 'writeFileInput',
1043+
acceptedTypes: '*',
1044+
placeholder: 'Store an existing file',
1045+
mode: 'basic',
1046+
condition: { field: 'operation', value: 'file_write' },
1047+
},
1048+
{
1049+
id: 'writeFileId',
1050+
title: 'File',
1051+
type: 'short-input' as SubBlockType,
1052+
canonicalParamId: 'writeFileInput',
1053+
placeholder: 'File from an earlier block',
1054+
mode: 'advanced',
1055+
condition: { field: 'operation', value: 'file_write' },
10351056
},
10361057
{
10371058
id: 'contentType',
@@ -1206,9 +1227,22 @@ export const FileV5Block: BlockConfig<FileParserV3Output> = {
12061227
const operation = params.operation || 'file_read'
12071228

12081229
if (operation === 'file_write') {
1230+
// Writing stores one file, so the single form.
1231+
const fileInput = normalizeFileInput(params.writeFileInput, { single: true })
1232+
// The contract counts any defined `content` as "text was provided", and
1233+
// an untouched Content box serializes as an empty string — so sending it
1234+
// unconditionally would make every file write collide with its own empty
1235+
// text box. The selected file is what disambiguates: with one present,
1236+
// an empty Content box means "not used" and is dropped, while a
1237+
// non-empty one is still forwarded so the contract can report that both
1238+
// were filled. With no file, `content` always goes through, which keeps
1239+
// writing a deliberately empty text file possible.
1240+
const contentText = typeof params.content === 'string' ? params.content : undefined
1241+
const omitContent = Boolean(fileInput) && !contentText
12091242
return {
12101243
fileName: params.fileName,
1211-
content: params.content,
1244+
...(omitContent ? {} : { content: params.content }),
1245+
...(fileInput ? { fileInput } : {}),
12121246
contentType: params.contentType,
12131247
workspaceId: params._context?.workspaceId,
12141248
}
@@ -1432,6 +1466,10 @@ export const FileV5Block: BlockConfig<FileParserV3Output> = {
14321466
fileType: { type: 'string', description: 'File type for fetch' },
14331467
fileName: { type: 'string', description: 'Name for a new file (write)' },
14341468
content: { type: 'string', description: 'File content to write' },
1469+
writeFileInput: {
1470+
type: 'json',
1471+
description: 'An existing file to store in the workspace, instead of text content',
1472+
},
14351473
contentType: { type: 'string', description: 'MIME content type for write' },
14361474
appendFileInput: { type: 'json', description: 'File to append to' },
14371475
appendContent: { type: 'string', description: 'Content to append to file' },
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
/**
2+
* @vitest-environment node
3+
*/
4+
import { describe, expect, it } from 'vitest'
5+
import { USER_FILE_ACCESSIBLE_PROPERTIES } from '@/lib/workflows/types'
6+
import { FunctionBlock } from '@/blocks/blocks/function'
7+
8+
describe('Function block file surface', () => {
9+
it('has no file configuration fields', () => {
10+
// Files reach the sandbox by being referenced in code as
11+
// <block.file.path> — the same way every other block output is referenced.
12+
// A dedicated field would be a second way to say the same thing, and would
13+
// need a home in the panel that the reference syntax does not.
14+
const ids = FunctionBlock.subBlocks.map((subBlock) => subBlock.id)
15+
16+
expect(ids).not.toContain('files')
17+
expect(ids).not.toContain('uploadedFiles')
18+
expect(ids).not.toContain('collectOutputFiles')
19+
expect(FunctionBlock.inputs).not.toHaveProperty('files')
20+
expect(FunctionBlock.inputs).not.toHaveProperty('collectOutputFiles')
21+
})
22+
23+
it('returns harvested files so downstream blocks can consume them', () => {
24+
expect(FunctionBlock.outputs.files).toMatchObject({ type: 'file[]' })
25+
})
26+
27+
it('offers path alongside base64 as a referenceable file property', () => {
28+
// This is what puts `.path` in the tag dropdown: block-outputs.ts maps the
29+
// list into `${path}.${prop}` suggestions.
30+
expect(USER_FILE_ACCESSIBLE_PROPERTIES).toContain('path')
31+
expect(USER_FILE_ACCESSIBLE_PROPERTIES).toContain('base64')
32+
})
33+
})

apps/sim/blocks/blocks/function.ts

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import { CodeIcon } from '@/components/icons'
22
import { isSandboxesEnabled } from '@/lib/core/config/env-flags'
33
import { CodeLanguage, getLanguageDisplayName } from '@/lib/execution/languages'
4+
import { SANDBOX_OUTPUT_DIR } from '@/lib/execution/remote-sandbox/sandbox-paths'
45
import type { BlockConfig } from '@/blocks/types'
56
import type { CodeExecutionOutput } from '@/tools/function/types'
67

@@ -17,6 +18,9 @@ export const FunctionBlock: BlockConfig<CodeExecutionOutput> = {
1718
- Shell code runs CLI commands in a remote sandbox.
1819
- To import third-party packages or add curated CLI tools, create a sandbox in Settings > Sandboxes and select it under the block's advanced options. Without one, only the default image's packages and commands are available.
1920
- Can reference workflow variables using <blockName.output> syntax as usual within code. Avoid XML/HTML tags.
21+
- To read a file from an earlier block, reference its path: <blockName.files[0].path> mounts the file and resolves to its location on the sandbox filesystem, which any language can open. Use <blockName.files[0].base64> instead when you only want the contents inline in JavaScript.
22+
- Anything the code writes to ${SANDBOX_OUTPUT_DIR} is returned as \`files\`, ready to attach to an email or upload without any extra step.
23+
- Referencing a file path runs the block in the remote sandbox, so it is slower to start than a plain local JavaScript run.
2024
`,
2125
docsLink: 'https://docs.sim.ai/workflows/blocks/function',
2226
category: 'blocks',
@@ -174,5 +178,9 @@ try {
174178
type: 'string',
175179
description: 'Console log output and debug messages from function execution',
176180
},
181+
files: {
182+
type: 'file[]',
183+
description: `Files the code wrote to ${SANDBOX_OUTPUT_DIR}, ready to attach or upload downstream`,
184+
},
177185
},
178186
}

apps/sim/executor/types.ts

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -403,6 +403,21 @@ export interface ExecutionContext {
403403
*/
404404
toolBindingLabelCache?: Map<string, string | null>
405405

406+
/**
407+
* Files produced during this execution, indexed by {@link UserFile.id}, so a
408+
* model can name one by id in a tool argument and the runtime can hydrate it
409+
* into the full object.
410+
*
411+
* Needed because a file an agent has just seen — a Gmail attachment fetched
412+
* moments ago in the same turn — lives only in that turn's tool results, not
413+
* in any block state or workspace row, so nothing else can resolve it. The
414+
* index only *selects*; every read is still authorized on its own.
415+
*
416+
* A Map for the same reason as {@link toolBindingLabelCache}: `blockCtx` is a
417+
* shallow clone per block execution, so only a shared reference survives.
418+
*/
419+
executionFilesById?: Map<string, UserFile>
420+
406421
blockStates: ReadonlyMap<string, BlockState>
407422
executedBlocks: ReadonlySet<string>
408423

0 commit comments

Comments
 (0)