Package a local directory code_source_path for AI Runtime tasks - #6110
Open
vinchenzo-db wants to merge 9 commits into
Open
Package a local directory code_source_path for AI Runtime tasks#6110vinchenzo-db wants to merge 9 commits into
vinchenzo-db wants to merge 9 commits into
Conversation
Let ai_runtime_task.code_source_path point at a local directory, not just a pre-built tarball. A new aicode mutator (run in the build phase, before libraries.ExpandGlobReferences/ReplaceWithRemotePath) detects a local-directory value, packages it into a reproducible, content-addressed tarball — honoring .gitignore and the top-level sync.include/exclude globs — uploads it to the user's ~/.air/repo_snapshots directory, and rewrites code_source_path to the uploaded (de-/Workspace-prefixed) path. Because it runs first, PR #5922's artifact-style collection then sees an already-remote path and skips it, so the two compose: a directory is packaged by the CLI, a pre-built .tgz still flows through the artifact path. Also synthesizes a requirements.yaml next to the task's command_path from the job's serverless environments[] spec, so the AI Runtime harness sets up the workload environment. command_path translation itself is provided by #5922. Verified end-to-end on a live workspace: a plain bundle deploy of a directory code_source_path yields a job whose run terminates SUCCESS; sync.exclude and .gitignore entries are absent from the uploaded snapshot; unchanged code skips re-upload. Co-authored-by: Isaac
Contributor
Approval status: pending
|
Collaborator
Integration test reportCommit: e306181
8 interesting tests: 4 SKIP, 3 KNOWN, 1 RECOVERED
Top 3 slowest tests (at least 2 minutes):
|
The aicode mutator packages a local directory code_source_path at deploy. A
pre-built tarball delivered via an `artifacts` block (code_source_path pointing
at a local *file*) must instead flow through the standard artifact-upload path,
exactly as the two-path design intends. Validate rejected such a path outright
("code_source_path not found" — the artifact tarball does not exist yet at
initialize time), breaking the existing bundle/artifacts/ai_runtime_code_source
acceptance test.
Both Validate and collectLocalCodeSources now skip any local code_source_path
that does not resolve to an existing directory, leaving it for the artifact
uploader. The packaging-specific constraints (git_source, immutable_folder) only
apply once the path is confirmed to be a directory this mutator will package.
Co-authored-by: Isaac
- Remove the never-set `client` filer field from packageAndUpload and synthesizeRequirements (only ever nil); build the workspace filer directly. - Read the current user directly (resolved in the initialize phase) instead of the defensive nil-guarded helper. - Surface a stat error other than not-exist (e.g. unreadable directory) instead of silently treating it as "not a directory to package"; shared isExistingDir helper used by collect and Validate. - Pass GitSource into validateTask (nil-check inside) to keep validation logic contained. - Collect a single direct-task code_source_path pattern, matching Validate and SynthesizeRequirements (for_each_task is unsupported until all three gain it). - SynthesizeRequirements writes requirements.yaml next to every deployment's command_path (deduped by directory), not just the first. Co-authored-by: Isaac
Reworks the local_code_source acceptance test per review: - Assert the uploaded tarball's contents via list_code_snapshot.py: gitignored files (ignored_by_git.txt, data/) are excluded, *.log is excluded via sync.exclude, and data/model.bin is force-included via sync.include. - Use print_requests.py instead of hand-rolled jq (testing.md rule); extend its --del-body to also strip the top-level raw_body so the binary tarball upload doesn't dump into the golden. - Add a cache-miss case: editing a file changes the content hash and re-uploads. Co-authored-by: Isaac
print_requests.py path filters and the workspace export in list_code_snapshot.py were rewritten by Git Bash's MSYS path conversion on Windows (e.g. /repo_snapshots/ -> C:/Program Files/Git/repo_snapshots/), failing the test. Use // leading filters (matching the existing convention) and set MSYS_NO_PATHCONV for the export. Co-authored-by: Isaac
Split subprocess.run kwargs onto separate lines to satisfy ruff format. Co-authored-by: Isaac
Reworks PackageAndUpload to place the code snapshot inside the bundle instead of uploading it to a shared ~/.air/repo_snapshots workspace cache. The mutator now writes the content-addressed tarball into the bundle's sync tree (.air_snapshots/) and rewrites code_source_path to the workspace path it will occupy once synced; it performs no workspace write itself. Normal bundle file sync uploads it during the deploy phase. Why: - Build phase must not mutate the workspace (it runs before `bundle plan`). The old filer upload violated that; now the only build-phase work is local prepare-and-archive, and the upload happens in the deploy phase via file sync. - No deploy-time side-effect outside the bundle: `bundle destroy` removes the snapshot like any other bundle file, so the home-dir cache can no longer compound across bundles/deploys. - Dedup is preserved: the name is content-addressed and bundle file sync is incremental, so unchanged code keeps the same synced path and is not re-uploaded. Tradeoff: the cache is now per-bundle rather than shared across bundles. See the package doc for the (rejected) TranslatePaths alternative and why archiving stays in the build phase. Co-authored-by: Isaac
The snapshot is now injected into the bundle sync root as an in-memory overlay file rather than written under the working tree. This keeps the deploy artifact inside the bundle (synced to <root>/files/.air_snapshots/, removed by `bundle destroy`, content-addressed so unchanged code is not re-uploaded) while leaving the user's working tree clean — no generated .tar.gz appears in their checkout. Adds libs/vfs.Overlay: a vfs.Path wrapper that serves a set of in-memory files in addition to a base path, participating in Open/Stat/ReadDir/ReadFile and fs.WalkDir so bundle file sync uploads the overlaid files transparently. PackageAndUpload builds each snapshot in memory and swaps b.SyncRoot for an overlay carrying them. Co-authored-by: Isaac
ben-hansen-db
self-requested a review
July 31, 2026 23:19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Let ai_runtime_task.code_source_path point at a local directory, not just a pre-built tarball. A new aicode mutator (run in the build phase, before libraries.ExpandGlobReferences/ReplaceWithRemotePath) detects a local-directory value, packages it into a reproducible, content-addressed tarball — honoring .gitignore and the top-level sync.include/exclude globs — uploads it to the user's ~/.air/repo_snapshots directory, and rewrites code_source_path to the uploaded (de-/Workspace-prefixed) path. Because it runs first, PR #5922's artifact-style collection then sees an already-remote path and skips it, so the two compose: a directory is packaged by the CLI, a pre-built .tgz still flows through the artifact path.
Also synthesizes a requirements.yaml next to the task's command_path from the job's serverless environments[] spec, so the AI Runtime harness sets up the workload environment. command_path translation itself is provided by #5922.
Verified end-to-end on a live workspace: a plain bundle deploy of a directory code_source_path yields a job whose run terminates SUCCESS; sync.exclude and .gitignore entries are absent from the uploaded snapshot; unchanged code skips re-upload.
Co-authored-by: Isaac
Changes
Adds an aicode bundle mutator that packages a local code directory referenced by an AI Runtime task's code_source_path and uploads it during bundle deploy, then wires it into the build/initialize phases.
Why
The SDK jobs.AiRuntimeTask.code_source_path field expects a workspace or UC-volume path to an uploaded code archive, and its doc comment states the CLI is responsible for packaging the user's local code into that archive. Nothing in DABs did that yet: a user pointing code_source_path at a local directory had no way to bundle deploy a runnable AI Runtime job.
This mutator implements that contract for bundles. It deliberately reuses DABs' existing local-artifact upload plumbing and runs before ReplaceWithRemotePath, so the two upload paths compose without special-casing: a directory is packaged and uploaded by this mutator (which then presents an already-remote path that the artifact collector skips), while a pre-built tarball delivered via an artifacts block still flows through the artifact path as a file. Content-addressing keeps redeploys of unchanged code a no-op. SynthesizeRequirements closes the loop so the deployed workload actually has its environment set up — without it the run fails during setup.
Tests
End-to-end validation
Verified on a live A10 staging workspace (dbc-04ac0685-8857) that the aicode mutator packages a local-directory code_source_path at bundle deploy and that the deployed AI Runtime workload runs to SUCCESS on real GPU. Covered all three code-source shapes the mutator must handle, plus the artifact-tarball path it must not claim.
What was validated
Seam: PackageAndUpload claims a local directory
Expected: dir → content-addressed .tar.gz, uploaded to ~/.air/repo_snapshots, code_source_path rewritten to
the remote archive
Result: ✅
────────────────────────────────────────
Seam: PackageAndUpload skips a local file
Expected: a pre-built tarball (artifacts block) flows through the standard artifact upload, not aicode
packaging
Result: ✅ (bundle/artifacts/ai_runtime_code_source acceptance test)
────────────────────────────────────────
Seam: command_path translation
Expected: rewritten to its absolute synced workspace path
Result: ✅
────────────────────────────────────────
Seam: SynthesizeRequirements
Expected: writes requirements.yaml next to command_path, derived from environments[].spec
Result: ✅
────────────────────────────────────────
Seam: GPU workload
Expected: extracts code, installs deps, runs on A10, exits SUCCESS
Result: ✅
Deployed ai_runtime_task after bundle deploy (mutator output)
{
"code_source_path": "/Users//.air/repo_snapshots/code_source/code_source_5659f5b55c3427b1.tar.gz",
"command_path": "/Workspace/Users//.bundle/convert-e2e-smoke/dev/files/command.sh"
}
code_source_path was ./code_source (a local directory) in the bundle — the mutator packaged it, uploaded the content-addressed archive, and rewrote the field to the remote path.
Reproduce