Skip to content

Add Go module collection support#14

Draft
dagger-codex[bot] wants to merge 4 commits into
mainfrom
codex/go-collections
Draft

Add Go module collection support#14
dagger-codex[bot] wants to merge 4 commits into
mainfrom
codex/go-collections

Conversation

@dagger-codex

@dagger-codex dagger-codex Bot commented Jun 1, 2026

Copy link
Copy Markdown

Summary

Go workspaces get three selection dimensions — go-module, go-directory, go-test — with batched test execution:

$ dagger list go-test --go-directory=app/sub
TestSubDouble
TestSubZero

$ dagger check --go-test=TestSubDouble --go-test=TestSubZero
ok  example.com/app/sub  0.001s        # one go test -run '^(TestSubDouble|TestSubZero)$'

GoModule.testDirs exposes test directory discovery as a collection keyed by path; each directory exposes its tests as a collection keyed by name (go test -list). The collection-level run is a batch check that shadows the per-test run: selecting several tests compiles and runs them in a single go test invocation, and dimension filters push down so only selected directories enumerate.

Go.modules() returns a GoModules collection (keyed by workspace-relative module root path) instead of a plain list, making Go modules first-class artifacts: enumerable, selectable, and filterable through the standard collection surface.

$ dagger list go-module
./app
./lib

$ dagger call modules subset --keys ./app keys
./app

Authored with Dang's @keys/@get directives; the engine projects the public surface (keys, list, get(key), subset(keys)). Internal callers (lintAll, testAll, generateAll) iterate via the raw backing members, which stay visible in-module.

Rebased on main: the original marker-file plumbing was dropped in favor of main's skip-path configuration, which the collection simply carries to get.

Verification

Requires an engine with collection support (dagger/dagger#13299) — CI cannot pass until a release ships it. Validated against a dev engine built from that PR:

  • e2e:module-introspection-check passes, including new assertions: get(key:) returns the requested module, subset(keys:) narrows exactly.
  • Consumer workspace UX: dagger list go-module, dagger call modules keys / subset --keys ... keys work against this module.

Notes

  • Test directory discovery (pre-existing go-includes behavior) does not surface test files at a module root; only subdirectories containing _test.go files become test directories.
  • The check-selection flow (dagger check --go-test=…) additionally requires the plans-lite layer in [1.0-beta] modules-v2: add artifacts and collections dagger#13299.
  • modules() discovers **/go.mod across the whole workspace; when the module source directory lives inside the workspace, its own embedded go.mods appear too. Pre-existing behavior, unchanged here.

@dagger-codex dagger-codex Bot force-pushed the codex/go-collections branch from d24d89a to 50ad694 Compare June 1, 2026 21:54
@dagger-codex dagger-codex Bot changed the title Add collection support to the Go module Add Go module collection support Jun 1, 2026
Signed-off-by: Solomon Hykes <solomon@dagger.io>
@shykes shykes force-pushed the codex/go-collections branch from 50ad694 to ab183bb Compare June 10, 2026 08:36
Layer two more collections under GoModules, giving Go workspaces three
selection dimensions: go-module, go-directory, and go-test.

GoModule.testDirs exposes the existing test directory discovery as a
collection keyed by workspace-relative path. Each GoDirectory exposes
its tests as a collection keyed by test name, enumerated with
go test -list; skipped directories produce an empty collection.

Test execution is batched: the collection-level run executes one
go test -run '^(A|B)$' over the current subset, and shadows the
per-test run so that selecting several tests by name
(dagger check --go-test=A --go-test=B) compiles and runs them in a
single invocation. Filters push down, so only selected directories
enumerate their tests.

Note: test directory discovery (pre-existing go-includes behavior)
does not surface test files at a module root; only subdirectories
containing _test.go files become test directories.

Signed-off-by: Solomon Hykes <solomon@dagger.io>
shykes added 2 commits June 11, 2026 16:25
GoDirectory.tests used to spin up a Go container and parse
'go test -list' stdout to enumerate test names. Replace that with a
single Workspace.directory(...).search(pattern: "^func Test\\w+\\(")
over the directory's *_test.go files and parse names from the matched
lines in Dang.

This is much faster (no compile, no container exec) and matches the
same set go test -list does for top-level test functions. Test methods
on testify-style suites are intentionally not listed: they cannot be
selected with go test -run independently of their parent function.

Add moduleIntrospectionCheck assertions covering the new GoTestDirs
and GoTests collection surface end-to-end: keys/get round-trip on both
levels, multi-test discovery from a single *_test.go file (cross-
include-b has two top-level tests), and that a skip-configured
directory produces an empty tests collection.

Signed-off-by: Solomon Hykes <solomon@dagger.io>
Every function with a Workspace receiver or argument is invalidated
unconditionally. GoDirectory.tests() was crossing Workspace once per
test directory, which on dagger/dagger meant ~hundreds of invalidated
calls and ~12 minutes to enumerate go-test.

Thread a single Directory snapshot of all **/*_test.go files from the
Workspace boundary (Go.modules / Go.module) down through GoModules ->
GoModule -> GoTestDirs -> GoDirectory. GoDirectory.tests() now searches
that pre-materialized Directory and filters results by filePath. The
search arguments are identical for every directory, so the search hits
the dagql cache (Directory receiver -> content-addressed) after the
first call; per-directory restriction is pure-Dang on the cached result.

Net: one Workspace crossing and one ripgrep per session, no matter how
many directories the workspace contains.

Signed-off-by: Solomon Hykes <solomon@dagger.io>
@shykes shykes force-pushed the codex/go-collections branch from 7e8e180 to 6ce779b Compare June 12, 2026 05:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant