Skip to content

fix(build): kill dead compiled artifacts + tighten Dockerfile context + fix checksums path (closes #147)#148

Merged
initializ-mk merged 2 commits into
mainfrom
fix/issue-147-build-output
Jun 11, 2026
Merged

fix(build): kill dead compiled artifacts + tighten Dockerfile context + fix checksums path (closes #147)#148
initializ-mk merged 2 commits into
mainfrom
fix/issue-147-build-output

Conversation

@initializ-mk

Copy link
Copy Markdown
Contributor

Closes #147.

Summary

Five interlocking bugs in the `forge build` output that compound to a bloated runtime image and a silently-disabled build-integrity check. Bundle is one PR because they all answer the same question — "what goes into the container image vs. what stays operator-side."

Bug-by-bug

1. Prompt-builder body duplication

`forge-skills/compiler/compiler.go:50` appended the full SKILL.md body once per tool entry. A SKILL.md with N `## Tool:` sections parses into N `SkillEntry` values that all share the same `Body` — so the bundled `code-review-github` skill (4 tools) shipped 4× repeats. On `aibuilderdemo`, `compiled/prompt.txt` was 1199 lines for what should be ~250.

Fix: track seen bodies in a set; emit each body once. Per-skill JSON entries still carry the full `Body` because forgecore SDK consumers may need it for non-prompt purposes.

2. `compiled/prompt.txt` and `compiled/skills/skills.json` are dead

The runtime never opens either file. `runner.discoverSkillFiles()` (runner.go:2619) re-globs `skills/*/SKILL.md` on every startup. Both files were written by the build, copied into the container, and never read.

Fix: stop generating them in `forge-cli/build/skills_stage.go`; delete `forge-cli/skills/writer.go`; remove the matching `COPY` lines from `Dockerfile.tmpl`; remove the required-files check in `validate_stage.go`. External library consumers (`forgecore.Compile`) still receive the in-memory `CompiledSkills` struct directly.

3. Dockerfile `COPY . .` leaks operator-side artifacts into `/app`

Pre-fix the only exclusions were `.env*`, `.enc`, `.key`, `.pem`. Everything else landed in `/app/`: `k8s/.yaml`, `build-manifest.json`, `compiled/security-audit.json`, `Dockerfile` itself (recursive), `.dockerignore` itself, `.local-bins/` (build-only stash).

Fix: expand `.dockerignore` with documented sections separating secrets from operator-side artifacts. The runtime image now contains only the 8 files the runner actually opens: `forge.yaml`, `guardrails.json`, `-config.yaml`, `agent.json`, `policy-scaffold.json`, `skills/`, `checksums.json`.

4. `checksums.json` integrity check silently skipped in container

`runner.go:245` looked at `/.forge-output/checksums.json`. Inside a Forge container, `WorkDir = /app` and the file lives at `/app/checksums.json` directly (no `.forge-output/` prefix — the build-output dir is flattened into `/app` by the `COPY . .` design). `VerifyBuildOutput` took the silent `os.IsNotExist` branch and integrity verification was effectively disabled in every Forge container.

Fix: add a fallback that picks the flattened container layout when the operator-side layout is absent. Both layouts now work end-to-end.

5. Redundant explicit `COPY`s in the template

`Dockerfile.tmpl:77-78` did `COPY compiled/skills/ /app/skills/` and `COPY compiled/prompt.txt /app/compiled/prompt.txt` after `COPY . .` had already copied them. The second was a literal no-op. Both targeted artifacts that are now gone (Bug 2), so these lines would have failed builds anyway.

Removed.

End-to-end repro

User's `aibuilderdemo` agent at `/Users/ibreakthecloud/Data/initializ/Workspace/agentdemo/aibuilderdemo`.

Pre-fix Post-fix
Files in `.forge-output/` 21 19 (lost: `compiled/prompt.txt`, `compiled/skills/skills.json`)
`compiled/prompt.txt` size 1199 lines (file deleted)
Files baked into `/app/` (incl. dead and operator-side) ~21 8
`COPY compiled/...` lines in Dockerfile 2 dead/redundant 0
`checksums.json` integrity verified at container startup silently skipped runs
Operator-side artifacts still on disk for `forge package` / `kubectl apply` yes yes

Files changed

  • `forge-skills/compiler/compiler.go` — dedup the prompt-builder via seen-bodies set
  • `forge-skills/compiler/compiler_issue147_test.go` — new, 3 tests (basic dedup, distinct bodies preserved, realistic 4-tool shape)
  • `forge-cli/build/skills_stage.go` — stop calling `WriteArtifacts` and stop registering the dead files
  • `forge-cli/build/skills_stage_test.go` — flipped from "must exist" to "must NOT exist"
  • `forge-cli/build/validate_stage.go` — drop the required-files check for the now-absent compiled artifacts
  • `forge-cli/build/dockerfile_stage.go` — expand `.dockerignore` with documented exclusions
  • `forge-cli/build/dockerfile_stage_issue147_test.go` — new, 3 tests (exclusions present, runtime files NOT excluded, dead `COPY`s gone from rendered Dockerfile)
  • `forge-cli/runtime/runner.go` — fallback to `WorkDir/checksums.json` when `WorkDir/.forge-output/checksums.json` doesn't exist
  • `forge-cli/runtime/verify_issue147_test.go` — new, 4 tests covering both layouts + end-to-end verify
  • `forge-cli/skills/writer.go` — deleted (dead)
  • `forge-cli/templates/Dockerfile.tmpl` — removed the `COPY compiled/skills/` and `COPY compiled/prompt.txt` lines

Test plan

  • `go test ./...` from `forge-skills/`, `forge-core/`, `forge-cli/` — all pass
  • `golangci-lint run` on touched packages — 0 issues
  • End-to-end build on `aibuilderdemo` — succeeds, output matches the table above
  • Generated Dockerfile inspected by hand — no dead `COPY`s, no recursive `Dockerfile` copy, runtime files all present
  • Generated `.dockerignore` excludes operator-side artifacts AND keeps runtime files

Out of scope (tracked separately)

  • `COPY --from=bins /usr/local/bin/ /usr/local/bin/` semantics. The current bins stage `apt install`s curl/git/jq (which land in `/usr/bin/`) but the application stage only copies `/usr/local/bin/` — those binaries may not survive into the runtime image. Will investigate + file separately.
  • `VerifyBuildOutput` mismatches only warn; should probably fail startup. Posture decision, not a bug fix.

… fix checksums path (closes #147)

Five interlocking bugs in the `forge build` output that compound to a
bloated runtime image and a silently-disabled integrity check:

1. **Prompt-builder body duplication.** A SKILL.md with N `## Tool:`
   sections parses into N SkillEntry values that all share the same
   Body. The compiler's prompt builder appended the body once per
   entry, so the bundled code-review-github skill (4 tools) shipped
   4× repeats and aibuilderdemo's compiled/prompt.txt was 1199 lines
   for what should be ~250. Dedup via a seen-bodies set; per-skill
   JSON entries still carry the full Body for SDK consumers.

2. **compiled/prompt.txt and compiled/skills/skills.json are dead.**
   The runtime never opens either file — runner.discoverSkillFiles()
   re-globs skills/SKILL.md on every startup. Stopped generating
   them; deleted forge-cli/skills/writer.go; pruned the matching
   COPY lines from the Dockerfile template and the validate stage's
   required-files check. External library consumers
   (forgecore.Compile) still receive the in-memory CompiledSkills
   struct directly.

3. **Dockerfile `COPY . .` leaked operator-side artifacts into
   /app.** Expanded .dockerignore to exclude k8s/, build-manifest.json,
   the entire compiled/ directory, the Dockerfile itself, the
   .dockerignore itself, and .local-bins/. The runtime image now
   contains only forge.yaml, guardrails.json, <channel>-config.yaml,
   agent.json, policy-scaffold.json, skills/, and checksums.json.

4. **checksums.json integrity check silently skipped in container.**
   The runner looked at <WorkDir>/.forge-output/checksums.json,
   which never exists inside a Forge container — the file lands at
   /app/checksums.json directly. Added a fallback that picks the
   flattened layout when the operator-side layout is absent.

5. **Removed redundant COPY lines from Dockerfile.tmpl** that would
   have failed after fix #2 anyway (the targets no longer exist).

End-to-end verified on the user's aibuilderdemo agent:
- Pre-fix: 21 files in .forge-output, compiled/prompt.txt 1199 lines,
  image contained k8s/ + build-manifest.json + security-audit.json +
  recursive Dockerfile + dead compiled/ artifacts.
- Post-fix: 19 files (compiled/prompt.txt + compiled/skills/skills.json
  removed), image contains only the 8 files the runtime opens,
  operator artifacts intact on disk for forge package / kubectl apply.

Tests pin all four invariants:
- TestCompile_BodyDeduplication_* (3 tests)
- TestSkillsStage_WithSkills updated to assert artifacts NOT created
- TestDockerignore_ExcludesOperatorSideArtifacts +
  TestDockerignore_KeepsRuntimeFiles +
  TestDockerfile_NoLongerCopiesDeadCompiledArtifacts
- TestResolveChecksumsDir_* (3 tests) +
  TestVerifyBuildOutput_ContainerLayoutWorksEndToEnd
…t} are not generated

The integration test still required the now-removed artifacts to
exist. Updated to assert their *absence* and to verify the in-memory
SkillsCount instead — matching the post-#147 contract.
@initializ-mk initializ-mk merged commit 2990092 into main Jun 11, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(build): dead compiled artifacts + leaky Dockerfile + broken checksums path

1 participant