Skip to content

test(opencode): validate native memory embeddings and replacement persistence - #113

Draft
xnoto wants to merge 7 commits into
mainfrom
test/opencode-memory-runtime
Draft

xnoto wants to merge 7 commits into
mainfrom
test/opencode-memory-runtime

Conversation

@xnoto

@xnoto xnoto commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Credential-free runtime probe for the owner-approved memory pilot, using the exact published OpenCode image and opencode-mem@2.26.0. The local embedding gate failed; pilot activation is blocked. No chart, runtime image, cluster, Secret, provider, or production configuration is changed.

The probe initializes the stock OpenCode/plugin, writes one synthetic memory through the real local embedding path, searches it, and—only if successful—replaces the container and verifies the same memory on the retained isolated home volume. No TEI, hosted embeddings, paid inference, or production data.

Issue: N/A — explicit owner rollout-and-validation approval. Maintainer: makeitworkcloud / Steven Welch (xnoto).

Type of change

  • CI / reusable workflow
  • Documentation

Validation

  • Required runtime check passes — FAILED, not waived or masked.
  • Existing chart hygiene/static tests and change detection passed in run 35521375954; package/GitOps jobs skipped for this .github-only change.
  • No generated centrally maintained files hand-edited.

Runtime run 35521375973, head a34d8ef1646470c5d24275660c72d314f63b617b, verified Linux/amd64 and image digest sha256:ecc3bf96ee55dad226d9cde50d79aaa8a1215c47860c0fcdc71570461bf438b8, initialized the pinned plugin via /config, and reached the memory API. The actual embedding write returned success:false because the present ONNX binding could not load shared library ld-linux-x86-64.so.2.

No memory was successfully created. Search and replacement-persistence assertions were therefore not reached; do not claim persistence success. Earlier runs failed in the test fixture's volume permissions before OpenCode started and were inconclusive; the scoped ownership repair allowed the real native-loading failure to surface.

Adversarial/security reviews ADVANCE and QA COVERED for the test design; DevOps DESIGN review ADVANCE. These do not override the red runtime result. No local execution was performed.

Impact and rollout

Test-owned files only under .github; no chart bump, package publication, or live mutation. Probe containers use the pinned image, explicit non-root HOME/XDG settings, read-only root, dropped capabilities, no-new-privileges, no published ports, and UUID-scoped disposable data. Root preparation is restricted to CHOWN on that fresh test volume. The fixed API marker is synthetic, not a production credential; no credential files are read. Resources are cleaned up on exit.

The fixture enables only the plugin's localhost test API and disables capture. Even a future PASS would prove only local embeddings, API memory storage/search, and same-volume container replacement—not LLM capture, provider entitlement, Kubernetes Service/PVC behavior, production Secret wiring, or disaster recovery. Transitive npm/model downloads are non-hermetic and explicitly documented.

Keep draft and do not activate the pilot. Resolve the proven runtime loader incompatibility through a separately reviewed remedy and rerun this gate. No remote/provider/image fallback, TEI, libc shim, custom image, registration, exec, sync, or restart was introduced. Existing production remains untouched by this PR.

Safety and secrets

  • No production secrets, decrypted values, state, kubeconfigs, tokens, or private endpoints; API marker is synthetic test data
  • No local OpenTofu operations run or claimed
  • Failure classification, cleanup, and rollout boundaries described above

AI-assisted; primary and independent specialist scrutiny applied.

Synthetic, bounded CI-only gate on test/opencode-memory-runtime: runs the
pinned published opencode image hardened and read-only with the
opencode-mem@2.26.0 plugin, proves the local embedding path really executes
(POST /api/memories + GET /api/search q/tag contracts verified at the pinned
releases), and proves memory id/content survive a same-volume container
replacement. No chart changes, no cluster exposure, no credentials; token
value is a public synthetic fixture. Exits nonzero on any failure with
native-load vs transport/auth classification.
Hardening and correctness fixes, all script-side (workflow unchanged):
explicit HOME/XDG env (stock image has no uid-1000 passwd home);
world-readable synthetic config dir for the ro bind; explicit
serve --hostname/--port; least-privilege non-recursive volume chown
container; global permission {"*":"deny"}; bounded CalledProcessError
handling; filtered container-log evidence actually printed with
auth/token/env lines excluded; classification priority
auth/transport/install over explicit-phrase native suspicion
(no bare onnxruntime/libsql/musl words, suspected-not-proof wording);
finite >=0.6 similarity gate rejecting bool/NaN; /config allow asserts
the pinned plugin spec (string or [spec, options]); per-call timeouts
capped to remaining budget with deadline-independent cleanup; unique
per-run container/volume names from uuid.
…esses

Pre-PR fixes from primary and security review. The runtime containers now
override the entrypoint with /bin/sh -ec to copy the two synthetic configs
from a read-only /config-source bind into the writable HOME named volume
(matching the production chart's writable config seeding; a read-only
config bind differs from production and risks false negatives), then exec
the pinned opencode serve. All non-cleanup subprocesses are capped to the
remaining global deadline (docker pull and prepare included) with float
timeouts and a 1s wget -T floor; cleanup and failure-path evidence
gathering keep fixed timeouts via enforce_deadline=False. Workflow paths
add opencode-server/** per infra review. New test-only README documents
invocation, pins, non-hermetic transitive/model resolution, gate-decision
scope (branch protection untouched), and the no-paid-inference/no-creds
posture.
Single error-handling patch: the per-container docker inspect + logs
evidence loop in the failure path is wrapped in try/except Fail so a
diagnostic failure cannot replace the original exception or skip
classification. A bounded diagnostics-unavailable marker is printed and
the original failure detail is preserved for classification. Cleanup in
finally is unchanged; no config or workflow changes.
QA/DevOps pre-PR additions. The probe now pulls the pinned digest for an
explicit linux/amd64 platform, inspects only public identity fields
(.Id, .RepoDigests, .Architecture, .Os -- never Env), asserts linux+amd64
and that the pinned manifest digest is represented in RepoDigests, prints
the public identity fields for exact runtime evidence, and fails with
clear identity diagnostics and no fallback on mismatch; the prepare and
both runtime containers pin the same platform. Existing assertions are
unchanged. README now states the CI probe enables the plugin web API and
disables capture, that a PASS covers only local embedding, API
storage/retrieval, and same-volume container replacement (not LLM
capture/provider, Kubernetes Service/PVC, or production Secret wiring),
records maintainer/owner and the owner-approved rollout validation
request, and notes alternatives (static Helm tests, upstream Node tests
not on the exact image) with no custom image or new service. Workflow
unchanged; checkout pin and primary-verified version comment retained.
@github-actions

github-actions Bot commented Sep 20, 2026

Copy link
Copy Markdown

Chart CI passed

Repository hygiene and Helm validation passed. View the workflow run.

Diagnostic-only fix for the run 35520640681 blind spot; runtime assumptions
unchanged. wait_json now inspects the container lifecycle state (.State.Status,
.State.ExitCode, .State.Error only -- never Env or files) whenever a probe
wget fails and fails fast with that metadata plus the public CONTAINER_INIT
fixture when the container is exited/dead instead of retrying for the full
window; a still-running container keeps its warmup window. Failure evidence
now prints the bounded tail (last 40 non-blank lines, 1500 chars) of the
synthetic-only container logs excluding credential-shaped lines -- plain
shell errors, usage text, ENOENT, and permission-denied lines are kept
rather than filtered by narrow error keywords. Docker state and logs are
captured in the except path before the finally-block cleanup removes the
containers. No runtime/config/workflow changes and no cause fix yet.
Run 35521152338 evidence: mkdir /home/opencode/.config/ failed with
Permission denied before opencode exec -- a test-fixture volume ownership
issue, not native compatibility. The first mount of the fresh named volume
copies the image's pre-created root-owned subdirectories up into it, so the
previous chown of the volume root alone was insufficient. The throwaway
preparer (same pinned image, root, CAP_CHOWN only, no new capability) now
mkdir -p's the HOME/XDG skeleton (.config/opencode, .cache, .local/share,
.local/state) and chown -R 1000:1000's it, wholly scoped to the fresh
UUID-named disposable volume, and prints safe stat metadata (uid:gid mode
path for the volume root and .config/opencode only) before/after with no
file contents. CONTAINER_INIT gains a uid-1000 writability preflight
(test -w ... || exit 9 with an explicit stderr marker) before any copy or
exec, so fixture errors fail fast. Runtime stays non-root with no grants;
this bounded fsGroup-aligned repair assumes and proves nothing about
embeddings. No chart, workflow, or other config changes.
@xnoto

xnoto commented Sep 20, 2026

Copy link
Copy Markdown
Contributor Author

Runtime result: rollout blocked

Probe run 35521375973 at head a34d8ef1646470c5d24275660c72d314f63b617b reached real local embedding execution and failed:

ONNX runtime failed to load despite pinned binding being present … Error loading shared library ld-linux-x86-64.so.2: No such file or directory

Verified sequence: exact published image digest and Linux/amd64 identity passed; disposable HOME ownership passed; OpenCode /config initialized the pinned plugin; plugin /api/health responded; POST /api/memories then returned success:false from native ONNX loading. Health alone was not accepted. No memory was successfully created, so vector search and replacement persistence were not validated.

Earlier runs failed in the test fixture before OpenCode started; those were inconclusive and were corrected without changing the runtime image. The final run establishes the missing-loader failure on the actual intended image, not a generic network or auth timeout.

Existing chart checks passed separately in run 35521375954; packaging/GitOps jobs skipped because this is a .github-only change. Runtime check intentionally remains red. No expected-failure masking, workflow dispatch, merge, publication, pilot registration, cluster exec, sync, or restart was performed. CI test containers/volume were cleaned up.

Keep this PR draft as reproducible gate evidence. Do not activate the staged pilot until a separately reviewed runtime remedy passes this test. No TEI, hosted embedding fallback, custom image, or libc shim has been introduced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant