Hosted snapshot: drop Docker (A1), rabbitmq mgmt present-not-enabled (A3), guarantee /run/futureagi + /work layout (P5) - #66
Open
azain-commits wants to merge 36 commits into
Conversation
… into feat/hosted-harness-e2e-runtime-azain # Conflicts: # src/fi/alk/harness/job.py # tests/test_harness_architecture.py
Phase 1 of the hosted execution guest: the typed world surface (state/put/change/drop/call/query) scenario code receives, backed by a per-world logical database — baseline-measured state caps, read-only sub-handles, SQL statement guards, typed errors, and offline + docker test lanes. call() raises until the http_tool shim wire format is pinned (recorded contract defect). Known defects tracked in the implementation ledger. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…contract check(): empty/whitespace strings count as held; any non-string, non-False value is a broken check, not an agent failure. ready(): bare False and any non-string value are broken ready code; setup stays advisory. Bare state() excludes unmeasured tables the same way it excludes over-cap ones, so nothing the agent does during a call can change which tables raise; the typed refusal stays on the explicit selector. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
Typed models for the hosted bundle manifest: runtime kinds with the evidence seam, managed/source processes, seed stores with per-protocol sentinels, capability wiring, and the byte-exact inputs-digest helper. Strategy-engine pairing keys on the capability protocol; the manifest re-gains the resolved-secret sweep, scoped so v2's own secret_purposes field is not mistaken for a credential. Riders: runnable-check predicate parity, broken-value message truncation, cap-exclusion coverage. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
The complete §2e checklist as typed preflight errors — digest and filesystem verification scoped to the real directory, secret material scanning, unknown-field translation, placeholder vocabulary, secret purposes both directions, depends_on graph, engine catalog, reserved names, inputs-digest verification, and the compose gate ahead of all file-level checks so the dispositive code wins. seal_bundle_v2() is the normative §2d digest producer, pinned by a hand-computed byte vector. Model layer gains the user-assignment, engine-agreement, and reserved-configuration-name rules. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
The execution half of the provisioner: per-world/job-shared port allocation, closed-vocabulary env rendering, copy-based build trees, and process spawn — every customer process dropped to its declared unprivileged user, the customer checkout copied with symlinks preserved and escaping links rejected, process names constrained so a name can never escape its build/scratch directory. depends_on waits on all of a dependency's readiness probes; healthy() only ever demotes. Managed engines start with atomically-created credential files and are probed with real queries. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…ool, transport clients The guest's three reporting channels per outbound-channels v1.3 and seam contract v1.12: the capabilities loader with its closed rejection vocabulary, byte-exact canonicalization with fixed digest vectors, the crash-safe sequenced spool (fsync-first, watermark clamp, corruption degrade-not-wedge, process registry), and the events/results/artifacts transport clients with the closed HTTP error map, fence latching, deadline-bounded retries, and redaction before emit. Four cold review rounds; survivors recorded as known defects in .claude/harness-alk/reports/outbound-review-r4.md (fork-path lock pairing, refusal-log volume, stale v1.11 pin). Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…eceipts The scheduler that drains a job's scenarios across W provisioned worlds per seam contract v1.12: the world pool with §4.5b-serialized provider calls, degrade-tolerant start, pool-owned world_unhealthy emission and the §5.4 zero-ready grace rule; per-scenario reset-lease-run-grade with the v3.4 receipt vocabulary, retry-once on another world, exact skipped synthesis, and cancel awareness; scenario phases dispatched on a dedicated executor so a leaked thread can never starve the provider. Three cold review rounds, closed by a 23-mutant mutation run; survivors (S1–S10, KD-1–6) recorded in .claude/harness-alk/reports/p9-review-r3.md. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…, conformance The provisioner's second half per seam contract v1.12: seed/migration application from the verified bundle under the store's own user, secrets loaded and deleted before any customer process with purposes from job.json, baseline freeze for all three engines (explicit redis SAVE, node-name-free rabbitmq mnesia path), per-world clone and reset with polled readiness promotion, the two-world conformance gate that never raises, the healthy() port method (demote-only), dead shared-engine respawn, reverse-order termination with engine-appropriate signals, and typed §2f failure codes on every path. Plus the P5-round-2 rider cleanups (bundle_v2, process_preflight) and the run_world_check exception-message truncation in checks.py. Five cold review rounds; the round-5 gate closed on mutation evidence. Known defects (rabbitmq datadir_copy baselines are world-0-only — disclosed in-code with the definitions-export follow-up recorded; sentinel read-only guard is defense-in-depth; promote-poll cost) are consolidated in .claude/harness-alk/reports/p6-review-r5.md. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…suite's mutation blind spots Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
… latch, close serialization Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…ode contract, W from job parallelism Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…dent Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…scenario validation, single domain map Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…A3), guarantee /run/futureagi + /work dirs (P5) - A1: remove docker-ce/compose + DOCKER_HOST; process-based runtime per spine §0 - A3: strip enabled_plugins so rabbitmq management is present but disabled at boot - P5: create /run/futureagi (0700 svc-control), /work + /work/artifacts (0755 svc-control)
… into feat/hosted-harness-e2e-runtime-azain
…o feat/hosted-harness-e2e-runtime-azain # Conflicts: # src/fi/alk/harness/hosted_entrypoint.py
… scenarios Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…liver Wire register_with_platform() into BundleScenarioSource.build(): one provision + one begin call on the attempt's scenarios endpoint (single POST, body-level operation field — the route the platform actually serves), platform-assigned ids matched to documents BY scenario_key with hard guards (missing/unknown/duplicate key -> typed failure, never a partial assignment), and the full key set sent on begin. Scenario result receipts now carry the platform-assigned scenario_id, so they deliver instead of being dropped for an empty id. The P12 pinning test is split honestly: a delivery proof plus a drop-guard preservation test for the one path that can still see an unregistered scenario. Also guards empty scenario_key before the network call, keeping that deterministic content defect in the environment domain instead of letting the platform's 400 reclassify it as platform_sync. ScenariosClient path defaults collapse to "" (provision/begin suffixes would 404 against the real router). Cold review: CLEARS — 7/7 mutations killed, 823 passed / 0 failed x2. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
… PR dry-run) - scripts/daytona-snapshot.sh: env-driven publish from Dockerfile.hosted via the Daytona SDK; caches an SDK venv; prints name+digest to pin ALK_DAYTONA_SNAPSHOT [_DIGEST]. SNAP_DRY_RUN validates the Dockerfile/build-context without a key. - .github/workflows/daytona-snapshot.yml: workflow_dispatch publishes a named snapshot (DAYTONA_API_KEY secret); PR/push to runtime paths runs the dry run.
…o feat/hosted-harness-e2e-runtime-azain
Daytona forces the sandbox to a fixed non-root user (svc-control) and ignores os_user overrides, so the guest cannot setuid/chown to the bundle's declared svc-agent/svc-tools/svc-data users. Run every process uniformly as svc-control: - ProcessRuntimeProvider gains a construction-time require_declared_user (default True); provision() falls back to it when the caller passes nothing. - hosted_entrypoint build_provider default constructs the provider with a null user_resolver + require_declared_user=False. - Dockerfile.hosted bakes uv (astral) for source-process builds (uv sync). Bundles may still declare svc-* users (model validates); not enforced at runtime.
The gateway delivers /work/source read-only (chmod a-w) for integrity and copytree preserves that mode, so a source process's build workspace was read-only and 'python -m venv .venv' failed with EPERM. Restore owner-write on the copied build tree before running build_commands.
…data Completes the hosted voice pipeline so a graded call runs in Daytona: - hosted_call_runner.VoiceCallRunner: drives the proven voice case (run/sdk_voice via run.call.place_the_call) against the provisioned agent (livekit_agent_name from runtime.metadata) and reads the agent's HARNESS_TOOL_TRACE via _semantic_calls. - hosted_entrypoint: build_call_runner default -> VoiceCallRunner; peek_secret_map() populates caller-lane provider creds into os.environ before the provisioner deletes secrets.json (the voice case reads them from the env). - process_runtime: _control_metadata surfaces livekit_agent_name + tool_trace_path on EnvironmentRuntime.metadata per world (rendered from the control service's env).
CallSummary.started_at/ended_at must be RFC3339 UTC 'Z' millisecond strings (is_valid_rfc3339_millis); datetime.isoformat() renders +00:00/microseconds and fails receipt validation. Use outbound.format_rfc3339_millis.
CallRunner: ADC materialization+cleanup, dynamic voice_case, Vertex pre-check, P=1 root-cause preservation, named dispatch via engine. Provisioner: per-world livekit_agent_name+tool_trace_path metadata. Entrypoint: caller-lane creds population, 2-arg build_call_runner, VoiceCallRunner default. Snapshot script: prefer python3.12/3.11. Bundle authoring script persisted. 214+20 ALK tests green.
Replace the typed not-wired call seam with a real LiveKit runner when the job's connector is livekit: pre-dial validation of credentials and dispatch identity, an in-process SimulationSpec drive of the voice engine, deterministic per-scenario room naming, and a runner-owned call budget (config voice_call_timeout_seconds, default 300). Evidence follows the bundle-declared seam: tool_trace reads the world's database endpoint after the call; http_tool has no guest-side capture surface today and stays a typed stop rather than an invented proxy. Transcripts and recordings upload through the artifacts channel before the receipt references them, and artifact-level refusals degrade to null. Failure semantics are three distinct paths, each pinned by tests: an agent that never joins retires the world; any post-dial failure keeps its measured timing on the receipt; a silent zero-turn agent surfaces as missing evidence rather than a graded verdict or a false infrastructure failure. Voice credentials are captured from the job's secrets before the provisioner deletes them and exported once for the engine's own reads; values never appear in logs and stay inside the redaction set. Cold reviews: round 1 found 1 High (test-honesty) + 2 Medium, all fixed; round 2 CLEARS — 18 mutation runs killed or validated, 863 passed twice. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…rompt.md sdk_voice requires this for the caller spec; the hosted lane reads it from the bundle (matching the local lane's simulator_prompt.md), falling back to the scenario instruction.
Merges feat/hosted-execution-guest (ef4c5a7): CallRunnerImpl using SimulationRunner directly, purpose-filtered secret peek, connector-gated factory, per-scenario room naming, artifact upload. Retires hosted_call_runner.py; keeps uniform-user provisioner, per-world metadata, build-tree-writable, snapshot script fixes. 203 tests green.
Accept LiveKit URL from config or target-provider secret. Support Google API-key or Vertex ADC caller auth; materialize service-account JSON mode-0600. Derive simulator LLM/STT/TTS providers and models from job config/env with generic defaults. No target-agent/domain-specific runtime logic.
HarnessJob now requires runtime isolation; the test fixture predated that and would fail construction. Adds RuntimeRequirements(DEDICATED_VM, cpu_units=1).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The hosted Daytona snapshot (
Dockerfile.hosted+scripts/daytona-snapshot.shpublisher/CI) and the in-sandbox guest runtime it ships for the hosted harness e2e. Aligned to spinehosted-execution-seams§0. Net-new-and-owned here is the snapshot; the guest runtime is merged from base +feat/hosted-execution-guest(see convergence note).Snapshot (
Dockerfile.hosted)DOCKER_HOST; the guest runtime is process-based ("no container runtime in the sandbox")..ezfiles present butenabled_pluginsstripped — available and disabled at boot; the guest enables it and pins its port (amqp_port + 10000)./run/futureagi(0700 svc-control),/work+/work/artifacts(0755 svc-control) — matches the gateway writing control files there and the guest reading them.uvbaked in;scripts/daytona-snapshot.shpublishes the snapshot from the Dockerfile (env-driven name/resources,SNAP_DRY_RUN, prints name/state/digest +.envpin) with a manual-publish CI hook.Guest runtime shipped (merged)
World handle over per-world Postgres,
environment-bundle.v2models + seal, in-sandbox provisioner (build/spawn/readiness/seed/baseline/worlds/reset), hosted scheduler (world pool, scenario loop, retry, receipts), outbound channels (capabilities, canonicalization, spool), scenario-source adapter, platform receipt registration, and the real-voiceCallRunner(provider-neutral credentials — Vertex/Deepgram, tool_trace capture, agent-name metadata). Retired the stopgaphosted_sink.py.Verification
alk-hosted-v17live:svc-controluid 2000;/run/futureagi700,/work(/artifacts)755;docker→ NO_DOCKER; rabbitmq mgmt plugins present + all disabled, noenabled_plugins./run/futureagi/capabilities.json→ world builds over the agent's live Postgres + tools-API → real voice calls placed through self-hosted EU LiveKit via coturn → events + graded receipts post, artifacts upload201, sandbox verified-deleted at terminal.Convergence note (read before merging)
This branch merged the guest at an older point. The guest/CallRunner has since advanced on
feat/hosted-execution-guestand the integration branchtest/manual-e2e, which now carry the run-levelbuild/result/logfinalize uploads — the fix for themanifest_required_artifact_missing422 that currently blocks terminal ack. Recommend converging the ALK-guest side on the integration branch and rebuilding the shared snapshot from it; the snapshot pieces here (Dockerfile.hosted,daytona-snapshot.sh) are what's uniquely owned by this PR.