Repository navigation
Preserve provider continuations and add source-scoped Harness streaming - #4
Merged
Merged
Conversation
Expose transport-independent turn observations and a stateful SSE encoder for Chat Completions, Responses, and Messages. Retain identity, content positions, opaque continuation, and reported token categories; reject reordered blocks and malformed terminal output. Exercise fragmented streams, withheld tools, size limits, ordering, and independent round-trip oracles with fixed property cases.
Add source-scoped Session discovery and turn admission through the Scheme owner, with versioned protocol records and Rust final validation. Project complete provider observations through the existing broker output owner. Prepare every frame in a bounded snapshot before the final delivery guard. Cover cancellation, source withdrawal, failed preparation, and rich provider SSE without adding a second sender or database. Move meter issuance to the Rust boundary.
Register source discovery, stream ordering, config recovery, and prepared-snapshot challenges with independent runtime properties and bounded QNT campaigns. Preserve competing designs that expose actual counterexamples, including the weaker snapshot oracle that missed an escaped prefix. Document the retained local results and open HTTP, linked Scheme, and Magpie integration work. Keep generated verification logs outside this change.
Replace unqualified unwrap calls in the new stream, Session, encoding and ingress fixtures with contextual failures. Keep assertions, fixed workloads and lint rules unchanged. The preceding PR's structured Rust receipt identified these test-only diagnostics. All targets of the five touched crates now pass local Clippy under the HTTP/Arrow profile; remote CI remains a separate gate.
Mount an optional loopback compatibility listener on the existing Tokio/Tonic runtime. Configure its one Session and distinct credential through gateway.http in weaver.toml. Read the live model catalog and submit exact bounded request bytes through the existing source lane and final Rust admission. Prepare SSE on the same response-owned broker projection before release; deliver it through the same native receipt cursor as RPC. Join HTTP with existing listener shutdown and tracked owner drain. No source management, delegated-identity downgrade, second router, or new database is exposed. Add actual HTTP/TLS two-turn, credential, retry, withdrawal and ABA tests; native after-enqueue revocation/cancellation tests; fixed-seed credential and fragmented-body properties; and the SysML/QNT challenge with competing-design counterexamples. Linked Scheme, complete HTTP protocol qualification and stock Magpie interoperability remain explicit acceptance gaps.
Build the server package binaries together in the existing preparation entry. Keep the meter fallback on the same release/default/Arrow graph rather than recompiling a no-default-features variant inside provider integration. Retain Cargo-reported executable selection and ambiguity checks, including another prepared binary in the artifact-stream control. No CI script, stage or timeout is added. Remote Go closure remains to be verified.
Exercise Chat Completions, Responses and Messages with independent raw SSE fixtures, exact follow-up bytes, retained opaque reasoning/signatures, fresh owners over the same ledger, completed retries, shared-budget exhaustion and fact withdrawal. Keep the TLS peer listening through rejection checks and fail on any extra connection. Include a real queued-connection negative control. Bind static SSE inputs and their Git formatting rule to currentness, and preserve the required final event delimiter and leading notices. Validation: 144 active SDK/broker/server library tests passed with four workers and the original 120-second watchdog; three existing process fixtures remain ignored. Server library/test Clippy, strict architecture/Workflow/currentness, Go input controls and Cargo artifact controls passed. The Go integration package compiled without invoking its native-build TestMain. Actor selection remains a labeled double; linked Scheme, HTTP slow readers/process crash and stock Magpie/config restoration remain open.
Add four finite native slow-reader/reset/withdrawal controls with SQL and wire oracles. Compose transport backlog with the existing receipt model and register bounded strong campaigns plus weak-design witnesses. Validation: 145 library tests passed, 3 existing ignored; 18 Quint checks and both strong safety campaigns passed; paused-prefix weak variant violated the invariant; architecture, Workflow and source-currentness passed.
Retain the first provider response write failure while continuing to reject extra connections. Add a real TLS disconnect followed by a forbidden TCP connection, and bind its oracle lifetime to SysML and architecture trace. Restoring early return failed the new test with connection refused. The candidate passed 146 library tests (3 existing ignored), server Clippy, and architecture/Workflow/currentness checks under unchanged budgets. The separately attempted Scheme suite reports a local module-prelude export mismatch and is not counted as passing.
Run Chat, Responses and Messages through real child HTTP/native owners. Kill the first owner with SIGKILL after a complete tool terminal; recover the next turn from the same ledger without another Session registration. Independently assert retained opaque/tool identities, exact upstream bytes, shared roots and durable claims/usage, including replay, exhaustion and withdrawal rejection. Validation: 148 library tests passed, three existing ignored; the opaque-checkpoint corruption control fails and the restored process test passes. SessionTurnAdmission has 21 named checks plus a passing fixed 256-trace campaign. Architecture/Workflow/currentness gates pass. Linked Scheme and in-flight send crash cuts remain open.
Gate the existing TLS fixture after exact request receipt and before every response byte. SIGKILL each Chat/Responses/Messages owner, then reopen the same ledger in a fresh process. Verify exact retries do not resend or rederive, changed bytes conflict, withdrawal blocks a fresh key, and one unknown dispatch retains its original reservation with no release or terminal usage. Reuse the child entry, process helpers, source lane, ledger and ProviderRpc model; do not add runtime retry machinery or CI stages. Validation: 149 SDK/broker/server library tests passed with four workers within the original 120-second command budget, 3 existing ignored; Clippy and architecture/Workflow/source-currentness passed; ProviderRpc has 91 passing named checks. No child processes remain. Finer partial-write/commit cuts, linked Scheme and stock Magpie remain open.
Normalize matching bounded Bearer and API-key encodings to one dedicated credential while rejecting conflicting values, repeated fields and other realms. Keep constant-time comparisons and a fixed public conflict explanation without exposing arbitrary internal errors. Add explicit real Magpie CLI/Provider/listener acceptance for Chat, Responses and Messages with two tool/result turns, original config ownership, live model discovery, credential isolation and exact ledger/upstream-byte commitments. Update SysML, Quint and independent raw-header properties around the observed compatibility failure rather than preserving the old physical-count assumption. Validation: 152 active library tests pass, four ordinary-library exclusions (three existing Go-driven fixtures plus the separately executed external consumer); real consumer passes three protocols. Server Clippy and architecture/Workflow/currentness pass. Twelve HttpOperation checks and the fixed strong campaign pass; two weak campaigns and a native equality-removal control fail as intended. Real linked Scheme and full managed Agent config recovery remain open.
Allocate a separate check-consumers suite in SysML and the reusable workflow. Checkout a verified public Magpie revision with the standard action, then prepare its original nogui CLI and identical Cargo test graph separately. Keep eight-minute build stage bounds distinct from the two-minute protocol gate and retain existing structured preparation/job receipts. Expose prepare-consumers and check-consumers through weaver-dev, reuse its runner and published SDK setup, and record the checkout action commit output without another pin verifier or installer. Preserve the existing Linux/macOS Scheme matrix. Validation: normal local check-consumers passes with prepared products; preparation/runtime graph contract, existing modular CI/bootstrap checks and architecture/Workflow/source-currentness pass. No local cold install or full preparation was rerun. Hosted job outcome remains pending.
Use SDK revision 7c4dd3d8 for Cargo and CI setup, retaining the existing AOT package API. Declare the complete module closure bound required by the SDK without changing required/forbidden module validation. Validation: archive adapter 4 tests; server/broker/LLM 152 passed, 4 ignored; real Magpie three-protocol consumer 1 passed; governance-build Clippy and workspace format pass. Real linked Scheme remains unqualified; document the separate POO preparation failure and keep the Lab source migration out of this batch.
Pin public Lab PR 34 revision 5702335b, use its tao3k package namespaces, and pass the admitted init std-make-spec directly to std/build-script. Remove the retired call-with-init-build-spec/source-registration wrapper while retaining Cargo source observations. Keep one direct Lab dependency and unchanged protocol schema identifiers. Register a production linked Session selector gate in the existing qualification Cargo graph and SysML trace: one VM, warm reuse, A-B-A rebind, and 64 fixed-seed fact/route cases. This new native test is pending, not claimed passed. Validation: architecture/workflow/source-currentness and formatting pass. Targeted POO, Yuanxu, Parser and ASCENT builds naturally exit; actual POO and Yuanxu VM load probes pass. Orgize final installed-interface check fails with Unexpected import (), reproduced in its stage-parser step; complete dependency/native image qualification remains open. No cold-install loop, CI budget change or upstream Gambit edit.
Consume Lab ModulesSource from its published PR revision 32af9e08; use native mount/from windows, the POO type facade, Prelude declarations and Build.defbuild-script.spec. Compile and clean through the existing init selection owner with standard Make callbacks so cross-package source views, input checks and output locks remain owned there. Replace retired requires/link injection with native imports of the existing typed semantic objects. Fix the selector local definition and cyclic test input slots; keep one production SchemeCompiler across the 64-case native property via an affine RefCell borrow. Read standard Make string/gxc rows without weakening the fixed inventory. Validation: original Weaver Scheme build passes; gateway source suite 24 cases and publication/contract suite 7 cases pass with full CASE/SUITE/HARNESS markers. Actual Rust-linked selector passes one test with 64 fixed-seed cases on the original Cargo graph (0.31s execution); architecture/workflow/source-currentness and format pass. Retain the full check-scheme FAIL: generated replacement property hits its original 60s deadline; A/B isolation now correctly exposes 31 targets against the unchanged 40 expectation. Native C generation/objects completed before a Rust closure compile error, corrected and verified by focused execution with existing products. Full HTTP/source/recovery qualification remains open; no cold-install retry loop, compiler patch or relaxed gate.
Register source/artifact cut, ABA and cleanup counterexamples in SysML and the existing Quint entrypoint. Retain the unresolved 31/40 runtime qualification gate. Validation: 11 Quint cases; 256 bounded traces, 30 steps, seed 293, four threads; architecture/workflow/currentness checks; Go devrunner tests.
Align Lab, ASCENT, Orgize, Parser, Yuanxu, POO and the Scheme SDK through their native package declarations. The Lab Core gitlink follows the same Yuanxu revision. Retain qualification exceptions in the test application and invalidate old linked-selector qualification until replayed against this cohort. Validation: architecture, workflow and source-currentness pass; two upstream compiler-carrier regression cases pass. New-cohort native preparation and full runtime acceptance remain pending; original 31/40 and source-property gates remain unchanged.
Record completed native package builds, the aggregate-budget Orgize timeout, unstarted Lab build and clean persistent source synchronization. ABI output after timeout is not a pass.
Delegate build flags and Make dispatch to STD while retaining Yuanxu source-view and output guards. Real provider and qualification applications consume the init namespace; their import closure selects init without build roots. Publish native Interface imports and declarations. Move the debug qualification application into Gateway Build and remove Rust’s manual helper compilation. Keep production refusal and archive exclusions for current and former test modules. Validation: actual program spec and compile --debug; hot compile has zero compile events; 31 source/contract cases and source integration including direct init owner; 4 Rust application-boundary tests; architecture/workflow/currentness. Complete Rust C ABI image, A/B foreign custody and hosted CI remain pending.
Centralize Lab leaf capabilities, preserve native identities and document functional Scheme style. Capture detached retained-selector results; challenge deferred getters and mutable aliases with Scheme regressions and bounded Quint campaigns.
Keep stock Magpie unchanged and expose a bounded Rust magpie feature for Weaver-owned configuration intent, CAS, immutable decisions and listener custody. Reuse the native persistence, transaction observer and tracked runtime owners. Model and challenge restart, ABA, stale receipts, split commit and activation races.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds bounded Chat, Responses and Messages streaming, source-scoped Session turns and a thin HTTP compatibility ingress to the existing Rust runtime. Scheme/POO/gerbil-ascent supplies advisory selection; Rust retains source/version validation, reservations, dispatch, guarded delivery and recovery through the existing ledger. The Harness keeps tool execution ownership.
The optional Rust
magpiefeature (off by default) now exposesmagpie::ConnectionControlfor Weaver-owned connection configuration. It reuses the existing ledger for durable preparation, exact operation binding, CAS, immutable decisions, retirement clocks and crash recovery. HTTP startup leases the current matching committed profile; live listeners must drain before another generation can publish. The request path keeps its immutable prepared profile without a configuration query. Stock Magpie remains unchanged and connects through its standard HTTP Provider; there is no fork, patch layer, internal Go import or upstream configuration writer. This is explicit publication and restart activation, not hot replacement of an active listener.The loopback HTTP listener binds one Session and a dedicated operation credential. Discovery reads current source facts. Complete terminals preserve tools and opaque continuation; conflicting credentials and foreign credential realms refuse. Delivery shares the RPC receipt cursor and bounded transport credit. Independent peer/SQL observations cover fragmentation, backpressure, cancellation, withdrawal and process-kill recovery.
The standard-consumer test uses Magpie's original nogui CLI and Provider path for two tool/result turns in all three protocols. Weaver owns protocol acceptance; downstream clients only need wiring checks. Consumer preparation uses the normal Go/Cargo package entries, separate preparation fences and four runtime workers. Exact dependency revisions belong to package declarations, lockfiles and the workflow rather than architecture-document inventories.
Scheme follows the GHQ Marlin C ABI Module pattern: Build declares targets, autoload groups and link options; Core publishes the compiler plan and activation stub; Rust links through the SDK archive API and supplies host carriers. Temporary callback roots use the host root service with unwind cleanup. Compiled requests use the canonical typed Gateway publication without acquiring init's Build projection or invoking Make. Development source admission and Rust's final source/fact/intent checks remain distinct.
Core now retains reached libraries from already-selected packages across ordinary facades, including when installed SSI prerequisites exist. Native source confinement, original target options and output ownership remain intact. The package cohort is aligned through its existing upstream PRs, including Parser’s Core dependency and the resulting Orgize/Lab references. Native installation correctly refused the intermediate contradictory graph; no conflict check was bypassed. The original fourteen foreign Lab targets remain mandatory; the complete selection now contains forty-nine targets because of the explicitly added interface and C ABI files. The existing cache fallback reuses products for the same compiler; no cold-install runner or extra CI Bash driver is added.
Weaver keeps the default package language: its package metadata declares only identity and the Lab dependency, while its interfaces explicitly import POO bindings. Core's existing source view now preserves an enrolled package's native prelude as compiler forms instead of a metadata-only library stamp. An independent public-API regression reproduces the same illegal import with the previous implementation and passes with the fix. The dependency cohort is aligned through the existing upstream PRs.
Generated evidence and logs are removed from source control and retained only under ignored local cache or CI artifacts. Source tests, models and drivers remain versioned; historical Absent declarations and archived-evidence exceptions are removed. CI also uses the standard pinned Cargo cache action to retain dependency compilation, including useful partial builds after failure, without adding a Bash runner or changing the budgets.
The Lab package now prepares selected Module products before its public native Governance, Session, loop-Governor and Cedar libraries, so their Core Module prerequisites exist before public-library expansion. Core's existing native import-closure API supplies private dependencies and STD Make compiles them. This repairs the clean-runner missing-library failure without changing Yuanxu resolution or bypassing its identity checks. SysML and the system design distinguish public libraries from Source Module declarations.
The execution-capacity fixture now establishes its precondition through first-frame receipts while retaining sixteen resident streams, excess-admission rejection, source revocation and canonical observation. The one-second delivery lease and twelve-second aggregate deadline remain unchanged. This occupancy scenario does not qualify sixteen simultaneous cold admissions; the observed burst-expiry performance gap remains separate. The scenario distinction is explicit in SysML and the performance design. The cache action's default incremental-disable behavior is overridden so Cargo continues to use incremental compilation.
Validation and remaining gates:
The unchanged dependency cohort passed the original Scheme source/unit and integration entries on Ubuntu, including native refusal controls. Candidate-tree and hosted OpenSysML architecture/trace and workflow checks pass. Go control/routing/gRPC and its capacity fixture pass in the hosted job.
Lab's normal incremental package build and original production-build check pass using installed dependencies. Its diagnostic spec coalesces target identities while preserving original selected-Module options. No dependency cold reinstall, additional branch, resolver or CI stage was added for public libraries.
Core's publication and testing-source-admission gates pass. All four existing input-context mutations fail their original assertions, with positive controls before and after and unchanged source bytes. The native-language regression has a failing previous-implementation control and a passing fixed implementation; the public-library repair makes no further Core change.
The preceding Ubuntu baseline's native build and runtime gates passed with four workers. The authorized ABI migration reused cached package sources, rebuilt incompatible compiler products in 15m1s, and saved the new compiler-specific dependency cache. macOS remains enabled; its Scheme unit/selection timeouts and service-build failure remain an independent open boundary outside this session's Ubuntu requirement.
The previously accepted original local module-selection entry passes within its unchanged 120-second fence: cold A 41.245s, cold B 16.092s, hot B zero compilation, A cleanup isolation, unchanged B digests and actual B Governance/Cedar/Gateway use. All 49 targets, five selected Modules and zero preparation scopes remain asserted. First actual export demand realizes one Scope; repeated demand realizes none. Cleanup uses the existing outer lock exactly once. Candidate-tree architecture and workflow checks pass.
Previous Ubuntu baseline CI is closed: Scheme/Cedar FFI/streaming recovery, Rust, Go, Policy (Cedar/Quint/Lean), architecture, benchmarks/gRPC measurements and Magpie interoperability all pass. That accepted baseline is 0faaa43; GitHub tested its expected PR merge checkout, and the structured job receipt binds both identities. All original gates and four-worker settings are retained. Scheme unit 36.037s, module-selection 57.193s, source integration 10.547s, Rust C ABI/service build 290.470s, rooted SDK runtime 40.321s and gRPC integration 118.132s all exit zero. No extra local cold A/B, dependency reinstall, build-entry/CI complexity or Yuanxu change was added during closure.
Current feature-batch local acceptance passes: eight ledger cases (four real process cuts, four CAS contenders, ABA/replay, rollback/lost acknowledgment, corrupted receipt and source-role refusal); 59 service functional cases including HTTP/2 backpressure, cancellation, withdrawal and process recovery; both ASP gates; typed control API and actual listener-drain checks; original stock-Magpie three-protocol Provider acceptance. Ten directed Quint tests and the four-thread 256-trace/30-step campaign at seed 293 pass. Feature-off compilation, strict Clippy, formatting, architecture/requirement trace and workflow checks pass. Preparation and timed runtime are separate. The initial concurrent-build observation timeout and initial style refusals remain failed evidence; no gate, lease, queue bound or observation minimum was relaxed. No local cold A/B or cold dependency installation was repeated.
Feature-batch Ubuntu CI is closed for 9693281: all seven Ubuntu jobs pass (Rust/PBT/ASP, stock Magpie, Go/gRPC, Scheme/Cedar FFI/recovery, Policy, architecture and benchmarks). Rust tests and strict all-target Clippy explicitly enable
weaver-server/magpie. Stock Magpie is checked out from its original locked upstream revision and the actualmagpie,arrowfour-worker Provider gate passes in 4.066s. The job receipts bind this PR head to GitHub's expected merge checkout and retain source-currentness before/after checks. Ubuntu Scheme unit 58.189s, module-selection 89.364s, native service build 338.400s, SDK runtime 58.523s and gRPC streaming 165.318s pass with exit zero under their original gates. Existing compatible dependency products were restored; the compiler-migration cleanup was skipped. No additional local cold A/B, cold installation, rerun, upstream modification or CI change was performed during this closure.Scheme qualification reuses one standard Build owner for cold A/B, hot B and cleanup; nested interpreter/build processes, manual build threads and temporary package-link processes are removed. The independent B runtime reuses installed dependencies and verifies its actual product paths. The selector returns its validated request commitment instead of copying a mutable optional cache field. Private derivation reports avoid repeated ownership admission while public contracts, detached inputs/results and all 32x8 property cases remain. The qualification image explicitly enables Scheme, LLM and Arrow; default Kubernetes coverage stays in ordinary Rust CI. CI YAML, dependency pins and installer remain unchanged. At the user's request the uncalibrated 480-second image-build cutoff is removed; build duration is measured under the existing enclosing suite deadline, independently of unchanged runtime gates.
Magpie owns its CLI configuration; modifying it is outside this integration. Unified installation, cold-burst latency and a new hot-path latency result remain separate open work.
Environment/owner fix CI passes all eight jobs, including both Scheme platforms. The macOS receipt binds head 9704270 to its expected merge checkout: unit tests 29.710s, complete A/B selection 59.134s, full image preparation 134.542s, SDK runtime qualification 24.332s and gRPC integration 57.543s all pass. The prior calibration measured a 385.262s complete macOS image build with a Cargo dependency cache hit; its following Go consumer changed PATH and caused another SDK build/link attempt. Cargo preparation and artifact discovery now normalize Go's tool prefix identically. The removed 480-second cutoff remains a calibration change, not proof of a universal cold-build bound. The earlier local Apple linker failure is retained separately.
Cache ownership is corrected in a separate batch. Official setup-go caches modules/build outputs using go/go.sum; the runner no longer overrides them with repository-local Go caches. Release consumers reuse the existing Go job Cargo cache identity with one writer; Rust tests retain their existing identity; architecture-only jobs skip Rust caching; Scheme caches retain workspace/AOT products. Existing compiler/OS/architecture/lockfile invalidation and Cargo source/feature/header checks remain authoritative. The audit found 11 Cargo caches using 8.315 GiB plus two Scheme caches using 1.267 GiB; no existing cache is manually deleted. Full runner regression and actionlint pass locally. Cache-batch CI for dea7f93 is pending; the earlier green run does not qualify this new batch.