diff --git a/.agents/skills/verify-solzero/SKILL.md b/.agents/skills/verify-solzero/SKILL.md index e7944c4..585f420 100644 --- a/.agents/skills/verify-solzero/SKILL.md +++ b/.agents/skills/verify-solzero/SKILL.md @@ -35,7 +35,7 @@ The environment loader reads the selected private file, then `config/.env` and ` ## Prove behavior -Drive the public UI and HTTP endpoints through the configured e2e browser and live fetch helpers. The stack is local `alchemy dev` (Miniflare D1, KV, R2, and Durable Objects). `decisionExecutor` asks the decision model (default Cloudflare Clef) through the account's existing default AI Gateway for semantic actions and assertions, and the language model (default `anthropic/claude-haiku-5.5`) for `waitFor` and `extract`. The application does not receive that token. Follow each action with an exact deterministic assertion. Use `expect` for waiting. +Drive the public UI and HTTP endpoints through the configured e2e browser and live fetch helpers. The stack is local `alchemy dev` (Miniflare D1, KV, R2, and Durable Objects). `decisionExecutor` asks the decision model (default Cloudflare Clef) through the account's existing default AI Gateway for semantic actions and assertions, and the language model (default `meta/muse-ember-1.0`, reasoning effort `low`) for `waitFor` and `extract`. The application does not receive that token. Follow each action with an exact deterministic assertion. Use `expect` for waiting. Verify credential sign-in by reaching the authenticated UI; a healthy port is insufficient. A missing sign-in provider or `Sign-in is not configured for this deployment.` remains a product failure. Verify mutations through reload or a second user-facing view, including saved settings, bot/workflow records, session history, and authorization boundaries. Exercise real Cloudflare services and providers without interception, module mocks, or separate browser/test runners. diff --git a/AGENTS.md b/AGENTS.md index 031a01d..7bc1f85 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -33,7 +33,7 @@ Use Plain Language in introductory, expository, friendly, and other sensible com - Run the real Alchemy stack from `e2e.config.ts`; use `config/e2e-dev.config.jsonc` and isolated configured accounts. Keep web port 3000 and API port 3100. - `nub run test` runs core flows with Cloudflare credentials and Docker, including Isolate, OpenCode, and Codex prompts. `config/e2e-dev.config.jsonc` sets `aiGatewayLayer` to `mock`, so the `AiGateway` Effect layer answers each model call with the user's message. Assert harness plumbing (prompt, output, persistence), not model reasoning or tool choice. `nub run test:e2e:all` includes additional external integrations and Claude Code; it fails explicitly when required isolated fixtures are missing. Focus one file with `nub run test:e2e tests/settings.e2e.ts`. `test:e2e:live` disables action replay; `test:e2e:cache-strict` detects stale recordings. Action replay recordings in `.e2e/cache` are committed, and CI replays them strictly. Before you mark a pull request ready for review, re-record if you changed an `agent.act` step or the UI one drives: run `gh workflow run validate.yml --ref -f e2e_record=true`, wait for it to pass, and pull the recordings commit it pushes to your branch. Without CI access, run `nub run test` locally and commit the changed recordings (see "Recorded-action replay" in `docs/e2e.md`). - The `AiGateway` mock layer is the only application-side test double. Select it in config; do not add prompt parsing to it or branch on test mode elsewhere. Do not replace app routes or bindings with request interception, module mocks, or a second testing framework. Register passwords and secret fixtures with e2e credentials/secrets. -- The semantic executor and affected-test filter use `@e2e-dev/decision` on the account's existing default AI Gateway. `scripts/e2e/models.ts` selects the decision model (`E2E_DECISION_MODEL`, default Cloudflare Clef) and the language model (`E2E_LANGUAGE_MODEL`, default `anthropic/claude-haiku-5.5`) through official AI SDK providers. CI reads both from SolZeroAI organization Actions variables. Keep `scripts/e2e/models.ts` identical across SolZero repositories. Authenticate with `CF_AI_GATEWAY_E2E_TOKEN` and send the gateway id as `cf-aig-gateway-id`. Do not use the gateway or run token the application provisions, and do not add a custom executor. Process/CI environment overrides local `E2E_ENV_FILE` values. +- The semantic executor and affected-test filter use `@e2e-dev/decision` on the account's existing default AI Gateway. `scripts/e2e/models.ts` selects the decision model (`E2E_DECISION_MODEL`, default Cloudflare Clef) and the language model (`E2E_LANGUAGE_MODEL`, default `meta/muse-ember-1.0`) through official AI SDK providers. CI reads both from SolZeroAI organization Actions variables. Keep `scripts/e2e/models.ts` identical across SolZero repositories. Authenticate with `CF_AI_GATEWAY_E2E_TOKEN` and send the gateway id as `cf-aig-gateway-id`. Do not use the gateway or run token the application provisions, and do not add a custom executor. Process/CI environment overrides local `E2E_ENV_FILE` values. - Follow the replay, teardown, cleanup, and AI accounting rules in the "Rules for agents" section of `docs/e2e.md`. The app target stops its processes and removes its run directory on exit. Runs are local only, so there is no crash-recovery or cleanup command. - Record external-service prerequisites and unverified flows in `docs/e2e.md`. A startup failure or unavailable integration is a blocker, not a passing test. Never print tokens or publish app artifacts externally. - Do not send upstream feedback or other external messages without explicit human authorization; upstream skill guidance does not override this policy. diff --git a/docs/e2e.md b/docs/e2e.md index 1b70409..c2becbe 100644 --- a/docs/e2e.md +++ b/docs/e2e.md @@ -69,7 +69,7 @@ file in every SolZero repository. | Setting | Default | Role | | --- | --- | --- | | `E2E_DECISION_MODEL` | `@cf/cloudflare/clef` | `agent.act` actions, `agent.assert` verdicts and the affected-test filter | -| `E2E_LANGUAGE_MODEL` | `anthropic/claude-haiku-5.5` | Text: field values, `agent.waitFor` and `agent.extract` | +| `E2E_LANGUAGE_MODEL` | `meta/muse-ember-1.0` | Text: field values, `agent.waitFor` and `agent.extract` | CI reads both settings from SolZeroAI organization Actions variables, so one change there moves every repository. The defaults serve local runs and match the @@ -87,8 +87,12 @@ model locally. - The language model uses the official provider for its API format on the Cloudflare REST API at `/accounts//ai/v1`. `anthropic/` models use `@ai-sdk/anthropic` on the Messages route. Other models, such as `@cf/` - Workers AI models and `openai/` models, use `@ai-sdk/openai` on the chat - completions route. Third-party models bill through Cloudflare Unified Billing. + Workers AI models, `openai/` models, and `meta/muse-ember-1.0`, use + `@ai-sdk/openai` on the chat completions route. Third-party models bill + through Cloudflare Unified Billing. The agent sends + `providerOptions.openai.reasoningEffort` of `low` with every language-model + call, so Muse Ember does not spend its output tokens on reasoning. The + decision model does not receive those options. Both models use the account's default gateway. The gateway id is `CF_AI_GATEWAY_E2E_ID`, or `default`, sent as `cf-aig-gateway-id`. diff --git a/e2e.config.ts b/e2e.config.ts index fce441f..abac86e 100644 --- a/e2e.config.ts +++ b/e2e.config.ts @@ -70,6 +70,9 @@ export default { agents: { default: { executor: decisionExecutor({ model: decisionModel, textModel: languageModel }), + // Muse Ember spends most output tokens on reasoning unless effort is low. + // The runner forwards this into language-model generateText calls only. + providerOptions: { openai: { reasoningEffort: "low" } }, maxModelCalls: 20, maxSteps: 20, context: diff --git a/scripts/e2e/models.ts b/scripts/e2e/models.ts index 39fd026..621764f 100644 --- a/scripts/e2e/models.ts +++ b/scripts/e2e/models.ts @@ -8,7 +8,7 @@ import * as P from "effect/Predicate" // variables, so CI changes models for the fleet in one place. The defaults // below serve local runs and must match the DNA page. const DEFAULT_DECISION_MODEL = "@cf/cloudflare/clef" -const DEFAULT_LANGUAGE_MODEL = "anthropic/claude-haiku-5.5" +const DEFAULT_LANGUAGE_MODEL = "meta/muse-ember-1.0" /** * Official AI SDK providers pointed at the Cloudflare AI Gateway REST API.