Skip to content

Registries for agents and model providers; openai-compatible provider - #6

Open
mattteufel-e2b wants to merge 1 commit into
mainfrom
refactor/pluggable-agents-providers
Open

mattteufel-e2b wants to merge 1 commit into
mainfrom
refactor/pluggable-agents-providers

Conversation

@mattteufel-e2b

Copy link
Copy Markdown
Collaborator

Why

The adapter should be easy to point at any model endpoint and, later, any agent upstream supports, without threading vendor names through the runtime. Before this PR, codex/openhands/fireworks/bedrock appeared as literals in RunOptions, _model_config, _summary_wire_api, _agent_model_id, _require_runnable_policy, _benchmark_result, _already_completed, argparse choices, preflight output, and the bundle builder.

Design

Two registries, each a frozen dataclass plus a dict:

  • agents.AgentHarness (src/cybergym_e2b/agents.py): upstream --agent name; wire_api (responses | chat-completions); installs_tooling_at_runtime (drives the allowlist gate); bundle_scripts (files shipped into scripts/, e.g. the hardened Codex installer); model_id (OpenHands' openai/ prefix); optional turn_state (Codex trajectory reader that separates an interrupted turn from a graded failure).
  • providers.ModelProvider (src/cybergym_e2b/providers.py): endpoint(EndpointRequest) -> Endpoint(host, base_url) and key_env names. Fireworks and Bedrock Mantle are fixed entries. openai-compatible takes --model-base-url and --model-key-env, so OpenAI, vLLM, OpenRouter, or an internal gateway work with no code change. The credential still never enters the sandbox; E2B's proxy injects it on the endpoint host only.

Everything downstream reads the registries: CLI --agent/--provider choices, preflight's provider_keys_present, _require_runnable_policy, bundle contents, _benchmark_result, batch reuse, and the experiment fingerprint (bundle_scripts_sha256 map replaces remote_install_codex_sha256).

Adding an agent upstream already runs is one AGENTS entry. Adding a fixed provider is one PROVIDERS entry plus its host in the two packaged policies (only needed for allowlist modes). README has a section on this.

Flag changes (0.4.0)

  • --remote-install-codex PATH--bundle-script NAME=PATH (repeatable, agent-agnostic)
  • --bedrock-region--model-region
  • new: --provider openai-compatible --model-base-url URL --model-key-env NAME
  • api.openai.com added to model_hosts in both packaged policies

Verification

  • pytest -q: 62 passed (55 + 7 in tests/test_agents_providers.py)
  • ruff check, ruff format --check, check_public_tree.py, actionlint, uv build, twine check pass
  • Base template rebuilt from this branch resolves to the same recipe tag as the 2026-09-08 validation (cybergym-e2e-dind:recipe-3b3d5f9d827b70b3); templates.py is untouched
  • Live smoke/run from this branch are the next step and will be reported separately

🤖 Generated with Claude Code

…le provider

The runtime, CLI, bundle builder, network gate, and result parser no longer
know about Codex, OpenHands, Fireworks, or Bedrock by name. Two small
registries carry that knowledge:

- agents.AgentHarness: upstream --agent name, wire API, whether the agent
  installs tooling at runtime, scripts to ship in the bundle, model-ID
  rewrite, optional trajectory reader for interrupted turns.
- providers.ModelProvider: endpoint(host, base_url) from (wire API, region,
  base URL, key env) plus the credential env names.

CLI choices, preflight key reporting, the allowlist gate, bundle contents,
and the experiment fingerprint all derive from the registries.

New provider `openai-compatible` takes --model-base-url and --model-key-env
so any HTTPS OpenAI-compatible endpoint (OpenAI, vLLM, OpenRouter, ...) works
with no code change; api.openai.com added to both packaged policies.

Flag changes (0.4.0): --remote-install-codex -> --bundle-script NAME=PATH
(repeatable, agent-agnostic); --bedrock-region -> --model-region.
Fingerprint field remote_install_codex_sha256 -> bundle_scripts_sha256 map.

Tests 55 -> 62; new tests/test_agents_providers.py covers registry-derived
CLI choices, the custom provider, credential lookup, per-agent bundle
scripts, fingerprint coverage, and the Codex trajectory reader.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@cla-bot cla-bot Bot added the cla-signed label Sep 15, 2026
@mattteufel-e2b

Copy link
Copy Markdown
Collaborator Author

Live evidence from this branch (2026-09-15):

  • templates build-base / build-ffmpeg: resolved to cybergym-e2e-dind:recipe-3b3d5f9d827b70b3 and cybergym-e2e-ffmpeg:recipe-e8010b6665ef25fa, the same tags as the 2026-09-08 validation.
  • preflight ok for: smoke curl, run curl (bedrock), smoke ffmpeg, run curl (openai-compatible pointed at Fireworks). Tag-to-build receipts verified against E2B; HF access 302.
  • smoke curl/arvo_66012: completed, exit 0, benchmark.status=passed, outcome=ground_truth_passed, 5.4 min, sandbox iqbvaannfvx44956xi9cf. 316 resource samples, no observation_errors; the post-workload path from PR Land PR #2 on main, remove dead code, rewrite README for customers #3 (monitor stop → summary → metrics → collect) ran cleanly for the first time on a live sandbox.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant