A lightweight agentic reinforcement learning framework for everyone
English · 简体中文
- 10/2026 📣📣📣 HarborRL released
HarborRL is a lightweight framework for training tool-using agents on verifiable, sandboxed tasks. It makes model serving, trajectory generation, tool execution, verifier rewards, and policy updates into a single reproducible pipeline.
HarborRL supports flexible combinations of Models × Algorithms × Tasks × Harnesses (MATH):
- Models: RL training using Qwen and GLM as base models.
- Algorithms: GRPO and DAPO for RL training.
- Tasks: RL training on 200+ Harbor-format tasks in isolated workers, including Terminal-Bench, SWE-Bench, Deep-SWE, and more.
- Harnesses: RL training with 40+ harness frameworks, including Claude Code, Codex, LangGraph, SWE-agent, and more.
If you want an AI agent to help deploy this repository and verify training, ask it to first read and strictly follow docs/agents/environment-check.md.
The practiced workflow is to give the agent this prompt, replacing the bracketed values with your topology:
Read docs/agents/environment-check.md and follow it exactly.
Target: [CPU development only | native split CPU-rollout/GPU-training deployment]
Config: <Path to config.yaml>, or none
Topology: controller=<Your IP address>:<Port Number>;
rollout workers=<Rollout worker IP address>:<Port Number>;
GPU machine=<GPU machine IP address>:<Port Number>
First run read-only checks only. Return the required structured report and tell
me the smallest next command that fixes one blocker. Do not install, download,
pull an image, mount storage, or start a service without my explicit consent.
After each approved change, rerun the relevant checks. Finish with harborrl
doctor and launch_native --dry-run for a native deployment.
Confirm the result yourself against this checklist:
- The agent reports one acceptance level:
cpu_development,deployment_preflight, ortraining; it does not promote a CPU-only check or dry run to a higher level. - For CPU development,
uv run harborrl --helpanduv run pytestboth pass. - For native deployment, every statically verifiable rollout-worker, GPU, and
cross-machine check is
ready;harborrl doctorhas no failed checks; anduv run harborrl train <Path to config.yaml> --dry-runresolves the launch plan. - Every
unknownitem names what is deferred and why. Live gateway and task-image checks can legitimately remain deferred before training. - The agent waits for explicit consent before starting real training.
Use Python 3.10 or newer. Runtime dependencies, development dependencies, and the harborrl command are locked by pyproject.toml and uv.lock. The CPU development install does not require CUDA, Docker, model weights, Slime, Megatron-LM, or SGLang.
Install uv 0.9 or newer, then synchronize the locked CPU development environment:
uv sync --extra devExpected result: uv reads uv.lock, creates .venv/, and installs HarborRL with pytest and ruff. The command does not install CUDA kernels, Docker, model weights, Slime, Megatron-LM, or SGLang.
Run commands through the locked environment:
uv run harborrl --help
uv run pytestExpected result: uv executes the installed entry point and tests without activating the virtual environment manually. train accepts command composition or a YAML file; doctor validates a YAML deployment.
For native training, install the pinned backend stack in a writable location:
bash scripts/bootstrap_backends.sh <Path to HarborRL backends>
. <Path to HarborRL backends>/harborrl-backend.envAfter installation, choose one of the two configuration styles:
# Compose the four experiment dimensions.
uv run harborrl train -model qwen3-8B -harness claude-code -task terminal-bench -alg grpo
# Load a complete, reproducible experiment definition.
uv run harborrl train <Path to config.yaml>For a bootstrap-aware YAML launch, use:
bash scripts/launch_native.sh --config <Path to config.yaml> --backends <Path to HarborRL backends>The command style is a shorthand over config.yaml in the working directory (or the file selected with --config) and selects the four experiment dimensions. The YAML style is the complete launch definition: it also pins checkpoints, workers, serving settings, GPU topology, sampling limits, and output locations. Start from examples/native/train_qwen_native.yaml and keep site-specific files outside the repository.
Training is configured with one YAML file that has a fixed section and field set. Relative paths resolve against the YAML file, and repeated --set section.field=value overrides are parsed as YAML values and validated again. Start site-specific configurations from examples/native/train_qwen_native.yaml.
The configuration has three broad parts: execution is fixed to harbor_job; harbor, gateway, and deployment describe rollout workers, service endpoints, and GPU topology; tasks, harness, model, training, sampling, and output describe the experiment and its outputs.
HarborRL consumes Harbor-recognized native tasks directly and does not maintain private conversion formats for individual benchmarks. A typical task directory is:
task/
├── task.toml
├── instruction.md
├── environment/Dockerfile
└── tests/test.sh
Here, task.toml declares resources, the agent, and the verifier; instruction.md is the agent-facing instruction; environment/Dockerfile builds the isolated environment; and tests/test.sh verifies the result and writes reward.json or reward.txt. For training, tasks are locked into a catalog with their task ID, upstream revision, local path, content digest, and reward parsing rules; see examples/native/catalog.json.
The bundled runnable example is hello_world. Real training tasks are not repackaged by this repository: fetch Harbor-native datasets or tasks from Harbor Hub, and use the compatibility summary to understand validated coverage. After fetching a task root, validate it locally with harborrl hub inspect and lock it into your own catalog.
HarborRL is released under the MIT license.
If you use HarborRL for your research, please cite our technical report:
We thank Slime and Harbor for developing the foundational infrastructure for training and task execution.