docs: add self-hosting system configuration page #695
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,143 @@ | ||
| --- | ||
| title: "System configuration" | ||
| description: "Configure the LLM gateway, PeerDB replication, and Temporal workers for your self-hosted instance." | ||
| --- | ||
|
|
||
| ## In this page | ||
|
|
||
| A few parts of the stack are configured outside `.env`: the LLM gateway needs a `config.yaml` listing its providers, PeerDB needs its replication mirrors running, and Temporal workers can be tuned for throughput. This page covers all three. Set your secrets and provider keys in [Environment Variables](/docs/self-hosting/configuration/environment) first, since the config here references them. | ||
|
|
||
| ## LLM Gateway | ||
|
|
||
| The gateway is a Go proxy that routes every model call the platform makes. It reads a `config.yaml` that lists which providers it may use and which models each exposes. | ||
|
|
||
| <Warning> | ||
| Model calls fail until this file exists. The gateway ships with `config.example.yaml` (OpenAI enabled) but **not** a live `config.yaml`. You create one in the steps below. | ||
| </Warning> | ||
|
|
||
| <Steps> | ||
| <Step title="Copy the Example Config"> | ||
| ```bash | ||
| cp agentcc-gateway/config.example.yaml \ | ||
| agentcc-gateway/config.yaml | ||
| ``` | ||
| </Step> | ||
|
|
||
| <Step title="Add Providers and Keys"> | ||
| Edit `config.yaml`: uncomment the providers you want and reference their keys with `${VAR}` interpolation. Set the matching keys (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, …) in `.env`. See the provider examples below. | ||
| </Step> | ||
|
|
||
| <Step title="Mount the Config and Restart"> | ||
| Point the gateway volume at your `config.yaml` in `docker-compose.yml`: | ||
|
|
||
| ```yaml | ||
| volumes: | ||
| - ./agentcc-gateway/config.yaml:/app/config.yaml:ro | ||
| ``` | ||
|
|
||
| ```bash | ||
| docker compose up -d --force-recreate gateway | ||
| ``` | ||
| </Step> | ||
| </Steps> | ||
|
|
||
| <Warning> | ||
| `config.yaml` is gitignored and holds live API keys. Treat it as a secret. Never commit it. | ||
| </Warning> | ||
|
|
||
| ### Provider Examples | ||
|
|
||
| <Tabs> | ||
| <Tab title="OpenAI / Anthropic / Gemini"> | ||
| ```yaml | ||
| providers: | ||
| openai: | ||
| api_key: "${OPENAI_API_KEY}" | ||
| api_format: "openai" | ||
|
Check warning on line 56 in src/pages/docs/self-hosting/configuration/system.mdx
|
||
| models: [gpt-4o, gpt-4o-mini] | ||
|
|
||
| anthropic: | ||
| api_key: "${ANTHROPIC_API_KEY}" | ||
| api_format: "anthropic" | ||
| models: [claude-opus-4-5, claude-sonnet-4-5] | ||
|
|
||
| gemini: | ||
| api_key: "${GOOGLE_API_KEY}" | ||
| api_format: "gemini" | ||
|
Check warning on line 66 in src/pages/docs/self-hosting/configuration/system.mdx
|
||
| models: [gemini-2.0-flash, gemini-1.5-pro] | ||
| ``` | ||
| </Tab> | ||
| <Tab title="AWS Bedrock"> | ||
| ```yaml | ||
| providers: | ||
| bedrock: | ||
| api_key: "${AWS_SECRET_ACCESS_KEY}" | ||
| api_format: "bedrock" | ||
| region: "${AWS_REGION}" | ||
| access_key: "${AWS_ACCESS_KEY_ID}" | ||
| models: [anthropic.claude-3-5-sonnet-20241022-v2:0] | ||
| ``` | ||
| </Tab> | ||
| <Tab title="Vertex AI"> | ||
| ```yaml | ||
| providers: | ||
| vertex: | ||
| base_url: "https://us-central1-aiplatform.googleapis.com" | ||
| api_key: "${GOOGLE_ACCESS_TOKEN}" | ||
| api_format: "gemini" | ||
|
Check warning on line 87 in src/pages/docs/self-hosting/configuration/system.mdx
|
||
| headers: | ||
| x-gcp-project: "${GCP_PROJECT_ID}" | ||
| x-gcp-location: "us-central1" | ||
| models: [gemini-2.0-flash-001] | ||
| ``` | ||
|
|
||
| Vertex uses a Bearer token, not a static API key. Rotate `GOOGLE_ACCESS_TOKEN` with a sidecar that calls `gcloud auth print-access-token`. | ||
| </Tab> | ||
| </Tabs> | ||
|
|
||
| For routing rules, rate limits, caching, and the full config reference, see [Agent Command Center → Self-Hosted](/docs/command-center/deployment/self-hosted). | ||
|
|
||
| ## PeerDB Replication | ||
|
|
||
| PeerDB continuously replicates Postgres tables into ClickHouse (change-data-capture) so trace and eval analytics stay fast. It runs on its own. The only thing you typically touch is a first-boot timing fix. | ||
|
|
||
| <Warning> | ||
| **First-boot timing.** `peerdb-init` runs the moment the stack starts, sometimes before Django has finished its migrations. If mirrors show "not started" in the PeerDB UI, re-run init once the backend is up: | ||
|
|
||
| ```bash | ||
| docker compose logs -f backend # wait for "Application startup complete" | ||
| docker compose run --rm peerdb-init bash /setup.sh # re-run init | ||
| ``` | ||
| </Warning> | ||
|
|
||
| Verify at [http://localhost:3001](http://localhost:3001). Mirrors should move to `running` within seconds. Re-run the same init command after any upgrade that changes replicated tables. | ||
|
|
||
| ## Temporal Workers | ||
|
|
||
| Temporal runs the platform's background jobs and evaluation pipelines. How those jobs are distributed across workers depends on one flag. | ||
|
|
||
| **All-queue (default).** One worker polls every task queue. Controlled by `TEMPORAL_ALL_QUEUES=true` in `.env`. This is the right setup for most self-hosted deployments. | ||
|
|
||
| **Per-queue (dev overlay).** Six dedicated workers, one per queue, brought up by the [dev overlay](/docs/self-hosting/install#dev-overlay): | ||
|
Contributor
There was a problem hiding this comment. Install comes with the new sidebar, add it there |
||
|
|
||
| | Service | Queue | Typical concurrency | | ||
| |---|---|---| | ||
| | `worker-default` | `default` | 100 | | ||
| | `worker-tasks-s` | `tasks_s` | 200 | | ||
| | `worker-tasks-l` | `tasks_l` | 50 | | ||
| | `worker-tasks-xl` | `tasks_xl` | 10 | | ||
| | `worker-trace-ingestion` | `trace_ingestion` | 100 | | ||
| | `worker-agent-compass` | `agent_compass` | 50 | | ||
|
|
||
| Tune throughput with `TEMPORAL_MAX_CONCURRENT_ACTIVITIES` and `TEMPORAL_MAX_CONCURRENT_WORKFLOW_TASKS` in `.env`. The Temporal UI is available in dev mode at [http://localhost:8085](http://localhost:8085). | ||
|
|
||
| ## Dive Deeper | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card title="Production Checklist" icon="shield" href="/docs/self-hosting/production/checklist"> | ||
|
Contributor
There was a problem hiding this comment. Checklist is part of the new Production group, add it with the sidebar |
||
| Hardening, backups, and monitoring before going live. | ||
| </Card> | ||
| <Card title="Troubleshooting" icon="wrench" href="/docs/self-hosting/troubleshooting"> | ||
| Fixes for gateway, PeerDB, and Temporal errors. | ||
| </Card> | ||
| </CardGroup> | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No coming soon. Implement the new self-hosting sidebar and Support (plus the other missing pages) come with it: