A self-contained stack that exercises Observer's probe sources against real local services. One docker compose up, one observer apply, and you have a live dashboard with sixteen metrics across thirteen source types, including one that intentionally flips status every minute so you can watch Observer react. CloudWatch and ICMP are included as optional, commented-out blocks (see below).

Nothing is mocked. The agent genuinely connects to containers, runs SQL, completes gRPC health checks, receives OTLP pushes, and queries Loki and Elasticsearch. Credentials and raw query responses never leave your machine; the agent pushes readings and computed verdicts to Observer Cloud.

What's inside

Container	Purpose	Observer source
observer-agent	The Observer agent	everything below
prometheus	Self-scraping Prometheus	prometheus (PromQL), http
postgres, mysql, redis, mongo	Seeded databases	database × 4 kinds, tcp
grpc	agnhost gRPC health server	grpc
websocket	Echo server	websocket round-trip
loki + loki-logger	Log store + producer	loki (LogQL)
elasticsearch + es-seeder	Search + document producer	elasticsearch (aggs)
otlp-pusher	Gauge pusher	otlp (push)
(none)	The agent's own machine	host (CPU)
(none)	You	manual
(none, optional)	Your AWS account	cloudwatch

Plus external checks with no container at all: dns (resolve via 1.1.1.1) and tls_cert (certificate expiry).

icmp is not enabled: the published agent image is distroless and has no ping binary, so ICMP would only report no_data. The commented demo-icmp-postgres block in observer.yaml explains how to enable it with a custom agent image.

Prerequisites

• Docker with Compose
• An Observer account (free tier is fine)
• The Observer CLI
• ~3 GB free RAM (Elasticsearch is the hungry one)

Quickstart

1. Create an agent in the Observer console: Agents → New agent, name it exactly demo-agent, copy the obs_live_* key (it is shown once).

2. Start the stack:

cp .env.example .env      # paste your AGENT_KEY
docker compose up -d

3. Apply the config (create an API key with write:config scope under API keys in the console):

export OBSERVER_API_KEY=obs_pub_...
observer apply -f observer.yaml

Within a minute or two the console fills in: sixteen metrics, three services, two SLOs. Watch Queue depth (OTLP push): its value follows the clock's seconds, so it crosses healthy → degraded → unhealthy roughly once a minute. That's not a bug; that's the demo.

4. Optional: publish a status page. The pages: block at the bottom of observer.yaml ships commented out. Uncomment it, replace pick-your-own-unique-subdomain with a globally unique subdomain, and re-run observer apply. Your page goes live at https://<subdomain>.use.observer.

5. Optional: CloudWatch. uncomment the AWS blocks in .env, docker-compose.yml, and observer.yaml. Read-only credentials (cloudwatch:GetMetricData) suffice; the agent uses the standard AWS credential chain and never sends credentials to Observer Cloud.

Things worth trying

• Kill a database: docker compose stop mysql. Watch the MySQL metric go unhealthy, the TCP check stay scoped to postgres, and the Datastores service degrade.
• Kill the agent: docker compose stop observer-agent. After ~3× the push interval, Observer marks the metrics stale (no_data) rather than freezing the last green verdict.
• Flip the manual metric: the "Payment vendor" metric has no probe; set its status from the console to see operator-controlled components.
• Dry-run config changes: edit observer.yaml, then observer apply --dry-run -f observer.yaml shows the diff without touching anything. observer export --all round-trips your console state back into YAML.

Security notes

• Database DSNs live only in docker-compose.yml as environment variables on the agent. The config (observer.yaml) references them by name (DEMO_POSTGRES_DSN); Observer Cloud never sees a connection string.
• The database probes are read-only by construction: SELECT-only SQL parsing, a Redis command allowlist, and a MongoDB count-only spec.
• The OTLP receiver binds beyond loopback inside the compose network, so it requires a bearer token (OTLP_TOKEN).

Teardown

docker compose down -v

Then delete the config-managed objects from Observer:

: > empty.yaml && echo 'apiVersion: observer/v1' > empty.yaml
observer apply --prune -f empty.yaml

(or delete them from the console).

License

Apache-2.0. This repository is a read-only mirror published from the Observer monorepo; issues and PRs here are read, but changes land via the mirror.