feat: Django ASGI simulator + 2-leaf databus MQTT contract#12
Open
dotjae wants to merge 17 commits into
Open
Conversation
…-simulator Moves the full simulator stack into this repo: - sim/: Python simulator service with harness, tests, scenarios, and schedule - web/: nginx-served browser UI (fleet, operator, runs, schedule tabs) - broker/mosquitto-bridge.conf: WS-to-TCP bridge config for the browser MQTT client Adds ws-bridge, simulator, and web services to docker-compose.yml. Simulator defaults now reference the internal app/redis service names. Updates .gitignore to cover __pycache__, *.pyc, and .DS_Store.
Removes all original Python/Celery app files — this branch now contains only the simulator stack (web, simulator, ws-bridge, redis). Also removes app/celery_worker/celery_beat services from docker-compose.yml and restores DATABUS_BASE_URL default to host.docker.internal:8000.
Simulator was timing out connecting to host.docker.internal:1883 when no external databus broker is running. Default MQTT_HOST/PORT to the in-network ws-bridge:1884 so the stack runs standalone; the bridge still forwards to the host broker when one is available. Also tracks README.md.
Centralize every port/host the stack publishes or connects to into a root .env file (with .env.example template), so users can adapt the simulator to a databus deployment that uses different ports without editing config by hand. The three files with no native env-var support now render from the environment at container startup: - mosquitto bridge: rendered inline by the ws-bridge service command - nginx: nginx.conf.template via the image's envsubst entrypoint - browser: config.js.template -> window.SIM_CONFIG (MQTT_WS_PORT) Also wire SIM_CORS_ORIGINS in http_control.py (was hardcoded :8080), derive DATABUS_BASE_URL from DATABUS_HOST/DATABUS_HTTP_PORT in compose, gitignore .env, and document the knobs in a new README Configuration section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the simulator's Redis dependency so it stays fully independent of
databus's internal Redis (which need not be exposed). Run-lifecycle state is
now read over HTTP via GET /api/run/{run_id}/.
- databus_client: add get_run_state / get_run_hash (404 → None/{}, errors
logged and treated as not-found, mirroring the old RedisClient semantics)
- run_binder: poll databus.get_run_state; a 404 is "run lost" exactly as a
missing Redis key was (grace window + force-unbind unchanged)
- http_control: GET /run/{id} reads via databus.get_run_hash; drop redis_client
- simulator: drop the RedisClient context manager and wiring
- delete sim/redis_client.py and its tests; remove redis + fakeredis deps
- compose: drop REDIS_URL and the now-unused bundled redis service/volume
- tests: replace fakeredis fixtures with a fake databus client; add
get_run_state/get_run_hash unit tests
- docs: update README + .env.example to describe HTTP run-state polling
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bootable skeleton: sim_project (settings/urls/asgi with lifespan handler), simulator_app (runtime seam + empty domain/services/realtime/api packages), single-service docker-compose, Dockerfile (single daphne worker), healthz smoke test. No domain logic yet — Phase 2 fills the runtime. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…me, API) - domain: fleet (verbatim), kinematics (publish split into pure payload builder), control (transport-agnostic apply_control / apply_global_control) - services: databus_client, run_binder, scheduler ported ~verbatim - realtime: FleetConsumer (Channels), throttled broadcast replacing StatePublisher - runtime: lifespan-started tick_loop + binder + scheduler background tasks - api: DRF endpoints (/sim/*), control POST routes, /databus/ proxy - serve via uvicorn (daphne 4.x omits ASGI lifespan); daphne kept test-only - set DJANGO_SETTINGS_MODULE in asgi.py for bare uvicorn boot - 94 tests, 85% coverage; live WS+control smoke verified without databus Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- ws_client.js: one /ws/fleet/ socket exposes the old subscribe()/publish()
interface; inbound {type} routed to old topic subscribers, publish() shimmed
to POST /sim/control/* (replaces all MQTT.js browser usage)
- app.js rewired to ws_client; schedule payload (runs vs old entries) reconciled
- index.html served as Django template; assets via {% static %}; MQTT CDN +
config.js dropped
- WhiteNoise serves static under uvicorn/ASGI (no nginx)
- live boot verified: index + all static assets 200, no mqtt/config refs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- README: one-service architecture, uvicorn quickstart, full env-var table, WS/control/REST API reference, schedule format, operational invariants, manual E2E checklist, troubleshooting (all verified against code) - AGENTS.md: LLM-oriented module map, 'how do I' recipes, gotchas, invariants - fix(compose): schedule bind-mount now read-write from canonical data path so PUT /sim/schedule works Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- settings.py loads .env via python-dotenv (override=False; no-op in Docker) - derive MQTT_HOST/PORT from DATABUS_MQTT_*, DATABUS_BASE_URL from DATABUS_HOST/ DATABUS_HTTP_PORT so one .env drives both Docker and local runs - refresh .env.example to the single-service var set (drop stale nginx/ws-bridge ports); WEB_PORT moves the whole service - add scripts/dev.sh: sources .env, binds uvicorn to $WEB_PORT - README: .env works in both paths; ports table reorganized Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The standalone FastAPI simulator and nginx/MQTT web UI are fully replaced by the Django app (simulator_app/). Removed: - old sim modules: simulator, fleet, controller, state_publisher, databus_client, run_binder, scheduler, http_control (+ their tests) - old sim extras: schedule.yaml, Dockerfile, CONTRACTS/HANDOFF/AGENT_RUNBOOK docs - web/ (nginx UI) — replaced by simulator_app/static + templates Kept under sim/: the FSM test harness (harness/, scenarios/, test_fsm_graph, conftest, mappings.yaml, shapes.json, extract_shapes.py, pyproject/uv.lock). Verified: Django app 94 tests pass; harness runs via 'python -m harness'. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Superseded by the shipped implementation + README/AGENTS.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The harness was the last non-simulator code in the repo. Removed sim/ entirely (harness/, scenarios/, mappings/shapes data, its uv project). The simulator has its own data under simulator_app/data/ and does not depend on sim/. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The harness and old simulator/web code were removed; scrub the repo-layout tree and the out-of-scope harness note so the docs match the lean tree. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
databus stopped subscribing to transit/vehicle/+/progression and now recomputes occupancy_status server-side. Make the simulator a dumb sensor emitter that publishes only position and occupancy: - kinematics.build_vehicle_payloads: drop the progression leaf and the occupancy_status enum; keep raw occupancy_percentage. Remove the now-unused occupancy_status() helper and INCOMING_AT_RADIUS_M constant. Position unchanged; step_vehicle auto-dwell retained (observable via speed). - tests: assert position/occupancy only, no progression leaf, no occupancy_status on the wire. - docs (AGENTS.md, README.md): update topic diagrams, payload tables, and JSON examples to the 2-leaf contract; add docs/ contract spec.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the legacy databus-sim with a standalone Django ASGI simulator and aligns its telemetry with the current databus MQTT contract. The whole branch is a ground-up port; the most recent work migrates the wire format to the new 2-leaf contract.
Architecture (new)
sim_project+simulator_app) started via uvicorn lifespan.tick_loop(kinematics → telemetry),RunBinder.poll_loop,Scheduler.run_loop.fleet,kinematics,control), services (databus_client,run_binder,scheduler), realtime (Channels WebSocket broadcast/consumers), and a DRF-style API..envdrives ports/hosts for Docker and local; legacy Celery/Redis/notebook scaffolding removed.databus MQTT contract migration (latest commit)
The simulator is now a dumb sensor emitter — it publishes only what a real vehicle can sense:
progressionleaf entirely (databus no longer subscribes; stop status is computed server-side).occupancy_statusenum fromoccupancy(databus recomputes it fromoccupancy_percentageand discards edge values). Kept rawoccupancy_percentage.positionunchanged;step_vehicleauto-dwell retained (observable viaspeed).occupancy_status()helper andINCOMING_AT_RADIUS_M.Topics published:
transit/vehicle/<id>/{position,occupancy}.Test plan
pytest→ 96 passed.position/occupancyonly, noprogressionleaf, and nooccupancy_statuson the wire (test_kinematics.py,test_tick_loop.py).docs/databus-mqtt-contract-changes.md§5).Docs
AGENTS.md+README.mdupdated to the 2-leaf contract (topic diagrams, payload tables, JSON examples).docs/databus-mqtt-contract-changes.md(the contract spec driving the migration).