feat: export cost and governance data as CSV/JSONFeat/csv json export token governance - #146
Open
RjyavardhanSingh wants to merge 2 commits into
Conversation
Add GET /v1/export endpoint and Dashboard "Export run data" expander for FinOps / chargeback reporting. Supports CSV (spreadsheets) and JSON (pipelines) with filters for time range, agent, status, and tenant. Includes _coerce_epoch() to prevent SQLite type-mismatch when date strings are passed as timestamps. 25 new tests cover store query, HTTP integration, and type coercion.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
targets issue #145
On-demand export of run-records and governance events from the control plane's
SQLite store as CSV (finance/spreadsheets) and JSON (pipelines). Two surfaces:
an
GET /v1/exportHTTP endpoint and a Streamlit Dashboard "Export" expander.Why
The plane holds all cost and governance data but there was no way to pull it into
a spreadsheet, a cost tool, or an archive. This is the FinOps view (who spent what)
and the compliance audit trail (halts, MUTATE/INJECT decisions, which policy fired).
Changes
Backend —
src/tokenops/control/store.py_coerce_epoch()helper: defensively converts epoch floats, ISO datestrings,
datetime.date, anddatetimeobjects to epoch floats. Prevents theSQLite type-mismatch bug where
REAL >= TEXTsilently returns zero rows.Store.export_runs(): filtered query over therunstable supportingfrom_at,to_at,agent,status,tenant, andlimit(capped at 10 000).Ordered by
started_at DESC.Backend —
src/tokenops/control/http.pyGET /v1/exporton the FastAPI app.from_at,to_at,agent,status,tenant,format(json|csv),limit.run_id,agent,status,cost_micros,steps,started_at,ended_at,duration_s,dims,halt_reason,detector,governance_events.X-Total-Countheader; CSV includesContent-Disposition: attachment.Backend —
src/tokenops/server/app.pymount_export(app, store).UI —
src/tokenops/ui/views/dashboard.py_date_to_epoch()converts Streamlitdate_inputvalues to epoch floats(start-of-day for
from_at, end-of-day forto_at).Tests —
tests/test_export.pyStore.export_runs()unit tests (filters, limit, empty, governanceevents),
GET /v1/exportintegration tests (JSON, CSV, empty, limit cap),_coerce_epoch()unit tests (None, float, int, date string, datetime string,date object, datetime object), and regression tests for string-date filtering.
Also in this branch
src/tokenops/control/pricing.py: added Gemini model rates.src/tokenops/providers/openai.py: addedOPENAI_BASE_URLpassthrough.Acceptance criteria
tenantfilter).GET /v1/exportroute only.How to test
make install && make run(plane on :7700, dashboard on :8501)make demo(agents register runs intokenops.db)localhost:8501→ scroll to "Export run data" → pick filters → Download CSVcurl http://localhost:7700/v1/export?format=csvResults
CSV

JSON
[ { "run_id": "run-5", "agent": "researcher", "status": "error", "parent_run": null, "parent_span": null, "halt_reason": null, "detector": null, "cost_micros": 12000, "steps": 6, "started_at": 1789762067.1769028, "ended_at": 1789762077.1769028, "task": null, "dims": { "tenant": "globex" }, "governance_events": [ { "kind": "halt", "reason": "model not found", "policy": "pre_call_worst_case" } ] }, { "run_id": "run-4", "agent": "editor", "status": "completed", "parent_run": null, "parent_span": null, "halt_reason": null, "detector": null, "cost_micros": 3000, "steps": 3, "started_at": 1789761767.1769006, "ended_at": 1789761787.1769009, "task": null, "dims": { "tenant": "acme" }, "governance_events": [] }, { "run_id": "run-3", "agent": "writer", "status": "completed", "parent_run": null, "parent_span": null, "halt_reason": null, "detector": null, "cost_micros": 8000, "steps": 5, "started_at": 1789760867.1768982, "ended_at": 1789760917.1768985, "task": null, "dims": { "tenant": "globex" }, "governance_events": [] }, { "run_id": "run-1", "agent": "researcher", "status": "completed", "parent_run": null, "parent_span": null, "halt_reason": null, "detector": null, "cost_micros": 15000, "steps": 12, "started_at": 1789759067.1768858, "ended_at": 1789759167.1768873, "task": null, "dims": { "tenant": "acme" }, "governance_events": [ { "kind": "mutate", "reason": "cap applied", "policy": "pre_call_worst_case" } ] }, { "run_id": "run-2", "agent": "researcher", "status": "halted", "parent_run": null, "parent_span": null, "halt_reason": null, "detector": null, "cost_micros": 50000, "steps": 8, "started_at": 1789755467.1768959, "ended_at": 1789755567.1768959, "task": null, "dims": { "tenant": "acme" }, "governance_events": [ { "kind": "halt", "reason": "budget exceeded", "policy": "cost_budget" } ] } ]