You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The README carries the full story today (why the library exists, the serving contracts, the architecture figure, GPU-validated numbers, the pre-computed-contexts assessment) and it is getting long. As external adoption starts, a docs site with proper navigation, an API reference generated from docstrings, and a place for deployment guides will matter more than one very long README.
Scope
Pick the tooling consistent with sibling Vector projects (MkDocs Material or Sphinx + Furo), built and published from CI on pushes to main (GitHub Pages).
Split the README into: Getting started (engine in-process, serving, client), Concepts (contexts, admission, cache tiers, affinity, tenancy -- the architecture figure lives here), Deployment (Ray Serve on GKE, env vars from serve/factory.py's docstring, multi-replica, GPU fraction, spill), Operations (/readyz, /v1/tabctx/limits, benchmarks/probe_deployment.py, the head-pod redeploy gotcha), and Benchmarks (the validated numbers + baselines).
API reference from docstrings for tabctx, tabctx.client, tabctx.backends.base (the protocol to implement for a new model), tabctx.memory.
Keep the README as a short landing page that links into the site; keep CHANGELOG/ROADMAP where they are.
Why
The README carries the full story today (why the library exists, the serving contracts, the architecture figure, GPU-validated numbers, the pre-computed-contexts assessment) and it is getting long. As external adoption starts, a docs site with proper navigation, an API reference generated from docstrings, and a place for deployment guides will matter more than one very long README.
Scope
main(GitHub Pages).serve/factory.py's docstring, multi-replica, GPU fraction, spill), Operations (/readyz,/v1/tabctx/limits,benchmarks/probe_deployment.py, the head-pod redeploy gotcha), and Benchmarks (the validated numbers + baselines).tabctx,tabctx.client,tabctx.backends.base(the protocol to implement for a new model),tabctx.memory.Pointers
README.md,
docs/figures/architecture.svg(editable source), ROADMAP.md item 4.