Post-release A100 stress run + researcher deployment on inference-platform

[amrit110]

Why

Everything needed for a researcher-facing deployment is staged in VectorInstitute/inference-platform on branch test/tabicl-gke-onboard: the research overlay (helm-charts/ray-llm-app/values/tabctx-research-values.yaml: both models, one A100, spillover, 64Gi host RAM per the measured host-OOM boundary), the probe battery (tests/gke-tabicl-test/probe_*.py), and a parameterized onboard.sh. ROADMAP item 2 sequences a dedicated ephemeral A100 stress run after the release, followed by the production inference-platform deployment for Vector researchers.

What changed since that plan

• v0.10.0 was validated on the standing tabctx-gateway GKE cluster (one L4) with benchmarks/probe_deployment.py (24/24). That probe handles TABCTX_REQUIRE_TENANT=true and a renamed session header, which the inference-platform probes do not -- they send neither a tenant header nor the renamed session header, so they 401 against that gateway.
• Redeploying a same-named wheel requires restarting the head pod, not just the worker: Ray caches pip runtime envs by package spec per node, and the Serve controller ships the ingress-wrapped class from the head's cached env by value (CHANGELOG 0.10.0 "Validated").

Tasks

• Run the A100 stress battery (probe_stress_v09.py, probe_soak.py, bench_concurrency.py) against a v0.10.0 wheel on an ephemeral A100 cluster via onboard.sh (KEEP_ALIVE=false; it tears down on exit -- real A100 cost). Compare against benchmarks/baselines/v0.9.1-research-1replica.json.
• Bump the wheel filename pinned in both overlays (/wheels/tabctx-0.9.1-py3-none-any.whl) to 0.10.0.
• Either port the inference-platform probes to TabctxClient(tenant_id=..., session_header=...) or retire them in favour of benchmarks/probe_deployment.py plus the stress/soak scripts.
• Document the head-pod restart requirement in onboard.sh / the overlay comments, or make the wheel filename carry a build id so the pip spec changes per deploy.
• Production deployment for researchers on inference-platform.

Pointers

ROADMAP.md item 2, benchmarks/README.md, benchmarks/baselines/.