-
Notifications
You must be signed in to change notification settings - Fork 3.1k
Improve provider usage tracking across Codex, Claude, and Grok #4141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
jetblk
wants to merge
22
commits into
pingdotgg:main
from
jetblk:agent/provider-usage-reliability-grok
Closed
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
3c4f59e
feat(mobile): allow overriding iOS team and bundle ID via env
jetblk f4a295c
Provider Usage in the T3 Code mobile app (#1)
jetblk 6a18d70
fix(mobile): stop Usage & Limits pull-to-refresh spinner from hanging
jetblk 3d53c89
Merge remote-tracking branch 'upstream/main'
jetblk 0558103
ci: fork nightly release + JetBlk app name
jetblk ca7e884
ci(nightly): install @t3tools/scripts deps for the version stamper
jetblk 58dcede
ci(nightly): ship a self-contained bundle via pnpm deploy
jetblk 4e10b2c
fix(nightly): ship the web client; add node runbook
jetblk e0d9651
docs: add the node systemd unit referenced by nodes.md
jetblk 5f1104d
docs(nodes): correct asset size; document web client + channel switcher
jetblk 8d15ed1
ci(nightly): skip docs-only builds; publish an immutable pin target
jetblk db3a315
docs(nodes): show both releases in the pipeline diagram
jetblk 53d27a1
feat(web): show provider usage (#2)
jetblk d106a28
fix(web): move provider usage into model picker (#3)
jetblk 94a3380
ci(nightly): run on a cron, not on every push to main (#4)
jetblk ebe84c9
Merge remote-tracking branch 'upstream/main'
jetblk cc9b037
fix(provider-usage): follow upstream CLAUDE_CONFIG_DIR semantics for …
jetblk 9197369
fix(provider-usage): headline session → weekly → credits in strict pr…
jetblk c45742b
Merge remote-tracking branch 'upstream/main'
jetblk 27e6a1f
Merge remote-tracking branch 'upstream/main'
jetblk 60d1917
ci(nightly): check for changes every 3 hours, matching upstream
jetblk 5af3fa8
Improve provider usage tracking
jetblk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,262 @@ | ||
| # Fork-local nightly for jetblk/t3code. | ||
| # | ||
| # Upstream's release.yml is unusable here: it needs pingdotgg's self-hosted | ||
| # `blacksmith-*` runners, their Clerk/Cloudflare/Apple/Azure secrets, and it | ||
| # publishes the `t3` npm package we don't own. This builds the one artifact our | ||
| # nodes actually consume — the self-contained `vp pack` server bundle — and | ||
| # publishes it as a rolling `nightly` GitHub Release. | ||
| # | ||
| # Nodes pull that asset on service restart (restart == update), mirroring the | ||
| # `npx t3@nightly` behaviour we replaced. The repo is public, so nodes need no | ||
| # auth to download it. | ||
| # | ||
| # Checks every 3 hours (mirroring upstream's release.yml cadence), and only | ||
| # builds when main actually moved (see check_changes) — a quiet window costs a | ||
| # ~10s gate check, not a build. Need a build sooner? Dispatch it manually; that | ||
| # path always builds. | ||
| name: Fork nightly | ||
|
|
||
| on: | ||
| # Every 3 hours — same cadence upstream's release.yml uses for its nightly | ||
| # channel. The offset minute is deliberate: crons on the hour queue behind | ||
| # GitHub's peak load and get delayed or dropped. | ||
| schedule: | ||
| - cron: "37 */3 * * *" | ||
| # Manual runs skip the change gate below and always build — this is the | ||
| # "I want my merge on a node now" path. | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| # Never cancel in flight: the publish step deletes the `nightly` release before | ||
| # recreating it, and a cancel inside that window would leave nodes with no | ||
| # release to pull at all (their ExecStartPre is best-effort, so they would | ||
| # silently keep running a stale build). Upstream's release.yml likewise has no | ||
| # concurrency group. | ||
| concurrency: | ||
| group: fork-nightly | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| check_changes: | ||
| name: Check for changes since last nightly | ||
| # Only the cron needs a gate; a manual dispatch is an explicit "build now". | ||
| if: github.event_name == 'schedule' | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| outputs: | ||
| has_changes: ${{ steps.check.outputs.has_changes }} | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| steps: | ||
| # No checkout on purpose — the compare API answers this in seconds, so a | ||
| # quiet night costs ~10s instead of a full install. Every failure path | ||
| # here builds rather than skips: a missed build is worse than a spare one. | ||
| - id: check | ||
| name: Compare HEAD to the last published nightly | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
|
|
||
| build() { echo "has_changes=true" >> "$GITHUB_OUTPUT"; exit 0; } | ||
|
|
||
| # The rolling release records the exact commit it was built from. | ||
| last="$(gh release view nightly --repo "$GITHUB_REPOSITORY" \ | ||
| --json targetCommitish -q .targetCommitish 2>/dev/null || true)" | ||
| if [ -z "$last" ]; then | ||
| echo "No nightly release to compare against — building." | ||
| build | ||
| fi | ||
|
|
||
| if ! json="$(gh api "repos/${GITHUB_REPOSITORY}/compare/${last}...${GITHUB_SHA}" 2>/dev/null)"; then | ||
| echo "Compare against ${last} failed (force-push? deleted commit?) — building." | ||
| build | ||
| fi | ||
|
|
||
| # The compare API caps .files at 300. A diff that large is an upstream | ||
| # sync, which has code in it by definition — don't trust the truncated | ||
| # list, just build. | ||
| if [ "$(jq '.files | length' <<<"$json")" -ge 300 ]; then | ||
| echo "Compare truncated at 300 files — building." | ||
| build | ||
| fi | ||
|
|
||
| # Same intent as the old `paths-ignore: docs/**, **/*.md`, but applied | ||
| # to the whole day's diff rather than one push: a day of nothing but | ||
| # docs commits now skips, where paths-ignore judged each push alone. | ||
| # (Path filters only apply to push/pull_request, so they would be dead | ||
| # config under `schedule`.) | ||
| code="$(jq -r '.files[].filename' <<<"$json" | grep -vE '^docs/|\.md$' || true)" | ||
| if [ -z "$code" ]; then | ||
| echo "No code changes since ${last} — skipping this build." | ||
| echo "has_changes=false" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "Code changed since ${last}:" | ||
| echo "$code" | ||
| echo "has_changes=true" >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| build: | ||
| name: Build and publish server bundle | ||
| needs: [check_changes] | ||
| # `!failure() && !cancelled()` is what lets a *skipped* gate through (a | ||
| # skipped `needs` job otherwise skips its dependents) while still halting on | ||
| # a *broken* one — `always()` would build even when the gate errored. | ||
| if: | | ||
| !failure() && !cancelled() && | ||
| (github.event_name != 'schedule' || needs.check_changes.outputs.has_changes == 'true') | ||
| # Only this job publishes; the gate above needs read-only. | ||
| permissions: | ||
| contents: write | ||
| # GitHub-hosted: the fork has no access to upstream's blacksmith runners. | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 20 | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v6 | ||
|
|
||
| - name: Setup Vite+ | ||
| uses: voidzero-dev/setup-vp@v1 | ||
| with: | ||
| # package.json pins Node ^24.13.1 — required by the TS build scripts. | ||
| node-version-file: package.json | ||
| cache: true | ||
| run-install: | | ||
| args: | ||
| - --filter=t3... | ||
| # scripts/update-release-package-versions.ts imports | ||
| # @effect/platform-node, which only @t3tools/scripts pulls in. | ||
| - --filter=@t3tools/scripts... | ||
| # The server serves the web client from dist/client. | ||
| - --filter=@t3tools/web... | ||
|
|
||
| - id: version | ||
| name: Resolve fork version | ||
| shell: bash | ||
| env: | ||
| RUN_STARTED_AT: ${{ github.run_started_at }} | ||
| run: | | ||
| set -euo pipefail | ||
| # Track upstream's stable base from package.json (nightly versions are | ||
| # never committed upstream, so main always carries the last stable), | ||
| # then mark it as ours so it is obvious which build a node is running. | ||
| base="$(node -p "require('./apps/server/package.json').version")" | ||
| # Date the run, not the clock: bare `date -u` reads whenever this step | ||
| # happens to run, so a build starting at 23:59 UTC would stamp | ||
| # tomorrow. Mirrors upstream's NIGHTLY_DATE: github.run_started_at. | ||
| day="$(date -u -d "$RUN_STARTED_AT" +%Y%m%d)" | ||
| version="${base}-jetblk.${day}.${GITHUB_RUN_NUMBER}" | ||
| echo "version=$version" >> "$GITHUB_OUTPUT" | ||
| echo "Building $version" | ||
|
|
||
| - name: Stamp version into releasable packages | ||
| run: node scripts/update-release-package-versions.ts "${{ steps.version.outputs.version }}" | ||
|
|
||
| # Gate on typecheck + the provider-usage tests only. The full server suite | ||
| # has pre-existing ACP-transport failures unrelated to this fork; gating on | ||
| # it would block every release. | ||
| - name: Typecheck | ||
| run: vp run --filter t3 --filter @t3tools/contracts typecheck | ||
|
|
||
| # `vp test` is vitest; run it from each package so its own config applies. | ||
| - name: Test contracts (provider usage) | ||
| working-directory: packages/contracts | ||
| run: vp test run src/providerUsage.test.ts | ||
|
|
||
| - name: Test server (provider usage) | ||
| working-directory: apps/server | ||
| run: | | ||
| vp test run \ | ||
| src/provider/Layers/ClaudeUsage.test.ts \ | ||
| src/provider/Layers/CodexUsage.test.ts \ | ||
| src/provider/Layers/ProviderUsageService.test.ts | ||
|
|
||
| - name: Build web client | ||
| run: vp run --filter @t3tools/web build | ||
|
|
||
| - name: Build server bundle | ||
| run: vp run --filter t3 build:bundle | ||
|
|
||
| # `vp pack` only produces dist/bin.mjs. The server serves the browser UI | ||
| # from dist/client, so without this it answers "No static directory | ||
| # configured and no dev URL set." This mirrors what apps/server/scripts/ | ||
| # cli.ts `build` does (copy apps/web/dist -> dist/client); we do it inline | ||
| # rather than calling that script so we skip its *development* icon | ||
| # overrides, which would ship dev icons in a release build. | ||
| - name: Bundle web client into dist/client | ||
| run: | | ||
| set -euo pipefail | ||
| test -f apps/web/dist/index.html | ||
| rm -rf apps/server/dist/client | ||
| cp -r apps/web/dist apps/server/dist/client | ||
| test -f apps/server/dist/client/index.html | ||
|
|
||
| - name: Package self-contained bundle | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| echo "${{ steps.version.outputs.version }}" > apps/server/dist/VERSION | ||
| # `vp pack` externalises dependencies (and 5 of them use pnpm's | ||
| # `catalog:` protocol), so dist/ alone cannot run. `pnpm deploy` copies | ||
| # the package plus its *production* deps — including the native ones | ||
| # (node-pty, sqlite) — into one portable tree. Safe because every node | ||
| # is linux-x64, matching this runner; a non-linux node would need its | ||
| # own build. | ||
| corepack enable # packageManager pins pnpm@11.10.0 | ||
| pnpm deploy --filter t3 --prod --legacy deploy-out | ||
| tar -czf t3-server.tgz -C deploy-out . | ||
| du -h t3-server.tgz | ||
|
|
||
| # Two releases per build, on purpose: | ||
| # v<version> immutable, never deleted — the pin/rollback target. | ||
| # nightly rolling pointer, recreated each run — the stable URL nodes | ||
| # curl, so they never need to know the current version. | ||
| # The asset is uploaded twice (GitHub can't alias an asset across | ||
| # releases); that's the price of keeping both a fixed URL and a history. | ||
| - name: Publish releases | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| ver="${{ steps.version.outputs.version }}" | ||
|
|
||
| cat > notes.md <<EOF | ||
| Self-contained linux-x64 server bundle built from \`${GITHUB_SHA}\`. | ||
|
|
||
| Version: \`${ver}\` | ||
|
|
||
| Contains \`dist/\` (incl. the \`dist/client\` web UI) + production \`node_modules/\` | ||
| with native deps, so a node needs only Node 22.16+ — no install step. | ||
|
|
||
| **Nodes track the rolling \`nightly\` release**, so a \`systemctl --user restart | ||
| t3code.service\` always pulls the newest build: | ||
|
|
||
| \`\`\`bash | ||
| curl -fsSL https://github.com/${GITHUB_REPOSITORY}/releases/download/nightly/t3-server.tgz \\ | ||
| | tar xz -C ~/.local/share/t3-nightly | ||
| \`\`\` | ||
|
|
||
| **To pin or roll back** a node to this exact build, point its \`ExecStartPre\` at | ||
| this immutable tag instead: | ||
|
|
||
| \`\`\` | ||
| https://github.com/${GITHUB_REPOSITORY}/releases/download/v${ver}/t3-server.tgz | ||
| \`\`\` | ||
| EOF | ||
|
|
||
| # 1. Immutable versioned release (rollback target). | ||
| gh release create "v${ver}" t3-server.tgz \ | ||
| --title "Fork nightly ${ver}" \ | ||
| --notes-file notes.md \ | ||
| --prerelease \ | ||
| --target "${GITHUB_SHA}" | ||
|
|
||
| # 2. Rolling pointer. Recreated so the tag tracks the built commit. | ||
| gh release delete nightly --yes --cleanup-tag 2>/dev/null || true | ||
| gh release create nightly t3-server.tgz \ | ||
| --title "Fork nightly ${ver} (rolling)" \ | ||
| --notes-file notes.md \ | ||
| --prerelease \ | ||
| --target "${GITHUB_SHA}" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Provide a fallback for
RUN_STARTED_ATto prevent script failure.The static analysis tool
actionlintreports thatrun_started_atis not a defined property on thegithubcontext object. If this property evaluates to an empty string, thedate -u -d ""command will fail with aninvalid dateerror, breaking the workflow due toset -euo pipefail.Add a fallback to
"now"(${RUN_STARTED_AT:-now}) to ensure the script always succeeds, even if the context property is missing.🛡️ Proposed fix
env: RUN_STARTED_AT: ${{ github.run_started_at }} run: | set -euo pipefail # Track upstream's stable base from package.json (nightly versions are # never committed upstream, so main always carries the last stable), # then mark it as ours so it is obvious which build a node is running. base="$(node -p "require('./apps/server/package.json').version")" # Date the run, not the clock: bare `date -u` reads whenever this step # happens to run, so a build starting at 23:59 UTC would stamp # tomorrow. Mirrors upstream's NIGHTLY_DATE: github.run_started_at. - day="$(date -u -d "$RUN_STARTED_AT" +%Y%m%d)" + day="$(date -u -d "${RUN_STARTED_AT:-now}" +%Y%m%d)"📝 Committable suggestion
🧰 Tools
🪛 actionlint (1.7.12)
[error] 140-140: property "run_started_at" is not defined in object type {action: string; action_path: string; action_ref: string; action_repository: string; action_status: string; actor: string; actor_id: string; api_url: string; artifact_cache_size_limit: number; base_ref: string; env: string; event: object; event_name: string; event_path: string; graphql_url: string; head_ref: string; job: string; output: string; path: string; ref: string; ref_name: string; ref_protected: bool; ref_type: string; repository: string; repository_id: string; repository_owner: string; repository_owner_id: string; repository_visibility: string; repositoryurl: string; retention_days: number; run_attempt: string; run_id: string; run_number: string; secret_source: string; server_url: string; sha: string; state: string; step_summary: string; token: string; triggering_actor: string; workflow: string; workflow_ref: string; workflow_sha: string; workspace: string}
(expression)
🤖 Prompt for AI Agents
Source: Linters/SAST tools