Skip to content

perf(web): cut Cap Infra billing reads on the console, usage API, ledger and storage cron - #4142

Merged
richiemcilroy merged 28 commits into
mainfrom
perf/infra-billing-reads
Nov 13, 2026
Merged

richiemcilroy merged 28 commits into
mainfrom
perf/infra-billing-reads

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Cap Infra billing reads did the same work several times per request and some grew with every video, ledger row or account. This cuts the repeats and bounds the rest.

Measured on a seeded scratch database (MySQL 8.0, 1,000 apps, 50,000 videos with 40,000 in one app, 50,000 ledger rows with 40,000 in one account, 30,000 transcripts). Baseline is main, candidate is this branch with migration 0056 applied.

Path Round trips Rows examined Median
Console shell (every console page) 5 → 2 38,614 → 25 26.2 → 0.4 ms
Console overview, 40,000 video app 13 → 10 118,642 → 40,056 78.0 → 40.0 ms
Console overview, small app of the same owner 13 → 10 38,775 → 142 28.6 → 2.4 ms
Console usage page 8 → 5 98,733 → 40,750 71.6 → 41.2 ms
Console billing, first page 13 → 9 158,676 → 40,087 68.4 → 37.4 ms
Console billing, page 400 of 800 13 → 9 218,626 → 40,087 104.5 → 42.1 ms
Console keys page 5 → 3 38,614 → 28 25.7 → 0.6 ms
Console settings page 5 → 3 38,614 → 38,026 25.7 → 6.0 ms
GET /usage (handler, after the key lookup) 9 → 8 140,020 → 80,626 59.0 → 51.4 ms
Month speech usage query 1 → 1 20,000 → 606 16.2 → 0.6 ms
Auto top-up pass, 31 accounts switched on 96 → 9 1,093 → 324 19.8 → 2.5 ms
Storage cron, today's pass on the first run of the day (990 accounts, 199 charged) 3,183 → 2,413 52,804 → 52,860 326 → 317 ms
Storage cron, a scheduled run once yesterday and today are settled (both passes) 795 → 48 not measured 80 → 118 ms
Pro credit claims for one subscription this month 1 → 1 1,100 → 105 2.8 → 0.3 ms

Every figure a customer sees is unchanged on that data: GET /usage and GET /transactions bodies, stored and billing totals for 211 apps, and today's 199 charges and snapshots match main (amount, owed, videos and minutes by tier; balance after differs only by the yesterday charge the seed had left open). Billing pages 1, 400 and 800 show the same rows offset paging showed.

flowchart LR
  S[scheduled run] --> Y[finish yesterday from the first account]
  Y --> T[sweep today from the first account]
  Y -.-> A
  T -.-> A
  A[accounts after cursor, 200 by id] --> B[live apps of the page]
  B --> C[charged for the date?]
  C --> D[stored video of the pending apps]
  D --> E[last charge of idle accounts, one read]
  E --> F{time left?}
  F -- yes --> G[charge 10 at a time] --> A
  F -- no --> H[after: GET itself with date, after, hop + 1]
  H --> A
Loading

Storage totals, once per request

  • getAppStorage (lib/infra-storage.ts) returns the stored total (ready video) and the counted total (ready plus processing) from one read. The aggregate SQL is untouched.
  • getInfraBilling returns stored beside usage, so GET /usage and the console overview stop reading stored video a second time.
  • getInfraBilling and getInfraUploadAdmission accept { storage } to reuse totals a caller already holds.

Storage cron and auto top-up

  • app/api/cron/developer-storage/route.ts pages accounts 200 at a time on the account id, sets maxDuration = 300 and gives each phase a share of it (top-ups to 60 s, charging to 200 s, top-ups to 240 s).
  • A cut-off run continues itself: inside after() it requests the same route with the same Authorization, carrying date (the day being charged), after (the last account it attempted) and hop. The answer carries complete, continued and hop.
  • A continuation charges for the carried date, so the last run of a UTC day still finishes that day after midnight. It resumes after the carried account, so accounts that end up unpaid (they leave no ledger row) are not walked again. A page cut off part way resumes at its first unattempted account.
  • date must be today's or yesterday's UTC key and hop 1 to 24; anything else is a 400. At hop 24 the chain stops and logs an error.
  • chargeStorageDay stays idempotent per account and day, so a continuation never charges an account an earlier hop charged.
  • Every scheduled run first finishes yesterday, then sweeps today, in the same time budget, so a chain that breaks on the last run of a day (failed handoff, refused continuation, hop limit) is finished by the next day's first run. A chain carries whichever date it was on and, once that day is done, carries on into today.
  • Every visit that charges nothing (no storage owed, no short day to settle) records a marker for that app and day: a snapshot row with zero minutes, zero videos and nothing charged, inserted only if the app has no snapshot for the day (recordStorageDayVisits in lib/infra-storage-billing.ts, one statement per page, insert-if-absent on app_date_unique). A later charge the same day replaces it as before. Finishing a past day handles only accounts with no snapshot for it, which are exactly the ones the sweep never reached, so an app that was empty at every run of the day and whose first video becomes ready after midnight is not charged for that day.
  • Every snapshot written before this has more than zero minutes (the writer has always skipped empty apps), so the two readers of the table leave zero-minute rows out: the console Usage page history, chart and billed-day count (getInfraStorageSnapshots) and the agent API's stored minutes (AgentManagement.getDeveloperAppContext). Their output is unchanged.
  • Marker cost on the seeded data: 791 idle apps write 791 rows per day on the first visit, in one statement per page that has idle apps (about 13 ms per 200-row page); later runs write none, because the per-page snapshot read already shows the marker.
  • Finishing yesterday charges with yesterday's date key for the ledger reference and the snapshot. For accounts the sweep never reached, it counts only storage that existed on that day: videos created before the following UTC midnight (a createdBefore bound added to the stored video read, inside the same aggregate query), and accounts and apps created before it. Usage is otherwise read at run time, as the existing retry within a day already does, so a video deleted after midnight but before the run is not charged for that day, which is in the customer's favour and what a timely run would mostly have seen.
  • A day already settled stays settled: billed and short days have their ledger row, and an unpaid day's snapshot is now final for a past day (retryUnpaid: false). Unpaid accounts are left out of yesterday's pass by one snapshot read per page, so they open no transaction; on the seeded data 99 unpaid accounts cost 99 transactions on today's retry, as on main, and none on yesterday's pass. Today's retry of an unpaid day after a top-up is unchanged.
  • A handoff that is still running when the 10 second handoff wait ends is logged at info level with the next cursor; a refused or failed handoff is logged as an error.
  • Auto top-ups run before charging on the scheduled run and after charging on the hop that finishes, not on hops in between.
  • Accounts that store nothing no longer cost a read each: latestStorageCharges (lib/infra-standing.ts) fetches the last charge of a page's idle accounts in one query.
  • The largest parameter list drops from every account id (992 here) to 200.
  • runDueAutoTopUps selects only accounts that are due (switched on, card saved, balance under threshold after held credit), in pages by id, and takes a deadline.

Ledger

  • The Billing page pages by cursor (?before= and ?after=), 50 a page, with Newer and Older and no total. The "x to y of z" line is gone.
  • GET /api/developer/v1/transactions keeps limit and offset and still does not count.
count, main:   developer_credit_transactions type=ref key=account_created_idx rows=24290 Using index (40,000 rows, removed)
page 400, main: Using filesort, key=account_type_created_idx (about 60,000 rows)
page, branch:  type=range key=account_created_idx key_len=128 Using index condition; Backward index scan (51 rows)

Console

  • The shell loads the app list (id, name, environment) and the open app's credit account, in parallel. Domains were loaded and never shown; keys load on the overview and keys pages; the video count loads on the settings page only.

Migration 0056

CREATE INDEX `auto_top_up_idx` ON `developer_credit_accounts` (`autoTopUpEnabled`);
CREATE INDEX `reference_type_id_idx` ON `developer_credit_transactions` (`referenceType`,`referenceId`);
CREATE INDEX `dev_transcripts_app_completed_idx` ON `developer_transcripts` (`appId`,`status`,`completedAt`,`durationSeconds`,`billedMicroCredits`);
CREATE INDEX `dev_transcripts_app_status_held_idx` ON `developer_transcripts` (`appId`,`status`,`createdAt`,`deletedAt`,`reservedMicroCredits`);

Additive only. Its journal when is newer than every earlier entry, so drizzle's migrator applies it on an existing install as well as a fresh one. The code works without the indexes, so the deploy order is free.

Index Query Before After Write cost
auto_top_up_idx accounts due an auto top-up type=ALL rows=1000 type=ref key=auto_top_up_idx rows=31 one entry per account, rewritten only when the switch flips
reference_type_id_idx Pro credit claims of a subscription this month (#4140) accounts type=index rows=1000, then a probe per account type=ref key=reference_type_id_idx rows=100, then eq_ref one more entry per ledger insert; ledger rows are never updated
dev_transcripts_app_completed_idx month speech usage type=ref key=dev_transcripts_app_list_idx rows=14220 Using where type=range rows=606 Using where; Using index one entry per transcript, moved once when it completes
dev_transcripts_app_status_held_idx held transcript credit type=ref key=dev_transcripts_app_status_idx rows=8 Using where type=ref rows=8 Using where; Using index one entry per transcript, moved on each status change
  • reference_type_id_idx narrows the claim count to the month's claims. The subscription id is read from metadata, so the count still reads each of those rows; a real column for it would make the count per subscription.
  • Refund and dispute lookups filter on accountId and keep using account_ref_dedup_idx and account_type_created_idx.
  • The last auto top-up lookup (lastAttempt in lib/infra-auto-top-up.ts) now also filters on the ledger type each reference type is written with, so account_type_created_idx serves it: 1 row read instead of 40,000 for an account with a large ledger and no auto top-up, with or without the new index.
  • Redundant once this lands, not dropped here: dev_transcripts_app_status_idx (a prefix of the held index).

Not covered

  • No covering index for the storage aggregate on developer_videos; that ships with the aggregate change. The aggregate is still one pass over the app's videos (40,000 rows, 36 ms here), now run once.
  • GET /usage still runs its own count of uploading videos (40,000 rows here).
  • Upload completion still reads stored video twice; the { storage } option is the hook for removing the second read.
  • Old ?page=N Billing links now open the newest page.
  • The Billing pager markup changed and has not been checked in a browser; screenshots to follow.

RetriggerView in GreptileConfidence Score: 5/5

No new actionable issue was found that blocks merging this PR.

Summary

This PR reduces repeated billing reads and bounds background work.

  • Billing and usage pages reuse stored-video totals.
  • Each console page loads only the data it shows.
  • Storage billing finishes daily sweeps in bounded, resumable pages.
  • Auto top-ups select due accounts in timed pages.
  • The Billing page moves through ledger rows with cursors.
  • New database indexes narrow billing and transcript lookups.

No new, distinct actionable findings were accepted. The supplied previous threads were unnumbered, so they have no numbered ledger entries.

Reviews (10) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 1 potential issue (1 medium).

View 1 other finding in Capy.

Open in Capy

@sandchest-com

sandchest-com Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Sandchest

Sandchest reviewed 0850363: nothing exploitable found in this change.

Sandchest reviews every change for vulnerabilities an attacker could use · 6 credits · Review details

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 01:41 Inactive
@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
cap-status Skipped Skipped Nov 13, 2026 4:51am UTC

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 2 other findings in Capy.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 01:46 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 1 potential issue (1 medium).

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 02:08 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 04:11 Inactive
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 2 other findings in Capy.

Open in Capy

Comment thread apps/web/lib/infra-auto-top-up.ts Resolved
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 2 other findings in Capy.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 04:39 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 04:46 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 13, 2026 04:51 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@richiemcilroy
richiemcilroy merged commit ec9c7c5 into main Nov 13, 2026
41 checks passed
richiemcilroy added a commit that referenced this pull request Nov 13, 2026
Brings in #4142's paged due query and #4143's transcript ceiling. The auto top-up waiting rule is now one SQL fragment, autoTopUpWaits, read by both the locked check in runAutoTopUp and the cron's due query: an unresolved pending payment younger than 72 hours, a decline in the last 24 hours with no top-up after it, or ten top-ups in the last 24 hours. A pending payment past 72 hours releases the account so the sweep reconciles it. Transcript top-up triggers stay on the 402 low-balance refusal only, not the 429 ceiling.

This branch was successfully deployed

1 active and 1 inactive deployments
Preview – cap-web — 08503634 Deployed Nov 13, 2026 by vercel[bot]
Preview – cap-status — 08503634 Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants