Repository navigation
perf(web): cut Cap Infra billing reads on the console, usage API, ledger and storage cron - #4142
Merged
Merged
Conversation
…auto top-up accounts
… Infra console page shows
… the function limit
…ver scans every app
…ed across accounts
|
Sandchest reviewed Sandchest reviews every change for vulnerabilities an attacker could use · 6 credits · Review details |
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
…last attempted account
Member
Author
|
hey @greptileai, please re-review the PR |
Member
Author
|
hey @greptileai, please re-review the PR |
# Conflicts: # apps/web/app/(infra)/dashboard/infra/_components/shell.tsx
Member
Author
|
hey @greptileai, please re-review the PR |
richiemcilroy
added a commit
that referenced
this pull request
Nov 13, 2026
Brings in #4142's paged due query and #4143's transcript ceiling. The auto top-up waiting rule is now one SQL fragment, autoTopUpWaits, read by both the locked check in runAutoTopUp and the cron's due query: an unresolved pending payment younger than 72 hours, a decline in the last 24 hours with no top-up after it, or ten top-ups in the last 24 hours. A pending payment past 72 hours releases the account so the sweep reconciles it. Transcript top-up triggers stay on the 402 low-balance refusal only, not the 429 ceiling.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Cap Infra billing reads did the same work several times per request and some grew with every video, ledger row or account. This cuts the repeats and bounds the rest.
Measured on a seeded scratch database (MySQL 8.0, 1,000 apps, 50,000 videos with 40,000 in one app, 50,000 ledger rows with 40,000 in one account, 30,000 transcripts). Baseline is
main, candidate is this branch with migration 0056 applied.GET /usage(handler, after the key lookup)Every figure a customer sees is unchanged on that data:
GET /usageandGET /transactionsbodies, stored and billing totals for 211 apps, and today's 199 charges and snapshots matchmain(amount, owed, videos and minutes by tier; balance after differs only by the yesterday charge the seed had left open). Billing pages 1, 400 and 800 show the same rows offset paging showed.flowchart LR S[scheduled run] --> Y[finish yesterday from the first account] Y --> T[sweep today from the first account] Y -.-> A T -.-> A A[accounts after cursor, 200 by id] --> B[live apps of the page] B --> C[charged for the date?] C --> D[stored video of the pending apps] D --> E[last charge of idle accounts, one read] E --> F{time left?} F -- yes --> G[charge 10 at a time] --> A F -- no --> H[after: GET itself with date, after, hop + 1] H --> AStorage totals, once per request
getAppStorage(lib/infra-storage.ts) returns the stored total (ready video) and the counted total (ready plus processing) from one read. The aggregate SQL is untouched.getInfraBillingreturnsstoredbesideusage, soGET /usageand the console overview stop reading stored video a second time.getInfraBillingandgetInfraUploadAdmissionaccept{ storage }to reuse totals a caller already holds.Storage cron and auto top-up
app/api/cron/developer-storage/route.tspages accounts 200 at a time on the account id, setsmaxDuration = 300and gives each phase a share of it (top-ups to 60 s, charging to 200 s, top-ups to 240 s).after()it requests the same route with the sameAuthorization, carryingdate(the day being charged),after(the last account it attempted) andhop. The answer carriescomplete,continuedandhop.datemust be today's or yesterday's UTC key andhop1 to 24; anything else is a 400. At hop 24 the chain stops and logs an error.chargeStorageDaystays idempotent per account and day, so a continuation never charges an account an earlier hop charged.recordStorageDayVisitsinlib/infra-storage-billing.ts, one statement per page, insert-if-absent onapp_date_unique). A later charge the same day replaces it as before. Finishing a past day handles only accounts with no snapshot for it, which are exactly the ones the sweep never reached, so an app that was empty at every run of the day and whose first video becomes ready after midnight is not charged for that day.getInfraStorageSnapshots) and the agent API's stored minutes (AgentManagement.getDeveloperAppContext). Their output is unchanged.createdBeforebound added to the stored video read, inside the same aggregate query), and accounts and apps created before it. Usage is otherwise read at run time, as the existing retry within a day already does, so a video deleted after midnight but before the run is not charged for that day, which is in the customer's favour and what a timely run would mostly have seen.retryUnpaid: false). Unpaid accounts are left out of yesterday's pass by one snapshot read per page, so they open no transaction; on the seeded data 99 unpaid accounts cost 99 transactions on today's retry, as onmain, and none on yesterday's pass. Today's retry of an unpaid day after a top-up is unchanged.latestStorageCharges(lib/infra-standing.ts) fetches the last charge of a page's idle accounts in one query.runDueAutoTopUpsselects only accounts that are due (switched on, card saved, balance under threshold after held credit), in pages by id, and takes a deadline.Ledger
?before=and?after=), 50 a page, with Newer and Older and no total. The "x to y of z" line is gone.GET /api/developer/v1/transactionskeepslimitandoffsetand still does not count.Console
Migration 0056
Additive only. Its journal
whenis newer than every earlier entry, so drizzle's migrator applies it on an existing install as well as a fresh one. The code works without the indexes, so the deploy order is free.auto_top_up_idxtype=ALL rows=1000type=ref key=auto_top_up_idx rows=31reference_type_id_idxaccounts type=index rows=1000, then a probe per accounttype=ref key=reference_type_id_idx rows=100, theneq_refdev_transcripts_app_completed_idxtype=ref key=dev_transcripts_app_list_idx rows=14220 Using wheretype=range rows=606 Using where; Using indexdev_transcripts_app_status_held_idxtype=ref key=dev_transcripts_app_status_idx rows=8 Using wheretype=ref rows=8 Using where; Using indexreference_type_id_idxnarrows the claim count to the month's claims. The subscription id is read frommetadata, so the count still reads each of those rows; a real column for it would make the count per subscription.accountIdand keep usingaccount_ref_dedup_idxandaccount_type_created_idx.lastAttemptinlib/infra-auto-top-up.ts) now also filters on the ledgertypeeach reference type is written with, soaccount_type_created_idxserves it: 1 row read instead of 40,000 for an account with a large ledger and no auto top-up, with or without the new index.dev_transcripts_app_status_idx(a prefix of the held index).Not covered
developer_videos; that ships with the aggregate change. The aggregate is still one pass over the app's videos (40,000 rows, 36 ms here), now run once.GET /usagestill runs its own count of uploading videos (40,000 rows here).{ storage }option is the hook for removing the second read.?page=NBilling links now open the newest page.

Confidence Score: 5/5
No new actionable issue was found that blocks merging this PR.Summary
This PR reduces repeated billing reads and bounds background work.
No new, distinct actionable findings were accepted. The supplied previous threads were unnumbered, so they have no numbered ledger entries.
Reviews (10) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile