Repository navigation
feat(billing): daily profile usage from service_usage - #2681
Conversation
This comment has been minimized.
This comment has been minimized.
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Profiles land in service_usage through a new view (ProfileCount, ProfileSizeBytes, 16 bytes per frame plus 200 per stack row), ClickHouse migration 0068 and local schema v60. The billing chart's daily volume reads them as profilesGB, and a cluster whose service_usage predates the columns reads no profile usage instead of a 502.
3d242b7 to
12c492b
Compare
This comment has been minimized.
This comment has been minimized.
…enamed ProfileRowCount Tinybird refuses a view whose output lacks target columns, so the six signal views write 0 to ProfileRowCount and ProfileSizeBytes, and migration 0068 and the local v60 step recreate them. The row count is named for stack rows, which is what it counts. service_usage keeps an explicit forward query: without one Tinybird would rebuild it from sources that retain 14 to 90 days.
Maple review🟢 Confidence 10/10 · safe to merge. No defect found; the new view, the v59 to v60 step and the fallback are each pinned by a test. The one-shot Adds estimated continuous-profiling volume to the billing page's daily chart: a
Before merge
Fixed since the last review
|
|
Merged with 4 steps from "Before merge" still open. If they are done, tick them on the review comment; if not, now is the time.
|
|
Merged with 1 step from "Before merge" still open. If it is done, tick it on the review comment; if not, now is the time.
|
Follow-up to #2678: fills the billing page's per-day profiles band, which reads $0 until this lands.
Warehouse
0068_service_usage_profiles:ProfileRowCountandProfileSizeBytes(UInt64, default 0) onservice_usage, and aservice_usage_profiles_mvview fromprofiles, shaped like the logs view. Not required for ingest, no backfill.length(Frames) * 16 + 200. The chart rescales each day to the metered Autumn total, so only the daily shape matters.Read path:
dailySignalVolumeQueryselects profile bytes, andDailySpendServicesetsprofilesGB. A cluster without the new columns yet falls back to a query that reports 0 for profiles, so the chart still loads.Tests: domain migration tests, the compiled query and its SQL baseline,
DailySpendService(including the fallback), and a new chDB test (apps/cli/test/service-usage-profiles.test.ts, added to CI's list) showing profile rows land inservice_usageand that the real v59 to v60 local migration keeps old rows.Validated against prod Tinybird with the read-only
bun run tinybird:deploy check: "Deployment is valid" in us and eu. Changes:service_usage_profiles_mvnew;service_usageand its six signal views modified (each now emits0 AS ProfileRowCount, 0 AS ProfileSizeBytes, which Tinybird requires of every view writing the table).ProfileRowCountcounts stack rows, since a summing table cannot count distinct profiles.Migration 0068 adds the columns, then drops and recreates all seven
service_usageviews from the generated schema; local v60 does the same.After deploy:
service_usagecarries a forward query naming every column, with the two new ones defaulted. Without it, Tinybird rebuilds the table from source tables that keep 14 to 90 days, so its 365 days of history would be lost. Once this deployment is live it must go back toSELECT *(a follow-up PR), or the next rebuild would reset profile counts.Gap: on self-hosted ClickHouse and local stores, rows ingested in the brief window while the views are recreated are not counted in
service_usage(the same trade-off migration 0065 made).🤖 Generated with Claude Code
Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.Summary by CodeRabbit