fix(api): stop profile/dashboard track queries from seq-scanning tracks (perf regression from #932)#942
Merged
Conversation
#932 folded accepted-collaborator tracks into GET /users/:id/tracks (and the dashboard monthly-listens query) as `OR t.track_id IN (SELECT ... FROM track_collaborators ...)`. That OR across two access paths defeats the tracks(owner_id) index, so Postgres sequential-scanned the entire tracks table on every profile load — a severe regression on a top-QPS endpoint. Fix: fetch the user's accepted-collaboration track ids first (a tiny index-only lookup that returns nothing for ~all users), and only widen the query when the set is non-empty — via an explicit `= ANY($ids)` array (bitmap-OR of the owner_id index and the track_id PK), not a correlated subquery. For users with no collaborations the query is byte-identical to the pre-#932 owner-only query and plan. Same treatment for v1_users_listen_counts_monthly. Behavior is unchanged (tests for the merge, pending-hidden, and the no-collab path all pass). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Incident hotfix
After #932 deployed,
GET /users/:id/tracks(and the dashboard monthly-listens query) started sequential-scanning the entiretrackstable on every request, melting the DB on a top-QPS endpoint.Root cause
#932 merged accepted-collaborator tracks into the profile query as:
That
ORacross two different access paths (an indexed column ontracksvs. a semi-join subquery) defeats thetracks(owner_id)index. With the subquery's poor row estimate, the planner falls back to a seq scan oftracks(+ theaggregate_*joins, then sort) — per request. Not feature-gated, so it hit 100% of profile loads immediately, despite zero collaborations existing yet (frontend isn't out).Fix
Fetch the user's accepted-collaboration track ids first — a tiny index-only lookup on the covering
(collaborator_user_id, status, track_id)index that returns nothing for ~every user — then:t.track_id = ANY($ids)array, which the planner handles with a bitmap OR of theowner_idindex and thetrack_idPK — not a correlated subquery.Same treatment applied to
v1_users_listen_counts_monthly.Correctness
Behavior is unchanged. Passing:
TestGetUserTracks(no-collab path),TestAcceptedCollaborationAppearsOnProfile(merge still works),TestPendingCollaborationHiddenFromProfile,TestV1UsersListenCountsMonthly, the collaborator embed/notification tests.Note
This is the perf risk I'd flagged in the #932 review (the
OR … INpattern). If you need relief before this merges, reverting justv1_users_tracks.go+v1_users_listen_counts_monthly.goto their pre-#932 owner-only queries is a zero-risk instant mitigation — there are no real collaborations yet, so nothing is lost.🤖 Generated with Claude Code