storage: budget MySQL snapshot partition probing by table size - #38090
Draft
peterdukelarsen wants to merge 1 commit into
Draft
storage: budget MySQL snapshot partition probing by table size#38090peterdukelarsen wants to merge 1 commit into
peterdukelarsen wants to merge 1 commit into
Conversation
peterdukelarsen
changed the title
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 6, 2026 20:11
437fc36 to
6ae6b86
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 6, 2026 23:31
6ae6b86 to
62cf41b
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
2 times, most recently
from
August 7, 2026 00:05
d391fe2 to
51bc750
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
2 times, most recently
from
August 7, 2026 22:27
0545c4e to
a3d533a
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
2 times, most recently
from
August 8, 2026 00:08
453cc7f to
c9bbea1
Compare
Bound the prefix partitioner to a per-table probe query budget scaled to the estimated row count, 2500 requests per billion rows by default via the mysql_source_snapshot_partition_requests_per_billion_rows dyncfg, with a floor of 256 so small tables can afford their handful of splits. Probes run sequentially, so this caps the wall-clock that partitioning can add to snapshot start in proportion to the snapshot work it optimizes. An exhausted budget stops splitting early and leaves coarser buckets, never incorrect ones. Adds a mock test asserting the partitioner never exceeds its budget and still returns a valid ordered boundary list when truncated.
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 8, 2026 01:44
c9bbea1 to
1b2ebf2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bounds the partitioner's probe traffic with a request budget derived from the table's estimated row count, via the new
mysql_source_snapshot_partition_requests_per_billion_rowsdyncfg (floored at 256 requests). When the budget runs out the walk stops splitting and emits coarser partitions, so cost is capped without losing correctness. The budget also serves as the hard bound on the walk for degenerate key spaces.Adds mock tests for budget-bounded probing and for termination on non-advancing prefixes.
🤖 Generated with Claude Code