ZOOKEEPER-2789: Reassign ZXID to a 24-bit epoch / 40-bit counter with rolling upgrade#2417
Open
asdf2014 wants to merge 1 commit into
Open
ZOOKEEPER-2789: Reassign ZXID to a 24-bit epoch / 40-bit counter with rolling upgrade#2417asdf2014 wants to merge 1 commit into
asdf2014 wants to merge 1 commit into
Conversation
asdf2014
force-pushed
the
reassign_zxid
branch
2 times, most recently
from
27a7697 to
0c5778c
Compare
… rolling upgrade Merge current master into the reassign_zxid branch and rework the change into an opt-in, rolling-upgradeable wide-counter zxid layout. The 32-bit counter overflowing forces a leader re-election roughly every 49.7 days at 1k writes/s (ZOOKEEPER-1277). This adds a 24-bit epoch / 40-bit counter layout that pushes that to ~34.9 years while still allowing ~1915 years of hourly elections before the epoch overflows. - ZxidLayout / ZxidLayoutState: the legacy 32/32 and wide 24/40 layouts plus a per-QuorumPeer record of the epoch from which the ensemble is wide. The static ZxidUtils helpers remain the legacy-fixed layout for the on-the-wire epoch carrier zxids of the handshake. - Opt-in via zookeeper.wideCounterZxidEnabled (default off). A newly elected leader switches at its new epoch, persists it to a new zxidLayoutSwitchEpoch file, and announces it in the LEADERINFO handshake (learner protocol version bumped to 0x11000). Learners below 0x11000 are refused once switched. - The switch coincides with an epoch bump, so the numeric order of zxids is preserved; layoutFor(zxid) selects the decode layout per zxid so existing legacy snapshots/txnlogs are read correctly across the boundary without being rewritten. One-way switch with an anti-downgrade guard. - Replace hardcoded zxid bit operations with layout-aware calls across Leader, LearnerHandler, Learner, Follower, ObserverMaster, ZKDatabase, FastLeaderElection, Commands and LogChopper.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continues #262 (ZOOKEEPER-2789), reworked and rebased onto current master as a single commit.
What this does. The zxid stays a 64-bit
epoch | counter. This adds an opt-in wide-counter layout — 24-bit epoch / 40-bit counter — pushing the counter-rollover-forces-reelection horizon (ZOOKEEPER-1277) from ~49.7 days to ~34.9 years at 1k writes/s, while 24 bits of epoch still lasts ~1915 years at hourly elections. Off by default; enabled per-ensemble via rolling upgrade — no downtime, no reformat.Design:
zookeeper.wideCounterZxidEnabled(default off). The learner protocol version bumps0x10000→0x11000to advertise "I understand the wide layout"; once a leader has switched it refuses any learner below0x11000, so a half-upgraded ensemble can't feed wide zxids to a node that would misparse them.zxidLayoutSwitchEpochfile). Because the first wide zxid(switchEpoch<<40)numerically exceeds every legacy zxid of prior epochs, the total order of zxids is preserved — the numeric zxid comparisons in ZAB keep working untouched.ZxidLayoutState.layoutFor(zxid)picks the decode layout per zxid by comparing against the switch point, so a legacy new-epoch zxid(5,0)is still read as counter==0 andLearnerHandler.syncFollower()makes the right DIFF/TRUNC/SNAP call across the boundary.Upgrade validation.
WideCounterZxidUpgradeTestruns a real 3-node ensemble end to end: commit legacy data → rolling-restart with the flag → new leader switches → verify legacy data survives, wide+legacy zxids keep total order, a lagging follower re-syncs across the boundary, and the switch epoch reloads after a full restart.Merge & review. Rebased onto current master (~880 commits past the old merge-base); adapted to ZOOKEEPER-4925 and the removed loggraph module. I ran an adversarial backward-compat review over the whole change and fixed a real gap where a pre-3.4.6 learner could slip past the wide-layout refusal, plus a diagnostics issue in offline
zkTxnLogToolkit chop. New unit + integration tests throughout; checkstyle / RAT / spotbugs clean.