Resolve cigar2 overflow - #2072
Open
jkbonfield wants to merge 5 commits into
Open
Conversation
Successive large D or I values in a row could overflow and become the opposite type (I or D). Similarly we have overflow in other variables. This is due to 32-bit integers. We now hold these 32-bit values as 64-bit. While technically we could overflow 64-bit, we have bigger problems if that's happening and hts_pos_t is broken too. For CIGAR it would also take a rediculous number of CIGAR operations given the maximum length per op is 256MB. Signed-off-by: James Bonfield <jkb@sanger.ac.uk>
It was an odd mix of 4 and 2 space indents, with some misleading one-line if statements followed by indented non-conditional statements. Signed-off-by: James Bonfield <jkb@sanger.ac.uk>
- Mostly line wrapping to avoid long lines - Splitting compound statements on a line - Using multi-line ifs and else statements for readability - Put some duplicated code into variables, eg _cln(cigar[k]) - Remove a pointless "if (l3 > 0)" statement and assign direct (BAM_CPAD) - Using bam_cigar_type instead of numerous separate if statements Signed-off-by: James Bonfield <jkb@sanger.ac.uk>
Eg T+2AA-2CD for 2I1D1D CIGAR is currently reported as T+2AA-1C. While duplicate neighbouring operations may not be recommended, the rest of the pileup code does handle this, meaning this code is an outlier. Signed-off-by: James Bonfield <jkb@sanger.ac.uk>
Particularly with Clang, some of the previous refactoring had a big slow down of this function, by as much as 10%. This new refactoring should speed it up to be faster than the original (bar the slow-down of int to int64, which is another 10% hit for both gcc and clang) Signed-off-by: James Bonfield <jkb@sanger.ac.uk>
jkbonfield
force-pushed
the
resolve_cigar2_overflow
branch
from
August 18, 2026 15:51
632ff85 to
c58bd9e
Compare
Contributor
Author
|
Claude review caught something more for me to explore: Overflow check doesn't cover the full qpos computation — moderate, but relevant given the fix's purpose p->qpos (an int32_t) is assigned s->y + (pos - s->x), but the overflow guard only checks s->y itself. If s->y is, say, INT_MAX - 100 and (pos - s->x) is larger than 100 (which can happen for a single large CIGAR op, since op lengths can be up to ~2^28), the sum can exceed INT_MAX and silently wrap when stored into the int32_t field — while ret stays 1 (success), so the corrupted qpos is not caught and gets returned to the caller as valid. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is does two things.
Fixes overflow bugs with long consecutive indels. Eg 250000D x10. We now basically fail as it represents something we cannot store in p->indel due to size limitations. It shouldn't ever happen, but I was wary of it being a vector for curious bugs given it could turn deletions into insertions and vice versa. This doesn't seem to trigger really bad behaviour with our code, but it could potentially elsewhere.
Refactor the resolve_cigar2 function, and try and speed it up somewhat to mitigate the performance hit of step 1 (which is surprisingly high given it's just swapping int for int64 essentially).
On short read data, this is around 8% slower (clang) and 11% slower (gcc).
On long read data it's 2% slower (clang) and 5% slower (gcc).
Note those timings are
resolve_cigar2only, measured using-fno-inlineandperf recordto check function counters.