Skip to content

Commit 588d391

Browse files
committed
gh-144759: Fix undefined behavior from NULL pointer arithmetic in lexer
Guard against NULL pointer arithmetic in `_PyLexer_remember_fstring_buffers` and `_PyLexer_restore_fstring_buffers`. When `start` or `multi_line_start` are NULL (uninitialized in tok_mode_stack[0]), performing `NULL - tok->buf` is undefined behavior. Add explicit NULL checks to store -1 as sentinel and restore NULL accordingly.
1 parent eb6ebdb commit 588d391

File tree

2 files changed

+8
-4
lines changed

2 files changed

+8
-4
lines changed
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Fix undefined behavior in the lexer when ``start`` and ``multi_line_start``
2+
pointers are ``NULL`` in :c:func:`_PyLexer_remember_fstring_buffers` and
3+
:c:func:`_PyLexer_restore_fstring_buffers`. The ``NULL`` pointer arithmetic
4+
(``NULL - valid_pointer``) is now guarded with explicit ``NULL`` checks.

Parser/lexer/buffer.c

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,8 @@ _PyLexer_remember_fstring_buffers(struct tok_state *tok)
1313

1414
for (index = tok->tok_mode_stack_index; index >= 0; --index) {
1515
mode = &(tok->tok_mode_stack[index]);
16-
mode->start_offset = mode->start - tok->buf;
17-
mode->multi_line_start_offset = mode->multi_line_start - tok->buf;
16+
mode->start_offset = mode->start == NULL ? -1 : mode->start - tok->buf;
17+
mode->multi_line_start_offset = mode->multi_line_start == NULL ? -1 : mode->multi_line_start - tok->buf;
1818
}
1919
}
2020

@@ -27,8 +27,8 @@ _PyLexer_restore_fstring_buffers(struct tok_state *tok)
2727

2828
for (index = tok->tok_mode_stack_index; index >= 0; --index) {
2929
mode = &(tok->tok_mode_stack[index]);
30-
mode->start = tok->buf + mode->start_offset;
31-
mode->multi_line_start = tok->buf + mode->multi_line_start_offset;
30+
mode->start = mode->start_offset < 0 ? NULL : tok->buf + mode->start_offset;
31+
mode->multi_line_start = mode->multi_line_start_offset < 0 ? NULL : tok->buf + mode->multi_line_start_offset;
3232
}
3333
}
3434

0 commit comments

Comments
 (0)