Skip to content

fix: reject oversized state in deserialize - #370

Open
dekobon wants to merge 1 commit into
tree-sitter:masterfrom
dekobon:fix/scanner-deserialize-bounds
Open

dekobon wants to merge 1 commit into
tree-sitter:masterfrom
dekobon:fix/scanner-deserialize-bounds

Conversation

@dekobon

@dekobon dekobon commented Aug 3, 2026

Copy link
Copy Markdown

This PR seeks to fix #368 where in src/scanner.c:135-143, we have the following function:

void tree_sitter_cpp_external_scanner_deserialize(void *payload, const char *buffer, unsigned length) {
    assert(length % sizeof(wchar_t) == 0 && "Can't decode serialized delimiter!");

    Scanner *scanner = (Scanner *)payload;
    scanner->delimiter_length = length / sizeof(wchar_t);
    if (length > 0) {
        memcpy(&scanner->delimiter[0], buffer, length);
    }
}

Here the greater than zero check is insufficiently guarding memcpy.

The variable Scanner is { uint8_t delimiter_length; wchar_t delimiter[16] } (68 bytes total on Linux/macOS, of which 64 are the delimiter; 32 where wchar_t is 2 bytes).

Elsewhere in the file we check to ensure that delimiter_length <= MAX_DELIMITER_LENGTH, but only in this function are we able to break that contract.

I've taken this through ASan and have a PoC of breaking through this guard, but in interest of safety, I think this is all the information needed to see why we need a better guard here.

AI disclosure: used for debugging and running ASan, formatting, and verification - poorly written prose is 100% organic human

Thank you for the great work with tree-sitter!

Downstream issue prompting PR: dekobon/big-code-analysis#1058
Fixes #368

GrigoryEvko added a commit to GrigoryEvko/tree-sitter-cpp that referenced this pull request Sep 17, 2026
An object-like macro with a replacement list of one identifier expands to that identifier. When
the identifier is the name of a function-like macro, the group after the alias is an argument
list ([cpp.rescan] p1). In bde, `#define P_ BSLIM_TESTUTIL_P_` makes `P_(LINE)` a call of
`BSLIM_TESTUTIL_P_`. The seed gave such an alias only the object-macro row. The rule of tree-sitter#370 reads
an object row with no function row as a macro with no arguments, so the seed must also give the
alias the function row of its target.

The reader of the #define lines gives the identifier of such a list as the alias target. The
collector gives each alias the macro kinds of each name on its chain of aliases. A chain that
comes back to a name stops there.

Over the 64 corpus projects, 53,624 names are aliases. In 38 projects, 6,710 seed rows get the
word function-macro: WebKit 3,109, boost 2,120, llvm-project 420, mongo 217, root 112, pytorch 90,
godot 78, hhvm 58, and 57 or less in each of the 30 other projects. The collector adds no row,
removes no row, and changes no type word. The seed of simdjson does not change.

The scanner of the base reads a macro row as a row with a macro bit, so no tree changes, with a
seed or with no seed. One class stays open: an object-like macro with a list of more tokens that
ends with the name of a function-like macro. A line scan finds 97 such names.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tree_sitter_cpp_external_scanner_deserialize doesn't bound length against the size of the delimiter buffer

1 participant