Skip to content

Header-only layout, arch-aware patterns, IDA plugin - #4

Merged
scizzydo merged 19 commits into
masterfrom
scanner-overhaul
Oct 12, 2026
Merged

scizzydo merged 19 commits into
masterfrom
scanner-overhaul

Conversation

@scizzydo

@scizzydo scizzydo commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

Header-only layout, SIMD scanning, target-architecture-aware patterns, and an IDA 9.x plugin that searches databases with the same pattern syntax.

-Pattern.hpp  RuntimePattern.hpp  CompileTimePattern.hpp  XORPattern.hpp  ArmPattern.hpp
+CMakeLists.txt                    # INTERFACE target PatternScanner::PatternScanner (C++17)
+include/PatternScanner/
+├── Pattern.hpp                   # SSE2/NEON scan, find_all, match()/resolve(), patterns::arch
+├── RuntimePattern.hpp            # + target arch argument
+├── CompileTimePattern.hpp
+├── XORPattern.hpp                # masks XOR'd too, unmask()
+├── ArmPattern.hpp                # now builds as C++17
+├── ArmXORPattern.hpp             # new
+└── RuntimeArmPattern.hpp         # new: arm64 assembly parsed at runtime
+tests/                            # separate CMake project
+plugins/ida/                      # IDA plugin, builds against the user's SDK + ida-cmake

Consumers: #include <PatternScanner/XORPattern.hpp> with include/ on the include path, or target_link_libraries(app PRIVATE PatternScanner::PatternScanner) via add_subdirectory / FetchContent.

Per-pattern target architecture replaces the #ifdef PATTERNS_ARM64 branches, so /d /r, X<len> and /a follow the code being scanned rather than the compiler target:

RuntimePattern(text, arch = host_arch)   # x86 | x64 | arm64
ArmPattern / ArmXORPattern / RuntimeArmPattern -> always arm64
resolve(match, end, &is_value)           # address vs value read from the code

Parsing is shared: one byte-pattern parser (detail::parse_bytes) and one arm64 parser (detail::parse_arm) behind all pattern classes.

IDA plugin (plugins/ida): Shift-Alt-B searches next/previous from the cursor or lists all matches with their /d /r results; Ctrl-Shift-B repeats. Byte patterns follow the database's architecture.

search (dialog) / repeat
  pscan::parse(text, kind)          # RuntimePattern(db_arch) | RuntimeArmPattern
  pscan::for_each_match(seg by seg, loaded runs only)
    buffer at the segment's 4KB page offset   # /a and ADRP page math
    Pattern::match -> resolve -> map back to ea

Evidence

  • Before: master has no tests. ArmPattern.hpp didn't compile as C++17 despite the documented macros; on an arm64 host, byte-pattern /d decoded ADRP even for x86 code (and vice versa).
    After (Linux, clang 19 and gcc 14, fresh build dirs):
    tests/       clang 19/19 passed, gcc 19/19 passed
                 test_patterns (+ PATTERNS_NO_SIMD), test_cpp17, test_arm, example_usage,
                 test_artifacts_cpp{17,20}_{O0,O1,O2,O3,Os,O2_lto,O3_lto}
    plugins/ida  test_search (idalib, IDA 9.3) on x64 / x86 / arm64 ELF fixtures: passed
                 plugin loads in headless idat and registers both actions
    
  • test_artifacts: no pattern text from any compile-time type, no XOR plaintext bytes/words/masks, across optimization levels and LTO. Checked that it catches leaks by breaking the library on purpose:
    XOR decode without volatile  -> fails all 4 XOR types on clang (O2, O3, LTO) and gcc (O2, O3)
    macros built at runtime      -> fails COMPILETIME/ARM/XOR_PATTERN on clang C++17, every level
    
    A 40-pattern signature file (one function, like real usage): 0/40 leak on either compiler at any level or with LTO.
  • idalib test catches a scan buffer at the wrong page offset (ADRP result) and the wrong /a step (x86 vs x64 database).

Merge Danger

Door: two-way

Plain revert; nothing persistent changes.

Blast Radius: consumers

  • Header paths moved: #include "Pattern.hpp" from the repo root becomes #include <PatternScanner/Pattern.hpp> with include/ on the include path.
  • Pattern::find(bytes, size) is no longer virtual; subclasses override match() instead. Pattern's destructor is protected (deleting through Pattern* no longer compiles).
  • XORPattern::mask() now returns XOR'd bytes; use unmask() for the original.
  • On arm64 hosts, behaviour for existing code is unchanged (the default arch is still the compile target).
  • Byte patterns: ?8 was a whole-byte wildcard and is now a nibble (4? / ?8 are new). arm64: ADD/SUB immediates, SP register forms, shifted ORR/EOR/TST, and LDP/STP [Rn] now encode the way assemblers do, so patterns that silently never matched (or matched too much) now match exactly.
  • /a aligns to the absolute address, not to the start of the buffer passed in.
  • CI runs the library tests on Linux, Windows (MSVC) and macOS. Not tested: the plugin build on Windows / macOS. The plugin UI (dialog, chooser, menu, hotkeys) was only checked headless.

🤖 Generated with Claude Code

scizzydo and others added 19 commits October 2, 2026 18:37
Files were a mix of LF and CRLF, so edits from Windows showed as whole-file
diffs. Store everything as LF; checkouts get the platform's line endings.
No content changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Pattern: SSE2/NEON scan filtering on two anchor bytes, with a portable
  64-bit fallback (PATTERNS_NO_SIMD); aligned scans align to absolute
  addresses
- Compile-time patterns are consteval (PATTERNS_CONSTEVAL) so the pattern
  text never reaches the binary; string literals for every pattern type
  (_rtpattern, _ctpattern, _xorpattern, _armpattern)
- XORPattern: keys per definition site (source location / __COUNTER__) and
  per build; decoded only on the stack during find()
- Patterns are no longer padded; bounds checks when parsing X offsets
- tests/: CMake project with correctness tests, fuzzing against a naive
  scanner, benchmarks and a multi-pattern pointer table example

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Aligned scans on arm64 step by the 4-byte instruction size, but the
reference scanner used sizeof(void*), so the fuzz test failed there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found by checking patterns against clang-assembled instructions:
- Each line is now parsed on its own (trimmed, uppercased). Previously a
  mnemonic only ended at a space, so NOP/RET before another line, the line
  before the options, tab-indented lines and CRLF all became wildcards
- B/BL #imm and B/BL #? encoded offset 0 under a full mask
- ORR Xd, Xn, #imm with an immediate that fits in 32 bits had the wrong N bit
- ADD/SUB #imm, LSL #12 (with a space) left the shift unmasked
- BR/BLR ignored their register; RET Xn is supported
- /d: negative ADR/ADRP offsets were decoded as unsigned, ADRP + ADD into
  a different register returned just the page, and the ADRP follow-up
  could read past the scanned buffer

test_arm checks 70 concrete encodings, 65 wildcard match/no-match pairs,
line formats and /d. On Apple arm64 it also re-assembles its tables and
resolves /d on linker-relocated code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With a constexpr pattern, clang -O3 inlined find() and folded
pattern ^ keys into a constant, emitting the plaintext in __const (caught
by test_patterns in a CMake Release build). Reading the keys through
volatile keeps the XOR at runtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
__arm64__ is only defined by Apple's toolchain, so GCC/clang on Linux and
MSVC on Windows ARM64 silently got x86 /d semantics and alignment.
Pattern.hpp now defines PATTERNS_ARM64 from any of the three.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Pattern: public match()/resolve(), find_all() for many patterns over one range
- XORPattern: masks stored XOR'd with their own key stream; unmask() for originals
- ArmXORPattern: ArmPattern with XOR'd instruction words and masks
- Protected Pattern destructor, size_t loop indices
- tests: example_usage replaces example_pointer_table, test_tls (Windows),
  multi_scan uses match()/resolve(), ArmXORPattern tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Consumers add include/ to their include path and use
#include <PatternScanner/XORPattern.hpp>. Tests include the same way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Root CMakeLists.txt for add_subdirectory / FetchContent users. The tests
link the target instead of setting the include path themselves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- patterns::arch (x86/x64/arm64) decides /d /r, X<len> and /a per pattern instead
  of by #ifdef; defaults to the compile target, Arm patterns are always arm64.
  RuntimePattern takes it as an argument, for scanning another architecture's code.
- resolve() can report whether the result is an address or a value.
- RuntimeArmPattern: ArmPattern from text known at runtime.
- ArmPattern/ArmXORPattern compile as C++17 (no std::countr_zero, literals
  guarded), covered by test_cpp17.
- Arch-specific tests now run on every host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
plugins/ida: find next/previous or all matches, with /d /r results, following
the database's architecture. Builds against the user's IDA SDK via ida-cmake;
optional idalib test on x86/x64/arm64 ELF fixtures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- One byte pattern parser (detail::parse_bytes / Pattern::parse) behind
  RuntimePattern, CompileTimePattern, XORPattern and pattern_length; one arm64
  parser (detail::parse_arm) behind ArmPattern, RuntimeArmPattern,
  ArmXORPattern and total_instructions. ArmXORPattern no longer needs friend
  access to ArmPattern.
- test_artifacts: no pattern text for any compile-time type, no XOR plaintext
  bytes/words/masks, at -O0..-O3/-Os with debug info, C++17 and C++20. Built
  once per pattern so find() inlines and optimizers get the chance to fold.
  Replaces the narrower self-scan checks in test_patterns and test_arm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One build per optimization level instead of one per pattern: all patterns are
scanned from a single non-static function, the shape of a real signature file
and one in which gcc and clang both inline find() and would fold XOR
plaintext back in. LTO variants added. Every leak is reported, not only the
first. 14 builds per compiler instead of 60.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- STP (pre-index, signed offset, post-index): LDP with the load bit clear.
- MOV encoded as assemblers pick it: ADD #0 with SP, ORR from the zero
  register for registers, MOVZ/MOVN/ORR (bitmask) for immediates; MOV Rd, #?
  matches MOVZ or MOVN. Encodings checked against clang's assembler.
- Unsupported instructions still become wildcards, but the IDA plugin now lists
  them and asks before searching.
- Regression test: an IDA-printed prologue (STP/MOV) must skip a near-miss
  SUB SP prologue, in test_arm and in the idalib search test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ads and stores, PAC

One parser for single-register loads/stores (LDR/STR/LDRB/STRB/LDRH/STRH/LDRSW and the LDUR/STUR forms).
LDR/STR offsets the scaled form can't hold are encoded as LDUR/STUR, as assemblers do, instead of a wrong
immediate. FP/SIMD loads/stores and out-of-range offsets become wildcards rather than never-matching patterns.
Fixes ADD/SUB register-form wildcards never matching an odd Rm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes for PR #4, each checked against clang's assembler:

arm64 encoder
- ORR/AND/EOR/TST: shift amount after a space ("LSL #3") was read as 0, and #? never a wildcard.
- ADD/SUB/CMP/CMN immediates: negative values are the opposite operation, multiples of 0x1000 the LSL #12
  form; #? no longer pins the shift bit (missed SUB SP, SP, #1, LSL #12).
- ADD/SUB register with SP: extended UXTX/UXTW form, as assemblers emit.
- MOV Rd, #? no longer matches MOVK. LDP/STP [Rn] is the signed form with offset 0, not a partial wildcard.
- ADR #-1 was a wildcard. With Rd (Rt1) "?", the width comes from Rn (Rt2).
- A line starting with X followed by a letter (XPACLRI) isn't the X marker.

arm64 /d
- MOVZ shifted by hw instead of hw*16; MOVN decoded.
- One load/store decoder for all general-purpose immediate forms (LDRB/LDRSW/STRB/STRH/LDUR/STUR): signed
  offsets, pre/post-index, and STR offsets >= 0x4000 that threw.
- B.cond, CBZ/CBNZ and LDR (literal) resolve to their target.
- ADRP chains skip instructions that leave the register alone (another ADRP between ADRP and ADD).

Byte patterns
- Nibble wildcards ("4?", "?8"); "?8" used to silently become a whole-byte wildcard.
- /r /d reads use memcpy (misaligned loads were UB, caught by UBSan).
- stoi accumulates unsigned: long immediates no longer overflow a signed int.

IDA plugin: an over-long arm64 line threw out of the dialog handler (listing unsupported lines ran outside
the try) and would take IDA down. CI: also run the library tests on Linux.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scizzydo
scizzydo merged commit 767a9b3 into master Oct 12, 2026
3 checks passed
@scizzydo
scizzydo deleted the scanner-overhaul branch October 12, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant