Skip to content

fix(arm): enable neon fp16 kernels via march auto-detect and runtime dispatch - #723

Open
richyreachy wants to merge 12 commits into
alibaba:mainfrom
richyreachy:feat/turbo_arm_dispatch
Open

fix(arm): enable neon fp16 kernels via march auto-detect and runtime dispatch#723
richyreachy wants to merge 12 commits into
alibaba:mainfrom
richyreachy:feat/turbo_arm_dispatch

Conversation

@richyreachy

Copy link
Copy Markdown
Collaborator

Problem

The ARM -march auto-detect never enables -march=armv8.2-a+fp16, so __ARM_FEATURE_FP16_VECTOR_ARITHMETIC is never defined and all native NEON FP16 kernel paths are dead code:

  • _setup_armv8_march() only probed -march=armv8, which GCC rejects outright (cc1: error: bad value 'armv8'); the per-file NEON flag was -march=armv8-a, which does not imply FEAT_FP16 either.
  • As a result, all fp16 distance computations on aarch64 silently fell back to the cvt-to-fp32 path (ailego) or plain scalar kernels (turbo).

Simply bumping the global march flag is not an option: it would emit FP16 instructions unconditionally and SIGILL on ARMv8.0 CPUs (e.g. Graviton1, Cortex-A72 / Raspberry Pi 4). Runtime CPU feature detection is required, mirroring the existing x86 AVX512FP16 setup.

Changes

CMake (cmake/option.cmake)

  • Fix _setup_armv8_march() to probe armv8-a before armv8 (GCC compatibility).
  • Add setup_compiler_march_for_arm(VAR_NEON VAR_NEON_FP16): probes armv8.2-a+fp16 and falls back to armv8-a when the compiler cannot target FEAT_FP16.

Runtime detection (ailego/internal/cpu_features)

  • New CpuFeatures::FP16() + static_flags_.FP16: reads getauxval(AT_HWCAP) & HWCAP_ASIMDHP on Linux/aarch64, always true on Apple Silicon, compile-time macro fallback elsewhere.

ailego math kernels

  • Split native FP16 code into dedicated *_neonfp16.cc TUs (inner product, squared euclidean, MIPS euclidean), compiled with -march=armv8.2-a+fp16; baseline *_neon.cc TUs keep the cvt-to-fp32 path only.
  • Dispatch TUs are compiled with the FP16 march flag (mirroring the x86 avx512fp16 setup) and gate the NEONFP16 kernels behind CpuFeatures::static_flags_.FP16 at runtime.

turbo

  • New distance/neon_fp16/fp16/ kernel family (squared euclidean, cosine, inner product + batch variants) using native vfmaq_f16 (8 lanes/op, 4 independent accumulators, widened to FP32 only for the final reduction).
  • New kCpuFeatureNeonFp16 mask bit wired to static_flags_.FP16; registry rows added for kFp16 (all three metrics) and kRaw+fp16 ahead

@richyreachy

Copy link
Copy Markdown
Collaborator Author

分发链路

请求 kAuto 时按表中行序优先匹配,每行要过两道关——CpuSupports(kernel_arch) 和 HasRequiredCpuFeatures。在 ARM 上:

  • x86 行全部跳过:AVX512/AVX2/SSE2 行的 CpuSupports 在 ARM 上 flags 全为 false,直接过滤,不会误选
  • NEON-FP16 行(三种度量都有):需要同时满足
    • 编译期:ZVEC_HAVE_ARM_FP16_KERNEL 已定义(CMake 探测到编译器支持 armv8.2-a+fp16,且非 iOS/MSVC)
    • 运行期:flags.FP16 为 true(Linux 上查 HWCAP_ASIMDHP,Apple Silicon 恒 true)
      满足则命中原生 fp16 内核
  • 普通 NEON 行:只要求 flags.NEON(aarch64 上恒 true),无额外 feature mask。老 CPU(如 armv8.0 的 Cortex-A72/树莓派4)或宏未编译进来时回退到这里
  • scalar 行:兜底

所以 kAuto 请求下三种度量(SE/Cosine/IP)在任何 ARM 环境都能拿到非空内核,且不会 SIGILL——fp16 指令的翻译单元只有运行时确认 FEAT_FP16 存在才会被选中。显式请求 kNEON 也能成功(命中 NEON-FP16 或普通 NEON 行)。

@richyreachy

Copy link
Copy Markdown
Collaborator Author

编译期

必须全部满足以下条件,才会定义 ZVEC_HAVE_ARM_FP16_KERNEL=1 并给 neon_fp16/*.cc 注入 -march=armv8.2-a+fp16

  1. 非 iOSNOT IOS
  2. 非 MSVC 编译器(NOT MSVC
  3. AUTO_DETECT_ARCH=ON 或者是 Android 构建(Android 即使 build_android.sh 关掉了 AUTO_DETECT_ARCH 也生效,因为这个探测针对的是交叉工具链而非宿主机)
  4. 目标架构是 arm/arm64(由 HOST_ARCH 或回退的 CMAKE_SYSTEM_PROCESSOR 解析出 _TURBO_TARGET_ARCH
  5. 编译器探测通过check_cxx_source_compiles-march=armv8.2-a+fp16 下实际编译一段测试代码,要求:
    • __aarch64__ 目标(所以 32 位 armv7 必然失败,宏不会打开)
    • 定义了 __ARM_FEATURE_FP16_VECTOR_ARITHMETIC
    • vdupq_n_f16 / vfmaq_f16 等 intrinsic 能编译通过

任一条件不满足 → 宏关闭,则表行不存在,编译出的二进制里没有 FEAT_FP16 路径。

运行期

即使编译进去了,表行还带有 kCpuFeatureNeonFp16 掩码,dispatch 时经 HasRequiredCpuFeatures 检查:

运行环境 判定方式 结果
Linux aarch64(含 Android) getauxval(AT_HWCAP) & HWCAP_ASIMDHP 由内核上报,armv8.2+ 且实现了 FEAT_FP16 的 CPU 为 true
Apple Silicon(macOS aarch64) 直接返回 true A11 及之后全支持
其他 NEON 平台 信任编译期 __ARM_FEATURE_FP16_VECTOR_ARITHMETIC 编译目标带 fp16 就为 true
非 ARM false

CPU 不支持(如 Cortex-A72 / 树莓派 4,armv8.0)→ 该行被跳过,自动回退到普通 NEON 内核,不会执行非法指令。

典型场景对照

  • 生效:Linux/Android arm64 + GCC/Clang + armv8.2+ CPU(如骁龙 845 之后、Graviton2/3、鲲鹏 920);Apple Silicon Mac
  • 不生效:iOS 构建;32 位 armv7;老旧编译器探测失败;桌面 Linux 构建时手动 AUTO_DETECT_ARCH=OFF;运行在 armv8.0 CPU 上(编译进去但运行时回退)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant