[C++][R] Arrow 26.0.0 fails to build on CRAN macOS builders: bundled simdjson C++20 concepts don't compile with the macOS 11 SDK #55267

AI-generated summary - needs double checking before resolving, but underlying problem is one I've confirmed as real.

The R package arrow 26.0.0 fails to install on all three CRAN macOS flavours (r-release-macos-x86_64, r-oldrel-macos-x86_64, r-oldrel-macos-arm64). Arrow 25.0.0 built fine on the same machines.

Install logs:

• https://www.r-project.org/nosvn/R.check/r-release-macos-x86_64/arrow-00install.html
• https://www.r-project.org/nosvn/R.check/r-oldrel-macos-x86_64/arrow-00install.html
• https://www.r-project.org/nosvn/R.check/r-oldrel-macos-arm64/arrow-00install.html

All three fail identically while compiling arrow_json sources that include the bundled simdjson headers:

In file included from .../tools/cpp/src/arrow/extension/fixed_shape_tensor.cc:22:
In file included from .../_deps/simdjson-src/include/simdjson.h:43:
In file included from .../_deps/simdjson-src/include/simdjson/base.h:12:
.../_deps/simdjson-src/include/simdjson/concepts.h:122:10: error: expected concept name with optional arguments
    } -> std::convertible_to<typename std::remove_cvref_t<T>::value_type>;
         ^
.../_deps/simdjson-src/include/simdjson/concepts.h:143:8: error: no member named 'ranges' in namespace 'std'
  std::ranges::input_range<T> && !string_like<T> && !concepts::string_view_keyed_map<T>;
  ~~~~~^
...
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make[2]: *** [src/arrow/CMakeFiles/arrow_json.dir/extension/fixed_shape_tensor.cc.o] Error 1
make[2]: *** [src/arrow/CMakeFiles/arrow_json.dir/extension/opaque.cc.o] Error 1

Toolchain on the CRAN builders, from the logs:

• Compiler: AppleClang 14.0.3.14030022
• CMAKE_OSX_SYSROOT: /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk
• CMake 3.31.6, -std=gnu++20

Cause

simdjson became a dependency of ARROW_JSON in #50587 (GH-50581), so 26.0.0 is the first release where the R package's bundled C++ build compiles simdjson's headers. The libc++ shipped with the macOS 11 SDK does not have std::ranges and its <concepts> support is incomplete, so simdjson/concepts.h does not compile.

#50587 anticipated this and added a workaround in cpp/cmake_modules/ThirdpartyToolchain.cmake:

# The macOS 11.3 SDK has incomplete C++20 concepts support, which prevents
# simdjson headers from compiling. Disable simdjson concepts for this SDK.
if(CMAKE_OSX_SYSROOT AND CMAKE_OSX_SYSROOT MATCHES "MacOSX11\\.3\\.sdk$")
  message(STATUS "Disabling simdjson concepts for macOS SDK 11.3")
  target_compile_definitions(simdjson PUBLIC SIMDJSON_CONCEPT_DISABLED=1)
endif()

The guard matches only a sysroot path ending in MacOSX11.3.sdk. On the CRAN builders the SDK is installed as MacOSX11.sdk, so the regex never matches and SIMDJSON_CONCEPT_DISABLED is not defined. The logs contain no "Disabling simdjson concepts" line, confirming the branch was skipped.

This is also why the nightly test-r-macos-as-cran job didn't catch it: dev/tasks/r/github.macos.cran.yml installs the SDK at /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk and sets SDKROOT to that path, which the regex matches. The CI job and the real CRAN machines differ only in the directory name.

Related: #53945 is a separate C++20 library gap (std::bit_width) against the same SDK, but that code is not in 26.0.0, so it only affects nightlies.

Possible fixes

1. Replace the path match with a feature check (check_cxx_source_compiles on #include <ranges> plus a trivial std::convertible_to constraint) and define SIMDJSON_CONCEPT_DISABLED=1 when it fails. This covers any toolchain with incomplete concepts support rather than a single directory name.
2. Define SIMDJSON_CONCEPT_DISABLED=1 unconditionally for the bundled simdjson. Arrow appears not to use the concepts-only parts of the simdjson API, given that the existing workaround builds and passes CI with them disabled.
3. Minimal: loosen the regex to MacOSX11(\\.[0-9]+)?\\.sdk$.

Because this lives in cpp/, fixing it for CRAN requires a patch release of the R package (26.0.0.1) carrying the updated tools/cpp sources.

Component(s)

C++, R