diff --git a/config/pkg/tool/llvm-tools.yml b/config/pkg/tool/llvm-tools.yml index d80479434..e2b7f6a4d 100644 --- a/config/pkg/tool/llvm-tools.yml +++ b/config/pkg/tool/llvm-tools.yml @@ -3,7 +3,9 @@ llvm-tools: artifact: source: type: custom - binary: custom + binary: + linux-x86_64: { type: custom, base-url: 'https://dl.static-php.dev/v3/deps/llvm-tools/' } + linux-aarch64: { type: custom, base-url: 'https://dl.static-php.dev/v3/deps/llvm-tools/' } metadata: license-files: [llvm/LICENSE.TXT] license: Apache-2.0-with-LLVM-exception diff --git a/docs/en/develop/artifact-model.md b/docs/en/develop/artifact-model.md index ae03d18a2..4cb34f480 100644 --- a/docs/en/develop/artifact-model.md +++ b/docs/en/develop/artifact-model.md @@ -454,10 +454,26 @@ artifact: Delegates download logic entirely to a PHP class under `src/Package/Artifact/`. If `func` is not specified, the class's default download method is called. -- **Optional**: `func` — name of the specific method to invoke in the implementation class +- **Optional**: + - `func` — name of the specific method to invoke in the implementation class + - `base-url` — an extra base URL the implementation can read from its artifact config, for example to assemble a download URL around a filename it computes at runtime ```yaml artifact: source: type: custom + binary: + linux-x86_64: + type: custom + base-url: 'https://dl.static-php.dev/v3/deps/llvm-tools/' +``` + +A custom downloader receives the `Artifact` and reads the option through `getBinaryDownloadConfig()`: + +```php +public function downBinary(ArtifactDownloader $downloader, Artifact $artifact): DownloadResult +{ + $base_url = $artifact->getBinaryDownloadConfig()['base-url']; + // ... append the filename computed at runtime and download +} ``` diff --git a/docs/en/develop/system-build-tools.md b/docs/en/develop/system-build-tools.md index 81d8e8052..f3be79788 100644 --- a/docs/en/develop/system-build-tools.md +++ b/docs/en/develop/system-build-tools.md @@ -66,7 +66,7 @@ Doctor additionally checks: The Linux default is `ZigToolchain`. The toolchain declares the `zig` tool package, so every build installs it even when `spc doctor` was never run (Doctor can also install it when it is missing, and `--auto-fix` checks the rest of the environment). It also declares `llvm-compiler-rt`, which every build installs before anything else: it supplies the profile runtime, the `crtbegin`/`crtend` objects and `libclang_rt.cpu_model.a` that the zig-shipped Clang omits. Its sources come from a trimmed compiler-rt archive published under `dl.static-php.dev/v3/deps/compiler-rt/`, keyed by the clang version zig reports, so a build downloads about 140KB instead of the 150MB+ llvm-project tarball that LLVM has required since release 22. That archive is the only source: when the archive for the zig version in use is missing, the build fails with instructions for publishing it. Because compiler-rt sources are resolved from the clang zig ships, a machine that has no zig yet installs zig on its first build and picks compiler-rt up on the next one. The objects are compiled locally (a few seconds) and are rebuilt when zig is upgraded or when `SPC_TARGET` changes the target triple. When a musl target is built on a non-musl host, Doctor also checks the musl runtime/wrapper required by that target. The legacy `MuslToolchain` additionally expects the musl cross toolchain under `/usr/local/musl`. A native Alpine build does not need these cross-host components. -The optional `--with-llvm-tools` build option installs the `llvm-tools` package (`llvm-objcopy`, `llvm-strip`, `llvm-profdata`), which is needed for LTO, PGO and shared-library builds because the host binutils cannot process the archives/bitcode Zig produces. It is never required: without it the build prefers the host binutils (zig's own `objcopy` only implements a subset of the options) and reports a failing strip as a warning instead of aborting the build. No prebuilt `llvm-tools` binaries are published yet, so enabling the option currently falls back to building LLVM from source; point `OBJCOPY` and `STRIP` at a system LLVM installation instead if you need LLVM binutils today. The option is only honoured for Linux targets built with `ZigToolchain` and after `zig` is installed; otherwise the build prints a warning explaining why it was ignored. +Under `ZigToolchain`, `llvm-objcopy`, `llvm-strip` and `llvm-profdata` come from the `llvm-tools` package. Cross builds, where `SPC_TARGET` names a different architecture or OS than the host, require it, because the host binutils only understand the host architecture: the toolchain then declares `llvm-tools` alongside `llvm-compiler-rt`, and the build installs it automatically. Native builds keep using the host binutils (zig's own `objcopy` only implements a subset of the options), and a failing strip or debug-info split only produces a warning instead of aborting the build; run `bin/spc install-pkg llvm-tools` to use the LLVM tools there as well. The package downloads prebuilt static binaries from `dl.static-php.dev/v3/deps/llvm-tools/`, selected by the host architecture (not `SPC_TARGET`) and keyed by the clang version zig reports, and verifies them against the published SHA-256 checksum. When no archive is published for the zig version in use yet (or the checksum does not match), it falls back to building LLVM from source, which takes much longer. Like compiler-rt, it needs zig on disk to know that version, so a cross build on a machine without zig installs zig first and picks up `llvm-tools` on the next run. ::: tip The target string and the host distribution are different concepts. A glibc distribution can produce a musl target through Zig, while a `*-gnu.*` target remains dynamically linked to the requested glibc ABI. diff --git a/docs/en/guide/cli-reference.md b/docs/en/guide/cli-reference.md index 48e53e7ca..f5678bd8f 100644 --- a/docs/en/guide/cli-reference.md +++ b/docs/en/guide/cli-reference.md @@ -96,7 +96,6 @@ These flags apply only to the combined `build:php` target. To build a specific S |--------------------------------------|-------|--------------------------------------------------------------------------------------------------------| | `--no-strip` | | Keep debug symbols; do not strip the binary | | `--with-upx-pack` | | Compress the output binary with UPX (install first with `spc install-pkg upx`; Linux and Windows only) | -| `--with-llvm-tools` | | Use LLVM binutils (`llvm-objcopy`/`llvm-strip`/`llvm-profdata`) instead of the host ones (Zig toolchain, Linux only) | | `--disable-opcache-jit` | | Disable OPcache JIT | | `--with-config-file-path=` | | Directory where PHP looks for `php.ini` (default: `/usr/local/etc/php`) | | `--with-config-file-scan-dir=` | | Directory PHP scans for additional `.ini` files (default: `/usr/local/etc/php/conf.d`) | diff --git a/docs/en/guide/first-build.md b/docs/en/guide/first-build.md index 095b2d750..a05230abb 100644 --- a/docs/en/guide/first-build.md +++ b/docs/en/guide/first-build.md @@ -119,7 +119,6 @@ spc build:php "bcmath,phar,zlib,openssl,curl" --build-cli --build-micro | `--no-strip` | Keep debug symbols; do not strip the binary | | `-I key=value` | Hard-compile an INI option into PHP | | `--with-upx-pack` | Compress output with UPX (run `spc install-pkg upx` first) | -| `--with-llvm-tools` | Use LLVM binutils for Zig builds on Linux (LTO/PGO/shared output; needs the `llvm-tools` package, whose prebuilt binaries are not published yet) | Example — baking in a larger memory limit and disabling the `system` function: diff --git a/docs/zh/develop/artifact-model.md b/docs/zh/develop/artifact-model.md index 51875a829..ea4af42da 100644 --- a/docs/zh/develop/artifact-model.md +++ b/docs/zh/develop/artifact-model.md @@ -450,11 +450,27 @@ artifact: 完全自定义的下载逻辑，由 `src/Package/Artifact/` 目录下对应的 PHP 类实现。如果不指定 `func`，则调用类的默认下载方法。 -- **选填**：`func` — 调用实现类中的指定方法名 +- **选填**： + - `func` — 调用实现类中的指定方法名 + - `base-url` — 实现类可从其 artifact 配置中读取的额外基础 URL，例如配合运行时计算出的文件名拼接下载地址 ```yaml artifact: source: type: custom + binary: + linux-x86_64: + type: custom + base-url: 'https://dl.static-php.dev/v3/deps/llvm-tools/' +``` + +自定义下载器会收到 `Artifact`，并通过 `getBinaryDownloadConfig()` 读取该选填项： + +```php +public function downBinary(ArtifactDownloader $downloader, Artifact $artifact): DownloadResult +{ + $base_url = $artifact->getBinaryDownloadConfig()['base-url']; + // ... 拼接运行时计算出的文件名并下载 +} ``` diff --git a/docs/zh/develop/system-build-tools.md b/docs/zh/develop/system-build-tools.md index 18655d714..e02c14c17 100644 --- a/docs/zh/develop/system-build-tools.md +++ b/docs/zh/develop/system-build-tools.md @@ -66,7 +66,7 @@ Doctor 还会检查： Linux 默认使用 `ZigToolchain`。工具链会把 `zig` 工具 Package 声明为自己的依赖，因此即使从未运行过 `spc doctor`，每次构建也会自动安装它（Doctor 同样会在缺失时安装，`--auto-fix` 负责检查其余环境）。它还会声明 `llvm-compiler-rt`，每次构建都会先安装该 target：它提供 Zig 自带 Clang 所缺失的 profile 运行时、`crtbegin`/`crtend` 目标文件和 `libclang_rt.cpu_model.a`。其源码来自发布在 `dl.static-php.dev/v3/deps/compiler-rt/` 下的裁剪版 compiler-rt 归档，以 zig 报告的 clang 版本作为键，因此一次构建只需下载约 140KB，而不是 LLVM 自 22 版起才提供的 150MB+ llvm-project 单体包。该归档是唯一来源：若当前 zig 版本对应的归档尚未发布，构建会直接失败并提示如何发布它。由于 compiler-rt 的源码要按 zig 自带的 clang 版本来解析，尚无 zig 的机器会在第一次构建时装上 zig，并在下一次构建时补上 compiler-rt。这些对象在本地编译（只需几秒），并在 zig 升级或 `SPC_TARGET` 改变目标 triple 时重新构建。在非 musl 宿主机上构建 musl 目标时，Doctor 还会检查该目标所需的 musl 运行时/wrapper。旧式的 `MuslToolchain` 还要求 `/usr/local/musl` 下存在 musl 交叉工具链。原生 Alpine 构建不需要这些跨宿主机组件。 -可选的 `--with-llvm-tools` 构建选项会安装 `llvm-tools` 包（`llvm-objcopy`、`llvm-strip`、`llvm-profdata`）；LTO、PGO 和共享库构建需要它，因为系统 binutils 无法处理 Zig 产生的归档/bitcode。它并非必需：未安装时会优先使用系统 binutils（Zig 自带的 `objcopy` 只实现了部分选项），且 strip 失败只会输出警告而不会中断构建。目前尚未发布 `llvm-tools` 的预构建二进制，因此启用该选项会回退到从源码构建 LLVM；如果现在就想要 LLVM binutils，可改为把 `OBJCOPY`/`STRIP` 指向系统安装的 LLVM 工具。该选项仅在 Linux 目标 + `ZigToolchain` 且 `zig` 已安装时生效，其余情况构建过程会打印警告说明被忽略的原因。 +在 `ZigToolchain` 下，`llvm-objcopy`、`llvm-strip` 和 `llvm-profdata` 由 `llvm-tools` 包提供。交叉构建（`SPC_TARGET` 指定的架构或操作系统与宿主机不同）必须使用它，因为系统 binutils 只认识宿主机架构：此时工具链会在 `llvm-compiler-rt` 之外再声明 `llvm-tools`，构建过程会自动安装它。本机构建继续使用系统 binutils（Zig 自带的 `objcopy` 只实现了部分选项），strip 或拆分调试信息失败只会输出警告而不会中断构建；如需在本机构建中也使用 LLVM 工具，可运行 `bin/spc install-pkg llvm-tools`。该包会从 `dl.static-php.dev/v3/deps/llvm-tools/` 下载预构建的静态二进制：按宿主机架构（而非 `SPC_TARGET`）选择，以 zig 报告的 clang 版本作为键，并用发布的 SHA-256 校验值进行校验。若当前 zig 版本对应的归档尚未发布（或校验失败），则回退到从源码构建 LLVM，耗时会长得多。与 compiler-rt 一样，它需要磁盘上已有 zig 才能确定该版本，因此在尚无 zig 的机器上进行交叉构建时，会先安装 zig，并在下一次构建时补上 `llvm-tools`。 ::: tip 目标字符串和宿主发行版是两个不同概念。glibc 发行版可以通过 Zig 生成 musl 目标，而 `*-gnu.*` 目标仍会动态链接到指定的 glibc ABI。 diff --git a/docs/zh/guide/cli-reference.md b/docs/zh/guide/cli-reference.md index 406dd7b54..31bd95d2b 100644 --- a/docs/zh/guide/cli-reference.md +++ b/docs/zh/guide/cli-reference.md @@ -96,7 +96,6 @@ spc build:php [options] |--------------------------------------|------|--------------------------------------------------------| | `--no-strip` | | 保留调试符号，不精简二进制 | | `--with-upx-pack` | | 用 UPX 压缩产物（需先 `spc install-pkg upx`；仅 Linux 和 Windows） | -| `--with-llvm-tools` | | 使用 LLVM binutils（`llvm-objcopy`/`llvm-strip`/`llvm-profdata`）替代系统工具（仅 Zig 工具链、Linux） | | `--disable-opcache-jit` | | 禁用 OPcache JIT | | `--with-config-file-path=` | | PHP 查找 `php.ini` 的目录（默认：`/usr/local/etc/php`） | | `--with-config-file-scan-dir=` | | PHP 扫描追加 `.ini` 文件的目录（默认：`/usr/local/etc/php/conf.d`） | diff --git a/docs/zh/guide/first-build.md b/docs/zh/guide/first-build.md index c49b646be..018df63e6 100644 --- a/docs/zh/guide/first-build.md +++ b/docs/zh/guide/first-build.md @@ -121,7 +121,6 @@ spc build:php "bcmath,phar,zlib,openssl,curl" --build-cli --build-micro | `--no-strip` | 保留调试符号，不精简二进制 | | `-I key=value` | 硬编译 INI 选项到 PHP 中 | | `--with-upx-pack` | 用 UPX 压缩产物（需先 `spc install-pkg upx`） | -| `--with-llvm-tools` | Zig + Linux 构建时使用 LLVM binutils（LTO/PGO/共享库产物；依赖 `llvm-tools` 包，其预构建二进制尚未发布） | 硬编译 INI 的例子——预设更大的内存限制，并禁用 `system` 函数： diff --git a/src/Package/Artifact/llvm_tools.php b/src/Package/Artifact/llvm_tools.php index 531c2b2a9..b774adc51 100644 --- a/src/Package/Artifact/llvm_tools.php +++ b/src/Package/Artifact/llvm_tools.php @@ -4,10 +4,10 @@ namespace Package\Artifact; +use StaticPHP\Artifact\Artifact; use StaticPHP\Artifact\ArtifactDownloader; use StaticPHP\Artifact\Downloader\DownloadResult; use StaticPHP\Artifact\Downloader\Type\CheckUpdateResult; -use StaticPHP\Artifact\Downloader\Type\GitHubRelease; use StaticPHP\Artifact\Downloader\Type\GitHubTokenSetupTrait; use StaticPHP\Attribute\Artifact\CustomBinary; use StaticPHP\Attribute\Artifact\CustomSource; @@ -39,15 +39,32 @@ public function downSource(ArtifactDownloader $downloader): DownloadResult 'linux-x86_64', 'linux-aarch64', ])] - public function downBinary(ArtifactDownloader $downloader): DownloadResult + public function downBinary(ArtifactDownloader $downloader, Artifact $artifact): DownloadResult { - $asset = 'llvm-tools-' . SystemTarget::getTargetArch() . '-linux-musl-' . $this->clangVersion() . '.txz'; - return new GitHubRelease()->download('llvm-tools', [ - 'type' => 'ghrel', - 'repo' => 'static-php/hosted', - 'match' => preg_quote($asset, '|'), - 'extract' => '{pkg_root_path}/llvm-tools', - ], $downloader); + // The tools run on the build host, so the host architecture selects the archive, not + // SPC_TARGET; they are static musl executables and run on any Linux distribution. + $clang = $this->clangVersion(); + // Pre-built archives: llvm-tools-{host-arch}-linux-musl-{clang-version}.txz plus a .sha256 sidecar. + $filename = 'llvm-tools-' . GNU_ARCH . "-linux-musl-{$clang}.txz"; + $base_url = $artifact->getBinaryDownloadConfig()['base-url'] ?? null; + if (!is_string($base_url) || $base_url === '') { + throw new DownloaderException('llvm-tools: binary config for ' . SystemTarget::getCurrentPlatformString() . ' does not define a base-url'); + } + $url = $base_url . $filename; + $path = DOWNLOAD_PATH . DIRECTORY_SEPARATOR . $filename; + default_shell()->executeCurlDownload($url, $path, retries: $downloader->getRetry()); + + // Published next to the archive by the llvm-tools workflow of static-php/hosted. + $checksum = default_shell()->executeCurl("{$url}.sha256", retries: $downloader->getRetry()); + $expected = is_string($checksum) ? strtolower((string) strtok(trim($checksum), " \t")) : ''; + if (preg_match('/^[0-9a-f]{64}$/', $expected) !== 1) { + throw new DownloaderException("Cannot fetch the checksum of {$filename} from {$url}.sha256"); + } + if (($actual = hash_file('sha256', $path)) !== $expected) { + unlink($path); + throw new DownloaderException("Hash mismatch for downloaded {$filename}. Expected {$expected}, got {$actual}"); + } + return DownloadResult::archive($filename, ['url' => $url], extract: '{pkg_root_path}/llvm-tools', verified: true, version: $clang); } #[CustomSourceCheckUpdate('llvm-tools')] diff --git a/src/Package/Target/php.php b/src/Package/Target/php.php index 0ff423400..1d96008fd 100644 --- a/src/Package/Target/php.php +++ b/src/Package/Target/php.php @@ -24,13 +24,11 @@ use StaticPHP\Package\PackageInstaller; use StaticPHP\Package\PhpExtensionPackage; use StaticPHP\Package\TargetPackage; -use StaticPHP\Package\ToolPackage; use StaticPHP\Registry\ArtifactLoader; use StaticPHP\Registry\PackageLoader; use StaticPHP\Runtime\SystemTarget; use StaticPHP\Toolchain\Interface\ToolchainInterface; use StaticPHP\Toolchain\ToolchainManager; -use StaticPHP\Toolchain\ZigToolchain; use StaticPHP\Util\DependencyResolver; use StaticPHP\Util\FileSystem; use StaticPHP\Util\InteractiveTerm; @@ -139,7 +137,6 @@ public function init(TargetPackage $package): void $package->addBuildArgument('extensions', InputArgument::REQUIRED, 'Comma-separated list of static extensions to build'); $package->addBuildOption('no-strip', null, null, 'build without strip, keep symbols to debug'); $package->addBuildOption('with-upx-pack', null, null, 'Compress / pack binary using UPX tool (linux/windows only)'); - $package->addBuildOption('with-llvm-tools', null, null, 'Use LLVM binutils (llvm-objcopy/llvm-strip/llvm-profdata) instead of the host ones (zig toolchain, linux only)'); // php configure and extra patch options $package->addBuildOption('disable-opcache-jit', null, null, 'Disable opcache jit'); @@ -266,25 +263,6 @@ public function resolveBuild(TargetPackage $package, PackageInstaller $installer $installer->addInstallPackage('upx'); } - // Zig's bundled binutils only implement a subset of the options these post-processing - // steps need (zig-objcopy reports --only-keep-debug as unimplemented), and the host - // binutils are used otherwise. The LLVM binutils are the opt-in replacement - // (--with-llvm-tools), added through the normal resolve/download pipeline so the outer - // download options (--no-download, --dl-prefer-binary, ...) are honoured. - // llvm-tools is never required, and llvm-compiler-rt is declared by the toolchain itself. - if ($package->getBuildOption('with-llvm-tools')) { - if (SystemTarget::getTargetOS() !== 'Linux') { - logger()->warning('--with-llvm-tools is only available for Linux targets; ignoring it.'); - } elseif (ToolchainManager::getToolchainClass() !== ZigToolchain::class) { - logger()->warning('--with-llvm-tools requires the zig toolchain; ignoring it.'); - } elseif (!$this->hasInstalledZigPackage()) { - // llvm-tools is versioned against zig's clang, so its downloader needs zig on disk. - logger()->warning('--with-llvm-tools needs an installed zig; run `spc doctor` first and re-run the build. Ignoring it for now.'); - } else { - $installer->addInstallPackage('llvm-tools')->preferBinary('llvm-tools'); - } - } - return [...$extensions_pkg, ...$additional_packages]; } @@ -569,21 +547,4 @@ private function makeStaticExtensionString(PackageInstaller $installer): string logger()->debug("Static extension configure args: {$str}"); return $str; } - - /** - * Whether the zig tool package is installed on disk. - * - * Checked through the registry rather than PackageInstaller::isPackageInstalled(): this runs - * inside a #[ResolveBuild] callback, i.e. before the installer has populated its resolved - * package list, so asking the installer would re-enter resolvePackages(). A custom registry - * does not have to ship a `zig` package either, and the caller only needs to warn in that case. - */ - private function hasInstalledZigPackage(): bool - { - if (!PackageLoader::hasPackage('zig')) { - return false; - } - $zig = PackageLoader::getPackage('zig'); - return $zig instanceof ToolPackage && $zig->isInstalled(); - } } diff --git a/src/Package/Tool/llvm_tools.php b/src/Package/Tool/llvm_tools.php index aa55402ca..4dfe30af8 100644 --- a/src/Package/Tool/llvm_tools.php +++ b/src/Package/Tool/llvm_tools.php @@ -78,8 +78,10 @@ private function buildPrivateZlib(ToolPackage $package): string $sources = implode(' ', array_map(fn ($unit) => escapeshellarg("{$source}/{$unit}.c"), $units)); $cc = getenv('CC') ?: 'cc'; $ar = getenv('AR') ?: 'ar'; + // HAVE_UNISTD_H is what zlib's configure sets on Unix: without it gz*.c call lseek/read/close + // undeclared, which clang >= 16 (zig) rejects as an error. shell()->cd($obj_dir) - ->exec("{$cc} -c -O2 -fPIC -DHAVE_HIDDEN -I" . escapeshellarg($source) . " {$sources}") + ->exec("{$cc} -c -O2 -fPIC -DHAVE_HIDDEN -DHAVE_UNISTD_H -I" . escapeshellarg($source) . " {$sources}") ->exec("{$ar} rcs " . escapeshellarg("{$prefix}/lib/libz.a") . ' *.o'); foreach (['zlib.h', 'zconf.h'] as $header) { diff --git a/src/StaticPHP/Artifact/Artifact.php b/src/StaticPHP/Artifact/Artifact.php index b5118d350..bd04fc753 100644 --- a/src/StaticPHP/Artifact/Artifact.php +++ b/src/StaticPHP/Artifact/Artifact.php @@ -281,6 +281,19 @@ public function getDownloadConfig(string $type): mixed return $this->config[$type] ?? null; } + /** + * Get the binary download config for a platform, defaulting to the current one. + * + * Custom binary fetchers receive the Artifact, not their own config entry, so this is how + * they read extra options such as a `base-url` used to assemble the download URL. + */ + public function getBinaryDownloadConfig(?string $platform = null): ?array + { + $platform ??= SystemTarget::getCurrentPlatformString(); + $config = $this->config['binary'][$platform] ?? null; + return is_array($config) ? $config : null; + } + /** * Get source extraction directory. * diff --git a/src/StaticPHP/Artifact/DownloaderOptions.php b/src/StaticPHP/Artifact/DownloaderOptions.php index 47c960ef5..6dd33222d 100644 --- a/src/StaticPHP/Artifact/DownloaderOptions.php +++ b/src/StaticPHP/Artifact/DownloaderOptions.php @@ -101,4 +101,21 @@ public static function extractFromConsoleOptions(array $allOptions, string $pref return $result; } + + /** + * Add artifact names to a list-valued option such as binary-only without narrowing it: true, or + * null (the flag given without a value), already applies to every artifact and stays as it is. + * + * @param mixed $current Current option value (false when the option was not given) + * @param string[] $names Artifact names to add + * @return null|bool|string The merged option value + */ + public static function addToList(mixed $current, array $names): bool|string|null + { + if ($current === true || $current === null) { + return $current; + } + $existing = is_string($current) && $current !== '' ? parse_comma_list($current) : []; + return implode(',', array_values(array_unique([...$existing, ...array_values($names)]))); + } } diff --git a/src/StaticPHP/Config/ConfigValidator.php b/src/StaticPHP/Config/ConfigValidator.php index 2289b3e6b..e8dd828b2 100644 --- a/src/StaticPHP/Config/ConfigValidator.php +++ b/src/StaticPHP/Config/ConfigValidator.php @@ -127,7 +127,7 @@ class ConfigValidator 'pie' => [['repo'], ['extract', 'prefer-stable']], 'pecl' => [['name'], ['extract', 'prefer-stable']], 'php-release' => [['domain'], ['extract']], - 'custom' => [[], ['func']], + 'custom' => [[], ['func', 'base-url']], ]; /** diff --git a/src/StaticPHP/Package/PackageBuilder.php b/src/StaticPHP/Package/PackageBuilder.php index 5e890f252..3a3d69a26 100644 --- a/src/StaticPHP/Package/PackageBuilder.php +++ b/src/StaticPHP/Package/PackageBuilder.php @@ -195,17 +195,17 @@ public function extractDebugInfo(string $binary_path): string } elseif (SystemTarget::getTargetOS() === 'Linux') { $objcopy = $this->getBinUtil('objcopy'); try { + FileSystem::copy($binary_path, $debug_file); shell() - ->exec("{$objcopy} --only-keep-debug {$binary_path} {$debug_file}") + ->exec("{$objcopy} --strip-debug {$binary_path}") ->exec("{$objcopy} --add-gnu-debuglink={$debug_file} {$binary_path}"); } catch (SPCException $e) { - // A separate debug file is optional. Without the opt-in LLVM binutils the fallback - // may not implement --only-keep-debug (zig's objcopy reports it as unimplemented), + // A separate debug file is optional. Native builds do not require the LLVM binutils, // so degrade to "no debug file" instead of failing an otherwise good build. logger()->warning("Could not extract debug info from '{$binary_path}' with '{$objcopy}': {$e->getMessage()}"); - logger()->warning('Continuing without a separate debug file. Install the LLVM binutils with --with-llvm-tools to keep debug information.'); + logger()->warning('Continuing without a separate debug file. Install the LLVM binutils (`bin/spc install-pkg llvm-tools`) to keep debug information.'); if (is_file($debug_file)) { - unlink($debug_file); // a failed --only-keep-debug can leave an empty file behind + unlink($debug_file); // do not leave a debug file the build did not register } return ''; } @@ -219,9 +219,10 @@ public function extractDebugInfo(string $binary_path): string /** * Strip unneeded symbols from binary file. * - * The LLVM binutils are optional (--with-llvm-tools). Under the zig toolchain the host strip - * can choke on zig/LLVM bitcode, and a build must not fail because an optional tool is not - * installed: skipping strip only costs binary size. + * Under the zig toolchain the LLVM binutils are only required for cross builds (see + * ZigToolchain::getRequiredPackages()). Native builds use the host strip, which can choke on + * zig/LLVM bitcode, and a build must not fail because of that: skipping strip only costs + * binary size. */ public function stripBinary(string $binary_path): void { @@ -239,13 +240,14 @@ public function stripBinary(string $binary_path): void throw $e; } logger()->warning("Failed to strip '{$binary_path}' with '{$strip}': {$e->getMessage()}"); - logger()->warning('Continuing without strip (this only affects binary size). Install the LLVM binutils with --with-llvm-tools if it keeps happening.'); + logger()->warning('Continuing without strip (this only affects binary size). Install the LLVM binutils (`bin/spc install-pkg llvm-tools`) if it keeps happening.'); } } /** * True when the LLVM binutils would help but are not installed: the zig toolchain is active - * and the optional llvm-tools package is missing. + * and the llvm-tools package is missing (optional for native builds; a cross build that first + * had to install zig only gets it on its next run). * * Guarded because a custom registry does not have to ship `llvm-tools`; in that case there is * nothing to advise and the original strip error has to surface unchanged. @@ -269,9 +271,9 @@ private function getBinUtil(string $name): string $is_zig = ToolchainManager::getToolchainClass() === ZigToolchain::class; // Zig ships zig- wrappers for its own binutils, but they only implement a subset of - // the options (zig-objcopy reports --only-keep-debug as unimplemented). Treat them as - // absent so these post-processing steps use llvm-tools when installed and the host - // binutils otherwise, which is what make the LLVM binutils genuinely optional. + // the options. Treat them as absent so these post-processing steps use llvm-tools when + // installed and the host binutils otherwise, which is what lets native builds work + // without the LLVM binutils. if ($is_zig && $env === "zig-{$name}") { $env = ''; } @@ -286,8 +288,8 @@ private function getBinUtil(string $name): string if ($llvm_tools instanceof ToolPackage && $llvm_tools->isInstalled()) { return $llvm_tools->getBinary("llvm-{$name}"); } - // llvm-tools is optional: make it visible which binutil the build actually uses. - logger()->debug("llvm-tools is not installed; falling back to the host '{$name}' (enable the LLVM binutils with --with-llvm-tools)."); + // Optional for native builds: make it visible which binutil the build actually uses. + logger()->debug("llvm-tools is not installed; falling back to the host '{$name}' (install the LLVM binutils with `bin/spc install-pkg llvm-tools`)."); } return $name; diff --git a/src/StaticPHP/Package/PackageInstaller.php b/src/StaticPHP/Package/PackageInstaller.php index 07b039690..205995a2a 100644 --- a/src/StaticPHP/Package/PackageInstaller.php +++ b/src/StaticPHP/Package/PackageInstaller.php @@ -218,7 +218,12 @@ public function run(bool $disable_delay_msg = false): void 'source-only' => implode(',', array_map(fn ($x) => $x->getName(), $this->build_packages)), ]; if ($binary_only_packages !== []) { - $dl_opts['binary-only'] = implode(',', array_map(fn ($x) => $x->getName(), $binary_only_packages)); + // Merged rather than replaced: a caller's global binary-only (the doctor's pkg-config + // fix) has to stay global when the toolchain adds a binary-only package such as zig. + $dl_opts['binary-only'] = DownloaderOptions::addToList( + $dl_opts['binary-only'] ?? false, + array_map(fn ($x) => $x->getName(), $binary_only_packages) + ); } $downloader = new ArtifactDownloader($dl_opts, $this->interactive); $downloader->addArtifacts($this->getArtifacts())->download(); @@ -722,6 +727,11 @@ private function resolvePackages(): void $this->collectToolchainPackages(), static fn (string $package): bool => !in_array($package, $pkgs, true) )); + // They are fetched pre-built where a binary is published (llvm-tools): building them from + // source is only the fallback, unless the user explicitly asked for source-only downloads. + if ($toolchain_packages !== []) { + $this->preferBinary(...$toolchain_packages); + } $pkgs = [...$toolchain_packages, ...$pkgs]; // resolve dependencies string diff --git a/src/StaticPHP/Runtime/SystemTarget.php b/src/StaticPHP/Runtime/SystemTarget.php index d1c16b38f..24e4ba910 100644 --- a/src/StaticPHP/Runtime/SystemTarget.php +++ b/src/StaticPHP/Runtime/SystemTarget.php @@ -138,6 +138,26 @@ public static function getCanonicalTriple(): string }; } + /** + * Whether the target runs on a different OS or CPU architecture than the host (a cross build). + * + * The architecture comes from the canonical target triple rather than getTargetArch(), so + * targets without a dedicated architecture mapping (e.g. arm-linux-musleabihf) count as well. + */ + public static function isCrossCompiling(): bool + { + if (self::getTargetOS() !== PHP_OS_FAMILY) { + return true; + } + $arch = strtolower(explode('-', self::getCanonicalTriple())[0]); + $arch = match ($arch) { + 'amd64', 'x64' => 'x86_64', + 'arm64' => 'aarch64', + default => $arch, + }; + return $arch !== GNU_ARCH; + } + /** * Check if the target OS is a Unix-like system. */ diff --git a/src/StaticPHP/Toolchain/Interface/PackageRequirementsInterface.php b/src/StaticPHP/Toolchain/Interface/PackageRequirementsInterface.php index f3a5b2992..710039c83 100644 --- a/src/StaticPHP/Toolchain/Interface/PackageRequirementsInterface.php +++ b/src/StaticPHP/Toolchain/Interface/PackageRequirementsInterface.php @@ -11,7 +11,7 @@ * packages before the dependency graph is resolved, so they follow the normal download, extract, * build and install pipeline. That means their own isInstalled() decides whether they are * skipped or rebuilt, which is what keeps target-specific artifacts in sync when SPC_TARGET - * changes between builds. + * changes between builds. They prefer a published binary over the source, which stays the fallback. * * This is deliberately separate from ToolchainInterface so existing toolchains do not need to * implement it. diff --git a/src/StaticPHP/Toolchain/ZigToolchain.php b/src/StaticPHP/Toolchain/ZigToolchain.php index c65db9f2c..d58039860 100644 --- a/src/StaticPHP/Toolchain/ZigToolchain.php +++ b/src/StaticPHP/Toolchain/ZigToolchain.php @@ -92,8 +92,10 @@ public function getCompilerVersion(): ?string return null; } - [$code, $out] = shell()->execWithResult(escapeshellarg($zig->getBinary('zig')) . ' cc --version', false); - if ($code === 0 && preg_match('/clang version (\d+\.\d+\.\d+)/', implode("\n", $out), $match)) { + // -dumpversion prints the same version as `cc --version`, which leaves an empty a.o in the + // working directory (the user's project) on every run. + [$code, $out] = shell()->execWithResult(escapeshellarg($zig->getBinary('zig')) . ' cc -dumpversion', false); + if ($code === 0 && preg_match('/^(\d+\.\d+\.\d+)$/', trim(implode("\n", $out)), $match)) { $this->compiler_version = $match[1]; } return $this->compiler_version; @@ -133,7 +135,8 @@ public function isStatic(): bool } /** - * The packages the zig toolchain needs: zig itself, plus the compiler runtime zig does not ship. + * The packages the zig toolchain needs: zig itself, the compiler runtime zig does not ship and, + * for a cross build, the LLVM binutils. * * Declaring them here instead of in a doctor check keeps them in sync with the toolchain: * they join the normal install pipeline, their isInstalled() decides skip vs rebuild, and @@ -148,20 +151,24 @@ public function getRequiredPackages(): array // here installs zig on a machine that never ran `spc doctor`. $packages = ['zig']; - // compiler-rt is versioned against the clang bundled with zig and its source downloader runs - // `zig cc --version`, so it can only be resolved once zig is on disk. On a fresh machine the - // build that installs zig therefore defers compiler-rt to the next one; PackageInstaller - // reports that final state once per run, after the install loop. + // compiler-rt and llvm-tools are versioned against the clang bundled with zig, and their + // downloaders run zig to find out which one, so they can only be resolved once zig is on + // disk. On a fresh machine the build that installs zig therefore defers them to the next + // one; PackageInstaller reports that final state once per run, after the install loop. if ($this->checkRequirements() !== []) { return $packages; } - try { - $package = PackageLoader::getPackage('llvm-compiler-rt'); - } catch (WrongUsageException) { - return $packages; // custom registry without this target + // A custom registry does not have to ship either package. compiler-rt is only needed where + // the target really has a build step (currently Linux hosts). + if (PackageLoader::hasPackage('llvm-compiler-rt') && PackageLoader::getPackage('llvm-compiler-rt')->hasBuildFunctionForCurrentOS()) { + $packages[] = 'llvm-compiler-rt'; + } + // The host binutils only understand the host architecture, so stripping or splitting debug + // info off a cross-built binary needs the LLVM ones; native builds keep using the host tools. + if (SystemTarget::isCrossCompiling() && PackageLoader::hasPackage('llvm-tools')) { + $packages[] = 'llvm-tools'; } - // Only where the target really has a build step (currently Linux hosts). - return $package->hasBuildFunctionForCurrentOS() ? [...$packages, 'llvm-compiler-rt'] : $packages; + return $packages; } /** diff --git a/tests/StaticPHP/Artifact/ArtifactTest.php b/tests/StaticPHP/Artifact/ArtifactTest.php index 3a86ec3f9..50930a71c 100644 --- a/tests/StaticPHP/Artifact/ArtifactTest.php +++ b/tests/StaticPHP/Artifact/ArtifactTest.php @@ -109,6 +109,32 @@ public function testGetDownloadConfigReturnsNullForMissingSection(): void $this->assertNull($artifact->getDownloadConfig('non-existent')); } + // ==================== getBinaryDownloadConfig ==================== + + public function testGetBinaryDownloadConfigReturnsCurrentPlatformConfig(): void + { + $platform = $this->getCurrentPlatform(); + $expected = ['type' => 'custom', 'base-url' => 'https://example.com/deps/']; + $artifact = new Artifact('my-pkg', ['binary' => [$platform => $expected]]); + + $this->assertSame($expected, $artifact->getBinaryDownloadConfig()); + } + + public function testGetBinaryDownloadConfigReturnsNullWhenPlatformMissing(): void + { + $artifact = new Artifact('my-pkg', ['binary' => ['linux-x86_64' => ['type' => 'custom']]]); + + $this->assertNull($artifact->getBinaryDownloadConfig('definitely-not-a-platform')); + } + + public function testGetBinaryDownloadConfigAcceptsExplicitPlatform(): void + { + $expected = ['type' => 'custom', 'base-url' => 'https://example.com/']; + $artifact = new Artifact('my-pkg', ['binary' => ['linux-x86_64' => $expected]]); + + $this->assertSame($expected, $artifact->getBinaryDownloadConfig('linux-x86_64')); + } + // ==================== hasSource ==================== public function testHasSourceReturnsTrueWhenConfigHasSource(): void diff --git a/tests/StaticPHP/Artifact/DownloaderOptionsTest.php b/tests/StaticPHP/Artifact/DownloaderOptionsTest.php new file mode 100644 index 000000000..f55c20d01 --- /dev/null +++ b/tests/StaticPHP/Artifact/DownloaderOptionsTest.php @@ -0,0 +1,38 @@ +assertSame('zig', DownloaderOptions::addToList(false, ['zig'])); + $this->assertSame('zig', DownloaderOptions::addToList('', ['zig'])); + } + + public function testAddToListKeepsAGlobalFlag(): void + { + // The doctor's pkg-config fix passes binary-only => true; adding zig must not narrow it. + $this->assertTrue(DownloaderOptions::addToList(true, ['zig'])); + // The flag given on the command line without a value. + $this->assertNull(DownloaderOptions::addToList(null, ['zig'])); + } + + public function testAddToListMergesWithoutDuplicates(): void + { + $this->assertSame('upx,zig,llvm-tools', DownloaderOptions::addToList('upx,zig', ['zig', 'llvm-tools'])); + } + + public function testAddToListIgnoresKeysOfTheNames(): void + { + $this->assertSame('zig', DownloaderOptions::addToList(false, ['zig' => 'zig'])); + } +} diff --git a/tests/StaticPHP/Config/ConfigValidatorTest.php b/tests/StaticPHP/Config/ConfigValidatorTest.php index 56f61ea63..770d0a88a 100644 --- a/tests/StaticPHP/Config/ConfigValidatorTest.php +++ b/tests/StaticPHP/Config/ConfigValidatorTest.php @@ -545,6 +545,12 @@ public function testValidateAndLintArtifactsWithAllArtifactTypes(): void 'source' => [ 'type' => 'custom', ], + 'binary' => [ + 'linux-x86_64' => [ + 'type' => 'custom', + 'base-url' => 'https://example.com/deps/', + ], + ], ], ]; diff --git a/tests/StaticPHP/Runtime/SystemTargetTest.php b/tests/StaticPHP/Runtime/SystemTargetTest.php new file mode 100644 index 000000000..10264ec03 --- /dev/null +++ b/tests/StaticPHP/Runtime/SystemTargetTest.php @@ -0,0 +1,87 @@ +original_target = getenv('SPC_TARGET'); + } + + protected function tearDown(): void + { + putenv($this->original_target === false ? 'SPC_TARGET' : "SPC_TARGET={$this->original_target}"); + parent::tearDown(); + } + + public function testNativeTargetIsNotCrossCompiling(): void + { + putenv('SPC_TARGET'); + $this->assertFalse(SystemTarget::isCrossCompiling()); + + putenv('SPC_TARGET=native-native-musl -dynamic'); + $this->assertFalse(SystemTarget::isCrossCompiling()); + } + + public function testHostArchitectureTripleIsNotCrossCompiling(): void + { + putenv('SPC_TARGET=' . GNU_ARCH . '-' . $this->hostTripleSuffix()); + $this->assertFalse(SystemTarget::isCrossCompiling()); + } + + public function testArchitectureAliasMatchesTheHost(): void + { + $alias = GNU_ARCH === 'aarch64' ? 'arm64' : 'amd64'; + putenv("SPC_TARGET={$alias}-" . $this->hostTripleSuffix()); + $this->assertFalse(SystemTarget::isCrossCompiling()); + } + + public function testOtherArchitectureIsCrossCompiling(): void + { + $other = GNU_ARCH === 'aarch64' ? 'x86_64' : 'aarch64'; + putenv("SPC_TARGET={$other}-" . $this->hostTripleSuffix()); + $this->assertTrue(SystemTarget::isCrossCompiling()); + } + + public function testArchitectureWithoutMappingIsCrossCompiling(): void + { + // getTargetArch() falls back to the host for architectures it has no mapping for. + putenv('SPC_TARGET=arm-linux-musleabihf'); + $this->assertTrue(SystemTarget::isCrossCompiling()); + } + + public function testLibcVersionSuffixIsIgnored(): void + { + putenv('SPC_TARGET=' . GNU_ARCH . '-linux-gnu.2.17'); + $this->assertSame(PHP_OS_FAMILY !== 'Linux', SystemTarget::isCrossCompiling()); + } + + public function testOtherOperatingSystemIsCrossCompiling(): void + { + $suffix = PHP_OS_FAMILY === 'Linux' ? 'macos' : 'linux-musl'; + putenv('SPC_TARGET=' . GNU_ARCH . "-{$suffix}"); + $this->assertTrue(SystemTarget::isCrossCompiling()); + } + + private function hostTripleSuffix(): string + { + return match (PHP_OS_FAMILY) { + 'Linux' => 'linux-musl', + 'Darwin' => 'macos', + 'Windows' => 'windows-gnu', + default => $this->markTestSkipped('No target triple for host OS ' . PHP_OS_FAMILY), + }; + } +}