[RFC] 3.0 Updates - Package And Artifact Changes

[crazywhalecc]

[RFC] 3.0 Updates - Package and Artifact Changes

Package Config Updates

Since 2.x has many kind of things, like ext, lib, pkg, pre-built, etc.,
we will regard them as package with different package types in 3.0:

• ext -> php-extension package type
• lib -> library package type
• pkg -> target package type
• pre-built -> binary for artifact type

For example, the config will be replaced by pkg.*.json and artifact.*.json for different registries.

pkg.ext.json example:

{
  "ext-amqp": {
    "type": "php-extension",
    "php-extension": {
      "suppport": {
        "BSD": "wip"
      },
      "arg-type": "custom",
      "depends": [
        "librabbitmq"
      ],
      "depends@windows": [
        "ext-openssl"
      ]
    },
    "artifact": "amqp",
    "license": {
      "type": "file",
      "path": "LICENSE"
    }
  }
}

pkg.lib.json example:

{
  "curl": {
    "type": "library",
    "depends@windows": [
      "zlib",
      "libssh2",
      "nghttp2"
    ],
    "depends": [
      "openssl",
      "zlib"
    ],
    "suggests@windows": [
      "brotli",
      "zstd"
    ],
    "suggests": [
      "libssh2",
      "brotli",
      "nghttp2",
      "nghttp3",
      "ngtcp2",
      "zstd",
      "libcares",
      "ldap"
    ],
    "headers": [
      "curl"
    ],
    "frameworks": [
      "CoreFoundation",
      "CoreServices",
      "SystemConfiguration"
    ],
    "artifact": "curl",
    "license": {
      "type": "file",
      "path": "COPYING"
    }
  }
}

The biggest changes for package is we will regard PHP SAPI as different package type.

pkg.target.json example:

{
  "pkg-config": {
    "type": "target",
    "static-bins": [
      "pkg-config"
    ],
    "artifact": "pkg-config"
  },
  "php": {
    "type": "target",
    "artifact": "php-src"
  },
  "php-cli": {
    "type": "virtual-target",
    "depends": [
      "php"
    ]
  },
  "php-micro": {
    "type": "virtual-target",
    "artifact": "micro",
    "depends": [
      "php"
    ]
  }
}

The normal target must have artifact, but for virtual-target, it's optional.

All the target can be built by command ./spc build:<target-name> or ./spc install:<target-name>.

• The virtual target must be declared with attribute.
• target is superset of library, it can have static-bins, static-libs fields for build.
• When virtual-target contains artifact field, it means the target needs downloading something to install or build.

The attribute update will be described in another RFC.

For clearer definition of OS-specific dependencies, we will use the depends@<os> and suggests@<os> fields
to specify dependencies for different operating systems in pkg.*.json files.

Artifact Updates

We will treat all downloadable item as artifact in 3.0, including:

• Source code tarballs, archives, git repositories
• Pre-built binaries for libraries
• Not buildable items, including all old pkg.json packages

For example, the config will be replaced by artifact.*.json files.

This is libcares source changed example in artifact.lib.json:

{
  "libcares": {
    "binary": "hosted",
    "source": {
      "type": "ghrel",
      "repo": "c-ares/c-ares",
      "match": "c-ares-.+\\.tar\\.gz",
      "prefer-stable": true
    },
    "source-mirror": {
      "type": "filelist",
      "url": "https://c-ares.org/download/",
      "regex": "/href=\"\\/download\\/(?<file>c-ares-(?<version>[^\"]+)\\.tar\\.gz)\"/"
    }
  }
}

The major part changes are:

• It divides source and binary fields clearly.
• It unifies all source code download methods into source field. But the field in source is not changed.
• Use source-mirror instead of alt field.
• Add binary field to specify the binary download method.

And for binary part, we use hosted to represent the old provide-pre-built type. That is, its binaries are hosted by static-php-cli-hosted repo.

Behavior Changes in SPC

PHP Extension Type Unification

• In 2.x, we have different type for ext.json defines: addon, builtin, external.
  In 3.0, all of them are php-extension type.
• If we define artifact to php-extension, it means it's external extension, otherwise it's builtin or addon extension.
• These field, support, arg-type* will be in php-extension.
• The target field will be separated to php-extension.build-shared: boolean and php-extension.build-static: boolean.
• The build-with-php field will be remained in php-extension, but it only works for php-extension type.
• The cpp-extension will be replaced by lang: cpp field in package.
• The frameworks will be kept.
• The notes will be kept in php-extension field.
• The zend-extension will be kept in php-extension field.
• All the ext-depends will be merged into depends field.
• All the ext-suggests will be merged into suggests field.
• All the PHP extensions will be renamed to ext-{original_name}, but will keep convertion for backward compatibility.

SAPI as Target

• All the PHP SAPI will be target or virtual-target type.
• The SAPI build procedure will be defined with new attributes in src/SPC/Target/*.php.
• We keep original bin/spc build <extensions> behavior for backward compatibility, but it's recommended to use spc build:php-cli or spc build:php-micro instead.

Target Build Changes

• The downloading procedure will be automatically handled during build:<target-name> command unless user using --no-download option.
• The build:<target-name> will always build the target from source.
  • When target is not buildable (does not have source in artifact), it will throw error.
  • When target contains depends, it will try to install binaries (as well as download) first.
    If binaries for dependencies failed to download or install, it will try to build from source.
  • If some dependencies only provide binary method to install, throw error.
  • The build option definition for different target can be defined with #[TargetOption] attribute. No longer symfony manually defines.
• The install:<target-name> will try to download pre-built binaries first.
  • When target is not available for current OS or architecture, it will throw error.
  • When target is buildable, it will not fall back to build from source.
• The library type of pkg.json can also be built with build:libs, like current version in 2.x.

Download Changes

We no longer keep download command as a must step before build or install.
But since we still separate package and artifact, the download command will keep working for downloading for artifact sources by default.

• When using download XXX,YYY command, it will download the source code by default.
• When using --for-packages=XXX option, it will regard XXX as package, and resolve all the artifacts for the packages, then download them.
• When using --binary-only option, it will only download the binary artifacts for the packages.
• When using --source-only option, it will only download the source code artifacts for the packages.
• When using --prefer-binary option, it will try to download binary artifacts first, if not available, then download source code artifacts.

Another major changes is version system. We will have a version getter for some important and widely-used packages, like php, openssl, curl, etc.

• The version getter can be defined in src/SPC/{Library|Target|Extension}/*.php with #[VersionGetter] attribute.
• static-php-cli will maintain an official version metadata json file hosted on static-php.dev server.
  • That means you will have ./spc check-updates command to check your downloaded artifacts' versions.
  • You can also use ./spc update-cache command to update your downloaded artifacts to latest versions.

For security for supply chain, we will add sha256 field to major artifact packages.

• When sha256 field is defined in artifact, spc will verify the downloaded file's sha256 checksum.
• When artifact verify function defined with #[ArtifactVerifier] attribute, it will be used to verify the downloaded file.
• The spc binary will have signature to verify its integrity.

For convinience, we provide a converter for config files: https://gist.github.com/crazywhalecc/a3c2d9b94f027464ebd402998f5eb1d4

If you have any question or have better suggestions, please leave comment here with quote.