Give each package its own version and release only what changed

AGENTS.md

@@ -226,14 +226,20 @@ example. The IP Intelligence cloud examples read their key from
 
 The root `pom.xml` lists the six package trees as modules but is not their
 parent, so nothing from it reaches the published poms. Each tree keeps its own
-parent pom. They all carry the same version, and the libraries reference each
-other through `${project.version}`, so Maven resolves those dependencies from
-the reactor. The examples reference published versions unless the `dev`
-profile is active. Keep it that way: a reader of the examples should see the
-package versions a project of their own would use.
-
-Because the root is not a parent, `versions:set` has to run in each tree. CI
-does that. When changing versions by hand, run it in each of the six folders.
+parent pom. In the repository they all carry the same development version,
+and the libraries reference each other through `${project.version}`, so Maven
+resolves those dependencies from the reactor. The examples reference published
+versions unless the `dev` profile is active. Keep it that way, so that a
+reader of the examples sees the package versions a project of their own would
+use.
+
+Each published module is released at its own version, tagged
+`<package>/<version>`. The publish writes that version into the module's
+`pom.xml`, with the versions of its parent and of the packages it depends on,
+so leave the versions in the poms as they are. When a module is added, or one
+module starts to depend on another, add it to `ci/packages.json` as well, or
+the pull request build fails. `ci/README.md` describes how a release is
+planned.
 
 ## Data files and `project.root`
 

README.md

@@ -132,9 +132,12 @@ does the same.
 
 ## Versions
 
-The repository has one version, and every package is released with it. CI
-sets the version on each package tree before a release and tags the commit
-with the plain version, such as `4.5.127`.
+Each package has its own version, and a release publishes only the packages
+that changed since their last release and the packages that depend on them.
+A change only to documentation, tests, examples or the descriptive parts of a
+`pom.xml`, such as its links, releases nothing.
+Each is tagged with its name and version, such as `pipeline.core/4.5.131`.
+[ci/README.md](ci/README.md) describes how CI works out what to release.
 
 ## History
 

ci/README.md

@@ -11,10 +11,18 @@ repositories used to do for one package tree each.
   publish, the two on-premise engines that build a native library, and the
   generated accessors. The other scripts loop over it rather than naming
   folders themselves.
-- The root `pom.xml` builds every tree in one reactor but is not their parent,
-  so [build-package.ps1](build-package.ps1) sets the version in each tree
-  before it deploys from the root. The examples are built and tested but left
-  out of the deploy.
+- [packages.json](packages.json) lists every published module as a package,
+  named by its artifactId, with its folder, the other folders its build takes
+  files from, and the packages it depends on, its parent included. A parent's
+  folder is its `pom.xml` alone, because the rest of the folder is its
+  modules. [build-project.ps1](build-project.ps1) fails a pull request whose
+  poms refer to a package that `DependsOn` does not name, or add a module
+  that has no entry.
+- The root `pom.xml` builds every tree in one reactor but is not their parent.
+  [build-package.ps1](build-package.ps1) writes each published module's own
+  version into its `pom.xml`, with the versions of its parent and of the
+  packages it depends on, and deploys only the modules being released. The
+  examples are built and tested but left out of the deploy.
 - [build-project.ps1](build-project.ps1) and
   [run-unit-tests.ps1](run-unit-tests.ps1) use the `dev` profile, so a pull
   request tests the examples against the libraries it changes. The publish
@@ -31,9 +39,23 @@ repositories used to do for one package tree each.
   API, Device Detection and IP Intelligence and writes one results file per
   configuration. The Device Detection and IP Intelligence metrics are prefixed
   with the product, as both use the same names.
-- The repository has one version, tagged as a plain version such as `4.5.127`.
-  [get-next-package-version.ps1](get-next-package-version.ps1) moves only the
-  patch number.
+- Each package has its own version, tagged `<package>/<version>`, such as
+  `pipeline.core/4.5.131`. A publish releases a package when a file under its
+  folders changed since its last tag, and every package that depends on it,
+  each at its last tag with the patch number moved. Changes only to Markdown,
+  `LICENSE` files, or files under folders named `docs`, `tests`, `test` or
+  `examples`, which takes in each module's `src/test`, release nothing, and
+  [release-plan-filter.ps1](release-plan-filter.ps1) leaves out a `pom.xml`
+  whose only change is to its `name`, `description`, `url`, `licenses`,
+  `developers`, `scm` or `issueManagement`, or to its comments and layout.
+  Its Pester 5 tests are
+  [release-plan-filter.Tests.ps1](release-plan-filter.Tests.ps1).
+  [release.ps1](release.ps1) takes that plan from common-ci's
+  `steps/release-plan.ps1`, and the publish scripts build, check and tag from
+  it. [check-version-bump.ps1](check-version-bump.ps1) and
+  [check-release-interval.ps1](check-release-interval.ps1) check each package
+  against its own tags, for a major or minor move and for a release less than
+  a week after its last.
 
 ## Secrets
 

ci/build-package.ps1

@@ -7,7 +7,15 @@ param(
 $ErrorActionPreference = "Stop"
 $PSNativeCommandUseErrorActionPreference = $true
 
-. "$PSScriptRoot/packages.ps1"
+. "$PSScriptRoot/release.ps1"
+
+# Builds the packages this publish releases, each at its own version from the
+# release plan (see release.ps1). $Version is common-ci's label for the run and
+# is not used.
+$plan = @(Get-BuildPlan $RepoName)
+Write-Host "Building $($plan.Count) packages:"
+$plan | Format-Table Name, Previous, Version, Reason -AutoSize | Out-String -Width 200 |
+    Write-Host
 
 # Put the native libraries built on every platform into the engine they came
 # from, so each engine's jar carries all of its own and none of the other's.
@@ -21,46 +29,25 @@ foreach ($package in $NativePackages) {
         }
 }
 
-# The root pom.xml is not the parent of the package trees, so setting the
-# version at the root does not reach them. Set it in each tree, the examples
-# included, so the whole repository carries the version being released.
-foreach ($package in $Packages) {
-    Write-Host "Setting the version of '$($package.Path)' to '$Version'"
-    Push-Location "$RepoName/$($package.Path)"
-    try {
-        mvn versions:set @MavenOptions "-DnewVersion=$Version" -DgenerateBackupPoms=false
-    } finally {
-        Pop-Location
-    }
-}
-
-# The examples are never published. They are left out of the deploy so that
-# the last project in the reactor is one the publishing plugin is configured
-# for. Maven 3 does not exclude a folder's modules with the folder, so each of
-# them is named.
-$repoPath = (Resolve-Path $RepoName).Path
-$examples = foreach ($package in $Packages | Where-Object { -not $_.Publish }) {
-    Get-ChildItem -Path "$RepoName/$($package.Path)" -Recurse -Filter pom.xml |
-        Where-Object { $_.FullName -notmatch '[\\/](target|node_modules)[\\/]' } |
-        ForEach-Object { '!' + [IO.Path]::GetRelativePath($repoPath, $_.DirectoryName).Replace('\', '/') }
-}
+# Every published module gets its own version, and the versions of its parent
+# and of the packages it depends on, including those this publish does not
+# release, which stay at their last release.
+Set-PomVersions $RepoName (Get-BuildVersions $RepoName $plan)
 
-# The publishing plugin writes the one bundle for the whole reactor under the
-# first tree that publishes. common-ci copies the bundle from where a single
-# tree puts it, below target at the root, so a link at that path leads to the
-# bundle. The link is made before the deploy, which Linux allows.
-$firstPublishing = ($Packages | Where-Object { $_.Publish } | Select-Object -First 1).Path
-New-Item -ItemType Directory -Force -Path "$RepoName/target" | Out-Null
-New-Item -ItemType SymbolicLink -Force -Path "$RepoName/target/central-publishing" -Target "../$firstPublishing/target/central-publishing" | Out-Null
+# Only the modules in the plan are deployed, so the one publishing bundle holds
+# only them. A package they depend on that is not released is taken from Maven
+# Central at its last release, the version their poms now require.
+$modules = @($plan | ForEach-Object { Get-ModulePath $_ })
 
-# common-ci sets the version of the root pom.xml, deploys from the root to a
-# local staging folder with signing, and copies the result and the publishing
-# bundle into the package folder.
+# common-ci deploys from the root to a local staging folder with signing, and
+# copies the result and the publishing bundle into the package folder. The
+# versions are already set, so it is given no folder to set them in.
 ./java/build-package.ps1 `
     -RepoName $RepoName `
     -Name $Name `
     -Version $Version `
-    -ExtraArgs "-DskipNativeBuild=true", "-pl", ($examples -join ',') `
+    -VersionPaths @() `
+    -ExtraArgs "-DskipNativeBuild=true", "-pl", ($modules -join ',') `
     -JavaGpgKeyPassphrase $Keys['JavaGpgKeyPassphrase'] `
     -JavaPGP $Keys['JavaPGP'] `
     -CodeSigningKeyVaultName: $Keys['CodeSigningKeyVaultName'] `

ci/build-project.ps1

@@ -9,6 +9,14 @@ $PSNativeCommandUseErrorActionPreference = $true
 
 Write-Host "JAVA_HOME: $ENV:JAVA_HOME"
 
+# A publish releases a package's dependents only when packages.json names
+# them, so a module or dependency it leaves out fails here, before the change
+# is merged, rather than at the publish.
+$gaps = @(Get-PackagesJsonGaps $RepoName)
+if ($gaps.Count -gt 0) {
+    throw "ci/packages.json does not match the poms:`n$($gaps -join "`n")"
+}
+
 # One build of the whole reactor, native libraries included. The dev profile
 # builds the examples against the libraries here rather than the versions
 # published to Maven Central, so a change to a library is tested by the

ci/check-release-interval.ps1

@@ -0,0 +1,42 @@
+param (
+    [Parameter(Mandatory)][string]$RepoName,
+    [string]$Repository,
+    [string]$HowToAllow
+)
+$ErrorActionPreference = "Stop"
+. "$PSScriptRoot/release.ps1"
+
+# The shared publish runs this in place of its own check, because each package
+# has its own version and tags (see release.ps1). Every package about to be
+# published goes through the shared steps/check-release-interval.ps1, run from
+# the common-ci root, against its own '<package>/<version>' tags, so each
+# package gets at most one new version a week. That refuses a package
+# released less than a week ago unless ALLOW_EARLY_RELEASE is 'true', from the
+# date in WEEKLY_RELEASES_FROM.
+
+$plan = @(Get-ReleasePlan $RepoName)
+if ($plan.Count -eq 0) {
+    Write-Host "No package is to be published, so there is nothing to check."
+    exit 0
+}
+
+$arguments = @{ RepoName = $RepoName; Repository = $Repository }
+if ($HowToAllow) {
+    $arguments.HowToAllow = $HowToAllow
+}
+
+$refused = @()
+foreach ($release in $plan) {
+    Write-Host "$($release.Name):"
+    $global:LASTEXITCODE = 0
+    ./steps/check-release-interval.ps1 @arguments -TagPrefix $release.Name
+    if ($LASTEXITCODE -ne 0) {
+        $refused += $release.Name
+    }
+}
+
+if ($refused) {
+    Write-Host "Nothing is released, because these packages were released less than a week ago: $($refused -join ', ')."
+    exit 1
+}
+exit 0

ci/check-version-bump.ps1

@@ -0,0 +1,36 @@
+param (
+    [Parameter(Mandatory)][string]$RepoName,
+    [string]$Version
+)
+$ErrorActionPreference = "Stop"
+. "$PSScriptRoot/release.ps1"
+
+# The shared publish runs this in place of its own check, because each package
+# has its own version (see release.ps1). Every package about to be published
+# goes through the shared steps/check-version-bump.ps1, run from the common-ci
+# root, against its own '<package>/<version>' tags. That refuses a major or
+# minor move unless ALLOW_VERSION_BUMP is 'true'. $Version is common-ci's label
+# for the run.
+
+Write-Host "The publish worked out $Version, its highest. Each package is checked on its own."
+
+$plan = @(Get-ReleasePlan $RepoName)
+if ($plan.Count -eq 0) {
+    Write-Host "No package is to be published, so there is nothing to check."
+    exit 0
+}
+
+$refused = @()
+foreach ($release in $plan) {
+    $global:LASTEXITCODE = 0
+    ./steps/check-version-bump.ps1 -Version $release.Version -RepoName $RepoName -TagPrefix $release.Name
+    if ($LASTEXITCODE -ne 0) {
+        $refused += $release.Name
+    }
+}
+
+if ($refused) {
+    Write-Host "Nothing is published, because these packages were refused: $($refused -join ', ')."
+    exit 1
+}
+exit 0

ci/get-next-package-version.ps1

@@ -3,31 +3,21 @@ param (
     [Parameter(Mandatory)][string]$VariableName
 )
 $ErrorActionPreference = "Stop"
+. "$PSScriptRoot/release.ps1"
 
-# The whole repository has one version, tagged as a plain version such as
-# 4.5.127. Only the patch number moves here. The major and minor numbers
-# change only when someone edits FirstVersion before the first release or
-# pushes a tag by hand.
-
-# The version used while the repository has no version tag. It is above every
-# version the absorbed repositories published, so the first release from here
-# is an ordinary upgrade for each package.
-$FirstVersion = '4.5.127'
-
-$tags = @(git -C $RepoName tag --list |
-    Where-Object { $_ -match '^\d+\.\d+\.\d+$' } |
-    Sort-Object { [version]$_ })
-$headTags = @(git -C $RepoName tag --points-at HEAD |
-    Where-Object { $_ -match '^\d+\.\d+\.\d+$' })
-
-if ($headTags) {
-    # Already released, so the publish sees the tag and does nothing.
-    $next = $headTags | Sort-Object { [version]$_ } | Select-Object -Last 1
-} elseif ($tags) {
-    $highest = [version]$tags[-1]
-    $next = "$($highest.Major).$($highest.Minor).$($highest.Build + 1)"
+# common-ci's publish carries one version through its jobs. This repository
+# releases each package at its own version (see release.ps1), so that one
+# version is only a label for the run, being the highest version this publish
+# releases. Whether anything is released is decided by
+# package-update-required.ps1, and each package's version and tag by
+# build-package.ps1 and publish-package.ps1, all from the same release plan.
+$plan = @(Get-ReleasePlan $RepoName)
+if ($plan.Count -gt 0) {
+    $label = ($plan.Version | Sort-Object { [version]$_ })[-1]
 } else {
-    $next = $FirstVersion
+    # Nothing to release. Any valid version will do, because
+    # package-update-required.ps1 stops the publish before it is used.
+    $label = '4.5.0'
 }
 
-Set-Variable -Scope Global -Name $VariableName -Value $next
+Set-Variable -Scope Global -Name $VariableName -Value $label

ci/package-update-required.ps1

@@ -0,0 +1,23 @@
+param (
+    [Parameter(Mandatory)][string]$RepoName,
+    [string]$Version
+)
+$ErrorActionPreference = "Stop"
+. "$PSScriptRoot/release.ps1"
+
+# Called by common-ci's publish in place of its own check, because a plain tag
+# lookup cannot tell this repository's packages apart. Exits 0 when the release
+# plan has something in it and 1 when it is empty, as common-ci's check does.
+# $Version is common-ci's label for the run and is not used here. Each
+# package's version is checked by check-version-bump.ps1.
+
+$plan = @(Get-ReleasePlan $RepoName)
+if ($plan.Count -eq 0) {
+    Write-Output "No package changed since its last release, so there is nothing to publish"
+    exit 1
+}
+
+Write-Output "This publish releases $($plan.Count) packages, in this order:"
+$plan | Format-Table Name, Previous, Version, Reason -AutoSize | Out-String -Width 200 |
+    Write-Output
+exit 0