diff --git a/.github/instructions/onebranch-pipeline-design.instructions.md b/.github/instructions/onebranch-pipeline-design.instructions.md index 7a24fd3799..75ea6ea303 100644 --- a/.github/instructions/onebranch-pipeline-design.instructions.md +++ b/.github/instructions/onebranch-pipeline-design.instructions.md @@ -34,7 +34,7 @@ Defined in `stages/build-stages.yml`. Four build stages plus validation, ordered - **`build_abstractions`** (Stage 2) — Abstractions; `dependsOn: build_independent`; downloads Logging artifact - **`build_dependent`** (Stage 3) — SqlClient and Extensions.Azure in parallel; `dependsOn: build_abstractions`; downloads Abstractions + Logging artifacts - **`build_addons`** (Stage 4) — AKV Provider; `dependsOn: build_dependent`; downloads SqlClient + Abstractions + Logging artifacts -- **`sqlclient_package_validation`** — Validates signed SqlClient package; `dependsOn: build_dependent`; runs in parallel with Stage 4 +- **`package_validation`** (Stage 5) — Validates every package produced by the run; `dependsOn` all four build stages plus `compute_versions` Each build job copies PDB files into `$(JOB_OUTPUT)/symbols/` so they are included in the auto-published pipeline artifact alongside the NuGet packages in `$(JOB_OUTPUT)/packages/`. @@ -46,7 +46,7 @@ Stage conditional rules: ## Job Templates - **`build-buildproj-job.yml`** — Shared build.proj-driven package job used for all shipped packages. Flow: build via `build.proj` → optional ESRP DLL signing → pack via `build.proj` → optional ESRP NuGet signing → copy outputs for APIScan/artifacts -- **`validate-signed-package-job.yml`** — Validates signed MDS package (signature, strong names, folder structure, target frameworks) +- **`validate-packages-job.yml`** — Validates every package produced by the run. Downloads all package artifacts into one tree and validates them together, so `tools/PackageValidator` can apply its cross-package rules (the SqlClient family must share one version, and inter-package dependency ranges must agree); validating per package would silently skip those findings. Runs on Windows because Authenticode verification has no Linux equivalent - **`publish-nuget-package-job.yml`** — Reusable release job using OneBranch `templateContext.type: releaseJob` with `inputs` for artifact download; pushes via `NuGetCommand@2` - **`publish-symbols-job.yml`** — Reusable symbols job: downloads a build artifact, locates PDBs under `symbols/`, and invokes `publish-symbols-step.yml` @@ -56,6 +56,19 @@ When adding a new package to the OneBranch flow: - Add version variables to `variables/common-variables.yml` - Add artifact name variables to `variables/onebranch-variables.yml` +## Package Validation Stage + +- Defined in `stages/build-stages.yml`; produces stage `package_validation` +- Consumes the package and file versions published by `compute_versions` and asserts the produced packages carry exactly those values, so nothing is re-derived +- All packages are validated together in one job so `tools/PackageValidator` can apply cross-package rules; the SqlServer artifact and its expectations are conditional on `buildSqlServer` +- Expectations use the validator's `[id=]value` form: the SqlClient family version is applied as a wildcard (proving the family agrees, and catching the case where all packages are consistently wrong), with `Microsoft.SqlServer.Server` as a per-id override +- When SqlServer is not built its expectations are **omitted entirely** rather than passed empty — the validator rejects an expectation with an empty value +- Gate categories are derived from `isOfficial`: `error`, `missing-symbols`, `dependency-inconsistency`, `delay-signed`, and `unsigned` always, plus `package-unsigned` on official runs only. The `error` severity covers only error-severity findings, so each warning/info category must be named explicitly — `missing-symbols`, `dependency-inconsistency`, and `delay-signed` are warnings, and `unsigned` and `package-unsigned` are info. Strong-name signing is unconditional in `build-buildproj-step.yml`, so the two strong-name categories gate everywhere; NuGet package signing is ESRP and official-only, so `package-unsigned` would fire on every non-official run +- The validator runs twice: once with `--json` and no gate so the report exists even for a failing run, then once human-readable with the gate so failures appear in the job log +- Signature verification (`dotnet nuget verify --all`, Authenticode) runs on official builds only, and verifies that signatures are *trusted* — PackageValidator reports only their presence, from metadata +- The release stage `dependsOn: package_validation`, so a package that fails validation is never published +- Step and job logic lives in `scripts/validate-packages.ps1`, `scripts/verify-package-signatures.ps1`, and `scripts/verify-assembly-signatures.ps1`, each with Pester tests under `scripts/tests/` + ## Symbols Publishing Stage - Defined in `stages/publish-symbols-stage.yml`; produces stage `publish_symbols` diff --git a/eng/pipelines/onebranch/jobs/validate-packages-job.yml b/eng/pipelines/onebranch/jobs/validate-packages-job.yml new file mode 100644 index 0000000000..f1501c7d1d --- /dev/null +++ b/eng/pipelines/onebranch/jobs/validate-packages-job.yml @@ -0,0 +1,206 @@ +################################################################################# +# Licensed to the .NET Foundation under one or more agreements. # +# The .NET Foundation licenses this file to you under the MIT license. # +# See the LICENSE file in the project root for more information. # +################################################################################# + +# Validates every NuGet package produced by this run. +# +# All packages are downloaded into a single tree and validated together in one job, rather than one +# job per package, so that PackageValidator can apply its cross-package rules: the SqlClient family +# must share a single version, and their inter-package dependency ranges must agree. +# +# The job runs on Windows because Authenticode verification has no equivalent on the Linux agents. +# PackageValidator itself is cross-platform, so only the signature checks are OS-bound. + +parameters: + # Package Parameters ----------------------------------------------------- + + - name: abstractionsArtifactsName + type: string + + - name: akvProviderArtifactsName + type: string + + - name: azureArtifactsName + type: string + + - name: loggingArtifactsName + type: string + + - name: sqlClientArtifactsName + type: string + + - name: sqlServerArtifactsName + type: string + + # Version Parameters ----------------------------------------------------- + # Pre-computed by the compute-versions stage. Validation asserts the produced packages carry + # exactly these versions, so nothing here is re-derived. + + - name: sqlClientPackageVersion + type: string + + - name: sqlClientFileVersion + type: string + + - name: sqlServerPackageVersion + type: string + + - name: sqlServerFileVersion + type: string + + # Behaviour Parameters --------------------------------------------------- + + # Whether Microsoft.SqlServer.Server was built this run. When false there is no SqlServer + # artifact to download and no SqlServer package in the drop to validate. + - name: buildSqlServer + type: boolean + + # True for official builds, which sign their packages and assemblies. Signature verification is + # skipped otherwise, because non-official runs deliberately produce unsigned output. + - name: isOfficial + type: boolean + +jobs: + - job: validate_packages + displayName: 'Validate Packages' + + pool: + type: windows + + # 1ES auto-injects Roslyn into any job holding a DotNetCoreCLI build task, which here is only + # the PackageValidator tool build -- never shipped, so out of SDL scope. + templateContext: + sdl: + roslyn: + enabled: false + + variables: + - name: ob_outputDirectory + value: '$(JOB_OUTPUT)' + + # This job inspects already-built packages and produces no assemblies, so it has nothing for + # APIScan or BinSkim to scan and no shipping component to describe in an SBOM. The build + # jobs cover all three for the packages they produce. + - name: ob_sdl_apiscan_enabled + value: false + - name: ob_sdl_binskim_enabled + value: false + - name: ob_sdl_sbom_enabled + value: false + + # Every package artifact is downloaded beneath this root, each into its own subdirectory so + # that identically-named files from different packages cannot collide. + - name: packagesRoot + value: '$(Pipeline.Workspace)/validate-packages' + + - name: extractRoot + value: '$(Pipeline.Workspace)/validate-extract' + + steps: + - template: /eng/pipelines/onebranch/steps/script-output-environment-variables-step.yml@self + + # Only the packages themselves are needed, not the full build output each artifact carries. + - task: DownloadPipelineArtifact@2 + displayName: 'Download Packages - Logging' + inputs: + artifactName: '${{ parameters.loggingArtifactsName }}' + targetPath: '$(packagesRoot)/Logging' + patterns: '**/*.*nupkg' + + - task: DownloadPipelineArtifact@2 + displayName: 'Download Packages - Abstractions' + inputs: + artifactName: '${{ parameters.abstractionsArtifactsName }}' + targetPath: '$(packagesRoot)/Abstractions' + patterns: '**/*.*nupkg' + + - task: DownloadPipelineArtifact@2 + displayName: 'Download Packages - SqlClient' + inputs: + artifactName: '${{ parameters.sqlClientArtifactsName }}' + targetPath: '$(packagesRoot)/SqlClient' + patterns: '**/*.*nupkg' + + - task: DownloadPipelineArtifact@2 + displayName: 'Download Packages - Azure' + inputs: + artifactName: '${{ parameters.azureArtifactsName }}' + targetPath: '$(packagesRoot)/Azure' + patterns: '**/*.*nupkg' + + - task: DownloadPipelineArtifact@2 + displayName: 'Download Packages - AkvProvider' + inputs: + artifactName: '${{ parameters.akvProviderArtifactsName }}' + targetPath: '$(packagesRoot)/AkvProvider' + patterns: '**/*.*nupkg' + + - ${{ if eq(parameters.buildSqlServer, true) }}: + - task: DownloadPipelineArtifact@2 + displayName: 'Download Packages - SqlServer' + inputs: + artifactName: '${{ parameters.sqlServerArtifactsName }}' + targetPath: '$(packagesRoot)/SqlServer' + patterns: '**/*.*nupkg' + + # PackageValidator targets net10.0, which the repo's global.json already pins. + - template: /eng/pipelines/common/steps/install-dotnet.yml@self + + - template: /eng/pipelines/onebranch/steps/validate-packages-step.yml@self + parameters: + packagesPath: '$(packagesRoot)' + reportPath: '$(JOB_OUTPUT)/validation/package-validation.json' + sqlClientPackageVersion: '${{ parameters.sqlClientPackageVersion }}' + sqlClientFileVersion: '${{ parameters.sqlClientFileVersion }}' + # Omitted when SqlServer is not built: its package is absent from the drop, and the + # validator rejects an expectation with an empty value. + ${{ if eq(parameters.buildSqlServer, true) }}: + sqlServerPackageVersion: '${{ parameters.sqlServerPackageVersion }}' + sqlServerFileVersion: '${{ parameters.sqlServerFileVersion }}' + # The error severity covers only error-severity findings, so every warning/info category + # this job relies on must be named explicitly: missing-symbols, dependency-inconsistency + # and delay-signed are warnings, and unsigned and package-unsigned are info. + # + # Strong-name signing is unconditional in build-buildproj-step.yml, so delay-signed and + # unsigned gate on every run. NuGet package signing is ESRP and runs on official builds + # only, so package-unsigned would fire on every non-official build. + ${{ if eq(parameters.isOfficial, true) }}: + failOn: + - error + - missing-symbols + - dependency-inconsistency + - delay-signed + - unsigned + - package-unsigned + ${{ else }}: + failOn: + - error + - missing-symbols + - dependency-inconsistency + - delay-signed + - unsigned + + # Signature verification, official builds only. PackageValidator reports strong-name and + # NuGet signature *presence* cross-platform; these steps additionally verify that the + # signatures are trusted, which requires the Windows trust store. + - ${{ if eq(parameters.isOfficial, true) }}: + - task: PowerShell@2 + displayName: 'Verify NuGet package signatures' + inputs: + targetType: filePath + pwsh: true + filePath: $(REPO_ROOT)/eng/pipelines/onebranch/scripts/verify-package-signatures.ps1 + arguments: >- + -PackagesPath "$(packagesRoot)" + + - task: PowerShell@2 + displayName: 'Verify assembly Authenticode signatures' + inputs: + targetType: filePath + pwsh: true + filePath: $(REPO_ROOT)/eng/pipelines/onebranch/scripts/verify-assembly-signatures.ps1 + arguments: >- + -PackagesPath "$(packagesRoot)" + -ExtractPath "$(extractRoot)" diff --git a/eng/pipelines/onebranch/jobs/validate-signed-package-job.yml b/eng/pipelines/onebranch/jobs/validate-signed-package-job.yml deleted file mode 100644 index 818b902cf7..0000000000 --- a/eng/pipelines/onebranch/jobs/validate-signed-package-job.yml +++ /dev/null @@ -1,395 +0,0 @@ -################################################################################# -# Licensed to the .NET Foundation under one or more agreements. # -# The .NET Foundation licenses this file to you under the MIT license. # -# See the LICENSE file in the project root for more information. # -################################################################################# -parameters: - # The name of the pipeline artifact to download that contains the SqlClient NuGet package. - - name: artifactName - type: string - - # List of versions of dotnet that are *allowed to exist* in the NuGet package. Separators do not - # matter as the folders in lib, runtime, etc are simply checked to see if they exist in this - # string. - - name: expectedDotnetVersions - type: string - default: 'net462;net8.0;net9.0;netstandard2.0' - - # Expected file version of the assemblies within the package. This should be of the form: - # (major).(minor).(patch).(buildNumber) - - name: expectedFileVersion - type: string - - # List of folders that are *allowed to exist* in the NuGet package. Separators do not matter as - # the folders are simply checked to see if they exist in this string. - - name: expectedFolderNames - type: string - default: 'lib;ref;runtimes' - - # Expected NuGet package version. Used to build the installation path. This should be of the - # form: (major).(minor).(patch)[-preview(preview_number)] - - name: expectedPackageVersion - type: string - - # True if this build is an official build. This will be used to gate some checks - # that only apply to official builds, such as signature verification. - - name: isOfficial - type: boolean - -jobs: - - job: validate_nuget_package - displayName: "Validate NuGet package" - - pool: - type: windows # read more about custom job pool types at https://aka.ms/obpipelines/yaml/jobs - isCustom: true - name: ADO-1ES-Pool - vmImage: "ADO-MMS22-SQL19" - - variables: # More settings at https://aka.ms/obpipelines/yaml/jobs - - # This job installs and inspects an already-built package rather than producing assemblies, - # so it has no APIScan software name/version to report. The build jobs scan those assemblies. - - name: ob_sdl_apiscan_enabled - value: false - - # Likewise it produces no package, and sets no sbomPackage* values for globalSdl to resolve. - - name: ob_sdl_sbom_enabled - value: false - - # Path within the downloaded artifact where NuGet packages are located. - - name: artifactPath - value: '$(Pipeline.Workspace)\${{ parameters.artifactName }}' - - # Path to the SqlClient NuGet package after installation. This path will only exist once the package - # been installed. - - name: nugetPackageInstallPath - value: '$(Pipeline.Workspace)\nugetPackageInstalls\Microsoft.Data.SqlClient.${{ parameters.expectedPackageVersion }}' - - # Root folder where NuGet package will be installed locally - - name: nugetPackageInstallRoot - value: '$(Pipeline.Workspace)\nugetPackageInstalls' - - steps: - - template: '/eng/pipelines/onebranch/steps/script-output-environment-variables-step.yml@self' - - - task: NuGetToolInstaller@1 - displayName: "Install NuGet" - - - powershell: | - echo "> 1. List all local cache directory paths" - nuget locals all -List - - echo "> 2. Clear all files from local cache directories" - nuget locals all -Clear - displayName: "Clear local cache" - - # Download NuGet packages from the specified build artifact. - - download: current - artifact: ${{ parameters.artifactName }} - patterns: "**/*.*nupkg" - displayName: "Download NuGet Package" - - # Verify secure signatures on the NuGet packages. - # NOTE: Packages will only be signed if the build is official. - - ${{ if eq(parameters.isOfficial, true) }}: - - powershell: | - # Propagate parameters to PS variables ####################### - $artifactPath = "${{ variables.artifactPath }}" - echo "artifactPath= $artifactPath" - - # Discover packages ########################################## - $packageFiles = Get-ChildItem -Path $artifactPath -Recurse -File -Include *.nupkg,*.snupkg - if ($packageFiles.Count -eq 0) - { - Write-Error "No NuGet package files were found under '$artifactPath'." - } - - # Verify package signatures ################################## - echo "> 1. Verify signature of source package(s)" - $packageFiles | Where-Object { $_.Extension -eq ".nupkg" } | ForEach-Object { - nuget verify -All $_.FullName - } - - echo "> 2. Verify signature of symbols package(s)" - $packageFiles | Where-Object { $_.Extension -eq ".snupkg" } | ForEach-Object { - nuget verify -All $_.FullName - } - displayName: "Verify nuget signature" - - # Install NuGet package to the temporary directory - - powershell: | - # Propagate pipeline to PS variables ############################# - $artifactPath = "${{ variables.artifactPath }}" - echo "artifactPath= $artifactPath" - - $expectedPackageVersion = "${{ parameters.expectedPackageVersion }}" - echo "expectedPackageVersion= $expectedPackageVersion" - - $nugetPackageInstallRoot = "${{ variables.nugetPackageInstallRoot }}" - echo "nugetPackageInstallRoot= $nugetPackageInstallRoot" - - # Find the SqlClient NuGet package ############################### - Get-ChildItem "$artifactPath" -Recurse - - $packagePaths = @(Get-ChildItem -Path $artifactPath -Recurse -File -Filter "Microsoft.Data.SqlClient.$expectedPackageVersion.nupkg") - if ($packagePaths.Count -eq 0) - { - Write-Error "Unable to find Microsoft.Data.SqlClient.$expectedPackageVersion.nupkg under '$artifactPath'." - } - if ($packagePaths.Count -gt 1) - { - Write-Error "Multiple Microsoft.Data.SqlClient.$expectedPackageVersion.nupkg files were found under '$artifactPath'." - } - - $packageSource = Split-Path -Path $packagePaths[0].FullName -Parent - echo "Found package path: $($packagePaths[0].FullName)" - echo "Using package source: $packageSource" - - # Install NuGet Package ########################################## - echo "> 1. Installing Microsoft.Data.SqlClient NuGet package..." - Install-Package ` - -Name "Microsoft.Data.SqlClient" ` - -Source "$packageSource" ` - -Destination $nugetPackageInstallRoot ` - -Force ` - -SkipDependencies - - echo "> 2. Listing contents of installed Microsoft.Data.SqlClient NuGet package:" - Write-Host $nugetPackageInstallRoot - Get-ChildItem $nugetPackageInstallRoot - displayName: "Install NuGet Package" - - # Find all DLL files in the installed NuGet package, verify each is signed with a strong name - - powershell: | - # Propagate pipeline to PS variables ############################# - $nugetPackageInstallPath = "${{ variables.nugetPackageInstallPath }}" - echo "nugetPackageInstallPath= $nugetPackageInstallPath" - - # Verify strong name signing ##################################### - echo "> 1. Verifying strong name signing of DLLs ..." - - # @TODO: This path seems brittle to VS upgrades, can we make it more flexible? - $snPath = "C:\Program Files (x86)\Microsoft SDKs\Windows\*\bin\NETFX 4.8.1 Tools\sn.exe" - - $dllFiles = Get-ChildItem -Path $nugetPackageInstallPath -Recurse -Filter *.dll - $badDlls = @() - foreach ($file in $dllFiles) - { - # Run sn.exe to verify the strong name on each dll - $result = & $snPath -vf $file.FullName - Write-OutPut $result - - # if the dll is not valid, it would be delay signed or test-signed which is not meant for production - if($result[$result.Length-1] -notlike "* is valid") - { - $badDlls += $result[$result.Length-1] - } - } - if($badDlls.Count -gt 0) - { - Write-OutPut "Error: Invalid dlls are detected. Check the list below:" - foreach($dll in $badDlls) - { - Write-Output $dll - } - Exit -1 - } - Write-Host "Strong name has been verified for all dlls" - displayName: "Verify assembly strong names" - - # Validate that the folders in the nuget are expected - # @TODO: This does not verify we are not missing any folders, only that the folders that - # exist are expected to exist. - - powershell: | - # Propagate pipeline to PS variables ############################# - $expectedFolderNames = "${{ parameters.expectedFolderNames }}" - echo "expectedFolderNames= $expectedFolderNames" - - $nugetPackageInstallPath = "${{ variables.nugetPackageInstallPath }}" - echo "nugetPackageInstallPath= $nugetPackageInstallPath" - - # Verify folders are expected #################################### - Get-ChildItem -Path $nugetPackageInstallPath -Directory | select Name | foreach { - if($expectedFolderNames.contains($_.Name)){ - Write-Host expected folder name verfied: $_.Name - } - } - displayName: "Verify NuGet Root Folder Structure" - - # Validate that the folders within the root folders of the nuget are expected - # @TODO: This does not verify we are not missing any folders, only that the folders that - # exist are expected to exist. - - powershell: | - # Propagate pipeline to PS variables ############################# - $expectedDotnetVersions = "${{ parameters.expectedDotnetVersions }}" - echo "expectedDotnetVersions= $expectedDotnetVersions" - - $nugetPackageInstallPath = "${{ variables.nugetPackageInstallPath }}" - echo "nugetPackageInstallPath= $nugetPackageInstallPath" - - # Verify folders are expected #################################### - # Checks the version of DotNetFramework and DotNet - $countErr = 0 - $countPass = 0 - $excludNamesFromRuntimeFolder = 'lib','win','unix' - - Get-ChildItem -Path $nugetPackageInstallPath -Directory | foreach { - $parentname=$_.Name - Write-Host $_.FullName -ForegroundColor yellow - - if($_.Name -ne 'runtimes') { - Get-ChildItem -Path $_.FullName -Directory | select Name | foreach { - if($expectedDotnetVersions.Contains($_.Name)){ - Write-Host "`tExpected version verified in $parentname": $_.Name -ForegroundColor green - $countPass += 1 - } - else{ - Write-Host "`tUnexpected version detected in $parentname": $_.Name - $countErr += 1 - } - } - } - - elseif ($_.Name -eq 'runtimes'){ - Get-ChildItem -Depth 3 -Path $_.FullName -Exclude $excludNamesFromRuntimeFolder -Directory | foreach{ - if('${{ parameters.expectedDotnetVersions }}'.Contains($_.Name)){ - Write-Host "`tExpected version verfied in $parentname": $_.Name - $countPass += 1 - } - else{ - Write-Host "`tUnexpected version detected": $_.Name -ForegroundColor Red - $countErr += 1 - } - } - } - else{ - Write-Host "`tUnknown folder " $_.Name -ForegroundColor Red - Exit -1 - } - } - - Write-Host "_______________" - Write-Host "Expected: $countPass" - Write-Host "Unexpected: $countErr" - Write-Host "_______________" - if ($countErr -ne 0) - { - Write-Host "Unexpected versions are detected!" -ForegroundColor Red - Exit -1 - } - displayName: "Verify NuGet DotNet Versions " - - - powershell: | - # Propagate pipeline to PS variables ############################# - $nugetPackageInstallPath = "${{ variables.nugetPackageInstallPath }}" - echo "nugetPackageInstallPath= $nugetPackageInstallPath" - - # Verify DLL Hierarchy ########################################### - foreach( $folderName in (Get-ChildItem -Path $nugetPackageInstallPath -Directory).Name) - { - # List all Childerns of the Path - Get-ChildItem -Path $nugetPackageInstallPath\$folderName -Recurse -File - $subFiles = Get-ChildItem -Path $nugetPackageInstallPath\$folderName -Recurse -File - - foreach($file in $subFiles) - { - if($subFiles[0].Name -like "*.dll" ) - { - Write-Host $subFiles[0].Name -ForegroundColor Green - Write-Host $subFiles[1].Name -ForegroundColor Green - if(($folderName -eq 'lib') -or ($folderName -eq 'ref')) - { - if($subFiles[2].Name -like "*.dll") - { - Write-Host $subFiles[2].Name -ForegroundColor Green - } - else - { - $subFiles[2].Name - Write-Host "Expected file pattern for localization did not match to *.dll" -ForegroundColor Red - Exit -1 - } - } - } - else - { - $subFiles[0].Name - $subFiles[1].Name - Write-Host "Expected file pattern did not match to *.dll" -ForegroundColor Red - Exit -1 - } - } - } - displayName: 'Verify all DLLs unzipped match "expected" hierarchy' - - # Verify that all DLLs are authenticode signed - # NOTE: This signing is only performed on official builds. - - ${{ if eq(parameters.isOfficial, true) }}: - - powershell: | - # Propagate pipeline to PS variables ############################# - $nugetPackageInstallPath = "${{ variables.nugetPackageInstallPath }}" - echo "nugetPackageInstallPath= $nugetPackageInstallPath" - - # Verify authenticode signature of DLLs ########################## - $dlls = Get-ChildItem -Path $nugetPackageInstallPath -Recurse -Include *.dll - foreach ($status in $dlls | Get-AuthenticodeSignature) - { - if ($status.Status -eq "Valid") - { - Write-Host $status.Status $status.Path - } - else - { - Write-Host "dll status of '$status.Path' is not valid!" -ForegroundColor Red - $status - Exit -1 - } - } - displayName: "Verify all dlls status are Valid" - - - powershell: | - # Propagate pipeline to PS variables ############################# - $expectedFileVersion = "${{ parameters.expectedFileVersion }}" - echo "expectedFileVersion= $expectedFileVersion" - - $expectedPackageVersion = "${{ parameters.expectedPackageVersion }}" - echo "expectedPackageVersion= $expectedPackageVersion" - - $nugetPackageInstallPath = "${{ variables.nugetPackageInstallPath }}" - echo "nugetPackageInstallPath= $nugetPackageInstallPath" - - # Validate ProductVersion and FileVersion fields ################# - $failed = 0 - foreach ( $pVersion in Get-ChildItem *.dll -Path $nugetPackageInstallPath -Recurse | ForEach-Object versioninfo ) - { - if ($pVersion.ProductVersion -Like $expectedPackageVersion + '*') - { - Write-Host -ForegroundColor Green "Correct ProductVersion detected for $($pVersion.FileName): $($pVersion.ProductVersion)" - } - else - { - Write-Host -ForegroundColor Red "Wrong ProductVersion detected for $($pVersion.FileName); expected: $expectedPackageVersion; found: $($pVersion.ProductVersion)" - $failed = 1 - } - - if ($pVersion.FileVersion -eq $expectedFileVersion) - { - Write-Host -ForegroundColor Green "Correct FileVersion detected for $($pVersion.FileName): $($pVersion.FileVersion)" - } - else - { - Write-Host -ForegroundColor Red "Wrong FileVersion detected for $($pVersion.FileName); expected $expectedFileVersion; found: $($pVersion.FileVersion)" - $failed = 1 - } - - # @TODO: We should do a check for assembly version here. - } - - if ($failed -ne 0) - { - Exit -1 - } - - Get-ChildItem *.dll -Path $nugetPackageInstallPath -Recurse | ForEach-Object VersionInfo | Format-List - displayName: 'Verify "File Version" matches expected values for DLLs' diff --git a/eng/pipelines/onebranch/scripts/tests/README.md b/eng/pipelines/onebranch/scripts/tests/README.md index 3585312d53..b2ea3754c1 100644 --- a/eng/pipelines/onebranch/scripts/tests/README.md +++ b/eng/pipelines/onebranch/scripts/tests/README.md @@ -37,10 +37,16 @@ Invoke-Pester ./publish-symbols.Tests.ps1 -Output Detailed | Request bodies | Registration body, default publish flags, flag overrides | | Error handling | Token failure, registration failure, publish failure, status failure — all verify expanded URI in error message | | Status validation | Detects Failed/Cancelled results, respects PublishToInternal/PublishToPublic flags, passes on Succeeded/Pending | +| Package validation | Wildcard vs per-id version expectations, SqlServer omitted when unbuilt, gate tokens, report written before gating, exit-code handling | +| Package signatures | Every package and symbol package verified, all failures reported before throwing | +| Assembly signatures | Package expansion, native binaries under `runtimes/` included, stale expansions replaced, all unsigned assemblies reported | ## Notes - All external calls (`az`, `Invoke-RestMethod`) are mocked — no network access or Azure credentials are required. - Script-level version tests mock `dotnet`; package-composition tests invoke the real MSBuild `GetVersionsSqlClient` and `GetVersionsSqlServer` targets. +- `Get-AuthenticodeSignature` is Windows-only, so the assembly-signature tests declare a stub when + it is absent. Only the signature lookup is substituted; package expansion and reporting run for + real against packages built in the test's temporary directory. - Tests validate scripts in the parent directory relative to this directory. diff --git a/eng/pipelines/onebranch/scripts/tests/validate-packages.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/validate-packages.Tests.ps1 new file mode 100644 index 0000000000..626dd2dca9 --- /dev/null +++ b/eng/pipelines/onebranch/scripts/tests/validate-packages.Tests.ps1 @@ -0,0 +1,188 @@ +<# +.SYNOPSIS + Pester tests for validate-packages.ps1. +#> + +BeforeAll { + $scriptPath = Join-Path $PSScriptRoot '..' 'validate-packages.ps1' + + # Stands in for the built PackageValidator.dll; the script only checks that it exists. + $script:validatorPath = Join-Path $TestDrive 'PackageValidator.dll' + Set-Content -LiteralPath $script:validatorPath -Value 'stub' + + $script:packagesPath = Join-Path $TestDrive 'packages' + New-Item -ItemType Directory -Force -Path $script:packagesPath | Out-Null + Set-Content -LiteralPath (Join-Path $script:packagesPath 'Microsoft.Data.SqlClient.7.1.0.nupkg') -Value 'stub' + Set-Content -LiteralPath (Join-Path $script:packagesPath 'Microsoft.SqlServer.Server.1.1.0.nupkg') -Value 'stub' + + $script:reportPath = Join-Path $TestDrive 'out' 'report.json' + + function Invoke-ValidatePackages { + param( + [string]$PackagesPath = $script:packagesPath, + [string]$ValidatorPath = $script:validatorPath, + [string]$SqlClientPackageVersion = '7.1.0-preview3.26238.3', + [string]$SqlClientFileVersion = '7.1.0.26238', + [string]$SqlServerPackageVersion = '', + [string]$SqlServerFileVersion = '', + [string[]]$FailOn = @('error') + ) + + & $scriptPath ` + -ValidatorPath $ValidatorPath ` + -PackagesPath $PackagesPath ` + -ReportPath $script:reportPath ` + -SqlClientPackageVersion $SqlClientPackageVersion ` + -SqlClientFileVersion $SqlClientFileVersion ` + -SqlServerPackageVersion $SqlServerPackageVersion ` + -SqlServerFileVersion $SqlServerFileVersion ` + -FailOn $FailOn ` + -DotnetPath 'dotnet' *>&1 | Out-String + } + + # Captures the arguments of each invocation so tests can assert on what the validator was + # asked to do, and controls the exit code of each run. + function Set-DotnetMock { + param( + [int]$GateExitCode = 0, + [int]$ReportExitCode = 0 + ) + + $global:validatePackagesInvocations = @() + Mock -CommandName 'dotnet' -MockWith { + $global:validatePackagesInvocations += , @($args) + # The first run carries --json and never gates; the second applies the gate. + if ($args -contains '--json') { + $global:LASTEXITCODE = $ReportExitCode + return '{ "packages": [], "summary": {} }' + } + + $global:LASTEXITCODE = $GateExitCode + return 'validator output' + }.GetNewClosure() + } +} + +AfterAll { + Remove-Variable -Name 'validatePackagesInvocations' -Scope Global -ErrorAction SilentlyContinue +} + +Describe 'validate-packages.ps1 Expectations' { + BeforeEach { + Set-DotnetMock + } + + It 'applies the SqlClient family versions as wildcard expectations' { + Invoke-ValidatePackages | Out-Null + + $gateArgs = $global:validatePackagesInvocations | Where-Object { $_ -notcontains '--json' } | Select-Object -First 1 + $gateArgs | Should -Contain '*=7.1.0-preview3.26238.3' + $gateArgs | Should -Contain '*=7.1.0.26238' + } + + It 'omits SqlServer expectations when its versions are not supplied' { + Invoke-ValidatePackages | Out-Null + + $gateArgs = $global:validatePackagesInvocations | Where-Object { $_ -notcontains '--json' } | Select-Object -First 1 + ($gateArgs -join ' ') | Should -Not -Match 'Microsoft\.SqlServer\.Server=' + } + + It 'adds SqlServer expectations as a per-id override when supplied' { + Invoke-ValidatePackages -SqlServerPackageVersion '1.1.0-preview1.26238.3' -SqlServerFileVersion '1.1.0.26238' | Out-Null + + $gateArgs = $global:validatePackagesInvocations | Where-Object { $_ -notcontains '--json' } | Select-Object -First 1 + $gateArgs | Should -Contain 'Microsoft.SqlServer.Server=1.1.0-preview1.26238.3' + $gateArgs | Should -Contain 'Microsoft.SqlServer.Server=1.1.0.26238' + } + + It 'passes each gate token as its own --fail-on argument' { + Invoke-ValidatePackages -FailOn @('error', 'missing-symbols', 'package-unsigned') | Out-Null + + $gateArgs = $global:validatePackagesInvocations | Where-Object { $_ -notcontains '--json' } | Select-Object -First 1 + $joined = $gateArgs -join ' ' + $joined | Should -Match '--fail-on error' + $joined | Should -Match '--fail-on missing-symbols' + $joined | Should -Match '--fail-on package-unsigned' + } + + It 'splits a single comma-separated gate token, as an Azure Pipelines argument line supplies it' { + Invoke-ValidatePackages -FailOn 'error, missing-symbols' | Out-Null + + $gateArgs = $global:validatePackagesInvocations | Where-Object { $_ -notcontains '--json' } | Select-Object -First 1 + $joined = $gateArgs -join ' ' + $joined | Should -Match '--fail-on error' + $joined | Should -Match '--fail-on missing-symbols' + $joined | Should -Not -Match 'error,' + } + + It 'writes the JSON report before applying the gate' { + Invoke-ValidatePackages | Out-Null + + # The reporting run must come first so the report survives a failing gate. + $firstInvocation = $global:validatePackagesInvocations | Select-Object -First 1 + $firstInvocation | Should -Contain '--json' + Test-Path -LiteralPath $script:reportPath | Should -BeTrue + } + + It 'does not gate the reporting run' { + Invoke-ValidatePackages -FailOn @('error') | Out-Null + + $reportArgs = $global:validatePackagesInvocations | Where-Object { $_ -contains '--json' } | Select-Object -First 1 + ($reportArgs -join ' ') | Should -Not -Match '--fail-on' + } +} + +Describe 'validate-packages.ps1 Exit Codes' { + It 'succeeds when the validator reports no gating findings' { + Set-DotnetMock -GateExitCode 0 + + $output = Invoke-ValidatePackages + $output | Should -Match 'Package validation passed' + } + + It 'fails when a gate is tripped' { + Set-DotnetMock -GateExitCode 2 + + { Invoke-ValidatePackages -FailOn @('error', 'missing-symbols') } | + Should -Throw '*matched the gate (error, missing-symbols)*' + } + + It 'reports an unexpected validator failure distinctly from a tripped gate' { + Set-DotnetMock -GateExitCode 1 + + { Invoke-ValidatePackages } | Should -Throw '*exited unexpectedly with code 1*' + } + + It 'fails the reporting run before gating so the real cause is not obscured' { + Set-DotnetMock -ReportExitCode 1 + + { Invoke-ValidatePackages } | Should -Throw '*failed while writing the JSON report (exit code 1)*' + + # The gating run must not have been reached. + $global:validatePackagesInvocations.Count | Should -Be 1 + } +} + +Describe 'validate-packages.ps1 Error Handling' { + BeforeEach { + Set-DotnetMock + } + + It 'throws when the validator is missing' { + { Invoke-ValidatePackages -ValidatorPath (Join-Path $TestDrive 'absent.dll') } | + Should -Throw '*PackageValidator was not found*' + } + + It 'throws when no packages are found' { + $empty = Join-Path $TestDrive 'empty' + New-Item -ItemType Directory -Force -Path $empty | Out-Null + + { Invoke-ValidatePackages -PackagesPath $empty } | Should -Throw '*No .nupkg files were found*' + } + + It 'rejects a half-supplied SqlServer expectation' { + # Supplying only one would assert a package version without its file version. + { Invoke-ValidatePackages -SqlServerPackageVersion '1.1.0' } | + Should -Throw '*must be supplied together*' + } +} diff --git a/eng/pipelines/onebranch/scripts/tests/verify-assembly-signatures.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/verify-assembly-signatures.Tests.ps1 new file mode 100644 index 0000000000..4d4946a5fb --- /dev/null +++ b/eng/pipelines/onebranch/scripts/tests/verify-assembly-signatures.Tests.ps1 @@ -0,0 +1,149 @@ +<# +.SYNOPSIS + Pester tests for verify-assembly-signatures.ps1. + +.NOTES + Get-AuthenticodeSignature is a Windows-only cmdlet, so a stub is declared when it is absent. + This lets the tests run on any platform while still exercising the script's real expansion and + reporting logic; only the signature lookup itself is substituted. +#> + +BeforeAll { + $scriptPath = Join-Path $PSScriptRoot '..' 'verify-assembly-signatures.ps1' + + if (-not (Get-Command 'Get-AuthenticodeSignature' -ErrorAction SilentlyContinue)) { + function Get-AuthenticodeSignature { + param([Parameter(Mandatory = $true)][string[]]$FilePath) + throw 'stub must be mocked' + } + } + + Add-Type -AssemblyName System.IO.Compression.FileSystem + + # Builds a real .nupkg so the script's expansion path is genuinely exercised. + function New-TestPackage { + param( + [string]$Name, + [string[]]$AssemblyPaths = @('lib/net8.0/Test.dll'), + [string[]]$OtherPaths = @() + ) + + $staging = Join-Path $TestDrive "staging-$Name" + if (Test-Path -LiteralPath $staging) { Remove-Item -LiteralPath $staging -Recurse -Force } + New-Item -ItemType Directory -Force -Path $staging | Out-Null + + foreach ($relative in ($AssemblyPaths + $OtherPaths)) { + $full = Join-Path $staging $relative + New-Item -ItemType Directory -Force -Path (Split-Path -Parent $full) | Out-Null + Set-Content -LiteralPath $full -Value 'stub' + } + + $packagePath = Join-Path $script:packagesPath "$Name.nupkg" + if (Test-Path -LiteralPath $packagePath) { Remove-Item -LiteralPath $packagePath -Force } + [System.IO.Compression.ZipFile]::CreateFromDirectory($staging, $packagePath) + return $packagePath + } + + function Invoke-VerifyAssemblySignatures { + param( + [string]$PackagesPath = $script:packagesPath, + [string]$ExtractPath = $script:extractPath + ) + + & $scriptPath -PackagesPath $PackagesPath -ExtractPath $ExtractPath *>&1 | Out-String + } +} + +AfterAll { + Remove-Variable -Name 'verifyAssemblySeen' -Scope Global -ErrorAction SilentlyContinue +} + +Describe 'verify-assembly-signatures.ps1' { + BeforeEach { + $script:packagesPath = Join-Path $TestDrive 'packages' + $script:extractPath = Join-Path $TestDrive 'extract' + foreach ($path in @($script:packagesPath, $script:extractPath)) { + if (Test-Path -LiteralPath $path) { Remove-Item -LiteralPath $path -Recurse -Force } + New-Item -ItemType Directory -Force -Path $path | Out-Null + } + } + + It 'expands packages and verifies every assembly they contain' { + New-TestPackage -Name 'PackageOne' -AssemblyPaths @('lib/net8.0/One.dll') | Out-Null + New-TestPackage -Name 'PackageTwo' -AssemblyPaths @('lib/net8.0/Two.dll', 'runtimes/win-x64/native/sni.dll') | Out-Null + Mock -CommandName 'Get-AuthenticodeSignature' -MockWith { + $FilePath | ForEach-Object { [pscustomobject]@{ Path = $_; Status = 'Valid' } } + } + + $output = Invoke-VerifyAssemblySignatures + $output | Should -Match 'All 3 assemblies are Authenticode signed' + } + + It 'finds native binaries under runtimes, not just managed assemblies' { + New-TestPackage -Name 'PackageOne' -AssemblyPaths @('runtimes/win-arm64/native/sni.dll') | Out-Null + $global:verifyAssemblySeen = @() + Mock -CommandName 'Get-AuthenticodeSignature' -MockWith { + $global:verifyAssemblySeen = $FilePath + $FilePath | ForEach-Object { [pscustomobject]@{ Path = $_; Status = 'Valid' } } + } + + Invoke-VerifyAssemblySignatures | Out-Null + ($global:verifyAssemblySeen -join ';') | Should -Match 'sni\.dll' + } + + It 'ignores non-assembly content' { + New-TestPackage -Name 'PackageOne' -AssemblyPaths @('lib/net8.0/One.dll') -OtherPaths @('README.md', 'lib/net8.0/One.xml') | Out-Null + Mock -CommandName 'Get-AuthenticodeSignature' -MockWith { + $FilePath | ForEach-Object { [pscustomobject]@{ Path = $_; Status = 'Valid' } } + } + + $output = Invoke-VerifyAssemblySignatures + $output | Should -Match 'All 1 assemblies are Authenticode signed' + } + + It 'fails when an assembly is not validly signed' { + New-TestPackage -Name 'PackageOne' -AssemblyPaths @('lib/net8.0/One.dll', 'lib/net8.0/Two.dll') | Out-Null + Mock -CommandName 'Get-AuthenticodeSignature' -MockWith { + $index = 0 + $FilePath | ForEach-Object { + $status = if ($index -eq 0) { 'Valid' } else { 'NotSigned' } + $index++ + [pscustomobject]@{ Path = $_; Status = $status } + } + } + + { Invoke-VerifyAssemblySignatures } | Should -Throw '*failed for 1 of 2 assemblies*' + } + + It 'reports every unsigned assembly rather than stopping at the first' { + New-TestPackage -Name 'PackageOne' -AssemblyPaths @('lib/net8.0/One.dll', 'lib/net8.0/Two.dll') | Out-Null + Mock -CommandName 'Get-AuthenticodeSignature' -MockWith { + $FilePath | ForEach-Object { [pscustomobject]@{ Path = $_; Status = 'NotSigned' } } + } + + { Invoke-VerifyAssemblySignatures } | Should -Throw '*failed for 2 of 2 assemblies*' + } + + It 'replaces a previous expansion so stale content cannot be verified' { + New-TestPackage -Name 'PackageOne' -AssemblyPaths @('lib/net8.0/One.dll', 'lib/net8.0/Stale.dll') | Out-Null + Mock -CommandName 'Get-AuthenticodeSignature' -MockWith { + $FilePath | ForEach-Object { [pscustomobject]@{ Path = $_; Status = 'Valid' } } + } + Invoke-VerifyAssemblySignatures | Out-Null + + # Repack the same package id with fewer assemblies; the stale one must not linger. + New-TestPackage -Name 'PackageOne' -AssemblyPaths @('lib/net8.0/One.dll') | Out-Null + $output = Invoke-VerifyAssemblySignatures + $output | Should -Match 'All 1 assemblies are Authenticode signed' + } + + It 'throws when no packages are found' { + { Invoke-VerifyAssemblySignatures } | Should -Throw '*No .nupkg files were found*' + } + + It 'throws when packages contain no assemblies' { + New-TestPackage -Name 'PackageOne' -AssemblyPaths @() -OtherPaths @('README.md') | Out-Null + + { Invoke-VerifyAssemblySignatures } | Should -Throw '*No assemblies were found*' + } +} diff --git a/eng/pipelines/onebranch/scripts/tests/verify-package-signatures.Tests.ps1 b/eng/pipelines/onebranch/scripts/tests/verify-package-signatures.Tests.ps1 new file mode 100644 index 0000000000..cadc1831a4 --- /dev/null +++ b/eng/pipelines/onebranch/scripts/tests/verify-package-signatures.Tests.ps1 @@ -0,0 +1,87 @@ +<# +.SYNOPSIS + Pester tests for verify-package-signatures.ps1. +#> + +BeforeAll { + $scriptPath = Join-Path $PSScriptRoot '..' 'verify-package-signatures.ps1' + + $script:packagesPath = Join-Path $TestDrive 'packages' + New-Item -ItemType Directory -Force -Path (Join-Path $script:packagesPath 'SqlClient') | Out-Null + New-Item -ItemType Directory -Force -Path (Join-Path $script:packagesPath 'SqlServer') | Out-Null + + # Symbol packages are signed too, so both extensions must be picked up, and the nested layout + # mirrors how each artifact is downloaded into its own subdirectory. + Set-Content -LiteralPath (Join-Path $script:packagesPath 'SqlClient' 'Microsoft.Data.SqlClient.7.1.0.nupkg') -Value 'stub' + Set-Content -LiteralPath (Join-Path $script:packagesPath 'SqlClient' 'Microsoft.Data.SqlClient.7.1.0.snupkg') -Value 'stub' + Set-Content -LiteralPath (Join-Path $script:packagesPath 'SqlServer' 'Microsoft.SqlServer.Server.1.1.0.nupkg') -Value 'stub' + + function Invoke-VerifyPackageSignatures { + param([string]$PackagesPath = $script:packagesPath) + + & $scriptPath -PackagesPath $PackagesPath -DotnetPath 'dotnet' *>&1 | Out-String + } + + # Fails verification only for packages whose name matches, so tests can make a subset unsigned. + function Set-DotnetMock { + param([string]$FailPattern = '') + + $global:verifyPackageInvocations = @() + Mock -CommandName 'dotnet' -MockWith { + $global:verifyPackageInvocations += , @($args) + $target = $args[-1] + if ($FailPattern -and $target -match $FailPattern) { + $global:LASTEXITCODE = 1 + return "unsigned" + } + + $global:LASTEXITCODE = 0 + return "verified" + }.GetNewClosure() + } +} + +AfterAll { + Remove-Variable -Name 'verifyPackageInvocations' -Scope Global -ErrorAction SilentlyContinue +} + +Describe 'verify-package-signatures.ps1' { + It 'verifies every package and symbol package found' { + Set-DotnetMock + + $output = Invoke-VerifyPackageSignatures + $output | Should -Match 'All 3 package signature\(s\) verified' + $global:verifyPackageInvocations.Count | Should -Be 3 + } + + It 'invokes dotnet nuget verify with --all' { + Set-DotnetMock + + Invoke-VerifyPackageSignatures | Out-Null + + $first = $global:verifyPackageInvocations | Select-Object -First 1 + ($first -join ' ') | Should -Match 'nuget verify --all' + } + + It 'fails when a package signature does not verify' { + Set-DotnetMock -FailPattern 'SqlServer' + + { Invoke-VerifyPackageSignatures } | Should -Throw '*Microsoft.SqlServer.Server.1.1.0.nupkg*' + } + + It 'checks every package before failing so all failures are reported' { + Set-DotnetMock -FailPattern '\.nupkg$' + + # Two of the three files are .nupkg; both must appear rather than only the first. + { Invoke-VerifyPackageSignatures } | Should -Throw '*failed for 2 of 3 package(s)*' + $global:verifyPackageInvocations.Count | Should -Be 3 + } + + It 'throws when no packages are found' { + Set-DotnetMock + $empty = Join-Path $TestDrive 'empty' + New-Item -ItemType Directory -Force -Path $empty | Out-Null + + { Invoke-VerifyPackageSignatures -PackagesPath $empty } | Should -Throw '*No package files were found*' + } +} diff --git a/eng/pipelines/onebranch/scripts/validate-packages.ps1 b/eng/pipelines/onebranch/scripts/validate-packages.ps1 new file mode 100644 index 0000000000..af5149c9f5 --- /dev/null +++ b/eng/pipelines/onebranch/scripts/validate-packages.ps1 @@ -0,0 +1,213 @@ +<# +.SYNOPSIS + Runs the PackageValidator tool over the NuGet packages produced by a OneBranch build. + +.DESCRIPTION + Invokes tools/PackageValidator once for a whole directory of packages rather than once per + package, because its most valuable checks are cross-package: every package in the SqlClient + family must carry the same version, and their inter-package dependency ranges must agree. + Validating one package at a time would silently skip all of those findings. + + The validator runs twice over the same inputs. The first run writes a machine-readable report + and never gates, so the report exists even when validation fails. The second run renders the + human-readable report and applies the gate, so a failing build shows its findings in its own + log rather than only in an artifact. + + Expected versions are supplied by the caller rather than derived here. The compute-versions + stage already computes every version the build stamps, and re-deriving them would reintroduce + the drift this validation exists to catch. + + Microsoft.SqlServer.Server is versioned separately from the SqlClient family, so its expected + versions are applied as a per-id override of the family wildcard. When it is not built in a + run, its package is absent from the drop and its expectations must be omitted entirely: the + validator rejects an expectation whose value is empty. + +.PARAMETER ValidatorPath + Path to the built PackageValidator.dll. Invoked through the managed assembly rather than the + native apphost so the same command works regardless of agent OS. + +.PARAMETER PackagesPath + Directory scanned recursively for .nupkg files. Sibling .snupkg files must sit beside their + .nupkg for symbol matching to resolve, which is how the build jobs publish them. + +.PARAMETER ReportPath + Path of the JSON report to write. Parent directories are created as needed. + +.PARAMETER SqlClientPackageVersion + Package version expected of every package in the SqlClient family, applied as a wildcard. + Pointing every package at one value is what proves they agree, and also catches the case where + all of them are consistently wrong. + +.PARAMETER SqlClientFileVersion + Assembly file version expected of every assembly in the SqlClient family. + +.PARAMETER SqlServerPackageVersion + Package version expected of Microsoft.SqlServer.Server. Omit when SqlServer is not built. + +.PARAMETER SqlServerFileVersion + Assembly file version expected of Microsoft.SqlServer.Server. Omit when SqlServer is not built. + +.PARAMETER FailOn + Finding severities and/or categories that fail the build. Run the validator with --help to see + the available categories. Note that missing-symbols is a warning and package-unsigned is info, + so neither is covered by the error severity and both must be named explicitly. + + Accepts either an array or a single comma-separated string, because an Azure Pipelines task + argument line collapses to one token and PowerShell's -File mode does not split it. + +.PARAMETER DotnetPath + dotnet executable to invoke. Defaults to the dotnet command resolved from PATH. This parameter + primarily supports isolated testing. + +.EXAMPLE + ./validate-packages.ps1 ` + -ValidatorPath ./PackageValidator.dll ` + -PackagesPath ./packages ` + -ReportPath ./out/report.json ` + -SqlClientPackageVersion 7.1.0-preview3.26238.3 ` + -SqlClientFileVersion 7.1.0.26238 ` + -FailOn error,missing-symbols + + Validates a family-only drop, failing on any error and on missing symbols. + +.NOTES + File Name : validate-packages.ps1 + Requires : PowerShell 7+ and the repository-pinned .NET SDK. + Called by : validate-packages-step.yml + + PackageValidator exit codes: + 0 - No gating findings. + 1 - The validator itself failed. + 2 - A --fail-on gate was tripped. +#> + +[CmdletBinding()] +param( + [Parameter(Mandatory = $true, HelpMessage = "Path to the built PackageValidator.dll.")] + [ValidateNotNullOrEmpty()] + [string]$ValidatorPath, + + [Parameter(Mandatory = $true, HelpMessage = "Directory scanned recursively for .nupkg files.")] + [ValidateNotNullOrEmpty()] + [string]$PackagesPath, + + [Parameter(Mandatory = $true, HelpMessage = "Path of the JSON report to write.")] + [ValidateNotNullOrEmpty()] + [string]$ReportPath, + + [Parameter(Mandatory = $true, HelpMessage = "Package version expected of the SqlClient family.")] + [ValidateNotNullOrEmpty()] + [string]$SqlClientPackageVersion, + + [Parameter(Mandatory = $true, HelpMessage = "File version expected of the SqlClient family.")] + [ValidateNotNullOrEmpty()] + [string]$SqlClientFileVersion, + + [Parameter(HelpMessage = "Package version expected of Microsoft.SqlServer.Server, when built.")] + [string]$SqlServerPackageVersion = "", + + [Parameter(HelpMessage = "File version expected of Microsoft.SqlServer.Server, when built.")] + [string]$SqlServerFileVersion = "", + + [Parameter(Mandatory = $true, HelpMessage = "Severities and/or categories that fail the build.")] + [ValidateNotNullOrEmpty()] + [string[]]$FailOn, + + [Parameter(HelpMessage = "dotnet executable to invoke.")] + [ValidateNotNullOrEmpty()] + [string]$DotnetPath = "dotnet" +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = "Stop" + +# Split on commas so a single "error,missing-symbols" token behaves like a two-element array. +$failOnTokens = @($FailOn -split ',' | ForEach-Object { $_.Trim() } | Where-Object { $_ }) +if ($failOnTokens.Count -eq 0) { + throw "FailOn must name at least one severity or category." +} + +Write-Host "=== Validate Packages Parameters ===" +Write-Host "ValidatorPath: ${ValidatorPath}" +Write-Host "PackagesPath: ${PackagesPath}" +Write-Host "ReportPath: ${ReportPath}" +Write-Host "SqlClientPackageVersion: ${SqlClientPackageVersion}" +Write-Host "SqlClientFileVersion: ${SqlClientFileVersion}" +Write-Host "SqlServerPackageVersion: ${SqlServerPackageVersion}" +Write-Host "SqlServerFileVersion: ${SqlServerFileVersion}" +Write-Host "FailOn: $($failOnTokens -join ', ')" +Write-Host "====================================" + +if (-not (Test-Path -LiteralPath $ValidatorPath)) { + throw "PackageValidator was not found at '${ValidatorPath}'." +} + +$packages = @(Get-ChildItem -Path $PackagesPath -Recurse -File -Filter *.nupkg -ErrorAction SilentlyContinue) +if ($packages.Count -eq 0) { + throw "No .nupkg files were found under '${PackagesPath}'." +} + +Write-Host "Validating $($packages.Count) package(s):" +$packages | ForEach-Object { Write-Host " $($_.Name)" } + +# A bare value applies to every package; an id=value pair overrides it for that package only. +$expectations = @( + "--expect-package-version", "*=${SqlClientPackageVersion}" + "--expect-file-version", "*=${SqlClientFileVersion}" +) + +# Both SqlServer versions travel together: supplying only one would assert half a package. +$hasSqlServerPackageVersion = -not [string]::IsNullOrWhiteSpace($SqlServerPackageVersion) +$hasSqlServerFileVersion = -not [string]::IsNullOrWhiteSpace($SqlServerFileVersion) +if ($hasSqlServerPackageVersion -ne $hasSqlServerFileVersion) { + throw "SqlServerPackageVersion and SqlServerFileVersion must be supplied together, or not at all." +} + +if ($hasSqlServerPackageVersion) { + $expectations += @( + "--expect-package-version", "Microsoft.SqlServer.Server=${SqlServerPackageVersion}" + "--expect-file-version", "Microsoft.SqlServer.Server=${SqlServerFileVersion}" + ) +} + +$gate = @() +foreach ($token in $failOnTokens) { + $gate += @("--fail-on", $token) +} + +Write-Host "Expectations: $($expectations -join ' ')" +Write-Host "Gate: $($gate -join ' ')" + +$reportDirectory = Split-Path -Parent $ReportPath +if ($reportDirectory) { + New-Item -ItemType Directory -Force -Path $reportDirectory | Out-Null +} + +# Reported before gating so the JSON exists even for a failing run. +& $DotnetPath $ValidatorPath $PackagesPath --json @expectations | + Set-Content -LiteralPath $ReportPath -Encoding utf8 +$reportExitCode = $LASTEXITCODE + +# This run is ungated, so any non-zero code means the validator itself failed and the report it +# produced cannot be trusted. Fail here rather than let the gated run obscure the real cause. +if ($reportExitCode -ne 0) { + throw "PackageValidator failed while writing the JSON report (exit code ${reportExitCode})." +} + +Write-Host "Wrote JSON report to ${ReportPath}" + +Write-Host "" +Write-Host "=== Package validation report ===" +& $DotnetPath $ValidatorPath $PackagesPath @expectations @gate +$exitCode = $LASTEXITCODE + +if ($exitCode -eq 0) { + Write-Host "" + Write-Host "Package validation passed." +} +elseif ($exitCode -eq 2) { + throw "Package validation failed: one or more findings matched the gate ($($failOnTokens -join ', '))." +} +else { + throw "PackageValidator exited unexpectedly with code ${exitCode}." +} diff --git a/eng/pipelines/onebranch/scripts/verify-assembly-signatures.ps1 b/eng/pipelines/onebranch/scripts/verify-assembly-signatures.ps1 new file mode 100644 index 0000000000..76c1495602 --- /dev/null +++ b/eng/pipelines/onebranch/scripts/verify-assembly-signatures.ps1 @@ -0,0 +1,99 @@ +<# +.SYNOPSIS + Verifies that every assembly shipped inside an official build's NuGet packages is Authenticode + signed. + +.DESCRIPTION + Expands each .nupkg beneath a directory and checks the Authenticode signature of every assembly + it contains, including native binaries under runtimes/. + + Packages are expanded rather than installed through NuGet so that every produced package is + covered without resolving dependencies, and so that the check does not depend on any single + package id. + + This complements PackageValidator, which reports strong-name state from assembly metadata + cross-platform. Authenticode verification requires the Windows trust store, so this script runs + only on Windows agents and only for official builds; non-official builds deliberately produce + unsigned assemblies. + + Every assembly is checked before failing, so a single run reports all unsigned assemblies + rather than stopping at the first. + +.PARAMETER PackagesPath + Directory scanned recursively for .nupkg files to expand. + +.PARAMETER ExtractPath + Directory the packages are expanded into. Each package is expanded into its own subdirectory so + that identically-named assemblies from different packages cannot collide. Existing content for + a package is replaced. + +.EXAMPLE + ./verify-assembly-signatures.ps1 -PackagesPath ./packages -ExtractPath ./extract + + Expands every package beneath ./packages and verifies the signature of each assembly. + +.NOTES + File Name : verify-assembly-signatures.ps1 + Requires : PowerShell 7+ on Windows (Get-AuthenticodeSignature). + Called by : validate-packages-job.yml +#> + +[CmdletBinding()] +param( + [Parameter(Mandatory = $true, HelpMessage = "Directory scanned recursively for .nupkg files.")] + [ValidateNotNullOrEmpty()] + [string]$PackagesPath, + + [Parameter(Mandatory = $true, HelpMessage = "Directory the packages are expanded into.")] + [ValidateNotNullOrEmpty()] + [string]$ExtractPath +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = "Stop" + +Write-Host "=== Verify Assembly Signatures Parameters ===" +Write-Host "PackagesPath: ${PackagesPath}" +Write-Host "ExtractPath: ${ExtractPath}" +Write-Host "=============================================" + +$packages = @(Get-ChildItem -Path $PackagesPath -Recurse -File -Filter *.nupkg -ErrorAction SilentlyContinue) +if ($packages.Count -eq 0) { + throw "No .nupkg files were found under '${PackagesPath}'." +} + +New-Item -ItemType Directory -Force -Path $ExtractPath | Out-Null + +Add-Type -AssemblyName System.IO.Compression.FileSystem +foreach ($package in $packages) { + $destination = Join-Path $ExtractPath $package.BaseName + if (Test-Path -LiteralPath $destination) { + Remove-Item -LiteralPath $destination -Recurse -Force + } + + Write-Host "Expanding $($package.Name)" + [System.IO.Compression.ZipFile]::ExtractToDirectory($package.FullName, $destination) +} + +$assemblies = @(Get-ChildItem -Path $ExtractPath -Recurse -File -Filter *.dll) +if ($assemblies.Count -eq 0) { + throw "No assemblies were found under '${ExtractPath}'." +} + +# Every assembly is checked before throwing so one run reports all failures. +$unsigned = @() +foreach ($signature in @(Get-AuthenticodeSignature -FilePath $assemblies.FullName)) { + if ($signature.Status -eq "Valid") { + Write-Host " OK $($signature.Path)" + } + else { + Write-Host " FAIL $($signature.Path) - $($signature.Status)" + $unsigned += $signature.Path + } +} + +if ($unsigned.Count -gt 0) { + throw "Authenticode verification failed for $($unsigned.Count) of $($assemblies.Count) assemblies." +} + +Write-Host "All $($assemblies.Count) assemblies are Authenticode signed." diff --git a/eng/pipelines/onebranch/scripts/verify-package-signatures.ps1 b/eng/pipelines/onebranch/scripts/verify-package-signatures.ps1 new file mode 100644 index 0000000000..74fa864c07 --- /dev/null +++ b/eng/pipelines/onebranch/scripts/verify-package-signatures.ps1 @@ -0,0 +1,72 @@ +<# +.SYNOPSIS + Verifies the NuGet signatures of every package produced by an official OneBranch build. + +.DESCRIPTION + Runs `dotnet nuget verify --all` over every .nupkg and .snupkg found beneath a directory, + confirming that each carries a valid, trusted signature. + + This complements PackageValidator, which reports signature *presence* from package metadata + cross-platform. Establishing that a signature is trusted requires the platform trust store, + which is why this runs separately and only on official builds. Non-official builds deliberately + produce unsigned packages, so verifying them would always fail. + + Every package is verified before failing, so a single run reports all unsigned packages rather + than stopping at the first. + +.PARAMETER PackagesPath + Directory scanned recursively for .nupkg and .snupkg files. + +.PARAMETER DotnetPath + dotnet executable to invoke. Defaults to the dotnet command resolved from PATH. This parameter + primarily supports isolated testing. + +.EXAMPLE + ./verify-package-signatures.ps1 -PackagesPath ./packages + + Verifies every package and symbol package beneath ./packages. + +.NOTES + File Name : verify-package-signatures.ps1 + Requires : PowerShell 7+ and the repository-pinned .NET SDK. + Called by : validate-packages-job.yml +#> + +[CmdletBinding()] +param( + [Parameter(Mandatory = $true, HelpMessage = "Directory scanned recursively for package files.")] + [ValidateNotNullOrEmpty()] + [string]$PackagesPath, + + [Parameter(HelpMessage = "dotnet executable to invoke.")] + [ValidateNotNullOrEmpty()] + [string]$DotnetPath = "dotnet" +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = "Stop" + +Write-Host "=== Verify Package Signatures Parameters ===" +Write-Host "PackagesPath: ${PackagesPath}" +Write-Host "============================================" + +$packages = @(Get-ChildItem -Path $PackagesPath -Recurse -File -Include *.nupkg, *.snupkg -ErrorAction SilentlyContinue) +if ($packages.Count -eq 0) { + throw "No package files were found under '${PackagesPath}'." +} + +# Every package is checked before throwing so one run reports all failures. +$failed = @() +foreach ($package in $packages) { + Write-Host "Verifying $($package.Name)" + & $DotnetPath nuget verify --all $package.FullName + if ($LASTEXITCODE -ne 0) { + $failed += $package.Name + } +} + +if ($failed.Count -gt 0) { + throw "NuGet signature verification failed for $($failed.Count) of $($packages.Count) package(s): $($failed -join ', ')" +} + +Write-Host "All $($packages.Count) package signature(s) verified." diff --git a/eng/pipelines/onebranch/stages/build-stages.yml b/eng/pipelines/onebranch/stages/build-stages.yml index afd1907859..34f88bd9b3 100644 --- a/eng/pipelines/onebranch/stages/build-stages.yml +++ b/eng/pipelines/onebranch/stages/build-stages.yml @@ -331,17 +331,44 @@ stages: packageVersion: '$(sqlClientPackageVersion)' # ==================================================================== - # Validation - # @TODO: Update validate-signed-package-job to compute expected versions from - # Versions.props (same as build jobs) instead of receiving them as parameters. + # Stage 5: Validation + # Validates every package produced by this run, together, so that + # cross-package rules (shared family version, dependency agreement) + # are actually exercised. Depends on all build stages. # ==================================================================== - # - stage: sqlclient_package_validation - # displayName: "SqlClient Package Validation" - # dependsOn: build_dependent - # jobs: - # - template: /eng/pipelines/onebranch/jobs/validate-signed-package-job.yml@self - # parameters: - # artifactName: '${{ parameters.sqlClientArtifactsName }}' - # expectedFileVersion: - # expectedPackageVersion: - # isOfficial: ${{ parameters.isOfficial }} + - stage: package_validation + displayName: "Validate Packages" + dependsOn: + - compute_versions + - build_independent + - build_abstractions + - build_dependent + - build_addons + + variables: + - name: sqlClientPackageVersion + value: $[ stageDependencies.compute_versions.compute_versions_job.outputs['versions.SqlClientPackageVersion'] ] + - name: sqlClientFileVersion + value: $[ stageDependencies.compute_versions.compute_versions_job.outputs['versions.SqlClientFileVersion'] ] + - name: sqlServerPackageVersion + value: $[ stageDependencies.compute_versions.compute_versions_job.outputs['versions.SqlServerPackageVersion'] ] + - name: sqlServerFileVersion + value: $[ stageDependencies.compute_versions.compute_versions_job.outputs['versions.SqlServerFileVersion'] ] + + jobs: + - template: /eng/pipelines/onebranch/jobs/validate-packages-job.yml@self + parameters: + abstractionsArtifactsName: '${{ parameters.abstractionsArtifactsName }}' + akvProviderArtifactsName: '${{ parameters.akvProviderArtifactsName }}' + azureArtifactsName: '${{ parameters.azureArtifactsName }}' + loggingArtifactsName: '${{ parameters.loggingArtifactsName }}' + sqlClientArtifactsName: '${{ parameters.sqlClientArtifactsName }}' + sqlServerArtifactsName: '${{ parameters.sqlServerArtifactsName }}' + + sqlClientPackageVersion: '$(sqlClientPackageVersion)' + sqlClientFileVersion: '$(sqlClientFileVersion)' + sqlServerPackageVersion: '$(sqlServerPackageVersion)' + sqlServerFileVersion: '$(sqlServerFileVersion)' + + buildSqlServer: ${{ parameters.buildSqlServer }} + isOfficial: ${{ parameters.isOfficial }} diff --git a/eng/pipelines/onebranch/stages/release-stages.yml b/eng/pipelines/onebranch/stages/release-stages.yml index d160c8906f..7ee8023cd9 100644 --- a/eng/pipelines/onebranch/stages/release-stages.yml +++ b/eng/pipelines/onebranch/stages/release-stages.yml @@ -112,6 +112,10 @@ stages: ${{ else }}: displayName: Release to NuGet Test dependsOn: + # Nothing is published unless every produced package passed validation. This stage also + # depends on all four build stages, but we keep the complete list here anyway to prevent + # regressions if package validation changes its dependencies. + - package_validation - ${{ if or(parameters.releaseSqlServer, parameters.releaseSqlClient) }}: - build_independent - ${{ if parameters.releaseSqlClient }}: diff --git a/eng/pipelines/onebranch/steps/validate-packages-step.yml b/eng/pipelines/onebranch/steps/validate-packages-step.yml new file mode 100644 index 0000000000..173df4d7ed --- /dev/null +++ b/eng/pipelines/onebranch/steps/validate-packages-step.yml @@ -0,0 +1,80 @@ +################################################################################# +# Licensed to the .NET Foundation under one or more agreements. # +# The .NET Foundation licenses this file to you under the MIT license. # +# See the LICENSE file in the project root for more information. # +################################################################################# + +# Builds and runs tools/PackageValidator over a directory of produced NuGet packages. +# +# The validator is invoked once for the whole directory rather than once per package, because its +# most valuable checks are cross-package: it confirms that every package in the SqlClient family +# carries the same version and that their inter-package dependency ranges agree. Running it per +# package would silently skip all of those findings. +# +# Two invocations are made over the same inputs. The first writes a machine-readable report and is +# ungated, so the artifact exists even for a failing run. It still fails the step if the validator +# itself errors, because the report it produced could not then be trusted. The second renders the +# human-readable report and applies the gate, so a failed build shows the findings in its own log. + +parameters: + # Directory scanned recursively for .nupkg files. Sibling .snupkg files must sit beside their + # .nupkg for symbol matching to resolve, which is how the build jobs publish them. + - name: packagesPath + type: string + + # Path of the JSON report to write. + - name: reportPath + type: string + + # Expected versions shared by the whole SqlClient family (Logging, Abstractions, SqlClient, + # Azure, AkvProvider), applied as wildcard expectations. Pointing every package at the same + # value is what proves they agree, and also catches every package being consistently wrong. + - name: sqlClientPackageVersion + type: string + + - name: sqlClientFileVersion + type: string + + # Expected versions for the separately-versioned Microsoft.SqlServer.Server, applied as a per-id + # override of the family wildcard. Left empty when SqlServer is not built this run: its package + # is then absent from the drop, and the validator rejects an expectation with an empty value. + - name: sqlServerPackageVersion + type: string + default: '' + + - name: sqlServerFileVersion + type: string + default: '' + + # Finding severities and/or categories that fail the build. Run the validator with --help to + # see the full set of categories. + - name: failOn + type: object + default: + - error + +steps: + - task: DotNetCoreCLI@2 + displayName: 'build.proj - BuildPackageValidator' + inputs: + command: build + projects: '$(REPO_ROOT)/build.proj' + arguments: >- + -t:BuildPackageValidator + -p:Configuration=Release + + - task: PowerShell@2 + displayName: 'Validate NuGet packages' + inputs: + targetType: filePath + pwsh: true + filePath: $(REPO_ROOT)/eng/pipelines/onebranch/scripts/validate-packages.ps1 + arguments: >- + -ValidatorPath "$(REPO_ROOT)/tools/PackageValidator/src/bin/Release/net10.0/PackageValidator.dll" + -PackagesPath "${{ parameters.packagesPath }}" + -ReportPath "${{ parameters.reportPath }}" + -SqlClientPackageVersion "${{ parameters.sqlClientPackageVersion }}" + -SqlClientFileVersion "${{ parameters.sqlClientFileVersion }}" + -SqlServerPackageVersion "${{ parameters.sqlServerPackageVersion }}" + -SqlServerFileVersion "${{ parameters.sqlServerFileVersion }}" + -FailOn "${{ join(',', parameters.failOn) }}"