Skip to content

(CAT-2840) Add Ruby 4.0 / Puppet 9 CI lane, source gems from PuppetCore - #391

Draft
LukasAud wants to merge 17 commits into
mainfrom
cat-2840-puppet9-ci
Draft

LukasAud wants to merge 17 commits into
mainfrom
cat-2840-puppet9-ci

Conversation

@LukasAud

Copy link
Copy Markdown
Contributor

Summary

Scoping work for CAT-2840 (Puppet 9 compatibility spike). Per discussion with Josh Cooper, main/2.x is already the line Puppet Agent ships for Puppet 9 in production (the earlier CAT-2327 caching regression was fixed forward via PA-7812 / #387), so this targets main only — 1.9.x is out of scope.

This mirrors the CI/Gemfile pattern already rolled out on ruby-pwsh (#385), puppetlabs_spec_helper, and bolt-private (#120):

  • Add a puppet_version: '9' / ruby: '4.0' matrix entry (plus jruby-10.1.1.0) alongside the existing Puppet 8 lane in ci.yml and nightly.yml.
  • Add fail-fast: false so a failure in the new, unproven Puppet 9 lane doesn't cancel the stable Puppet 8 results.
  • Route the puppet gem (and its transitive facter dependency) through the private PuppetCore source (rubygems-puppetcore.puppet.com) via PUPPET_FORGE_TOKEN, falling back to the default source otherwise. This is required, not optional — the public puppet gem on RubyGems tops out at 8.10.0, and Puppet 9 is only available via the private PuppetCore source.
  • Bump puppetlabs_spec_helper ~> 8.0~> 9.0 (published, CVE-fixed) and the Windows :integration group's ffi pin 1.15.51.17.1 (matches the existing open dependabot PR Update ffi requirement from 1.15.5 to 1.17.1 #364).

Known open questions (this is scoping/spike work, not a finished implementation)

  • Does PUPPET_FORGE_TOKEN already exist as a secret for this repo, or does it need adding (org-level or repo-level)?
  • Does ffi 1.17.1 conflict with the actual Puppet 9 gemspec's ffi constraint? The public Puppet 8 gemspec on puppetlabs/puppet@main still caps ffi at < 1.17.0 — untested against the private Puppet 9 gemspec.
  • Can ruby/setup-ruby install jruby-10.1.1.0? (Puppet Server 9.0 moved to JRuby 10.1; version confirmed to exist upstream, but untested with this action.)
  • Once CI actually runs, expect some spec failures to triage — e.g. Ruby 3.4+ changed Hash#inspect's default format ({:key=>"value"}{key: "value"}), which broke string-matching assertions in ruby-pwsh's test suite during the same migration; resource_api's specs haven't been audited for the same pattern yet.

Test plan

  • CI passes on the existing ruby 3.2 / jruby-9.4.2.0 | puppet 8 lanes (no regression)
  • CI passes on the new ruby 4.0 / jruby-10.1.1.0 | puppet 9 lanes
  • Windows integration group installs cleanly with ffi 1.17.1 on both Ruby lanes

Adds a puppet_version: '9' / ruby: '4.0' (and jruby-10.1.1.0) matrix
entry alongside the existing Puppet 8 lane in ci.yml and nightly.yml,
and routes the puppet/facter gems through the private PuppetCore
source (rubygems-puppetcore.puppet.com) via PUPPET_FORGE_TOKEN, since
neither is published to public RubyGems for Puppet 9 yet.

Mirrors the pattern already rolled out on ruby-pwsh (#385),
puppetlabs_spec_helper, and bolt-private (#120). Bumps
puppetlabs_spec_helper to ~> 9.0 and the Windows integration ffi pin
to 1.17.1 (matches the existing open dependabot PR #364) for Ruby 4
compatibility.

Known open questions, called out inline and in the PR description:
- whether 1.17.1 conflicts with the actual Puppet 9 gemspec's ffi
  constraint (public Puppet 8 gemspec still caps ffi at < 1.17.0)
- whether ruby/setup-ruby can install jruby-10.1.1.0
CI showed the unconditional ~> 9.0 bump broke the existing Puppet 8
jruby-9.4.2.0 lane: puppetlabs_spec_helper 9.0.0 requires Ruby >= 3.2,
but jruby-9.4.2.0 only implements the Ruby 3.1 interface. Condition
the version on RUBY_VERSION so that lane stays on 8.0 while Ruby 3.2,
Ruby 4.0, and jruby-10.1.1.0 (which implements Ruby 4.0 -- confirmed
via CI logs) get 9.0.
CI showed the Puppet 9 (PuppetCore-sourced) lanes reliably (not
flakily -- reproduced on re-run) fail at the "Rubocop" step with
"Could not find gem 'puppet (~> 9)' in locally installed gems",
immediately after "bundle install" in the prior step succeeded and
reported the gem installed. Puppet 8 lanes (no PuppetCore source
involved) never hit this with the same split-step structure.

Testing whether running install/rubocop/spec as one continuous shell
step (vs. three separate steps, each a fresh process boundary)
avoids whatever is causing bundler to lose track of the
PuppetCore-pinned puppet gem between steps. Not yet root-caused --
will revert to split steps once understood, this is here to observe
CI behavior.
…e install

The working ruby-pwsh#385 precedent (via the shared cat-github-actions
gem_ci.yml reusable workflow) uses `bundler-cache: true` on the
ruby/setup-ruby step rather than a separate manual `bundle install`
run step -- our ci.yml was doing the latter. Testing whether that's
the actual differentiator behind the reproducible "GemNotFound:
puppet (~> 9)" failure in the subsequent rubocop/spec steps (which
only happens on the PuppetCore-sourced Puppet 9 lanes, never on
Puppet 8). Replaces the imperative `bundle config set without
development` / `with integration` calls with equivalent BUNDLE_WITHOUT
/ BUNDLE_WITH env vars, which bundler-cache's internal install honors.
Root-caused via diagnostic: bundle install/bundle check both succeed
and confirm the lockfile is fully valid (puppet 9.0.0 correctly
resolved from PuppetCore), but `bundle exec` prints "Your Gemfile
contains scoped sources that don't implement a dependency API,
namely: * locally installed gems" and then fails with GemNotFound.
The PuppetCore source apparently doesn't implement Bundler's legacy
dependency API, which sends `bundle exec`'s re-verification down a
broken "locally installed gems" fallback lookup on every invocation,
even though nothing needs re-resolving.

`bundle config set frozen true` after install forces later `bundle
exec` calls to trust the already-valid lockfile instead of
re-triggering that fallback. Removes the earlier DIAGNOSTIC step.
Removed the diagnostic step and reverted the frozen-mode experiment
(neither fixed the underlying issue -- see PR description/discussion
for the full root-cause writeup). Kept the facter-pin removal as a
harmless simplification confirmed unnecessary by Gemfile.lock
evidence, and cleaned up its comment now that it's no longer framed
as a test.
…inux_tests, test if group exclusion is the trigger"

This reverts commit e96ccf5.
…urce_api self-dependency conflict"

This reverts commit 9a9ce67.
…9 to satisfy puppet 9.0.0's declared puppet-resource_api (~> 1.9) requirement -- checking if this is really what's causing the bundle exec failure
…to 1.9.99 to satisfy puppet 9.0.0's declared puppet-resource_api (~> 1.9) requirement -- checking if this is really what's causing the bundle exec failure"

This reverts commit 0226c9e.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant