Fix two issues by including the ruby platform variants in the lock file

[rwstauner]

What was the end-user or developer problem that led to this PR?

2 issues:

• When the lock file only contains platform specific variants but install is being done by a version of ruby that doesn't work with the platform version bundler will install the ruby variant... but then at setup time it errors because it looks for the platform variant and it isn't there. bundler doesn't need to install an alternative that doesn't satisfy it's own requirements. The first commit adds a test that verifies this behavior (the second commit updates the test to fail earlier but with a suggestion people can use to resolve the issue).
• It's been a long standing issue that you can't easily retain the ruby platform variant of some gems (by adding "ruby" to the list of platforms) if not all gems support it. This separates those two concepts by locking the ruby platform variant without requiring all gems to support the "ruby" platform. By updating your lockfile this way you can resolve the previous issue (and also stop having to edit the lock file by hand).

What is your fix for the problem, implemented in this PR?

Keep the ruby variants and their dependencies in the lockfile when they would be useful as a fallback regardless of whether the ruby platform is included.

Make sure the following tasks are checked

• Describe the problem / feature
• Write tests for features and bug fixes
• Write code to solve the problem
• Make sure you follow the current code style and write meaningful commit messages without tags

[rwstauner]

@hsbt What do you think about this?

[hsbt]

@rwstauner Thanks!

I'd like to confirm my understanding of what this PR intends.

First, the goal is that when a locked platform-specific variant becomes unusable, for example precompiled gems like grpc that declare an upper bound on required_ruby_version, bundler can fall back to the ruby variant because it is now locked as a fallback. Is that right? If so, this matches the direction I want Bundler to take. Users should be able to use a gem even when precompiled binaries restrict them, so I'd like to support this work.

Second, instead of adding "ruby" to PLATFORMS, which requires every gem in the bundle to support the ruby platform, the ruby variant is kept per gem for any gem that has one and whose dependencies resolve, so gems like sorbet-static no longer block this. Correct?

Third, since this changes lockfile contents on the next re-resolution and would flip-flop when older bundler versions re-lock the same file, I'm thinking this should ship in 4.1 rather than a 4.0.x patch release. Any concerns with that?

[rwstauner]

Yes, indeed, thank you. This keeps the ruby platform variant in the lockfile as a fallback without needing ruby to be in the PLATFORMS.

This makes it easier to use a different ruby version and/or platform without needing to disable frozen/deployment mode. I have a setup where I dynamically set force_ruby_platform for one gem based on an env var, so that it is false on developer machines but true in CI. This setup has proven to work just fine, so long as I hand-edit the lock file to ensure the ruby platform variant of the gem stays in the lock file. With this PR that fallback information will stay in the lock file without needing to be manually modified.

A similar situation happens for all of our gems when attempting to upgrade ruby. Some precompiled gems may support 3.4 but not 4.0. As long as the ruby variant is present this is not an issue. Even more so for testing against development ruby versions. I currently need to unset DEPLOYMENT mode for that just so that it can discard the precompiled versions and switch to all the ruby variants, but that shouldn't be necessary if the lockfile retains all that information.

I don't believe changing bundler versions would introduce any conflicts in the lockfile, it's just saving additional information.
If you use a newer version of bundler to write the lock file, doesn't bundler generally update itself to match the version recorded in the lockfile?
At that point the only reason to downgrade would be if you find an unexpected change in bundler behavior, at which point it would just remove those extra lines that you apparently didn't need before.
So if bundler was upgraded and subsequently downgraded it should only be an issue if it wants to remove lines that I used to keep in there by hand. And hand-editing the lockfile was always dangerous (or at least precarious when working with a large app or team).

All that said I don't have a strong opinion on what version of ruby this should be released with. If I can use it, even by pointing at a development version of bundler, I will.

[rwstauner]

I suppose we could also consider putting the behavior behind a config flag so that we can opt into it, and we change it to the default behavior in 4.1

[rwstauner]

Maybe I should clarify: Are we talking about ruby 4.1 or bundler 4.1? Is the intention to keep those aligned?