Repository navigation
Avoid parsing sys.version when detecting the Python implementation - #624
Open
LorenzoMartini wants to merge 1 commit into
Open
LorenzoMartini wants to merge 1 commit into
LorenzoMartini wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Importing cloudpickle can raise
ValueErroron Python builds that include distributor text insys.version, such as3.13.13 | packaged by example | (...) [...]. The import-timeplatform.python_implementation()call parses that string to determine the interpreter implementation.Use
sys.implementation.nameinstead. It provides the implementation identifier directly, independently of the version string's formatting, and is available on all supported Python versions. The comparisons use its lowercase identifiers ("pypy"and"cpython"). The same change is applied to implementation checks in both test modules, and the unusedplatformimports are removed.There is a concrete example of this formatting variation in conda-forge's Python branding patch: it adds a
packaged bynote and also patches theplatformparser to accept it. PEP 421 describes the motivation for exposing implementation information directly.The regression test adds a generic
packaged bynote tosys.versionin a fresh interpreter, then imports cloudpickle and round-trips a lambda. It fails with the original implementation and passes with this change.This follows the request to contribute the fix upstream in apache/spark#60244, so Spark can receive it through a future cloudpickle release.