fix: read static discovery documents as UTF-8 - #2820
Open
MohammedAlkindi wants to merge 2 commits into
Open
MohammedAlkindi wants to merge 2 commits into
MohammedAlkindi wants to merge 2 commits into
Conversation
The packaged discovery documents are UTF-8, but get_static_doc opened
them with no encoding, so they were decoded with the host's locale
encoding instead.
On a default Windows install (cp1252) this makes
build("run", "v1beta1") raise UnicodeDecodeError, and jobs.v2.json
decodes to mojibake with no error at all.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the get_static_doc function in googleapiclient/discovery_cache/init.py to explicitly use UTF-8 encoding when opening and reading static discovery documents, ensuring consistent behavior across different host locales. Additionally, a new unit test GetStaticDocTest has been added to tests/test_discovery_cache.py to verify this behavior using a temporary directory and mock patch. There are no review comments to address.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
get_static_docopens the packaged discovery documents with no encoding, so they decode with the host locale codec instead of UTF-8.On a default Windows install (cp1252) that is not cosmetic. Of the 599 documents
get_static_doccan open, one raises and one decodes silently into the wrong characters:So
build("run", "v1beta1")fails outright, andbuild("jobs", "v2")returns corrupted descriptions with nothing to signal it. A third packaged file,index.json, also fails to decode, but it is a directory index thatget_static_docnever opens.The fix passes
encoding="utf-8". The added test writes a document containing U+201C and U+201D, pointsDISCOVERY_DOC_DIRat it, and asserts the round trip.Verification. Windows, Python 3.13.13, locale cp1252.
noxis not installed here, so I ran pytest directly.tests/test_discovery_cache.py: 1 passed, 1 skipped (FileCacheTest, oauth2client absent), and the new test fails on the pre-fix source with the charmap error above.tests/test_discovery.py: 168 passed, 2 skipped, 3 failed (test_tests_should_be_run_with_strict_positional_enforcementand twoUniversemTLS tests), identical with this change reverted.