Repository navigation
Detect RFC 8785 duplicate keys that differ only in encoding - #1108
Closed
Shaisolaris wants to merge 1 commit into
Closed
Shaisolaris wants to merge 1 commit into
Shaisolaris wants to merge 1 commit into
Conversation
Member
|
Not very convinced by this. String hash keys not encoded in UTF-8 are supposed to be invalid. We should just raise. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With
rfc8785: true, a Hash whose String keys have the same text in different encodings is written with that key twice:The two keys are distinct Hash keys, but both are written as the UTF-8 string
"é", so the canonical output has a duplicate property name. RFC 8785 output is I-JSON, which does not allow duplicate names.The sort proc already converts keys to UTF-8 and checks for duplicates when an
as_jsonhook is given. It now does the same when a String or Symbol key is not ASCII or valid UTF-8, even withoutas_json, so this case raises the usual "detected duplicate key"GeneratorError. Hashes whose keys are all ASCII or UTF-8 still skip the conversion.Tests: added
test_rfc8785_duplicate_keys_in_different_encodingstotest/json/json_generator_test.rb. It fails on master and passes with the change. Locally (CRuby 3.3, C extension) the rfc8785 tests and the 11 test files undertest/jsonthat I ran pass.