Repository navigation
Fall back to the built-in hash for hashable objects with unreadable slots - #675
Open
tony1233200 wants to merge 1 commit into
Open
tony1233200 wants to merge 1 commit into
tony1233200 wants to merge 1 commit into
Conversation
…lots Objects that declare __slots__ but never populate them (e.g. bson ObjectId, which declares '__id' without exposing it) produced an empty attribute dict, so every such object hashed to the same constant and ignore_order comparisons treated different values as equal. When the attribute representation is vacuous and the object is hashable, use the built-in hash instead. Fixes qlustered#560
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fall back to the built-in hash for hashable objects with unreadable slots
Fixes #560.
Problem
DeepHashcould not distinguishbson.ObjectIdvalues:On the reported version this surfaced as
Can not hash the following items: [ObjectId(...)]. On currentdevthe warning is gone but the result is worse: everyObjectIdhashes to the same constant, soignore_ordercomparisons silently treat different ids as equal. The issue's suggestion was to allow custom hash functions, or alternatively to try Python's built-inhash()before giving up — this PR implements the alternative.Root cause
DeepHash._prep_obj()turns an object into a dict via__dict__or__slots__.ObjectIdhas no__dict__and declares__slots__ = ('__id',)without populating it, so_get_slots_dict()silently yields{}and the object hashes as the constant"obj{}".Change
deepdiff/deephash.py,_prep_obj()only:__slots__(i.e. the representation is vacuous), and the object is hashable, fall back to the built-inhash()("objhash:<hash>", still passed through the normalapply_hashstep). Unhashable objects keep the previous empty representation; objects whose slot strategies raiseAttributeError(e.g. theBadtest fixture whose__getattr__raises) still end upunprocessed, exactly as before.Tests
tests/test_hash.py:test_hashable_object_with_unreadable_slots(distinct opaque objects hash differently, same object stable within a run) andtest_hashable_slots_object_hashes_by_value(equal objects share a hash, different ones do not, and set diffs distinguish them).bson.ObjectId(pymongo installed in a scratch venv; no new test dependency): before the fix both ids hashed to961aa767...; after the fix they hash differently, equal ids still share a hash, andDeepDiff({oid1}, {oid3}, ignore_order=True)correctly reportsset_item_added/set_item_removed.Bad-object and slots tests still pass (test_bad_attribute,test_slots_with_no_initialized_attributes,test_slots_deepdiff_comparison).pytest tests/— 1346 passed, 45 skipped, 0 failed.flake8on the touched files reports no new findings vs the clean tree.Known limitation (not changed by this PR)
Bare
ObjectIdvalues still compare equal under default settings in plainDeepDiff, because the only readable attribute is the dunder slot__id, which the defaultignore_private_variables=Truefilters out. Withignore_private_variables=Falsethe difference is reported (root.__idvalues_changed), and listignore_orderdiffs then report it too. That filtering is intended default behaviour for private attributes, so I left it alone; the hash path fixed here is what #560 asked about.Notes