Repository navigation
Set CKRecord system fields in MockCloudDatabase - #573
lukaskubanek wants to merge 5 commits into
Conversation
`CKRecord.copyWithMockSystemFields(…)` returns a copy with the given values stored in the real system fields, so they behave like server-assigned values without swizzling. Supports `modificationDate` only for now and is not yet used by `MockCloudDatabase`.
Records saved in a batch now receive an incrementing modification date, mirroring the CloudKit server. The sync engine's modification date checks therefore run in tests as they do in production, which changes an expectation in one merge conflict test.
`MockCloudDatabase` now stores the change tag in the real system field as well, while the sync machinery still reads `_recordChangeTag` for now.
`MockCloudDatabase` and `MockSyncEngine` now read `CKRecord.recordChangeTag` from the record's system field instead of the `_recordChangeTag` mock field, which is removed along with its `isTesting` persistence branches. Change tags become strings but remain digit-based for now, to be replaced later.
Matches the format CloudKit uses for `CKRecord.recordChangeTag`.
|
Hey @lukaskubanek, Stephen and I chatted about this quite a bit today and we're still trying to wrap our heads around a number of things. First, and easiest to discuss, it does not seem necessary to make any changes to Second, I still don't see why something like this: extension CKRecord {
package var _modificationDate: Date? {
get { modificationDate ?? self[#function] }
set { self[#function] = newValue }
}
}…couldn't work to simulate Third, and finally, we definitely agree that the mock database really should be simulating I understand your reasoning about 3-way merging, but consider the following. There are 3 devices (A, B, C), and A makes a change at t=1, B makes a change at t=2 and C makes a change at t=3. Further suppose that A and B sync at t=4 so that their conflicts are merged and the timestamp is moved to t=4 with your suggestion. And finally C syncs at t>4, but because its change was at t=3 its edit is discarded. This is happening even though C made its change after A and B, and the reason it is happening is because the conflict resolution is updating the timestamp to be the newest time. So it really does seem we would like that test to stay at 30 and for the |
|
Oh, and one more thing. It also seems like it would be ideal to use |
|
Hey @lukaskubanek, here is our approach to FWIW, the timestamp test failure was actually a legitimate bug in the sync engine. The place where we compared Would love to get your thoughts on this approach. |
|
@mbrandonw Thanks for digging in! Here are some initial thoughts before going to bed. Will have more to say tomorrow.
It works, but it requires all (current and future) call sites in the library to go through the underscored property. This makes the library aware of and coupled to the mock infrastructure. Another downside is that the mock value is stored as a regular record field, so it leaks into The approach presented in this PR, unlike the one from #574, doesn’t come with these downsides and is self-contained. Its cost is relying on CloudKit’s private archive keys, which seem to be pretty stable and only ever touched in tests.
When the change tag is accessed from the library, which I think is needed for properly detecting conflicts, it’s in the same situation as
Yes, this change is purely cosmetic. The idea, which I hope you share, is to bring the mock cloud database as close to the actual behavior as possible.
I agree and I’ve found even more of them and have fixes in my fork. The thing is that I wanted to first get tests ready that document the current behavior including the bugs (see also #414) and then submit the fixes. That’s why I’ve been focusing on the test infrastructure. But it’s really hard to pick the exact amount of changes to submit. I’ll reply regarding the remaining topics tomorrow. |
|
Hey @lukaskubanek, thanks for the clarifications!
I think we are willing to take this downside to not resort to swizzling or encoding/decoding tricks. Further, we could technically swizzle in just debug mode to
True, but that's easy enough to handle.
Your approach has a similar downside in that you have to make sure to use the special
Yep, but we are willing to accept that tradeoff.
Ok cool, just wanted to make sure I understand. We certainly can take that base-36 change as a self-contained PR if you have the time, or I can get to it sometime soon.
Yeah sorry it's been difficult to land things. I would be curious to see if you have one (or a few) failing tests that demonstrate some behavior we are getting wrong (ideally based on some of the PRs we will be merging soon) so that we can understand a single issue at a time. |
|
@mbrandonw OK, so my understanding is that you’re going forward with the underscore approach in #574. While not my favored approach, I’m happy to see the
Just so we’re on the same page. I moved away from swizzling (in #411) to the encoding/decoding trick in this PR. So I considered swizzling ruled out already as you’ve requested before.
Allow me one last comment on this and then I’ll sign off. The difference I see with this is that you’d be nudged to use the former when you want to set the mock values, which happens only in the
That makes more sense, yes.
You’re likely referring to the section “Impact on Tests” from #411. It’s been a while since I wrote that and I since gained a better understanding of what’s going on. So, let me give an updated stance on this matter. I agree that the described scenario is problematic and that a merge of A and B should not promote the field timestamp to t=4 and thus ignore the change from C at t=3. I think a better approach is to use the max of the per-field timestamps, i.e. t=2 from B, and leave other untouched fields alone. And that is regardless of the merge policy, be it the current last edit wins or a potential customizable one in the future. So, the flip in the test was wrong. In hindsight, I should have used
As referenced above, I have a draft of rewritten merge conflict tests ready that show all possible scenarios, document the current behavior, and uncover two more categories of bugs. I know it goes against your requirement of presenting a single issue at a time, but I feel this needs to be considered more holistically. And since these are tests, not production code, I wanted to ask if there is a remote chance of #414 being accepted in its current shape (assuming rebased onto your latest changes)? I spent quite a lot of time distilling these scenarios and I’d like to contribute them to improve the library, but I’m totally lost when it comes to the approach I should take.
I mean I’d still be willing to, but I’m not sure if it’s helpful. Since you’re already in that code it might be easier if it comes from your side. The implementation is simple enough and it can be found in this PR. Feel free to close this PR. I’d still be glad to hear your thoughts on my comments above, though. |
Yeah, sorry, I understand you moved away from swizzling. I was just re-iterating all the possible solutions we had chatted about in the past.
Point definitely taken, and we can always switch to something more robust in the future!
Yeah definitely. Before rebasing let me and Stephen chat about it first and we will get back to you.
Yep, no problem! |
I actually went ahead and rebased #414 onto
Since the base-36 format for change tags didn’t make it to the latest release, I went ahead, extracted those changes from my branch, and submitted them as #580. With that, I’m closing this PR in favor of #574. |
Supersedes #411, which populated
CKRecord.modificationDateinMockCloudDatabasevia swizzling. Motivation and test impact are documented there. This PR replaces the mechanism, as requested in review, and also includes the planned follow-up for handlingrecordChangeTag.Beyond aligning
MockCloudDatabasewith the actual server behavior, this is a prerequisite for the conflict resolution work, which will rely on these system fields to detect and handle conflicts.Approaches Tried
CKRecord: CloudKit logsBUG IN CLIENT OF CLOUDKIT: CKRecord should not be subclassed, so subclassing doesn’t seem to be supported.copy(with:)must be swizzled too so that associated objects survive). Concerns regarding static analysis were also raised in the review of the initial PR.Changes
CKRecord.copyWithMockSystemFields(recordChangeTag:modificationDate:), which creates a newCKRecordinstance with the system fields set to the provided values. It archives the receiver and unarchives it through a customNSKeyedUnarchiversubclass that substitutes the two archived keys. The values land in the real system fields, so they are returned through the regular properties, survivecopy()and persist to SQLite with no mock-specific plumbing.modificationDateandrecordChangeTaginMockCloudDatabaseon every save and when touching a parent record. Change tags are now zero-based base-36 strings, matching CloudKit’s format.MockSyncEnginewithisNewerChangeTag(than:), which compares by length before lexicographic order, since base-36 strings such as"10"and"z"don’t sort correctly on their own.recordChangeTaginMockCloudDatabaseandMockSyncEngine, and remove_recordChangeTag, itsisTestingpersistence branches and theCustomDumpfilter that hid the mock field in snapshots.Naming
I went back and forth between
settingMockSystemFields(…), which follows the Swift naming convention for non-mutating variants, andcopyWithMockSystemFields(…). I settled on the latter becauseCKRecordis an Objective-C class, the name sits naturally next tocopy(with:), and the method takes the place of explicitcopy()calls in the mock. Happy to hear your take.Impact on Tests
Mainly still the same change as in #411:
isCompleted🗓️: 30becomes60in one merge conflict test. There is much more to untangle regarding storing the in-flight record as the last-known server record on send, but I’d like to discuss that separately. The key point here is that the test is now exercising the same behavior as production.Disclaimer: I used Claude Code to help me write the code and work through the large amount of prep work I accumulated over the past months.