Repository navigation
fix(backup): report an unset completed_at as None - #2217
RaphaelFakhri wants to merge 4 commits into
Conversation
Weaviate serializes the completedAt of a backup or export that is still running as Go's zero time, 0001-01-01T00:00:00.000Z. list_backups() and export.get_status() returned it as datetime(1, 1, 1) instead of None.
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
|
To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge. |
|
I agree with the Weaviate Contributor License Agreement. |
dirkkul
left a comment
There was a problem hiding this comment.
could you also add a few asserts to the existing integration tests, in addition to your unit tests? two of them can see an operation while it is still running:
test_list_backupinintegration/test_backup_v4.pylists the backups right aftercreatetest_create_export_without_waitinginintegration/test_export.pypollsget_statusuntil it returnsSUCCESS
in both, completed_at should be None while the backup or export is still running. it's fine if a run only sometimes catches the operation before it finishes, so the tests need no other changes.
possible asserts
# test_list_backup, after list_backups
backup = next(b for b in backups if b.backup_id.lower() == backup_id.lower())
if backup.status in (BackupStatus.STARTED, BackupStatus.TRANSFERRING):
assert backup.completed_at is None
# test_create_export_without_waiting, in the polling loop
if status.status == ExportStatus.SUCCESS:
assert status.completed_at is not None
break
assert status.completed_at is Nonereviewed with claude code, I went through every comment below myself
| from weaviate.backup.backup_location import _BackupLocationConfig | ||
|
|
||
|
|
||
| def _zero_time_to_none(value: Optional[datetime]) -> Optional[datetime]: |
There was a problem hiding this comment.
since #2244 was merged, main has a second zero-time check, for a user's created_at. _parse_optional_timestamp in weaviate/users/base.py compares against its own _GO_ZERO_TIME. could we move your _zero_time_to_none into weaviate/util.py and call it from _parse_optional_timestamp too? then the client has one zero-time check, and export.py imports it from util instead of a private helper from backup.py.
how I checked
I merged main into this branch locally and made the change, which touches 4 files at +12/-16. import weaviate still works, since util.py imports nothing from backup or users, and pytest test/ gives 574 passed, 1 skipped. The change leaves the timezone import in users/base.py unused, so that import goes too.
def _parse_optional_timestamp(value: Optional[str]) -> Optional[datetime]:
if value is None:
return None
return _zero_time_to_none(datetime.fromisoformat(value.replace("Z", "+00:00")))There was a problem hiding this comment.
Done in d7e618d, after merging main. _zero_time_to_none is now in weaviate/util.py. backup.py, export.py and _parse_optional_timestamp in weaviate/users/base.py import it from there, and _GO_ZERO_TIME and the unused timezone import are gone. pytest test/ gives 576 passed, 1 skipped.
|
@dirkkul Added both asserts in d7e618d, as in your suggestion. I ran |
What this changes
Weaviate serializes the
completedAttimestamp of a backup or export that has not finished yet as Go's zero time,0001-01-01T00:00:00.000Z. The client parsed that value as-is, so for a running backup or export:client.backup.list_backups(...)returnedcompleted_at=datetime(1, 1, 1, tzinfo=UTC).client.export.get_status(...)returnedcompleted_at=datetime(1, 1, 1, tzinfo=UTC).Both fields are typed
Optional[datetime]with aNonedefault, so code that checkscompleted_at is Noneto detect an unfinished operation, or that computes a duration from it, gets a wrong answer.This change adds a field validator to
BackupListReturn.completed_atandExportStatusReturn.completed_atthat maps the zero time toNone. Real timestamps pass through unchanged.Fixes #2216
Reproduction
Against Weaviate 1.40.0 with
backup-filesystemandEXPORT_ENABLED=true:The raw response from
GET /v1/backups/filesystemwhile the backup runs:[{"classes":["Article"],"completedAt":"0001-01-01T00:00:00.000Z","id":"bk1","startedAt":"2026-10-21T06:49:36.565Z","status":"STARTED"}]Tests
test/test_backup.py: an unsetcompletedAtmaps toNone; a set one is kept.test/test_export.py(new): the same two cases forExportStatusReturn.pytest test/test_backup.py test/test_export.py: 2 of the new tests fail without the fix and all 7 pass with it.pytest testpasses (546 passed, 1 skipped).ruff format,ruff check, andflake8are clean on the changed files.