Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/requirements/integration.in
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ PyYAML>=6.0.3
tzdata>=2026.5
opentelemetry-api>=1.45.1
opentelemetry-sdk>=1.45.1
tomli>=2.0.1; python_version<"3.11"
boto3>=1.43.109
azure-storage-blob>=12.19.0
google-api-python-client>=2.100.0
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: [ "3.10", "3.11", "3.12", "3.13", "3.14" ]
python-version: [ "3.11", "3.12", "3.13", "3.14" ]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-stable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: [ "3.10", "3.11", "3.12", "3.13", "3.14" ]
python-version: [ "3.11", "3.12", "3.13", "3.14" ]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ automation_file/
- Keep `dependencies` and `[project.optional-dependencies]` (`dev`) in sync across both TOMLs; `tests/test_dev_toml_parity.py` fails when those, the entry points, `requires-python`, `[build-system]` or `[tool.setuptools]` differ. The base `dependencies` carry no cloud SDK and no GUI toolkit: each backend's SDK, `pyarrow` and `PySide6` live in an extra (`s3`, `azure`, `gdrive`, `dropbox`, `sftp`, `smb`, `fsspec`, `onedrive`, `box`, `parquet`, `gui`; `ftp` and `webdav` are empty; `all` lists every one). Do not move one into `dependencies`, and do not import one at module level: code asks for it at the moment of use with `automation_file.core.optional.require_module(name, extra=...)`, which raises `OptionalDependencyException` naming the extra. `tests/test_optional_dependencies.py` fails when the package cannot be imported without them, when importing it loads one, or when the extras and `all` drift apart. A new optional package needs its extra in both TOMLs, its line in `all`, and its entry in `core.optional.EXTRAS`.
- **Patch releases bump themselves.** The publish workflow runs `scripts/stable_release.py bump`, which raises the patch in both `stable.toml` and `dev.toml`, then builds, uploads to PyPI and commits the bump back to `main` tagged as `vX.Y.Z`. Do not hand-bump the patch before merging to `main`. The next publish run is skipped via a commit-message guard (`chore: bump version`), so the bump itself never re-triggers publishing. The dev channel takes its number from PyPI, so never hand-bump `dev.toml` for a patch either.
- **A MINOR or MAJOR release is written in the pull request.** Set the version to `X.Y.0` in both `stable.toml` and `dev.toml` in the pull request that goes to `main`. When a file's `MAJOR.MINOR` is above the newest `vX.Y.Z` tag's, the script publishes the version as written instead of adding a patch; it stops the job when the stable version would not be above the newest tag. What counts as MINOR and MAJOR is in `docs/source/Eng/usage/api_policy.rst`.
- CI: GitHub Actions — a `lint` job on Ubuntu (Python 3.12), then `pytest` on Windows across Python 3.10 / 3.11 / 3.12 / 3.13 / 3.14. One workflow per branch: `.github/workflows/ci-dev.yml`, `.github/workflows/ci-stable.yml`.
- CI: GitHub Actions — a `lint` job on Ubuntu (Python 3.12), then `pytest` on Windows across Python 3.11 / 3.12 / 3.13 / 3.14. One workflow per branch: `.github/workflows/ci-dev.yml`, `.github/workflows/ci-stable.yml`.
- CI steps: `lint` (ruff check + ruff format --check + mypy) → `pytest` with coverage, installed with `.[all,test]` → uploads `coverage.xml` as an artifact. Two more jobs follow `lint`: `minimal` installs `.[test]` only and runs the whole suite (the tests of a missing extra skip), and `extras` installs each extra on its own. `publish-dev` needs all four. A `package` job builds the sdist and the wheel and runs `twine check`; it gates nothing. `.github/workflows/integration.yml` (§ Testing) runs next to them and gates nothing either.
- Stable publishing lives in a separate workflow (`.github/workflows/publish.yml`) that runs on push to `main`: picks the version in both TOMLs (`scripts/stable_release.py`), copies `stable.toml` to `pyproject.toml`, builds the sdist + wheel, `twine upload` via `PYPI_API_TOKEN`, then commits + tags + pushes and creates `gh release create v<version> --generate-notes`.
- Dev publishing is the `publish-dev` job at the end of `ci-dev.yml`. It runs only on a push to `dev`, after `lint` and `pytest` pass: `scripts/dev_release.py prepare` writes `pyproject.toml` from `dev.toml` with one patch above the newest `automation_file_dev` on PyPI, the job builds and runs `twine check`, and it uploads (same `PYPI_API_TOKEN`) only when the commit is still the tip of `dev` and the wheel differs from the newest published one. Nothing is committed back.
Expand Down Expand Up @@ -136,7 +136,7 @@ python -m automation_file --help

## Conventions

- Python 3.10+ — use `X | Y` union syntax, not `Union[X, Y]`.
- Python 3.11+ — use `X | Y` union syntax, not `Union[X, Y]`.
- Use `from __future__ import annotations` at the top of every module for deferred type evaluation.
- Exception hierarchy: all custom exceptions inherit from `FileAutomationException`; never `raise Exception(...)` directly.
- Logging: use `file_automation_logger` from `automation_file.logging_config`. Never `print()` for diagnostics.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -372,7 +372,7 @@ Upgrading from a release that bundled everything: install `automation_file[all]`
what you had.

Requirements:
- Python 3.10+
- Python 3.11+
- Base dependencies: `requests`, `tqdm`, `watchdog`, `cryptography`, `prometheus_client`, `defusedxml`,
`PyYAML`, `opentelemetry-api`, `opentelemetry-sdk`, `je_action_core` (the action executor
shared with APITestka, LoadDensity and MailThunder)
Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -368,7 +368,7 @@ pip install "automation_file[all,dev]" # 另含 ruff、mypy、pre-commit、pyt
从先前包含所有包的版本升级时:安装 `automation_file[all]` 即可保留原有的全部功能。

要求:
- Python 3.10+
- Python 3.11+
- 基础依赖:`requests`、`tqdm`、`watchdog`、`cryptography`、`prometheus_client`、`defusedxml`、
`PyYAML`、`opentelemetry-api`、`opentelemetry-sdk`、`je_action_core`(与 APITestka、
LoadDensity、MailThunder 共用的 action 执行器)
Expand Down
2 changes: 1 addition & 1 deletion README.zh-TW.md
Original file line number Diff line number Diff line change
Expand Up @@ -368,7 +368,7 @@ pip install "automation_file[all,dev]" # 另含 ruff、mypy、pre-commit、pyt
從先前內含所有套件的版本升級時:安裝 `automation_file[all]` 即可保留原有的全部功能。

需求:
- Python 3.10+
- Python 3.11+
- 基礎相依套件:`requests`、`tqdm`、`watchdog`、`cryptography`、`prometheus_client`、`defusedxml`、
`PyYAML`、`opentelemetry-api`、`opentelemetry-sdk`、`je_action_core`(與 APITestka、
LoadDensity、MailThunder 共用的 action 執行器)
Expand Down
2 changes: 1 addition & 1 deletion architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ storage.observe → events.storage_bridge → StorageError (only for a failing b
warns, the name stays for at least two minor releases, a major release removes it
(`docs/source/Eng/usage/api_policy.rst`; CLAUDE.md § Conventions).
- Only the three action shapes in §3. Extend through the registry, not by subclassing the executor.
Python 3.10+, `X | Y` unions, `from __future__ import annotations` (CLAUDE.md § Conventions).
Python 3.11+, `X | Y` unions, `from __future__ import annotations` (CLAUDE.md § Conventions).
- Exceptions derive from `FileAutomationException`. Log through `file_automation_logger`; no
`print()` diagnostics and no runtime `assert` (§ Conventions; § Code quality › Logging, printing, assertions).
- Every user-supplied URL goes through `validate_http_url`. Never `verify=False`. Downloads have size
Expand Down
4 changes: 2 additions & 2 deletions automation_file/app/dashboard_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
from collections import Counter
from collections.abc import Callable
from dataclasses import dataclass, field
from datetime import datetime, timezone
from datetime import UTC, datetime
from typing import Any

from automation_file.app.audit_service import AuditService
Expand Down Expand Up @@ -111,7 +111,7 @@ def brief_run(view: dict[str, Any]) -> dict[str, Any]:


def _now() -> str:
return datetime.now(timezone.utc).isoformat(timespec="seconds")
return datetime.now(UTC).isoformat(timespec="seconds")


class DashboardService:
Expand Down
8 changes: 4 additions & 4 deletions automation_file/audit/record.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
import uuid
from collections.abc import Mapping
from dataclasses import dataclass, field, fields
from datetime import datetime, timedelta, timezone
from datetime import UTC, datetime, timedelta
from typing import TYPE_CHECKING, Any

from automation_file.core.audit import AuditException
Expand All @@ -34,7 +34,7 @@
#: The ``source`` of the records made from storage operations.
STORAGE_SOURCE = "storage"

_EPOCH = datetime(1970, 1, 1, tzinfo=timezone.utc)
_EPOCH = datetime(1970, 1, 1, tzinfo=UTC)
_MICROSECOND = timedelta(microseconds=1)
_ZULU = "Z"
_UTC_OFFSET = "+00:00"
Expand All @@ -56,7 +56,7 @@ def _new_id() -> str:


def _now() -> datetime:
return datetime.now(timezone.utc)
return datetime.now(UTC)


def to_microseconds(moment: datetime) -> int:
Expand Down Expand Up @@ -95,7 +95,7 @@ def parse_time(value: object, name: str = "timestamp") -> datetime:
raise AuditException(f"{name} must be a time, got {value!r}")
if value.utcoffset() is None:
raise AuditException(f"{name} needs a time zone, got {value.isoformat()!r}")
return value.astimezone(timezone.utc)
return value.astimezone(UTC)


def _check_text(name: str, value: object, *, optional: bool) -> None:
Expand Down
4 changes: 2 additions & 2 deletions automation_file/audit/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
from abc import ABC, abstractmethod
from collections.abc import Mapping
from dataclasses import dataclass, fields
from datetime import datetime, timedelta, timezone
from datetime import UTC, datetime, timedelta
from types import TracebackType
from typing import Any, TypeVar

Expand Down Expand Up @@ -207,7 +207,7 @@ def purge_cutoff(older_than_seconds: float) -> datetime:
if not older_than_seconds > 0:
raise AuditException("older_than_seconds must be positive")
try:
return datetime.now(timezone.utc) - timedelta(seconds=older_than_seconds)
return datetime.now(UTC) - timedelta(seconds=older_than_seconds)
except (OverflowError, ValueError) as err:
raise AuditException(f"older_than_seconds is out of range: {older_than_seconds!r}") from err

Expand Down
10 changes: 2 additions & 8 deletions automation_file/core/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@

from __future__ import annotations

import sys
from collections.abc import Iterable
from pathlib import Path
from typing import Any
Expand All @@ -68,14 +67,9 @@


def _load_tomllib() -> Any:
if sys.version_info >= (3, 11):
import tomllib
import tomllib

return tomllib
# pragma: no cover - exercised only on Python 3.10 runners
import tomli # pylint: disable=import-error # declared in *.toml for Python<3.11

return tomli
return tomllib


class ConfigException(FileAutomationException):
Expand Down
2 changes: 1 addition & 1 deletion automation_file/core/manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def write_manifest(
"version": _MANIFEST_VERSION,
"algorithm": algorithm,
"root": str(root_path.resolve()),
"created_at": dt.datetime.now(dt.timezone.utc).isoformat(),
"created_at": dt.datetime.now(dt.UTC).isoformat(),
"files": files,
}
Path(manifest_path).parent.mkdir(parents=True, exist_ok=True)
Expand Down
2 changes: 1 addition & 1 deletion automation_file/core/plugins.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,5 +82,5 @@ def _iter_entry_points() -> list[EntryPoint]:
return list(entry_points(group=ENTRY_POINT_GROUP))
except TypeError:
# importlib.metadata before 3.10 used a different API; the project
# targets 3.10+, so this branch exists only as defensive padding.
# targets 3.11+, so this branch exists only as defensive padding.
return list(entry_points().get(ENTRY_POINT_GROUP, [])) # pylint: disable=no-member
4 changes: 2 additions & 2 deletions automation_file/events/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
import uuid
from collections.abc import Mapping
from dataclasses import dataclass, field
from datetime import datetime, timezone
from datetime import UTC, datetime
from enum import Enum
from typing import Any, ClassVar

Expand Down Expand Up @@ -64,7 +64,7 @@ def at_least(self, other: Severity) -> bool:


def _now() -> datetime:
return datetime.now(timezone.utc)
return datetime.now(UTC)


def _event_id() -> str:
Expand Down
4 changes: 2 additions & 2 deletions automation_file/integrity/manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@

import json
from collections.abc import Mapping
from datetime import datetime, timezone
from datetime import UTC, datetime
from typing import Any

from automation_file.exceptions import StorageURIException
Expand Down Expand Up @@ -131,7 +131,7 @@ def _from_legacy(document: Mapping[str, Any], origin: str) -> Snapshot:
root=_legacy_root(document.get("root")),
backend=LOCAL_SCHEME,
algorithm=name,
created_at=created_at or datetime.now(timezone.utc),
created_at=created_at or datetime.now(UTC),
entries=tuple(_legacy_entry(path, meta, name) for path, meta in files.items()),
)
except IntegrityException as error:
Expand Down
4 changes: 2 additions & 2 deletions automation_file/integrity/monitor.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
import threading
from collections.abc import Callable, Iterable, Mapping
from dataclasses import dataclass, fields
from datetime import datetime, timezone
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any, TypedDict

from automation_file.events.bus import EventBus, event_bus
Expand Down Expand Up @@ -551,7 +551,7 @@ def _record_failure(self, error: Exception, *, repeat: bool) -> dict[str, Any]:
signature: _Signature = (("error", message),)
announce = repeat or signature != self._announced
self._last_summary, self._last_error = summary, message
self._last_run = datetime.now(timezone.utc)
self._last_run = datetime.now(UTC)
self._announced = signature
if announce:
with correlation_scope():
Expand Down
4 changes: 2 additions & 2 deletions automation_file/integrity/remediation.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@

from collections.abc import Iterable
from dataclasses import dataclass
from datetime import datetime, timezone
from datetime import UTC, datetime

from automation_file.exceptions import FileAutomationException
from automation_file.integrity.detector import Change, ChangeKind
Expand Down Expand Up @@ -119,7 +119,7 @@ def apply(self, changes: Iterable[Change], engine: HashEngine) -> list[Remediati
"""
if not self._policy.active:
return []
stamp = datetime.now(timezone.utc).strftime(_STAMP_FORMAT)
stamp = datetime.now(UTC).strftime(_STAMP_FORMAT)
steps: list[RemediationStep] = []
for change in changes:
steps.extend(self._handle(change, stamp, engine))
Expand Down
4 changes: 2 additions & 2 deletions automation_file/integrity/report.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from __future__ import annotations

from dataclasses import dataclass, field
from datetime import datetime, timezone
from datetime import UTC, datetime
from typing import Any

from automation_file.integrity.detector import Change, ChangeKind
Expand All @@ -17,7 +17,7 @@


def _now() -> datetime:
return datetime.now(timezone.utc)
return datetime.now(UTC)


@dataclass(frozen=True)
Expand Down
8 changes: 4 additions & 4 deletions automation_file/integrity/snapshot.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
from bisect import bisect_left
from collections.abc import Iterable, Iterator, Mapping, Sequence
from dataclasses import dataclass, field, replace
from datetime import datetime, timezone
from datetime import UTC, datetime
from typing import Any

from automation_file.exceptions import FileNotExistsException, StorageURIException
Expand All @@ -29,16 +29,16 @@


def _now() -> datetime:
return datetime.now(timezone.utc)
return datetime.now(UTC)


def as_utc(moment: datetime | None) -> datetime | None:
"""Return ``moment`` as an aware UTC time; a naive one is taken to be UTC already."""
if moment is None:
return None
if moment.tzinfo is None:
return moment.replace(tzinfo=timezone.utc)
return moment.astimezone(timezone.utc)
return moment.replace(tzinfo=UTC)
return moment.astimezone(UTC)


def parse_timestamp(value: object, name: str) -> datetime | None:
Expand Down
4 changes: 2 additions & 2 deletions automation_file/pipeline/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
import threading
from collections.abc import Callable, Mapping
from dataclasses import dataclass, field
from datetime import datetime, timezone
from datetime import UTC, datetime
from enum import Enum
from typing import Any

Expand Down Expand Up @@ -75,7 +75,7 @@ class RunStatus(str, Enum):

def utc_now() -> datetime:
"""Return the current time as a timezone-aware UTC ``datetime``."""
return datetime.now(timezone.utc)
return datetime.now(UTC)


def _iso(moment: datetime | None) -> str | None:
Expand Down
2 changes: 1 addition & 1 deletion automation_file/scheduler/cron.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def resolve_timezone(name: str | None) -> dt.tzinfo | None:
)
key = name.strip()
if key.upper() == _UTC:
return dt.timezone.utc
return dt.UTC
try:
return zoneinfo.ZoneInfo(key)
except zoneinfo.ZoneInfoNotFoundError as error:
Expand Down
2 changes: 1 addition & 1 deletion automation_file/scheduler/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@


def _utc_now() -> dt.datetime:
return dt.datetime.now(dt.timezone.utc)
return dt.datetime.now(dt.UTC)


def _checked_name(name: Any) -> str:
Expand Down
4 changes: 2 additions & 2 deletions automation_file/scheduler/runs.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import threading
from collections import deque
from dataclasses import dataclass, field
from datetime import datetime, timezone
from datetime import UTC, datetime
from enum import Enum
from typing import Any

Expand Down Expand Up @@ -59,7 +59,7 @@ class TriggerKind(str, Enum):

def as_utc(moment: datetime) -> datetime:
"""Return ``moment`` as an aware UTC ``datetime``; a naive one is taken as local time."""
return moment.astimezone(timezone.utc)
return moment.astimezone(UTC)


def _iso(moment: datetime | None) -> str | None:
Expand Down
6 changes: 3 additions & 3 deletions automation_file/storage/dropbox_storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

import contextlib
from collections.abc import Callable, Hashable, Iterable, Iterator
from datetime import datetime, timezone
from datetime import UTC, datetime
from pathlib import Path
from typing import Any, BinaryIO

Expand Down Expand Up @@ -148,8 +148,8 @@ def _utc(moment: Any) -> datetime | None:
if not isinstance(moment, datetime):
return None
if moment.tzinfo is None:
return moment.replace(tzinfo=timezone.utc)
return moment.astimezone(timezone.utc)
return moment.replace(tzinfo=UTC)
return moment.astimezone(UTC)


def _file_info(path: str, metadata: Any, files: Any) -> FileInfo | None:
Expand Down
Loading
Loading