Skip to content

CNCF .project/ upstream spec drift blocking PR CI (feature 030 reconcile) #397

Description

@mlieberman85

What

The -m integration CI job under .github/workflows/ci.yml is currently blocked by:

```
FAILED tests/darnit/context/test_dot_project_upstream.py::TestUpstreamSpecSync::test_upstream_spec_unchanged
E Failed: Upstream .project/ spec has changed!
E
E Tracked hash: afa3d17914ab7eac38b92d9db64338c3530848411f29c7589a26ba5fb9c625d9
E Current hash: e9adbac38ab1d4d54fa2cb109a664f412653d112e70c6ec68c5793b75dc6bbd8
```

CNCF pushed a change to https://github.com/cncf/automation/tree/main/utilities/dot-project between the last main-branch CI run and now.

The test module docstring states "It does NOT block PRs - it's informational to alert maintainers when the upstream spec evolves." That intent isn't holding up in CI right now.

Why it's blocking

Two orthogonal things:

  1. Marker filter isn't working as designed. The upstream tests are @pytest.mark.upstream at the class level, and the ci.yml command is pytest tests/ -m integration. Locally, -m integration correctly deselects the upstream tests (0 selected, 5 deselected). On CI, all 5 collect and run. Root cause unclear -- may be a pytest 9.x collection edge case with class-level markers, or a workflow env difference.

  2. CNCF spec drifted. Even if the marker filter were working, a nightly reconcile would still be needed per feature 030's runbook (specs/030-dot-project-spec-sync/quickstart.md).

Suggested actions (either unblocks CI)

  • Short term: change ci.yml line 44 to `pytest tests/ -m "integration and not upstream" -v --tb=short`. Restores the intended "informational, not gating" semantics. Fixes the drift for feat(stores): pluggable per-artifact backends via typing.Protocol #396 and any future PR opened while upstream is in flux.
  • Also short term: run the feature 030 reconcile (`pytest tests/darnit/context/test_dot_project_upstream.py -v --update-hash`), review the CNCF diff, adjust the reader if a new required field landed. Landing this in a small standalone PR keeps its history separate from other work.
  • Long term: figure out why the class-level upstream mark isn't filtering under `-m integration` in the CI environment (pytest version? plugin? env var?). If it's a genuine pytest quirk, either move upstream tests out of the integration path or add a `pytestmark = pytest.mark.upstream` at module level.

Not this issue

Feature 033 (#396) touches darnit.stores and unrelated seams. It doesn't modify the .project/ reader or the upstream sync test, and the failure reproduces on any branch opened right now.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions