Skip to content

tier1 conftest applies integration marker to every collected test, not just tier1 #395

Description

@mlieberman85

Summary

tests/darnit/parity/tier1/conftest.py:32 implements pytest_collection_modifyitems intending to mark every parity-tier1 test as integration (per PR #370 review fix). The implementation iterates the ENTIRE items collection pytest passes in, not just items under the conftest's own scope:

def pytest_collection_modifyitems(config: pytest.Config, items: list) -> None:
    integration_mark = pytest.mark.integration
    for item in items:
        item.add_marker(integration_mark)

As a result, when pytest runs the full test tree (e.g., CI's pytest tests/ -m integration), every collected test in the workspace gets the integration marker applied by this hook -- including tests that the author never intended to mark.

Symptom

PR #384's CI failure (2026-08-23) was a real example: tests/darnit/context/test_dot_project_upstream.py::TestUpstreamSpecSync::test_upstream_spec_unchanged is a @pytest.mark.upstream-only canary (docstring: "It does NOT block PRs -- it's informational to alert maintainers when the upstream spec evolves"). CI's -m integration step should NOT have collected it. But it did, because tier1's hook globally added integration, and #384 got blocked on unrelated CNCF spec drift until #386 landed the reconcile.

The tier2 conftest at tests/darnit/parity/tier2/conftest.py:15 has the same over-broad iteration and needs the same fix.

Fix

Constrain each hook to items under its own conftest's directory:

CONFTEST_DIR = Path(__file__).parent

def pytest_collection_modifyitems(config: pytest.Config, items: list) -> None:
    integration_mark = pytest.mark.integration
    for item in items:
        try:
            item_path = Path(str(item.fspath))
        except Exception:
            continue
        if CONFTEST_DIR in item_path.parents or item_path == CONFTEST_DIR:
            item.add_marker(integration_mark)

Same shape applied to tier2's conftest.

Regression guard

Add a test that runs pytest tests/darnit/context/test_dot_project_upstream.py -m integration --collect-only and asserts zero tests are collected. That would have caught this class of leak.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions