Do not pick up before the trigger condition fires.
Trigger: #1480 reports. That spike classifies every core-tier section as
invariant policy or process layer, and the section this issue adds to is
classified as moving. Writing the rule first means writing it into a file whose
home is undecided, and the split would then have to carry it twice.
If #1480 is rejected and the core tier stays as it is, this becomes ready
immediately.
testing-tier-by-directory in templates/base/core/testing.md tells a project
to derive the test tier from the directory:
Derive the tier from the directory with a single collection hook
(pytest pytest_collection_modifyitems, or the runner's equivalent):
tests/* -> unit, tests/integration/ -> integration, tests/e2e/ -> e2e.
The rule is sound. What it does not say is that moving files into those
directories is constrained by how the runner imports them, and that the two
plausible import modes forbid opposite things. A project meets both constraints
at the moment it moves the files, which is the worst moment to discover them.
The two constraints
Measured against pytest 9.1.1 on a project adopting this rule, with a
throwaway fixture rather than by reading documentation:
A helper is invisible one level down. Under --import-mode=prepend, only
a module's own directory goes on the search path. A helper at tests/helper.py
imported as from helper import X resolves from tests/test_a.py and raises
ModuleNotFoundError from tests/unit/test_a.py. So a suite that shares
helpers by bare import cannot be subdivided at all without first restructuring
how helpers are reached.
Two modules may not share a basename. The same mode refuses them, and the
run stops at collection:
import file mismatch:
imported module 'test_stream' has this __file__ attribute:
.../tests/integration/test_stream.py
which is not the same as the test file we want to collect:
.../tests/unit/test_stream.py
That matters because a natural layout puts one test module per source module
in each tier, and any source module with tests in both tiers then needs the
same basename twice.
The escape swaps one for the other. --import-mode=importlib permits the
repeated basename and breaks every bare helper import instead. They are
mutually exclusive, so a project has to choose before it moves anything.
Why this is worth stating in the rule
The failures are loud, which is the only good news. Both stop collection, so a
half-finished migration cannot report green.
But the choice is not reversible cheaply. Making the test root a package
resolves both -- helpers become importable by dotted path from any depth, and
repeated basenames become legal -- at the cost of rewriting every bare helper
import. That is a decision to take before the first file moves, and the rule
as written gives a reader no reason to think there is a decision there at all.
One project measured this on adoption: fourteen bare helper imports across five
helpers, and three source modules carrying tests in both tiers. Neither number
is visible from the rule, and both decide the layout.
Suggested addition
A short paragraph under testing-tier-by-directory stating that the tier
layout has an import prerequisite, that the two modes forbid opposite things,
and that making the test root a package satisfies both. The specific mode
names are Python's; the shape generalises to any runner that resolves test
modules by path.
The check is cheap and worth carrying with it -- create two same-named modules
in two tier directories, run the suite, and confirm the runner's behaviour
before committing to a layout.
Do not pick up before the trigger condition fires.
Trigger: #1480 reports. That spike classifies every core-tier section as
invariant policy or process layer, and the section this issue adds to is
classified as moving. Writing the rule first means writing it into a file whose
home is undecided, and the split would then have to carry it twice.
If #1480 is rejected and the core tier stays as it is, this becomes ready
immediately.
testing-tier-by-directoryintemplates/base/core/testing.mdtells a projectto derive the test tier from the directory:
The rule is sound. What it does not say is that moving files into those
directories is constrained by how the runner imports them, and that the two
plausible import modes forbid opposite things. A project meets both constraints
at the moment it moves the files, which is the worst moment to discover them.
The two constraints
Measured against pytest 9.1.1 on a project adopting this rule, with a
throwaway fixture rather than by reading documentation:
A helper is invisible one level down. Under
--import-mode=prepend, onlya module's own directory goes on the search path. A helper at
tests/helper.pyimported as
from helper import Xresolves fromtests/test_a.pyand raisesModuleNotFoundErrorfromtests/unit/test_a.py. So a suite that shareshelpers by bare import cannot be subdivided at all without first restructuring
how helpers are reached.
Two modules may not share a basename. The same mode refuses them, and the
run stops at collection:
That matters because a natural layout puts one test module per source module
in each tier, and any source module with tests in both tiers then needs the
same basename twice.
The escape swaps one for the other.
--import-mode=importlibpermits therepeated basename and breaks every bare helper import instead. They are
mutually exclusive, so a project has to choose before it moves anything.
Why this is worth stating in the rule
The failures are loud, which is the only good news. Both stop collection, so a
half-finished migration cannot report green.
But the choice is not reversible cheaply. Making the test root a package
resolves both -- helpers become importable by dotted path from any depth, and
repeated basenames become legal -- at the cost of rewriting every bare helper
import. That is a decision to take before the first file moves, and the rule
as written gives a reader no reason to think there is a decision there at all.
One project measured this on adoption: fourteen bare helper imports across five
helpers, and three source modules carrying tests in both tiers. Neither number
is visible from the rule, and both decide the layout.
Suggested addition
A short paragraph under
testing-tier-by-directorystating that the tierlayout has an import prerequisite, that the two modes forbid opposite things,
and that making the test root a package satisfies both. The specific mode
names are Python's; the shape generalises to any runner that resolves test
modules by path.
The check is cheap and worth carrying with it -- create two same-named modules
in two tier directories, run the suite, and confirm the runner's behaviour
before committing to a layout.