Skip to content

fix(impact): let declared inputs select pytest tests - #320

Closed
tyza66 wants to merge 1 commit into
DiffCI:mainfrom
tyza66:fix/python-declared-inputs
Closed

tyza66 wants to merge 1 commit into
DiffCI:mainfrom
tyza66:fix/python-declared-inputs

Conversation

@tyza66

@tyza66 tyza66 commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Closes #162.

The issue's "or" branch applied: declared inputs did not work for pytest, so this PR contains the test that shows it plus the fix.

The bug

declaredTests (src/repo/impact.ts:355) builds its candidate list from knownTestPaths, which walks profile.tests — the test locations the JavaScript runners declare — and from the Go adapter's goTestPackages, which #83 added for exactly this reason. The pytest adapter reports its files on profile.pythonTests, which neither reads. So a declaration like

{ "inputs": [{ "files": ["tests/data/**"], "tests": ["tests/test_prices.py"] }] }

matched no candidate, and the change came back as Declared input tests/data/prices.csv selects no tests; full validation required.

The fix

The Python adapter's test files join the candidates under the same condition the Go ones use (the adapter is in the profile). The globs still filter, so nothing is selected that the declaration did not name.

The test

tests/repo/python-declared-inputs.test.ts is the Python counterpart of the Go testdata case in go-runtime-inputs.test.ts:

  • pyproject.toml with [tool.pytest.ini_options] testpaths = ["tests"], a tests/data/prices.csv, a tests/test_prices.py that reads it with open(...), and an unrelated tests/test_pricing.py;
  • changing tests/data/prices.csv must select tests/test_prices.py with reason DECLARED_INPUT, must not select tests/test_pricing.py, and must not fall back;
  • a second case keeps the old behaviour honest: with no declaration, the same change still falls back to the full suite.

Without the source change the first case reports 0 selected tests; with it, both pass.

docs/using-diffci.md now has the Python example beside the JavaScript ones under "Declare inputs the graph can't see". The new test file is in release-files.json, which audit:boundary enforces (379 files).

Verification

  • npm run typecheck, npm run build, npm run audit:boundary: clean.
  • npm test: 1056 pass, 4 skipped, and the same 8 Rust/Cargo tests that fail on unmodified main on this Windows machine (link.exe "missing operand after '\377\376'" while linking the generated fixture crates).
  • npx tsx --test tests/repo/python-declared-inputs.test.ts: 2 pass.

Commits are signed off (git commit -s).

Copilot AI balanced review requested due to automatic review settings November 18, 2026 08:01

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

A `diffci.json` declaration whose `tests` names a pytest file selected nothing,
so the full-suite fallback it was meant to narrow stayed. `declaredTests` builds
its candidates from `knownTestPaths`, which walks `profile.tests` - the test
locations the JavaScript runners declare - and from the Go adapter's
`goTestPackages`, which DiffCI#83 added for the same reason. The pytest adapter
reports its files on `profile.pythonTests`, which neither source reads, so a
`tests` glob like `tests/test_prices.py` matched nothing and DiffCI reported the
declaration as stale ("Declared input ... selects no tests").

The Python adapter's test files now join the candidates under the same
condition the Go ones use: the adapter is present in the profile. Nothing is
selected that the declaration did not name - the globs still filter the list.

`tests/repo/python-declared-inputs.test.ts` is the Python counterpart of the Go
testdata case in `go-runtime-inputs.test.ts`: a pytest file reads
`tests/data/prices.csv` by name, a declaration maps the file to the test, and
changing the data file must select that test with `DECLARED_INPUT` and no other
test, without falling back. A second case keeps the old behaviour honest: with
no declaration the same change still falls back to the full suite. The first
case fails without the source change (0 selected tests) and passes with it.

docs/using-diffci.md gains the Python example next to the JavaScript ones, and
the new test file is in release-files.json, which audit:boundary enforces.

Verified with npm run typecheck, npm run build and npm run audit:boundary;
`npm test` gives 1056 passes and the same 8 Rust/Cargo failures this machine
has on unmodified main (link.exe "missing operand after '\377\376'").

Signed-off-by: tyza66 <83948628+tyza66@users.noreply.github.com>
@tyza66
tyza66 force-pushed the fix/python-declared-inputs branch from afff2f9 to 3ac972a Compare November 18, 2026 08:10
@adityankale190895

Copy link
Copy Markdown
Member

Thanks @tyza66, and sorry this one overlapped. Declared inputs already select pytest tests on current main: #272 (30c9fcb) changed knownTestPaths so any graph node that looks like a test is a candidate, whatever profile.tests says.

I checked by running this PR's python-declared-inputs.test.ts against unmodified main: both cases pass without the impact.ts change. I also tried a variant with python_files = ["check_*.py"] and check_prices.py, and it passes on main too, with or without the extra pythonTests candidates. #319, which merged a few minutes ago (e41c538), adds the same test file and the same docs example and closes #162, so I'm closing this as already covered.

Your earlier fallback-naming work (#144, #155, #156) is still credited in the README and on diffci.com/contributors. If you'd like another piece, the open good first issues are a good place to look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Show that declared inputs work for pytest, and document it

3 participants