Skip to content

feat(python): read pytest.toml and the native [tool.pytest] table - #318

Merged
adityankale190895 merged 4 commits into
DiffCI:mainfrom
sarankumar1325:pytest-toml
Nov 18, 2026
Merged

adityankale190895 merged 4 commits into
DiffCI:mainfrom
sarankumar1325:pytest-toml

Conversation

@sarankumar1325

Copy link
Copy Markdown
Contributor

Fixes #160

Changed behavior

readPytestSettings follows pytest 9.2's CONFIG_LOADERS_BY_NAME order: pytest.toml, .pytest.toml, pytest.ini, .pytest.ini, pyproject.toml, tox.ini, setup.cfg.

  • pytest.toml / .pytest.toml: settings are read from the [pytest] table in native TOML mode. Like pytest.ini, the file wins even when it is empty or has no [pytest] table, and the defaults apply. Options written above any table make pytest exit with pytest configuration must be under a [pytest] table, so they set unreadable.
  • [tool.pytest] in pyproject.toml: it is read in native TOML mode. The header patterns are exact, so [tool.pytest] and [tool.pytest.ini_options] never match each other.
    • When both tables have entries, pytest raises Cannot use both [tool.pytest] … and [tool.pytest.ini_options], so unreadable is set.
    • When one of the two is empty, the other is used, as _config_from_tool_pytest does.
    • An empty [tool.pytest] with no ini_options doesn't match, so the search continues to tox.ini.
    • An ini_options key written inside [tool.pytest] is a form this reader doesn't parse, so it sets unreadable.
  • Native TOML mode: python_files, testpaths, pythonpath and addopts must be lists of strings, read with pyproject.toml: a comment or ] inside a pytest list drops entries #158's readTomlStringArray. Each addopts item is one argument and isn't split again, matching _getini_toml. Any other type is a TypeError in pytest, so it sets unreadable.
  • Detection and fallback: pytest.toml and .pytest.toml are added to PROJECT_FILES and to the Python entries of CONFIG_PATTERNS (/^\.?pytest\.(?:ini|toml)$/). A change to either falls back as a configuration change, the same way pytest.ini does.
  • Docs: the Python section of docs/using-diffci.md lists the new files and the new cannot read causes.

Validation

  • tests/repo/python-pytest-toml.test.ts is new. It covers:
    • each file and its [pytest] table, including -p plugins from a native addopts list,
    • precedence across all seven files,
    • an empty pytest.toml, or one without [pytest], stopping the search,
    • top-level options,
    • [tool.pytest] vs [tool.pytest.ini_options] (exact matching, both set, one empty, an ini_options key),
    • non-list native values,
    • detection, and the config-change fallback for both names.
  • Checked against a real pytest 9.2.0 with pytest --co: pytest.toml beats pyproject.toml; top-level options in .pytest.toml, both tables set, and a string python_files in [tool.pytest] all error out; an empty [tool.pytest] falls through to ini_options.
  • Mutation-checked: swapping the order of the two TOML files, or dropping any of the new unreadable cases, the empty-table checks, or the CONFIG_PATTERNS change, makes a test fail.
  • npm test: 1297 pass, 0 fail. npm run typecheck, npm run build and npm run audit:boundary pass; the new test file is in release-files.json.

pytest 9 looks for pytest.toml and .pytest.toml before pytest.ini, and
reads a native [tool.pytest] table in pyproject.toml. Both are now read in
pytest's order with native TOML types; a value pytest would reject, or a
pyproject.toml with both [tool.pytest] and [tool.pytest.ini_options], sets
unreadable. A change to either file falls back as a configuration change.

Fixes DiffCI#160

Signed-off-by: sarankumar1325 <120080287+sarankumar1325@users.noreply.github.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-18T07:08:41.939747Z 0d11ab7 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Comment thread src/repo/adapters/python.ts Fixed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0d11ab7439

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/repo/adapters/python.ts Outdated
// strings, and each `addopts` item is one argument. Any other type is an error in pytest, so it can't be read.
const readNative = (file: string, body: string) => {
for (const key of keys) {
const array = new RegExp(`^${key}\\s*=\\s*(?=\\[)`, "m").exec(body);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Match native TOML option keys using TOML syntax

For a valid TOML table with an indented key ([pytest] followed by python_files = ['check_*.py']), this regex does not match because it requires the bare key at column 0. The entry passes hasEntries but never sets unreadable, so readNative silently returns defaults and isPytestFile omits check_*.py; source changes can consequently select no tests that pytest actually collects. Parse the table with TOML or mark unrecognized entries unreadable.

Useful? React with 👍 / 👎.

Comment thread src/repo/adapters/python.ts Outdated
Comment on lines +189 to +190
const body = section(text, "pytest");
if (body !== undefined) return readNative(file, body);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Accept comments on native TOML table headers

For a valid pytest.toml or .pytest.toml header such as [pytest] # test selection, section() does not match because it requires only whitespace after ]. The top-level check then sees no options and this branch returns default settings; if the table defines custom python_files or testpaths, DiffCI can omit those tests and pass a source change without selecting them. Accept trailing TOML comments or fall back when a populated native table cannot be read.

Useful? React with 👍 / 👎.

Comment thread src/repo/adapters/python.ts Outdated
Comment on lines +188 to +190
if (file === "pytest.toml" || file === ".pytest.toml") {
const body = section(text, "pytest");
if (body !== undefined) return readNative(file, body);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Respect the pytest version when choosing loaders

If a repository runs pytest older than the 9.2 loaders implemented here, pytest ignores pytest.toml and [tool.pytest], but this adapter always gives them precedence (even an empty pytest.toml stops the search). When a legacy pytest.ini defines custom test patterns, DiffCI can instead build its graph from defaults and skip tests the runner actually collects. Gate these loaders on the repository's pytest version or fall back when it is unknown.

Useful? React with 👍 / 👎.

…eader

CodeQL flagged the moved header escape as incomplete (it escaped only . and :).
The headers are constants, but escape them fully so the helper is safe.

Signed-off-by: sarankumar1325 <120080287+sarankumar1325@users.noreply.github.com>
@adityankale190895

Copy link
Copy Markdown
Member

Thanks for the quick CodeQL fix. Codex's three P1s on 7d23b20 reproduce. The first two also affect [tool.pytest.ini_options] on main, so it's worth fixing them for every pytest table here:

  1. Indented keys. python_files = ['check_*.py'] under [pytest] is valid TOML but isn't matched, so the defaults are used. Match keys with ^[ \t]*key[ \t]*=, and set unreadable when a table has an entry DiffCI can't classify.
  2. Header comments. [pytest] # selection isn't recognised as the table. Accept [ \t]*(#.*)?$ after the ]. If a line loosely looks like a pytest header (spaces inside the brackets, quoted keys) but doesn't match exactly, set unreadable instead of continuing.
  3. pytest version. Before pytest 9, pytest.toml and [tool.pytest] are ignored. When either is chosen while a lower-precedence pytest config with options also exists, set unreadable, because the answer depends on the installed pytest. Repositories with only one config are unaffected.

Please add a test for each. Happy to merge once they fail closed.

Indented keys and a comment after a table header are valid TOML (and a
comment after an INI header is valid for pytest), so read them. A quoted
or dotted key, or a header with spaces or quotes inside the brackets,
can't be matched reliably, so mark the config unreadable. pytest before 9
ignores pytest.toml, .pytest.toml and [tool.pytest]; when one of those
wins while a file older pytest would read also exists, the selection
depends on the installed pytest, so mark it unreadable too.

Signed-off-by: sarankumar1325 <120080287+sarankumar1325@users.noreply.github.com>
@adityankale190895
adityankale190895 merged commit 78d4eb9 into DiffCI:main Nov 18, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pytest 9: read pytest.toml and the native [tool.pytest] table

3 participants