diff --git a/changelog/14791.improvement.rst b/changelog/14791.improvement.rst new file mode 100644 index 00000000000..f1156e32247 --- /dev/null +++ b/changelog/14791.improvement.rst @@ -0,0 +1,3 @@ +The :confval:`junit_family`, :confval:`junit_logging`, :confval:`junit_duration_report`, :confval:`console_output_style`, :confval:`empty_parameter_set_mark`, :confval:`assertion_text_diff_style` and :confval:`tmp_path_retention_policy` configuration options are now registered with the set of values they accept: an invalid value now fails with a clean usage error instead of being silently ignored or crashing later, and ``pytest --help`` lists the accepted values. + +:func:`config.getini ` now raises :class:`pytest.UsageError` instead of ``TypeError`` or ``ValueError`` when the value read from the configuration file does not match the registered type, so a bad value is always reported as a short usage error message instead of an internal error traceback. Asking for an option name that was never registered still raises ``ValueError``. diff --git a/doc/en/reference/reference.rst b/doc/en/reference/reference.rst index 4f2e0b70850..00fb0bae3ae 100644 --- a/doc/en/reference/reference.rst +++ b/doc/en/reference/reference.rst @@ -1443,7 +1443,7 @@ passed multiple times. The expected format is ``name=value``. For example:: .. versionadded:: 8.1 .. confval:: console_output_style - :type: ``str`` + :type: ``"classic" | "progress" | "count" | "times" | "progress-even-when-capture-no"`` :default: ``"progress"`` Sets the console output style while running tests: @@ -1568,7 +1568,7 @@ passed multiple times. The expected format is ``name=value``. For example:: .. confval:: empty_parameter_set_mark - :type: ``str`` + :type: ``"skip" | "xfail" | "fail_at_collect"`` :default: ``"skip"`` Allows to pick the action for empty parametersets in parameterization @@ -1733,7 +1733,7 @@ passed multiple times. The expected format is ``name=value``. For example:: .. confval:: junit_duration_report - :type: ``str`` + :type: ``"total" | "call"`` :default: ``"total"`` .. versionadded:: 4.1 @@ -1759,7 +1759,7 @@ passed multiple times. The expected format is ``name=value``. For example:: .. confval:: junit_family - :type: ``str`` + :type: ``"legacy" | "xunit1" | "xunit2"`` :default: ``"xunit2"`` .. versionadded:: 4.2 @@ -1811,7 +1811,7 @@ passed multiple times. The expected format is ``name=value``. For example:: .. confval:: junit_logging - :type: ``str`` + :type: ``"no" | "log" | "system-out" | "system-err" | "out-err" | "all"`` :default: ``"no"`` .. versionadded:: 3.5 @@ -2638,7 +2638,7 @@ passed multiple times. The expected format is ``name=value``. For example:: .. confval:: tmp_path_retention_policy - :type: ``str`` + :type: ``"all" | "failed" | "none"`` :default: ``"all"`` Controls which directories created by the `tmp_path` fixture are kept around, @@ -2768,7 +2768,7 @@ passed multiple times. The expected format is ``name=value``. For example:: .. confval:: assertion_text_diff_style - :type: ``str`` + :type: ``"ndiff" | "block"`` :default: ``"ndiff"`` Set how pytest renders diffs for string equality assertions. @@ -3654,7 +3654,7 @@ All the command-line flags can also be obtained by running ``pytest --help``:: [pytest] configuration options in the first pytest.toml|pytest.ini|tox.ini|setup.cfg|pyproject.toml file found: markers (linelist): Register new markers for test functions - empty_parameter_set_mark (string): + empty_parameter_set_mark ('skip' | 'xfail' | 'fail_at_collect'): Default marker for empty parametersets strict_config (bool): Any warnings encountered while parsing the `pytest` section of the configuration file raise errors @@ -3696,11 +3696,11 @@ All the command-line flags can also be obtained by running ``pytest --help``:: strict_parametrization_ids (bool): Emit an error if non-unique parameter set IDs are detected - console_output_style (string): + console_output_style ('classic' | 'progress' | 'count' | 'times' | 'progress-even-when-capture-no'): Console output: "classic", or with additional progress information ("progress" (percentage) | - "count" | "progress-even-when-capture-no" (forces - progress even when capture=no) + "count" | "times" | "progress-even-when-capture-no" + (forces progress even when capture=no) verbosity_test_cases (string): Specify a verbosity level for test case execution, overriding the main level. Higher levels will @@ -3713,10 +3713,9 @@ All the command-line flags can also be obtained by running ``pytest --help``:: How many sessions should we keep the `tmp_path` directories, according to `tmp_path_retention_policy`. - tmp_path_retention_policy (string): + tmp_path_retention_policy ('all' | 'failed' | 'none'): Controls which directories created by the `tmp_path` fixture are kept around, based on test outcome. - (all/failed/none) enable_assertion_pass_hook (bool): Enables the pytest_assertion_pass hook. Make sure to delete any previously generated pyc cache files. @@ -3726,25 +3725,24 @@ All the command-line flags can also be obtained by running ``pytest --help``:: truncation_limit_chars (string): Set threshold of CHARS after which truncation will take effect - assertion_text_diff_style (string): + assertion_text_diff_style ('ndiff' | 'block'): Choose how pytest renders diffs for string equality - assertions: ndiff or block + assertions verbosity_assertions (string): Specify a verbosity level for assertions, overriding the main level. Higher levels will provide more detailed explanation when an assertion fails. junit_suite_name (string): Test suite name for JUnit report - junit_logging (string): - Write captured log messages to JUnit report: one of - no|log|system-out|system-err|out-err|all + junit_logging ('no' | 'log' | 'system-out' | 'system-err' | 'out-err' | 'all'): + Write captured log messages to JUnit report junit_log_passing_tests (bool): Capture log information for passing tests to JUnit report: - junit_duration_report (string): - Duration time to report: one of total|call - junit_family (string): - Emit XML for schema: one of legacy|xunit1|xunit2 + junit_duration_report ('total' | 'call'): + Duration time to report + junit_family ('legacy' | 'xunit1' | 'xunit2'): + Emit XML for schema doctest_optionflags (args): Option flags for doctests doctest_encoding (string): diff --git a/src/_pytest/assertion/__init__.py b/src/_pytest/assertion/__init__.py index ad2454ab820..a883ede93d1 100644 --- a/src/_pytest/assertion/__init__.py +++ b/src/_pytest/assertion/__init__.py @@ -12,6 +12,7 @@ from _pytest.assertion import rewrite from _pytest.assertion import truncate from _pytest.assertion import util +from _pytest.assertion._typing import _AssertionTextDiffStyle from _pytest.assertion._typing import NO_TRUNCATION_BUDGET from _pytest.assertion._typing import TruncationBudget from _pytest.assertion.rewrite import assertstate_key @@ -61,12 +62,9 @@ def pytest_addoption(parser: Parser) -> None: ) parser.addini( "assertion_text_diff_style", - default=util.ASSERTION_TEXT_DIFF_STYLE_NDIFF, - help=( - "Choose how pytest renders diffs for string equality assertions: " - f"{util.ASSERTION_TEXT_DIFF_STYLE_NDIFF} or " - f"{util.ASSERTION_TEXT_DIFF_STYLE_BLOCK}" - ), + type=_AssertionTextDiffStyle, + default="ndiff", + help="Choose how pytest renders diffs for string equality assertions", ) Config._add_verbosity_ini( @@ -80,7 +78,8 @@ def pytest_addoption(parser: Parser) -> None: def pytest_configure(config: Config) -> None: - util.validate_assertion_text_diff_style(config) + # Eagerly validate the value; it is only read lazily when an assertion fails. + config.getini("assertion_text_diff_style") def register_assert_rewrite(*names: str) -> None: @@ -249,7 +248,7 @@ def pytest_assertrepr_compare( right=right, verbose=config.get_verbosity(Config.VERBOSITY_ASSERTIONS), highlighter=highlighter, - assertion_text_diff_style=util.get_assertion_text_diff_style(config), + assertion_text_diff_style=config.getini("assertion_text_diff_style"), truncation_budget=truncation_budget, ) return truncate.materialize_with_truncation(lines, config) or None diff --git a/src/_pytest/assertion/util.py b/src/_pytest/assertion/util.py index 3898a6f79a8..a6f124bb502 100644 --- a/src/_pytest/assertion/util.py +++ b/src/_pytest/assertion/util.py @@ -6,7 +6,6 @@ from collections.abc import Callable from collections.abc import Iterator from collections.abc import Sequence -from typing import Literal from unicodedata import normalize from _pytest import outcomes @@ -24,7 +23,6 @@ from _pytest.assertion.compare_text import _notin_text from _pytest.assertion.highlight import dummy_highlighter as dummy_highlighter from _pytest.config import Config -from _pytest.config import UsageError # The _reprcompare attribute on the util module is used by the new assertion @@ -40,32 +38,6 @@ # Config object which is assigned during pytest_runtest_protocol. _config: Config | None = None -ASSERTION_TEXT_DIFF_STYLE_INI = "assertion_text_diff_style" -ASSERTION_TEXT_DIFF_STYLE_NDIFF: Literal["ndiff"] = "ndiff" -ASSERTION_TEXT_DIFF_STYLE_BLOCK: Literal["block"] = "block" -ASSERTION_TEXT_DIFF_STYLE_CHOICES = ( - ASSERTION_TEXT_DIFF_STYLE_NDIFF, - ASSERTION_TEXT_DIFF_STYLE_BLOCK, -) - - -def get_assertion_text_diff_style(config: Config) -> _AssertionTextDiffStyle: - style = str(config.getini(ASSERTION_TEXT_DIFF_STYLE_INI)) - match style: - case "ndiff" | "block": - return style - case _: - choices = ", ".join( - repr(choice) for choice in ASSERTION_TEXT_DIFF_STYLE_CHOICES - ) - raise UsageError( - f"{ASSERTION_TEXT_DIFF_STYLE_INI} must be one of {choices}; got {style!r}" - ) - - -def validate_assertion_text_diff_style(config: Config) -> None: - get_assertion_text_diff_style(config) - def format_explanation(explanation: str) -> str: r"""Format an explanation. diff --git a/src/_pytest/config/__init__.py b/src/_pytest/config/__init__.py index 688cc8a9054..c5f75cebcb5 100644 --- a/src/_pytest/config/__init__.py +++ b/src/_pytest/config/__init__.py @@ -1739,6 +1739,9 @@ def getini(self, name: str) -> Any: If the specified name hasn't been registered through a prior :func:`parser.addini ` call (usually from a plugin), a ValueError is raised. + + If the value read from the configuration file does not match the + registered ``type``, a :class:`~pytest.UsageError` is raised. """ canonical_name = self._parser._ini_aliases.get(name, name) try: @@ -1784,21 +1787,28 @@ def _getini(self, name: str): value = selected.value mode = selected.mode - if not isinstance(type, tuple): - return self._getini_value(mode, name, canonical_name, type, value, default) - - # Union: try each member; the first one that accepts the value wins. - for member in type: - try: + # An invalid value is a user error, raised as UsageError so that it is + # reported as a short message rather than an internal error traceback. + try: + if not isinstance(type, tuple): return self._getini_value( - mode, name, canonical_name, member, value, default + mode, name, canonical_name, type, value, default ) - except (TypeError, ValueError): - pass - raise TypeError( - f"{self.inipath}: config option '{name}' expects one of " - f"{_ini_type_repr(type)}, got {builtins.type(value).__name__}: {value!r}" - ) + + # Union: try each member; the first one that accepts the value wins. + for member in type: + try: + return self._getini_value( + mode, name, canonical_name, member, value, default + ) + except (TypeError, ValueError): + pass + raise TypeError( + f"{self.inipath}: config option '{name}' expects one of " + f"{_ini_type_repr(type)}, got {builtins.type(value).__name__}: {value!r}" + ) + except (TypeError, ValueError) as e: + raise UsageError(str(e)) from e def _getini_value( self, diff --git a/src/_pytest/junitxml.py b/src/_pytest/junitxml.py index ac78f50e618..4dab8f0fb03 100644 --- a/src/_pytest/junitxml.py +++ b/src/_pytest/junitxml.py @@ -15,6 +15,7 @@ import os import platform import re +from typing import Literal import xml.etree.ElementTree as ET from _pytest import nodes @@ -33,6 +34,10 @@ xml_key = StashKey["LogXML"]() +_JunitLogging = Literal["no", "log", "system-out", "system-err", "out-err", "all"] +_JunitDurationReport = Literal["total", "call"] +_JunitFamily = Literal["legacy", "xunit1", "xunit2"] + def bin_xml_escape(arg: object) -> str: r"""Visually escape invalid XML characters. @@ -396,8 +401,8 @@ def pytest_addoption(parser: Parser) -> None: ) parser.addini( "junit_logging", - "Write captured log messages to JUnit report: " - "one of no|log|system-out|system-err|out-err|all", + "Write captured log messages to JUnit report", + type=_JunitLogging, default="no", ) parser.addini( @@ -408,12 +413,14 @@ def pytest_addoption(parser: Parser) -> None: ) parser.addini( "junit_duration_report", - "Duration time to report: one of total|call", + "Duration time to report", + type=_JunitDurationReport, default="total", - ) # choices=['total', 'call']) + ) parser.addini( "junit_family", - "Emit XML for schema: one of legacy|xunit1|xunit2", + "Emit XML for schema", + type=_JunitFamily, default="xunit2", ) @@ -422,14 +429,13 @@ def pytest_configure(config: Config) -> None: xmlpath = config.option.xmlpath # Prevent opening xmllog on worker nodes (xdist). if xmlpath and not hasattr(config, "workerinput"): - junit_family = config.getini("junit_family") config.stash[xml_key] = LogXML( xmlpath, config.option.junitprefix, config.getini("junit_suite_name"), config.getini("junit_logging"), config.getini("junit_duration_report"), - junit_family, + config.getini("junit_family"), config.getini("junit_log_passing_tests"), ) config.pluginmanager.register(config.stash[xml_key]) @@ -459,9 +465,9 @@ def __init__( logfile, prefix: str | None, suite_name: str = "pytest", - logging: str = "no", - report_duration: str = "total", - family="xunit1", + logging: _JunitLogging = "no", + report_duration: _JunitDurationReport = "total", + family: _JunitFamily = "xunit1", log_passing_tests: bool = True, ) -> None: logfile = os.path.expanduser(os.path.expandvars(logfile)) diff --git a/src/_pytest/mark/__init__.py b/src/_pytest/mark/__init__.py index 56c407ab371..5b6ae8aa83d 100644 --- a/src/_pytest/mark/__init__.py +++ b/src/_pytest/mark/__init__.py @@ -10,6 +10,7 @@ from typing import TYPE_CHECKING from .expression import Expression +from .structures import _EmptyParameterSetMark from .structures import _HiddenParam from .structures import EMPTY_PARAMETERSET_OPTION from .structures import get_empty_parameterset_mark @@ -124,7 +125,12 @@ def pytest_addoption(parser: Parser) -> None: ) parser.addini("markers", "Register new markers for test functions", "linelist") - parser.addini(EMPTY_PARAMETERSET_OPTION, "Default marker for empty parametersets") + parser.addini( + EMPTY_PARAMETERSET_OPTION, + "Default marker for empty parametersets", + type=_EmptyParameterSetMark, + default="skip", + ) @hookimpl(tryfirst=True) @@ -288,13 +294,8 @@ def pytest_configure(config: Config) -> None: config.stash[old_mark_config_key] = MARK_GEN._config MARK_GEN._config = config - empty_parameterset = config.getini(EMPTY_PARAMETERSET_OPTION) - - if empty_parameterset not in ("skip", "xfail", "fail_at_collect", None, ""): - raise UsageError( - f"{EMPTY_PARAMETERSET_OPTION!s} must be one of skip, xfail or fail_at_collect" - f" but it is {empty_parameterset!r}" - ) + # Eagerly validate the value; it is only read lazily during collection. + config.getini(EMPTY_PARAMETERSET_OPTION) def pytest_unconfigure(config: Config) -> None: diff --git a/src/_pytest/mark/structures.py b/src/_pytest/mark/structures.py index 5449b17a1c6..e14e1406336 100644 --- a/src/_pytest/mark/structures.py +++ b/src/_pytest/mark/structures.py @@ -14,6 +14,7 @@ import inspect from typing import Any from typing import final +from typing import Literal from typing import NamedTuple from typing import overload from typing import TYPE_CHECKING @@ -23,6 +24,7 @@ from .._code import getfslineno from ..compat import NOTSET from ..compat import NotSetType +from _pytest.compat import assert_never from _pytest.config import Config from _pytest.deprecated import check_ispytest from _pytest.deprecated import PARAMETRIZE_NON_COLLECTION_ITERABLE @@ -38,6 +40,7 @@ EMPTY_PARAMETERSET_OPTION = "empty_parameter_set_mark" +_EmptyParameterSetMark = Literal["skip", "xfail", "fail_at_collect"] # Singleton type for HIDDEN_PARAM, as described in: @@ -63,18 +66,18 @@ def get_empty_parameterset_mark( _fs, lineno = getfslineno(func) reason = f"got empty parameter set for ({argslisting})" - requested_mark = config.getini(EMPTY_PARAMETERSET_OPTION) - if requested_mark in ("", None, "skip"): - mark = MARK_GEN.skip(reason=reason) - elif requested_mark == "xfail": - mark = MARK_GEN.xfail(reason=reason, run=False) - elif requested_mark == "fail_at_collect": - raise Collector.CollectError( - f"Empty parameter set in '{func.__name__}' at line {lineno + 1}" - ) - else: - raise LookupError(requested_mark) - return mark + requested_mark: _EmptyParameterSetMark = config.getini(EMPTY_PARAMETERSET_OPTION) + match requested_mark: + case "skip": + return MARK_GEN.skip(reason=reason) + case "xfail": + return MARK_GEN.xfail(reason=reason, run=False) + case "fail_at_collect": + raise Collector.CollectError( + f"Empty parameter set in '{func.__name__}' at line {lineno + 1}" + ) + case unreachable: + assert_never(unreachable) class ParameterSet(NamedTuple): diff --git a/src/_pytest/terminal.py b/src/_pytest/terminal.py index e46f8fb4c20..e3de07a7c9b 100644 --- a/src/_pytest/terminal.py +++ b/src/_pytest/terminal.py @@ -76,6 +76,10 @@ _REPORTCHARS_DEFAULT = "fE" +_ConsoleOutputStyle = Literal[ + "classic", "progress", "count", "times", "progress-even-when-capture-no" +] + class MoreQuietAction(argparse.Action): """A modified copy of the argparse count action which counts down and updates @@ -274,8 +278,9 @@ def pytest_addoption(parser: Parser) -> None: parser.addini( "console_output_style", help='Console output: "classic", or with additional progress information ' - '("progress" (percentage) | "count" | "progress-even-when-capture-no" (forces ' - "progress even when capture=no)", + '("progress" (percentage) | "count" | "times" | "progress-even-when-capture-no" ' + "(forces progress even when capture=no)", + type=_ConsoleOutputStyle, default="progress", ) Config._add_verbosity_ini( @@ -289,6 +294,8 @@ def pytest_addoption(parser: Parser) -> None: def pytest_configure(config: Config) -> None: + # Eagerly validate the value; it is only read lazily during reporting. + config.getini("console_output_style") reporter = TerminalReporter(config, sys.stdout) config.pluginmanager.register(reporter, "terminalreporter") if config.option.debug or config.option.traceconfig: @@ -422,15 +429,18 @@ def _determine_show_progress_info( # do not show progress if we are showing fixture setup/teardown if self.config.getoption("setupshow", False): return False - cfg: str = self.config.getini("console_output_style") - if cfg in {"progress", "progress-even-when-capture-no"}: - return "progress" - elif cfg == "count": - return "count" - elif cfg == "times": - return "times" - else: - return False + cfg: _ConsoleOutputStyle = self.config.getini("console_output_style") + match cfg: + case "progress" | "progress-even-when-capture-no": + return "progress" + case "count": + return "count" + case "times": + return "times" + case "classic": + return False + case unreachable: + compat.assert_never(unreachable) @property def verbosity(self) -> int: diff --git a/src/_pytest/tmpdir.py b/src/_pytest/tmpdir.py index 9196006e500..745a3c95670 100644 --- a/src/_pytest/tmpdir.py +++ b/src/_pytest/tmpdir.py @@ -98,11 +98,7 @@ def from_config( f"tmp_path_retention_count must be >= 0. Current input: {count}." ) - policy = config.getini("tmp_path_retention_policy") - if policy not in ("all", "failed", "none"): - raise ValueError( - f"tmp_path_retention_policy must be either all, failed, none. Current input: {policy}." - ) + policy: RetentionType = config.getini("tmp_path_retention_policy") return cls( given_basetemp=config.option.basetemp, @@ -265,9 +261,8 @@ def pytest_addoption(parser: Parser) -> None: parser.addini( "tmp_path_retention_policy", - help="Controls which directories created by the `tmp_path` fixture are kept around, based on test outcome. " - "(all/failed/none)", - type="string", + help="Controls which directories created by the `tmp_path` fixture are kept around, based on test outcome.", + type=RetentionType, default="all", ) diff --git a/testing/python/metafunc.py b/testing/python/metafunc.py index 3dd07a0ffc8..654c99271c5 100644 --- a/testing/python/metafunc.py +++ b/testing/python/metafunc.py @@ -338,7 +338,7 @@ def func(y): class MockConfig: def getini(self, name): - return "" + return "skip" @property def hook(self): diff --git a/testing/test_assertion.py b/testing/test_assertion.py index 609c4b1a62f..bc5126d5a4b 100644 --- a/testing/test_assertion.py +++ b/testing/test_assertion.py @@ -8,6 +8,7 @@ import sys import textwrap from typing import Any +from typing import Literal from typing import NamedTuple import attr @@ -18,6 +19,7 @@ from _pytest.assertion import util from _pytest.assertion._compare_any import _compare_eq_cls from _pytest.assertion._compare_mapping import _compare_eq_mapping +from _pytest.assertion._typing import _AssertionTextDiffStyle from _pytest.assertion._typing import NO_TRUNCATION_BUDGET from _pytest.assertion._typing import TruncationBudget from _pytest.assertion.compare_text import _compare_eq_text @@ -31,7 +33,7 @@ def mock_config( verbose: int = 0, assertion_override: int | None = None, - assertion_text_diff_style: str = util.ASSERTION_TEXT_DIFF_STYLE_NDIFF, + assertion_text_diff_style: _AssertionTextDiffStyle = "ndiff", truncation_limit_lines: str = "0", truncation_limit_chars: str = "0", has_terminalreporter: bool = True, @@ -67,7 +69,7 @@ def get_verbosity(self, verbosity_type: str | None = None) -> int: raise KeyError(f"Not mocked out: {verbosity_type}") def getini(self, name: str) -> str: - if name == util.ASSERTION_TEXT_DIFF_STYLE_INI: + if name == "assertion_text_diff_style": return assertion_text_diff_style # Truncation defaults to disabled (``"0"``) so ``callop``-style # tests can compare against the full explanation; the dispatcher @@ -455,7 +457,7 @@ def callop( left: Any, right: Any, verbose: int = 0, - assertion_text_diff_style: str = util.ASSERTION_TEXT_DIFF_STYLE_NDIFF, + assertion_text_diff_style: _AssertionTextDiffStyle = "ndiff", ) -> list[str] | None: config = mock_config( verbose=verbose, @@ -468,7 +470,7 @@ def callequal( left: Any, right: Any, verbose: int = 0, - assertion_text_diff_style: str = util.ASSERTION_TEXT_DIFF_STYLE_NDIFF, + assertion_text_diff_style: _AssertionTextDiffStyle = "ndiff", ) -> list[str] | None: return callop( "==", @@ -504,7 +506,7 @@ def test_text_diff_ndiff_style(self) -> None: "eggs", util.dummy_highlighter, 0, - util.ASSERTION_TEXT_DIFF_STYLE_NDIFF, + "ndiff", ) ) == [ "- eggs", @@ -516,7 +518,7 @@ def test_text_diff_budget_caps_ndiff_input(self) -> None: bounded instead of growing with the input.""" left = "\n".join(f"left {i}" for i in range(1000)) right = "\n".join(f"right {i}" for i in range(1000)) - ndiff_style = util.ASSERTION_TEXT_DIFF_STYLE_NDIFF + ndiff_style: Literal["ndiff"] = "ndiff" capped = list( _compare_eq_text( left, @@ -609,7 +611,7 @@ def test_multiline_text_diff_block(self) -> None: assert callequal( "foo\nspam\nbar", "foo\neggs\nbar", - assertion_text_diff_style=util.ASSERTION_TEXT_DIFF_STYLE_BLOCK, + assertion_text_diff_style="block", ) == [ r"'foo\nspam\nbar' == 'foo\neggs\nbar'", "", @@ -628,7 +630,7 @@ def test_multiline_text_diff_block_preserves_blank_lines(self) -> None: assert callequal( "\nfoo\n", "\nbar", - assertion_text_diff_style=util.ASSERTION_TEXT_DIFF_STYLE_BLOCK, + assertion_text_diff_style="block", ) == [ r"'\nfoo\n' == '\nbar'", "", @@ -646,7 +648,7 @@ def test_single_line_text_diff_block(self) -> None: assert callequal( "spam", "eggs", - assertion_text_diff_style=util.ASSERTION_TEXT_DIFF_STYLE_BLOCK, + assertion_text_diff_style="block", ) == [ "'spam' == 'eggs'", "", @@ -1946,7 +1948,7 @@ def _explain_capped(self, left: object, right: object) -> list[str]: right=right, verbose=1, highlighter=util.dummy_highlighter, - assertion_text_diff_style=util.ASSERTION_TEXT_DIFF_STYLE_NDIFF, + assertion_text_diff_style="ndiff", truncation_budget=cap, ) return truncate.materialize_with_truncation(src, config) @@ -2159,7 +2161,7 @@ class NoCompare: NoCompare(2), util.dummy_highlighter, 0, - util.ASSERTION_TEXT_DIFF_STYLE_NDIFF, + "ndiff", ) ) == [] @@ -2888,9 +2890,9 @@ def test_text_diff(): """ ) pytester.makeini( - f""" + """ [pytest] - assertion_text_diff_style = {util.ASSERTION_TEXT_DIFF_STYLE_BLOCK} + assertion_text_diff_style = block """ ) @@ -2920,9 +2922,9 @@ def test_text_diff(): """ ) pytester.makeini( - f""" + """ [pytest] - assertion_text_diff_style = {util.ASSERTION_TEXT_DIFF_STYLE_BLOCK} + assertion_text_diff_style = block """ ) @@ -2958,7 +2960,8 @@ def test_ok(): assert result.ret == pytest.ExitCode.USAGE_ERROR result.stderr.fnmatch_lines( [ - "*ERROR: assertion_text_diff_style must be one of 'ndiff', 'block'; got 'side-by-side'" + "*ERROR: *: config option 'assertion_text_diff_style' expects one of " + "'ndiff' | 'block', got 'side-by-side'" ] ) diff --git a/testing/test_config.py b/testing/test_config.py index 5d19627bca7..4c4419500a5 100644 --- a/testing/test_config.py +++ b/testing/test_config.py @@ -1046,7 +1046,7 @@ def pytest_addoption(parser): ) config = pytester.parseconfig() with pytest.raises( - TypeError, match="Expected an int string for option ini_param" + UsageError, match="Expected an int string for option ini_param" ): _ = config.getini("ini_param") @@ -1085,7 +1085,7 @@ def pytest_addoption(parser): ) config = pytester.parseconfig() with pytest.raises( - TypeError, match="Expected a float string for option ini_param" + UsageError, match="Expected a float string for option ini_param" ): _ = config.getini("ini_param") @@ -1141,7 +1141,7 @@ def test_addini_union_type_invalid_value(self, pytester: Pytester) -> None: ) config = pytester.parseconfig() with pytest.raises( - TypeError, match=r"config option 'ini_param' expects one of int \| string" + UsageError, match=r"config option 'ini_param' expects one of int \| string" ): _ = config.getini("ini_param") @@ -1204,20 +1204,20 @@ def test_addini_literal_type_ini_and_override(self, pytester: Pytester) -> None: ) @pytest.mark.parametrize( - "value, exc, match", + "value, match", [ - ('"short"', ValueError, r"expects one of 'auto' \| 'long', got 'short'"), - ("5", TypeError, r"expects a string, got int: 5"), + ('"short"', r"expects one of 'auto' \| 'long', got 'short'"), + ("5", r"expects a string, got int: 5"), ], ids=["bad-choice", "bad-type"], ) def test_addini_literal_type_invalid_value( - self, pytester: Pytester, value: str, exc: type[Exception], match: str + self, pytester: Pytester, value: str, match: str ) -> None: pytester.makeconftest(self.LITERAL_CONFTEST) pytester.makepyprojecttoml(f"[tool.pytest]\nini_param = {value}") config = pytester.parseconfig() - with pytest.raises(exc, match=f"config option 'ini_param' {match}"): + with pytest.raises(UsageError, match=f"config option 'ini_param' {match}"): _ = config.getini("ini_param") UNION_LITERAL_CONFTEST = """ @@ -1263,7 +1263,7 @@ def test_addini_union_with_literal_invalid_value(self, pytester: Pytester) -> No pytester.makepyprojecttoml('[tool.pytest]\nini_param = "3"') config = pytester.parseconfig() with pytest.raises( - TypeError, + UsageError, match=r"config option 'ini_param' expects one of int \| 'auto', " r"got str: '3'", ): @@ -3164,7 +3164,7 @@ def pytest_addoption(parser): pytester.parseconfig() def test_type_errors(self, pytester: Pytester) -> None: - """Test all possible TypeError cases in getini.""" + """Test all invalid-type cases in getini, reported as UsageError.""" pytester.maketoml( """ [pytest] @@ -3208,49 +3208,51 @@ def pytest_addoption(parser): config = pytester.parseconfig() with pytest.raises( - TypeError, match=r"expects a list for type 'paths'.*got str" + UsageError, match=r"expects a list for type 'paths'.*got str" ): config.getini("paths_not_list") with pytest.raises( - TypeError, match=r"expects a list of strings.*item at index 0 is int" + UsageError, match=r"expects a list of strings.*item at index 0 is int" ): config.getini("paths_list_with_int") - with pytest.raises(TypeError, match=r"expects a list for type 'args'.*got int"): + with pytest.raises( + UsageError, match=r"expects a list for type 'args'.*got int" + ): config.getini("args_not_list") with pytest.raises( - TypeError, match=r"expects a list of strings.*item at index 1 is int" + UsageError, match=r"expects a list of strings.*item at index 1 is int" ): config.getini("args_list_with_int") with pytest.raises( - TypeError, match=r"expects a list for type 'linelist'.*got bool" + UsageError, match=r"expects a list for type 'linelist'.*got bool" ): config.getini("linelist_not_list") with pytest.raises( - TypeError, match=r"expects a list of strings.*item at index 1 is bool" + UsageError, match=r"expects a list of strings.*item at index 1 is bool" ): config.getini("linelist_list_with_bool") - with pytest.raises(TypeError, match=r"expects a bool.*got str"): + with pytest.raises(UsageError, match=r"expects a bool.*got str"): config.getini("bool_not_bool") - with pytest.raises(TypeError, match=r"expects an int.*got str"): + with pytest.raises(UsageError, match=r"expects an int.*got str"): config.getini("int_not_int") - with pytest.raises(TypeError, match=r"expects an int.*got bool"): + with pytest.raises(UsageError, match=r"expects an int.*got bool"): config.getini("int_is_bool") - with pytest.raises(TypeError, match=r"expects a float.*got str"): + with pytest.raises(UsageError, match=r"expects a float.*got str"): config.getini("float_not_float") - with pytest.raises(TypeError, match=r"expects a float.*got bool"): + with pytest.raises(UsageError, match=r"expects a float.*got bool"): config.getini("float_is_bool") - with pytest.raises(TypeError, match=r"expects a string.*got int"): + with pytest.raises(UsageError, match=r"expects a string.*got int"): config.getini("string_not_string") diff --git a/testing/test_junitxml.py b/testing/test_junitxml.py index 1018b858413..bee1cf2fc75 100644 --- a/testing/test_junitxml.py +++ b/testing/test_junitxml.py @@ -13,6 +13,9 @@ import xmlschema from _pytest.config import Config +from _pytest.junitxml import _JunitDurationReport +from _pytest.junitxml import _JunitFamily +from _pytest.junitxml import _JunitLogging from _pytest.junitxml import bin_xml_escape from _pytest.junitxml import LogXML from _pytest.monkeypatch import MonkeyPatch @@ -41,7 +44,7 @@ def __init__(self, pytester: Pytester, schema: xmlschema.XMLSchema) -> None: def __call__( self, *args: str | os.PathLike[str], - family: str | None = "xunit1", + family: _JunitFamily | None = "xunit1", suite_name: str = "pytest", ) -> tuple[RunResult, DomDocument]: if family: @@ -207,7 +210,7 @@ def test_node_repr(self, document: DomDocument) -> None: class TestPython: @parametrize_families def test_summing_simple( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -233,7 +236,7 @@ def test_xpass(): @parametrize_families def test_summing_simple_with_errors( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -262,7 +265,7 @@ def test_xpass(): @parametrize_families def test_hostname_in_xml( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -276,7 +279,7 @@ def test_pass(): @parametrize_families def test_timestamp_in_xml( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -319,7 +322,7 @@ def test_junit_duration_report( self, pytester: Pytester, monkeypatch: MonkeyPatch, - duration_report: str, + duration_report: _JunitDurationReport, run_and_parse: RunAndParse, ) -> None: # mock LogXML.node_reporter so it always sets a known duration to each test report object @@ -350,7 +353,7 @@ def test_foo(): @parametrize_families def test_setup_error( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -375,7 +378,7 @@ def test_function(arg): @parametrize_families def test_teardown_error( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -401,7 +404,7 @@ def test_function(arg): @parametrize_families def test_call_failure_teardown_error( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -432,7 +435,7 @@ def test_function(arg): @parametrize_families def test_skip_contains_name_reason( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -452,7 +455,7 @@ def test_skip(): @parametrize_families def test_mark_skip_contains_name_reason( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -475,7 +478,7 @@ def test_skip(): @parametrize_families def test_mark_skipif_contains_name_reason( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -499,7 +502,7 @@ def test_skip(): @parametrize_families def test_mark_skip_doesnt_capture_output( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -516,7 +519,7 @@ def test_skip(): @parametrize_families def test_classname_instance( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -536,7 +539,7 @@ def test_method(self): @parametrize_families def test_classname_nested_dir( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: p = pytester.mkdir("sub").joinpath("test_hello.py") p.write_text("def test_func(): 0/0", encoding="utf-8") @@ -549,7 +552,7 @@ def test_classname_nested_dir( @parametrize_families def test_internal_error( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makeconftest("def pytest_runtest_protocol(): 0 / 0") pytester.makepyfile("def test_function(): pass") @@ -570,9 +573,9 @@ def test_internal_error( def test_failure_function( self, pytester: Pytester, - junit_logging: str, + junit_logging: _JunitLogging, run_and_parse: RunAndParse, - xunit_family: str, + xunit_family: _JunitFamily, ) -> None: pytester.makepyfile( """ @@ -636,7 +639,7 @@ def test_fail(): @parametrize_families def test_failure_verbose_message( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -653,7 +656,7 @@ def test_fail(): @parametrize_families def test_failure_escape( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -681,7 +684,7 @@ def test_func(arg1): @parametrize_families def test_junit_prefixing( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -705,7 +708,7 @@ def test_hello(self): @parametrize_families def test_xfailure_function( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -725,7 +728,7 @@ def test_xfail(): @parametrize_families def test_xfailure_marker( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -748,7 +751,10 @@ def test_xfail(): "junit_logging", ["no", "log", "system-out", "system-err", "out-err", "all"] ) def test_xfail_captures_output_once( - self, pytester: Pytester, junit_logging: str, run_and_parse: RunAndParse + self, + pytester: Pytester, + junit_logging: _JunitLogging, + run_and_parse: RunAndParse, ) -> None: pytester.makepyfile( """ @@ -777,7 +783,7 @@ def test_fail(): @parametrize_families def test_xfailure_xpass( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -796,7 +802,7 @@ def test_xpass(): @parametrize_families def test_xfailure_xpass_strict( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -817,7 +823,7 @@ def test_xpass(): @parametrize_families def test_collect_error( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile("syntax error") result, dom = run_and_parse(family=xunit_family) @@ -864,7 +870,10 @@ def test_str_compare(): @pytest.mark.parametrize("junit_logging", ["no", "system-out"]) def test_pass_captures_stdout( - self, pytester: Pytester, run_and_parse: RunAndParse, junit_logging: str + self, + pytester: Pytester, + run_and_parse: RunAndParse, + junit_logging: _JunitLogging, ) -> None: pytester.makepyfile( """ @@ -887,7 +896,10 @@ def test_pass(): @pytest.mark.parametrize("junit_logging", ["no", "system-err"]) def test_pass_captures_stderr( - self, pytester: Pytester, run_and_parse: RunAndParse, junit_logging: str + self, + pytester: Pytester, + run_and_parse: RunAndParse, + junit_logging: _JunitLogging, ) -> None: pytester.makepyfile( """ @@ -911,7 +923,10 @@ def test_pass(): @pytest.mark.parametrize("junit_logging", ["no", "system-out"]) def test_setup_error_captures_stdout( - self, pytester: Pytester, run_and_parse: RunAndParse, junit_logging: str + self, + pytester: Pytester, + run_and_parse: RunAndParse, + junit_logging: _JunitLogging, ) -> None: pytester.makepyfile( """ @@ -940,7 +955,10 @@ def test_function(arg): @pytest.mark.parametrize("junit_logging", ["no", "system-err"]) def test_setup_error_captures_stderr( - self, pytester: Pytester, run_and_parse: RunAndParse, junit_logging: str + self, + pytester: Pytester, + run_and_parse: RunAndParse, + junit_logging: _JunitLogging, ) -> None: pytester.makepyfile( """ @@ -970,7 +988,10 @@ def test_function(arg): @pytest.mark.parametrize("junit_logging", ["no", "system-out"]) def test_avoid_double_stdout( - self, pytester: Pytester, run_and_parse: RunAndParse, junit_logging: str + self, + pytester: Pytester, + run_and_parse: RunAndParse, + junit_logging: _JunitLogging, ) -> None: pytester.makepyfile( """ @@ -1040,7 +1061,7 @@ def getini(self, name: str) -> str: class TestNonPython: @parametrize_families def test_summing_simple( - self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + self, pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makeconftest( """ @@ -1068,7 +1089,7 @@ def repr_failure(self, excinfo): @pytest.mark.parametrize("junit_logging", ["no", "system-out"]) -def test_nullbyte(pytester: Pytester, junit_logging: str) -> None: +def test_nullbyte(pytester: Pytester, junit_logging: _JunitLogging) -> None: # A null byte cannot occur in XML (see section 2.2 of the spec) pytester.makepyfile( """ @@ -1090,7 +1111,7 @@ def test_print_nullbyte(): @pytest.mark.parametrize("junit_logging", ["no", "system-out"]) -def test_nullbyte_replace(pytester: Pytester, junit_logging: str) -> None: +def test_nullbyte_replace(pytester: Pytester, junit_logging: _JunitLogging) -> None: # Check if the null byte gets replaced pytester.makepyfile( """ @@ -1449,7 +1470,7 @@ def test_x(i): @parametrize_families def test_root_testsuites_tag( - pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -1555,7 +1576,7 @@ def test_pass(): @parametrize_families -def test_global_properties(pytester: Pytester, xunit_family: str) -> None: +def test_global_properties(pytester: Pytester, xunit_family: _JunitFamily) -> None: path = pytester.path.joinpath("test_global_properties.xml") log = LogXML(str(path), None, family=xunit_family) @@ -1617,7 +1638,7 @@ class Report(BaseReport): @parametrize_families def test_record_testsuite_property( - pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makepyfile( """ @@ -1671,7 +1692,10 @@ def test_func1(record_testsuite_property): @pytest.mark.parametrize("suite_name", ["my_suite", ""]) @parametrize_families def test_set_suite_name( - pytester: Pytester, suite_name: str, run_and_parse: RunAndParse, xunit_family: str + pytester: Pytester, + suite_name: str, + run_and_parse: RunAndParse, + xunit_family: _JunitFamily, ) -> None: if suite_name: pytester.makeini( @@ -1757,7 +1781,7 @@ def test_esc(my_setup): @parametrize_families def test_logging_passing_tests_disabled_does_not_log_test_output( - pytester: Pytester, run_and_parse: RunAndParse, xunit_family: str + pytester: Pytester, run_and_parse: RunAndParse, xunit_family: _JunitFamily ) -> None: pytester.makeini( f""" @@ -1790,9 +1814,9 @@ def test_func(): @pytest.mark.parametrize("junit_logging", ["no", "system-out", "system-err"]) def test_logging_passing_tests_disabled_logs_output_for_failing_test_issue5430( pytester: Pytester, - junit_logging: str, + junit_logging: _JunitLogging, run_and_parse: RunAndParse, - xunit_family: str, + xunit_family: _JunitFamily, ) -> None: pytester.makeini( f""" @@ -1837,3 +1861,26 @@ def test_no_message_quiet(pytester: Pytester) -> None: result = pytester.runpytest("--junitxml=pytest.xml", "--quiet") result.stdout.no_fnmatch_line("* generated xml file: *") + + +@pytest.mark.parametrize( + ("name", "value"), + [ + ("junit_logging", "stdout"), + ("junit_duration_report", "setup"), + ("junit_family", "xunit3"), + ], +) +def test_invalid_junit_option_value(pytester: Pytester, name: str, value: str) -> None: + """Invalid junit option values fail with a clean usage error.""" + pytester.makeini( + f""" + [pytest] + {name} = {value} + """ + ) + result = pytester.runpytest("--junitxml=junit.xml") + assert result.ret == pytest.ExitCode.USAGE_ERROR + result.stderr.fnmatch_lines( + [f"*ERROR: *config option '{name}' expects one of *, got '{value}'"] + ) diff --git a/testing/test_mark.py b/testing/test_mark.py index 253cda94503..c70376e7015 100644 --- a/testing/test_mark.py +++ b/testing/test_mark.py @@ -7,6 +7,7 @@ from _pytest.config import ExitCode from _pytest.mark import MarkGenerator +from _pytest.mark.structures import _EmptyParameterSetMark from _pytest.mark.structures import EMPTY_PARAMETERSET_OPTION from _pytest.nodes import Collector from _pytest.nodes import Node @@ -996,9 +997,9 @@ def test_aliases(self) -> None: assert md.kwargs == {"three": 3} -@pytest.mark.parametrize("mark", [None, "", "skip", "xfail"]) +@pytest.mark.parametrize("mark", [None, "skip", "xfail"]) def test_parameterset_for_parametrize_marks( - pytester: Pytester, mark: str | None + pytester: Pytester, mark: _EmptyParameterSetMark | None ) -> None: if mark is not None: pytester.makeini( @@ -1014,8 +1015,8 @@ def test_parameterset_for_parametrize_marks( pytest_configure(config) result_mark = get_empty_parameterset_mark(config, ["a"], all) - if mark in (None, ""): - # normalize to the requested name + if mark is None: + # normalize to the default mark = "skip" assert result_mark.name == mark assert result_mark.kwargs["reason"].startswith("got empty parameter set ") @@ -1023,6 +1024,23 @@ def test_parameterset_for_parametrize_marks( assert result_mark.kwargs.get("run") is False +def test_parameterset_for_parametrize_marks_invalid(pytester: Pytester) -> None: + pytester.makeini( + f""" + [pytest] + {EMPTY_PARAMETERSET_OPTION}=dontcare + """ + ) + result = pytester.runpytest() + assert result.ret == pytest.ExitCode.USAGE_ERROR + result.stderr.fnmatch_lines( + [ + f"*ERROR: *: config option '{EMPTY_PARAMETERSET_OPTION}' expects one of " + "'skip' | 'xfail' | 'fail_at_collect', got 'dontcare'" + ] + ) + + def test_parameterset_for_fail_at_collect(pytester: Pytester) -> None: pytester.makeini( f""" @@ -1090,11 +1108,6 @@ def test(param): ) -def test_parameterset_for_parametrize_bad_markname(pytester: Pytester) -> None: - with pytest.raises(pytest.UsageError): - test_parameterset_for_parametrize_marks(pytester, "bad") - - def test_mark_expressions_no_smear(pytester: Pytester) -> None: pytester.makepyfile( """ diff --git a/testing/test_terminal.py b/testing/test_terminal.py index de1dfbbda3d..30208084ab2 100644 --- a/testing/test_terminal.py +++ b/testing/test_terminal.py @@ -2119,6 +2119,19 @@ def test_quiet(self, pytester: Pytester, test_files) -> None: result.stdout.fnmatch_lines([".F..F", "*2 failed, 3 passed in*"]) +def test_console_output_style_invalid(pytester: Pytester) -> None: + """An invalid console_output_style fails with a clean usage error.""" + result = pytester.runpytest("-o", "console_output_style=fancy") + assert result.ret == ExitCode.USAGE_ERROR + result.stderr.fnmatch_lines( + [ + "*ERROR: *config option 'console_output_style' expects one of " + "'classic' | 'progress' | 'count' | 'times' | " + "'progress-even-when-capture-no', got 'fancy'" + ] + ) + + class TestProgressOutputStyle: @pytest.fixture def many_tests_files(self, pytester: Pytester) -> None: diff --git a/testing/test_tmpdir.py b/testing/test_tmpdir.py index 90a84da5bb5..0b33a74b926 100644 --- a/testing/test_tmpdir.py +++ b/testing/test_tmpdir.py @@ -880,3 +880,22 @@ def _raise_oserror(): monkeypatch.setattr(getpass, "getuser", _raise_oserror) assert get_user() is None + + +def test_tmp_path_retention_policy_invalid(pytester: Pytester) -> None: + """An invalid tmp_path_retention_policy fails with a clean usage error.""" + pytester.makepyprojecttoml( + """ + [tool.pytest] + tmp_path_retention_policy = "compress" + """ + ) + pytester.makepyfile("def test(): pass") + result = pytester.runpytest() + assert result.ret == pytest.ExitCode.USAGE_ERROR + result.stderr.fnmatch_lines( + [ + "*ERROR: *config option 'tmp_path_retention_policy' expects one of " + "'all' | 'failed' | 'none', got 'compress'" + ] + )