test(commands): centralize all --help tests in a file to dedup code #1726

bearomorphism · 2025-12-13T03:03:04Z

Test improvements

Addressed the issues about missing skip_below_py_3_13 and missing forced consistent environment variables to reduce the chance of unnecessary file regression test failures.
Extracted duplicated logics.

codecov · 2025-12-13T03:04:55Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.84%. Comparing base (2072f8e) to head (ce12748).
⚠️ Report is 1 commits behind head on v4-11-0.

Additional details and impacted files

@@           Coverage Diff            @@
##           v4-11-0    #1726   +/-   ##
========================================
  Coverage    97.84%   97.84%           
========================================
  Files           60       60           
  Lines         2604     2604           
========================================
  Hits          2548     2548           
  Misses          56       56

Flag	Coverage Δ
unittests	`97.84% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

bearomorphism · 2025-12-13T03:34:18Z

tests/utils.py

-skip_below_py_3_13 = pytest.mark.skipif(
-    sys.version_info < (3, 13),
-    reason="The output message of argparse is different between Python 3.13 and lower than Python 3.13",
-)


The variable name itself does not reflect why we want to set this variable... this may make it hard for future contributors to understand the code correctly

bearomorphism · 2025-12-13T03:36:55Z

tests/commands/test_common_command.py

+    monkeypatch.setenv("COLUMNS", "80")
+    monkeypatch.setenv("TERM", "dumb")
+    monkeypatch.setenv("LC_ALL", "C")
+    monkeypatch.setenv("LANG", "C")
+    monkeypatch.setenv("NO_COLOR", "1")
+    monkeypatch.setenv("PAGER", "cat")


Copied from AI output. I know COLUMNS is necessary, but not sure for the rest

bearomorphism · 2025-12-14T03:27:18Z

@woile @Lee-W could you review this PR when you have a moment? This PR contains the resolution of error when someone is trying to update CLI --help description. Some other PRs updating CLI options have dependencies on this PR.

Thanks!

woile · 2025-12-14T09:01:13Z

I don't understand what this is supposed to fix

bearomorphism · 2025-12-14T11:48:05Z

I don't understand what this is supposed to fix

The behavior of --help of argparse is different in some cases between python 3.13 and below 3.13 as the message of original skip_below_py_3_13 says. Some of --help tests still uses skip_below_py_3_10 before this PR, and those tests still pass in the pipeline luckily.

When we try to add new options to the CLI, the help text changes, and usually we run pytest --regen-all to regenerate the file_regression test file, but the regenerated content may be different depends on the local environment (monkeypatch in this PR addresses this issue).

bearomorphism · 2025-12-14T12:00:38Z

The following pipeline result is a false alarm test failure caused by forgetting to skip the test below python 3.13.

https://github.com/commitizen-tools/commitizen/actions/runs/20207591917/job/58008470452?pr=1724

Test passes for python version 3.13, 3.14 but not on 3.10

This PR can fix it.

noirbizarre

Good to me, but I added some comment for possible later improvements

noirbizarre · 2025-12-27T17:18:12Z

tests/commands/test_common_command.py

+@pytest.mark.skipif(
+    sys.version_info < (3, 13),
+    reason="The output message of argparse is different between Python 3.13 and lower than Python 3.13",
+)


Can't we have different snapshots for Python <3.13 and >=3.13 ?
It seems weird to have some tests skipped for some Python version for things that are existing on those Python versions

Sounds good. I will check how to do it later and will do in another PR.

noirbizarre · 2025-12-27T17:24:07Z

tests/commands/test_common_command.py

+    # Force consistent terminal output
+    monkeypatch.setenv("COLUMNS", "80")
+    monkeypatch.setenv("TERM", "dumb")
+    monkeypatch.setenv("LC_ALL", "C")
+    monkeypatch.setenv("LANG", "C")
+    monkeypatch.setenv("NO_COLOR", "1")
+    monkeypatch.setenv("PAGER", "cat")


I think this could be a fixture like fixed_term for any test relying on output.
Basically, what it does:

COLUMNS fix the width of the term

TERM=dumb deactivate reactive output on libs supporting this (like spinners, progress bars...), better for static snapshots.

LC_ALL=C and LANG=C disable localization and utf-8 chars in most CLI tools, it ensure test snapshot doesn't fail on non-english setups

NO_COLOR is the now "standard" way of disabling ANSI color support (cf. https://no-color.org/)

PAGER=cat will avoid user defined pagers like bat to change the output.

will do in another PR

bearomorphism requested review from Lee-W, noirbizarre and woile as code owners December 13, 2025 03:03

github-actions bot added the pr-status: wait-for-review label Dec 13, 2025

bearomorphism force-pushed the test-help branch 2 times, most recently from 097ef1f to 6c3ef01 Compare December 13, 2025 03:09

This was referenced Dec 13, 2025

test: remove skip_below_py_3_10 #1725

Closed

feat(version): add MANUAL_VERSION, --next and --patch to version comm… #1724

Open

bearomorphism force-pushed the test-help branch from 6c3ef01 to ecd05a4 Compare December 13, 2025 03:15

test(commands): centralize all --help tests in a file to dedup code

ce12748

bearomorphism force-pushed the test-help branch from ecd05a4 to ce12748 Compare December 13, 2025 03:24

bearomorphism commented Dec 13, 2025

View reviewed changes

noirbizarre approved these changes Dec 27, 2025

View reviewed changes

noirbizarre mentioned this pull request Dec 28, 2025

Fix tox testing #1740

Open

1 task

bearomorphism merged commit c7b8d97 into commitizen-tools:v4-11-0 Dec 28, 2025
18 checks passed

bearomorphism deleted the test-help branch December 28, 2025 05:20

Uh oh!

test(commands): centralize all --help tests in a file to dedup code #1726

test(commands): centralize all --help tests in a file to dedup code #1726

Uh oh!

Conversation

bearomorphism commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test improvements

Uh oh!

codecov bot commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bearomorphism Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

bearomorphism Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

bearomorphism commented Dec 14, 2025

Uh oh!

woile commented Dec 14, 2025

Uh oh!

bearomorphism commented Dec 14, 2025

Uh oh!

bearomorphism commented Dec 14, 2025

Uh oh!

noirbizarre left a comment

Choose a reason for hiding this comment

Uh oh!

noirbizarre Dec 27, 2025

Choose a reason for hiding this comment

Uh oh!

bearomorphism Dec 28, 2025

Choose a reason for hiding this comment

Uh oh!

noirbizarre Dec 27, 2025

Choose a reason for hiding this comment

Uh oh!

bearomorphism Dec 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bearomorphism commented Dec 13, 2025 •

edited

Loading

codecov bot commented Dec 13, 2025 •

edited

Loading