Skip to content

The test gate's pytest probe never imports a pytest.py from the checkout - #733

Open
zlareb1 wants to merge 3 commits into
mainfrom
fix/test-gate-isolated-python
Open

zlareb1 wants to merge 3 commits into
mainfrom
fix/test-gate-isolated-python

Conversation

@zlareb1

@zlareb1 zlareb1 commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in a release)

Changelog entry (one or two sentences a user of Loom can read without the diff; the release pull request collects these into the release notes):

The Claude Code and Codex plugins' Stop-time test gate no longer runs a pytest.py or pytest/ found at the checkout root when it looks for a Python that has pytest.

Why

When the server's code.test_gate answer names a pytest test and the checkout has no virtualenv, hooks/loom-test-gate.mjs ran python3 -c "import pytest" (then python) with the checkout as its working directory. With -c, Python puts the working directory first on sys.path, so a pytest.py or pytest/ at the checkout root was imported, and its code run, at Stop. Checked here: in a directory holding a pytest.py that writes a file, python3 -c "import pytest" creates the file.

The probe now runs python3 -c "import pytest" (then python) from the plugin's own lib/ directory instead of the checkout. With -c, Python puts the working directory on sys.path, and an empty or relative PYTHONPATH entry (PYTHONPATH=:/opt/lib, ., src) resolves against it too; from the plugin's directory none of them reaches the checkout. The probe keeps the user's site-packages and absolute PYTHONPATH entries, which -I and -E would drop, and works on every Python version, which -P (3.11+) does not. Running it outside the checkout also keeps a python3 executable in the checkout from being the one started on Windows, where the working directory is searched first.

checkoutPython moves from the hook into lib/test-gate.mjs, beside venvPython, so it can be unit-tested; it takes an optional exec for the test.

Scope and risk

Plugin hook code: lib/test-gate.mjs and hooks/loom-test-gate.mjs in the Claude Code plugin, their synced Codex copies, both manifests (1.2.22 → 1.2.23, a version no other open PR uses), and tests/test-gate.test.mjs. The virtualenv lookup is unchanged (it only checks that .venv/bin/python, venv/bin/python or $VIRTUAL_ENV's interpreter exists; it never runs it). The hook still only names the interpreter to the agent; it never runs tests itself.

Compatibility, rollout and rollback

  • Wire: none.
  • Operator steps: none.
  • Rollback: revert; the previous probe returns.

Tests

  • New in integrations/claude-code-plugin/tests/test-gate.test.mjs: a checkout with a pytest.py that writes a marker file; checkoutPython leaves no marker. The marker test also sets PYTHONPATH to :.:src with a second pytest.py under src/. A second test checks the probe's arguments and that its working directory is outside the checkout. Both clear VIRTUAL_ENV while they run, since an active virtualenv is used before the probe.
  • node --test integrations/claude-code-plugin/tests/*.test.mjs: 611 passed.
  • scripts/codegen/agent_plugin_sync.py --check: copies current. Biome format and Prettier clean. tests/unit/test_codex_plugin.py and the other plugin and hook unit tests: 240 passed.

AI assistance in the code (leave one; writing this description with AI is fine):

  • AI-generated (an agent wrote most of the diff): Claude Code found the issue while checking the plugin's data-handling text, reproduced it, and wrote the fix and tests.

  • I can explain every line of this diff and how it interacts with the rest of Loom.

Checklist

  • make format-check, make lint, make typecheck and make test-unit pass; the integration suite passes if server behaviour changed. (Ran the plugin and plugin-related unit tests above; no server change.)
  • Generated files are regenerated in this PR, not by hand: the Codex plugin copies (python3 scripts/codegen/agent_plugin_sync.py).
  • A behaviour change to any file a plugin ships (hooks/, lib/, bin/, skills/, .mcp.json, the Codex plugin-defaults.json) bumps both plugin manifests' version, and no other open PR uses that version.
  • Repository text has no dates, people, host names, internal namespaces, ticket or decision numbers; the history goes in this description (tests/unit/test_public_copy.py).
  • Commits are signed off (git commit -s), as CONTRIBUTING.md asks.

🤖 Generated with Claude Code

zlareb1 and others added 3 commits November 16, 2026 07:48
…h, so a pytest.py in the checkout never runs at Stop

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
…m sys.path, and its tests clear VIRTUAL_ENV

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
… no sys.path or PYTHONPATH entry resolves into the checkout

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant