ci: separate extra installs by begumcig · Pull Request #622 · PrunaAI/pruna

begumcig · 2026-04-08T09:01:32Z

Description

Isolated whisper dependencies into a dedicated [whisper] extra. Moved ctranslate2 and whisper-s2t out of the default full install group into [whisper] to prevent side-effects from whisper_s2t imports breaking unrelated tests at collection time. The previously hard-skipped TestWhisperS2T is now gated behind requires_whisper instead.
Pin torch/torchvision bounds on the [intel] extra. intel-extension-for-pytorch is tightly coupled to the torch version; added explicit torch>=2.7.0,<2.9.0 and torchvision>=0.22.0,<0.24.0 constraints so installs don't silently break.
Also switch TestIPEXLLM to use opt_125m model and drop latency metric. The algorithm didn't work on a tiny model due to latent size being small; we also need a specific inference handler for it for the metrics, which can be done later, so removed the evaluation part for now.
IPEX is strictly coupled with the torch version and is not maintained. So makes sense to run it in the nightlies but not on the CI, as it requires its own setup.
Declared packaging conflicts between intel/awq and gptq/awq. These extras pull in incompatible transitive deps; the new [tool.uv.conflicts] entries surface the error at resolve time instead of runtime.
Added requires_* markers for optional-extra tests. New markers requires_awq, requires_gptq, requires_stable_fast, requires_vllm, requires_intel, requires_lmharness, and requires_whisper let CI select/deselect tests based on what's installed rather than relying on implicit skips or collection errors.
Removed the old HIGH_RESOURCE_FIXTURES list and runtime CUDA/multi-GPU detection. Tests without an explicit hardware mark are now auto-tagged cpu. Incompatible device_parametrized variants (e.g. a CUDA-only algorithm's CPU test) are deselected at collection time instead of skipping after expensive fixture setup. Added explicit pytestmark = pytest.mark.cpu to pure-CPU test modules (test_compatibility_symmetry, test_evalharness_metrics, test_registry, test_metrics) so they're correctly selected by -m "cpu".
Updated CI workflow. Dropped --extra vllm from the default uv sync in the setup action. Changed the pytest filter from -m "not (slow or style or high_cpu or cuda or distributed)" to the positive-selection form -m "cpu and not slow and not requires_intel".

Related Issue

Fixes #(issue number)

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Refactor (no functional change)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update

Testing

I added or updated tests covering my changes
Existing tests pass locally (uv run pytest -m "cpu and not slow")

For full setup and testing instructions, see the Contributing Guide.

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my code, especially for agent-assisted changes
I updated the documentation where necessary

Thanks for contributing to Pruna! We're excited to review your work.

New to contributing? Check out our Contributing Guide for everything you need to get started.

Note:

Draft PRs or PRs without a clear and detailed overview may be delayed.

Please mark your PR as Ready for Review and ensure the sections above are filled out.

Contributions that are entirely AI-generated without meaningful human review are discouraged.

First Prune (1-year OSS anniversary)

First Prune marks one year of Pruna’s open-source work. During the initiative window, qualifying merged contributions count toward First Prune. You can earn credits for our performance models via our API.

If you’d like your contribution to count toward First Prune, here’s how it works:

Initiative window: First Prune starts on March 31.
Issue assignment: For your PR to count toward First Prune, the related issue must be assigned to the contributor opening the PR. Issues are labeled with first-prune.
Open for review: Please open your PR and mark it ready for review by April 30 (end of April).
Review priority: We’ll make our best effort to review quickly any PR that is open and has a review request before April 30.
Credits: Each qualifying merged PR earns 30 credits. We’ll be in touch after all qualifying PRs for First Prune have been merged.
To get started: Have a look at all models. You’ll need to sign up on the dashboard before you can redeem your credits.

codacy-production · 2026-04-08T09:02:36Z

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity · 0 duplication

Metric Results

Complexity 0

Duplication 0

View in Codacy

_{TIP This summary will be updated as you push new changes. Give us feedback}

…n errors in test collection

gsprochette

Thanks for this absolute gold PR. Already super good, I left some comments for little improvements here and there, almost ready already

gsprochette · 2026-04-09T14:16:39Z

.github/actions/setup-uv-project/action.yml


    - shell: bash
-      run: uv sync --extra dev --extra lmharness --extra vllm
+      run: uv sync --extra dev --extra lmharness


why are we still installing lmharness? Shouldn't it be isolated as well?

gsprochette · 2026-04-09T14:18:17Z

.github/workflows/tests.yaml

        run: |
          echo "Running tests with up to 3 reruns on failure using $PYTEST_WORKERS workers..."
-          uv run pytest -n $PYTEST_WORKERS -m "not (slow or style or high_cpu or cuda or distributed)" --reruns 3 --reruns-delay 10 --maxfail=1
+          uv run pytest -n $PYTEST_WORKERS -m "cpu and not slow and not requires_intel" --reruns 3 --reruns-delay 10 --maxfail=1


I think this will run the style tests a second time, since they are already running during the linting job line 56

gsprochette · 2026-04-09T14:18:58Z

tests/algorithms/testers/awq.py



-@pytest.mark.slow
+@pytest.mark.requires_awq


is it not slow anymore?

i thought slow was a wrong tag here, as awq is not time intensive like gptq for instance, but how do you feel?

gsprochette · 2026-04-09T14:24:41Z

tests/algorithms/testers/upscale.py


-@pytest.mark.cuda
+# Takes too long to run on CPU, so we mark it as slow
+@pytest.mark.slow


You could also do something like

@classmethod def compatible_devices(cls) -> list[str]: # type: ignore[override] """Get the devices compatible with this algorithm, except CPU (too slow).""" return [d for d in super().compatible_devices() if d != "cpu"]

if the intent is to not run this test on cpu.
I do agree that the cuda mark is not the correct tool because it will just create a cpu test eitherway, and it will have both the cpu and cuda mark...

I don't think we need the slow thing now do we? maybe we still want it or maybe it should be a comment stating why we ignore the cpu case... I'll let you decide

gsprochette · 2026-04-09T14:30:22Z

tests/evaluation/test_registry.py

 from pruna.evaluation.metrics.metric_base import BaseMetric
 from pruna.evaluation.metrics.registry import MetricRegistry

+pytestmark = pytest.mark.cpu


this is redundant with the cpu default in conftest.py and not very explicit

gsprochette · 2026-04-09T14:31:24Z

tests/telemetry/test_metrics.py

    set_opentelemetry_log_level,
 )

+pytestmark = pytest.mark.cpu


This is redundant with the cpu default in conftest.py, and not very explicit

gsprochette · 2026-04-09T14:39:39Z

pyproject.toml

+# Intel extension is tightly coupled with the torch version
 intel = [
    "intel-extension-for-pytorch>=2.7.0",
+    "torch>=2.7.0,<2.9.0",


I get that it's tightly coupled with torch, but if intel-extension already declares it's torch dependency we shouldn't need to declare it here no? Same goes for torchvision

I see what you mean and I softly disagree. yes the version already defines which torch version it's supposed to use, but it doesn't define it in its dependencies. same for the torchvision version. So someone could install ipex with the absolutely wrong torch stack and get really cryptic errors. I think it's better to be explicit here.

ah, if the torch version is not in the package dependency then absolutely let's pin torch so that the intel install works

gsprochette

Looks great, let's merge it ! I just left one comment for you to decide about the "slow" mark on upscale :) 💅

gsprochette · 2026-04-10T14:00:35Z

tests/algorithms/testers/upscale.py


-@pytest.mark.cuda
+# Takes too long to run on CPU, so we mark it as slow
+@pytest.mark.slow


I don't think we need the slow thing now do we? maybe we still want it or maybe it should be a comment stating why we ignore the cpu case... I'll let you decide

gsprochette

Meant to approve, please refer to previous comment 🙃

begumcig force-pushed the ci/separate-extra-installs branch 2 times, most recently from 15f808b to 790b58a Compare April 8, 2026 14:34

begumcig added 2 commits April 8, 2026 14:46

refactor: add more descriptive install message for awq

8893d42

test: refactor the marks

d018be1

begumcig force-pushed the ci/separate-extra-installs branch from 790b58a to b72e644 Compare April 8, 2026 14:47

refactor: remove implicit redundant checks that obfuscate installatio…

2202c07

…n errors in test collection

begumcig force-pushed the ci/separate-extra-installs branch from b72e644 to 2202c07 Compare April 8, 2026 14:58

begumcig added 3 commits April 8, 2026 18:57

ci: modify collected algorithms tests to only run on desired device

438871d

test: change upscaler mark to something more accurate

8f94b3b

ci: add ipex to ci

a59e6c5

begumcig force-pushed the ci/separate-extra-installs branch 2 times, most recently from 437aac1 to 298914e Compare April 9, 2026 08:42

test: debug ipexllm test also remove it from the CI

026073c

begumcig force-pushed the ci/separate-extra-installs branch from 298914e to 026073c Compare April 9, 2026 11:09

begumcig requested a review from gsprochette April 9, 2026 11:38

gsprochette requested changes Apr 9, 2026

View reviewed changes

test: add proper marks to combo tests

e1089ac

begumcig requested a review from gsprochette April 10, 2026 13:51

gsprochette reviewed Apr 10, 2026

View reviewed changes

gsprochette self-requested a review April 10, 2026 14:07

gsprochette approved these changes Apr 10, 2026

View reviewed changes

begumcig force-pushed the ci/separate-extra-installs branch from 58db272 to c7e5727 Compare April 10, 2026 14:50

refactor: address pr comments

8292c33

begumcig force-pushed the ci/separate-extra-installs branch from c7e5727 to 8292c33 Compare April 10, 2026 14:55



		@pytest.mark.slow
		@pytest.mark.requires_awq

Conversation

begumcig commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue

Type of Change

Testing

Checklist

First Prune (1-year OSS anniversary)

Uh oh!

codacy-production bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Up to standards ✅

Uh oh!

gsprochette left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsprochette left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsprochette left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

begumcig commented Apr 8, 2026 •

edited

Loading

codacy-production bot commented Apr 8, 2026 •

edited

Loading