Do not use CodSpeedHQ/action on pull requests to a fork - #9761
Conversation
akx
left a comment
akx
left a comment
There was a problem hiding this comment.
I'd be fine with just running Codspeed on pull requests to or pushes on python-pillow/Pillow?
| - name: Run CodSpeed benchmarks on pull requests to a fork | ||
| if: github.event_name == 'pull_request' && github.event.repository.fork == true | ||
| run: | | ||
| python3 -m pip install -e . pytest-codspeed | ||
| python3 -m pytest -vv --codspeed Tests/benchmarks.py |
There was a problem hiding this comment.
I'm not sure this alternate is really needed. If you're not using the codspeed action (that does CPU simulation and other deep magic), you're not going to get comparable numbers anyhow.
| - name: Run CodSpeed benchmarks on pull requests to a fork | |
| if: github.event_name == 'pull_request' && github.event.repository.fork == true | |
| run: | | |
| python3 -m pip install -e . pytest-codspeed | |
| python3 -m pytest -vv --codspeed Tests/benchmarks.py |
| GHA_LIBWEBP_CACHE_HIT: ${{ steps.cache-libwebp.outputs.cache-hit }} | ||
|
|
||
| - name: Run CodSpeed benchmarks | ||
| if: github.event_name != 'pull_request' || github.event.repository.fork == false |
There was a problem hiding this comment.
Maybe simply
| if: github.event_name != 'pull_request' || github.event.repository.fork == false | |
| if: github.event.repository.fork == false |
in conjunction with the below (see rationale there).
WDYT?
This seems like the same thinking as #9946? |
Not really... looks like the CodSpeed action doesn't report anything on fork runs, see e.g. https://github.com/akx/Pillow/actions/runs/33734366520/job/100581356953, it just churns through running the benchmarks and then crashes and fails the run. |
|
Depends if you've enabled CodSpeed on your fork, compare: https://github.com/hugovk/Pillow/actions/runs/33500798014/job/99833423587 Can we have it not fail the run when it's not enabled? |
I can suggest a PR for the action, an input where we could feed But if it doesn't otherwise report the benchmark results, then it's just doing 20+ minutes of work for every push just to drop the results on the floor. (I guess it could maybe do a GitHub Step Summary instead...) EDIT: Tuumasta toimeen, as they say. |
3 participants
Second version of #9749
While we can configure the Benchmarks workflow in our own repositories, perhaps it is not ideal that it fails when pull requests are made to a fork? For example, akx#8. The receiving user might believe that the changes in my pull request are causing this workflow to fail.