Skip to content

[CI] Use HF Trusted Publisher OIDC tokens #15106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
DN6 wants to merge 2 commits into
base: main
Choose a base branch
from
+49 −25
Open

[CI] Use HF Trusted Publisher OIDC tokens #15106

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d737167
[CI] Use HF Trusted Publisher OIDC tokens
DN6 Oct 22, 2026
86712af
Merge branch 'main' into ci-oidc-tokens
sayakpaul Oct 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ jobs:
torch_models_cuda_benchmark_tests:
permissions:
contents: read
id-token: write
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL_BENCHMARK }}
name: Torch Core Models CUDA Benchmarking Tests
Expand Down Expand Up @@ -50,7 +51,7 @@ jobs:
diffusers-cli env
- name: Diffusers Benchmarking
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
run: |
cd benchmarks && python run_all.py

Expand Down
23 changes: 15 additions & 8 deletions .github/workflows/nightly_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ jobs:
run_nightly_tests_for_torch_pipelines:
permissions:
contents: read
id-token: write
name: Nightly Torch Pipelines CUDA Tests
needs: setup_torch_cuda_pipeline_matrix
strategy:
Expand Down Expand Up @@ -93,7 +94,7 @@ jobs:
diffusers-cli env
- name: Pipeline CUDA Test
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
# https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms
CUBLAS_WORKSPACE_CONFIG: :16:8
run: |
Expand All @@ -116,6 +117,7 @@ jobs:
run_nightly_tests_for_other_torch_modules:
permissions:
contents: read
id-token: write
name: Nightly Torch CUDA Tests
runs-on:
group: aws-g4dn-2xlarge
Expand Down Expand Up @@ -150,7 +152,7 @@ jobs:
- name: Run nightly PyTorch CUDA tests for non-pipeline modules
if: ${{ matrix.module != 'examples'}}
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
# https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms
CUBLAS_WORKSPACE_CONFIG: :16:8
run: |
Expand All @@ -163,7 +165,7 @@ jobs:
- name: Run nightly example tests with Torch
if: ${{ matrix.module == 'examples' }}
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
# https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms
CUBLAS_WORKSPACE_CONFIG: :16:8
run: |
Expand All @@ -188,6 +190,7 @@ jobs:
run_torch_compile_tests:
permissions:
contents: read
id-token: write
name: PyTorch Compile CUDA tests

runs-on:
Expand Down Expand Up @@ -216,7 +219,7 @@ jobs:
diffusers-cli env
- name: Run torch compile tests on GPU
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
run: |
pytest -n 1 --max-worker-restart=0 --dist=loadfile -m compile --make-reports=tests_torch_compile_cuda tests/
- name: Failure short reports
Expand All @@ -233,6 +236,7 @@ jobs:
run_big_gpu_torch_tests:
permissions:
contents: read
id-token: write
name: Torch tests on big GPU
strategy:
fail-fast: false
Expand Down Expand Up @@ -262,7 +266,7 @@ jobs:
diffusers-cli env
- name: Selected Torch CUDA Test on big GPU
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
# https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms
CUBLAS_WORKSPACE_CONFIG: :16:8
BIG_GPU_MEMORY: 40
Expand All @@ -287,6 +291,7 @@ jobs:
torch_minimum_version_cuda_tests:
permissions:
contents: read
id-token: write
name: Torch Minimum Version CUDA Tests
runs-on:
group: aws-g4dn-2xlarge
Expand Down Expand Up @@ -316,7 +321,7 @@ jobs:

- name: Run PyTorch CUDA tests
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
# https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms
CUBLAS_WORKSPACE_CONFIG: :16:8
run: |
Expand Down Expand Up @@ -345,6 +350,7 @@ jobs:
run_nightly_quantization_tests:
permissions:
contents: read
id-token: write
name: Torch quantization nightly tests
strategy:
fail-fast: false
Expand Down Expand Up @@ -390,7 +396,7 @@ jobs:
diffusers-cli env
- name: ${{ matrix.config.backend }} quantization tests on GPU
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
# https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms
CUBLAS_WORKSPACE_CONFIG: :16:8
BIG_GPU_MEMORY: 40
Expand All @@ -417,6 +423,7 @@ jobs:
run_nightly_pipeline_level_quantization_tests:
permissions:
contents: read
id-token: write
name: Torch quantization nightly tests
strategy:
fail-fast: false
Expand Down Expand Up @@ -445,7 +452,7 @@ jobs:
diffusers-cli env
- name: Pipeline-level quantization tests on GPU
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
# https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms
CUBLAS_WORKSPACE_CONFIG: :16:8
BIG_GPU_MEMORY: 40
Expand Down
6 changes: 4 additions & 2 deletions .github/workflows/pr_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ jobs:
run_fast_tests:
permissions:
contents: read
id-token: write
needs: [check_code_quality, check_repository_consistency]
strategy:
fail-fast: false
Expand Down Expand Up @@ -122,7 +123,7 @@ jobs:
shell: bash

env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: ${{ !github.event.pull_request.head.repo.fork && 'Diffusers-CI-Bot' || '' }}

@sayakpaul sayakpaul Oct 22, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is happening here? 👀

Because in some places I am seeing:

HF_OIDC_RESOURCE: Diffusers-CI-Bot

@DN6 DN6 Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PRs from forks do not have secrets injected into the env. This also applies to OIDC tokens. So if the OIDC resource is set to Diffusers-CI-Bot and no corresponding token is available, the Hub requests raises an OIDCError.

This section just sets the HF_OIDC_RESOURCE to an empty string, which would then set the HF token to None in the subsequent workflows, which is the current behaviour for forked PRs.


steps:
- name: Checkout diffusers
Expand Down Expand Up @@ -240,6 +241,7 @@ jobs:
run_lora_tests:
permissions:
contents: read
id-token: write
needs: [check_code_quality, check_repository_consistency]

name: LoRA tests with PEFT main
Expand All @@ -256,7 +258,7 @@ jobs:
shell: bash

env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: ${{ !github.event.pull_request.head.repo.fork && 'Diffusers-CI-Bot' || '' }}

steps:
- name: Checkout diffusers
Expand Down
9 changes: 6 additions & 3 deletions .github/workflows/pr_tests_gpu.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ jobs:
torch_pipelines_cuda_tests:
permissions:
contents: read
id-token: write
name: Torch Pipelines CUDA Tests
needs: setup_torch_cuda_pipeline_matrix
strategy:
Expand Down Expand Up @@ -166,7 +167,7 @@ jobs:

- name: PyTorch CUDA checkpoint tests on Ubuntu
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: ${{ !github.event.pull_request.head.repo.fork && 'Diffusers-CI-Bot' || '' }}
# https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms
CUBLAS_WORKSPACE_CONFIG: :16:8
run: |
Expand Down Expand Up @@ -197,6 +198,7 @@ jobs:
torch_cuda_tests:
permissions:
contents: read
id-token: write
name: Torch CUDA Tests
needs: [check_code_quality, check_repository_consistency]
runs-on:
Expand Down Expand Up @@ -239,7 +241,7 @@ jobs:

- name: Run PyTorch CUDA tests
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: ${{ !github.event.pull_request.head.repo.fork && 'Diffusers-CI-Bot' || '' }}
# https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms
CUBLAS_WORKSPACE_CONFIG: :16:8
run: |
Expand Down Expand Up @@ -268,6 +270,7 @@ jobs:
run_examples_tests:
permissions:
contents: read
id-token: write
name: Examples PyTorch CUDA tests on Ubuntu
needs: [check_code_quality, check_repository_consistency]
runs-on:
Expand Down Expand Up @@ -297,7 +300,7 @@ jobs:

- name: Run example tests on GPU
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: ${{ !github.event.pull_request.head.repo.fork && 'Diffusers-CI-Bot' || '' }}
run: |
uv pip install ".[training]"
pytest -n 1 --max-worker-restart=0 --dist=loadfile --make-reports=examples_torch_cuda examples/
Expand Down
12 changes: 8 additions & 4 deletions .github/workflows/push_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ jobs:
torch_pipelines_cuda_tests:
permissions:
contents: read
id-token: write
name: Torch Pipelines CUDA Tests
needs: setup_torch_cuda_pipeline_matrix
strategy:
Expand Down Expand Up @@ -96,7 +97,7 @@ jobs:
diffusers-cli env
- name: PyTorch CUDA checkpoint tests on Ubuntu
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
# https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms
CUBLAS_WORKSPACE_CONFIG: :16:8
run: |
Expand All @@ -118,6 +119,7 @@ jobs:
torch_cuda_tests:
permissions:
contents: read
id-token: write
name: Torch CUDA Tests
runs-on:
group: aws-g4dn-2xlarge
Expand Down Expand Up @@ -152,7 +154,7 @@ jobs:

- name: Run PyTorch CUDA tests
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
# https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms
CUBLAS_WORKSPACE_CONFIG: :16:8
run: |
Expand All @@ -177,6 +179,7 @@ jobs:
run_torch_compile_tests:
permissions:
contents: read
id-token: write
name: PyTorch Compile CUDA tests

runs-on:
Expand Down Expand Up @@ -205,7 +208,7 @@ jobs:
diffusers-cli env
- name: Run torch compile tests on GPU
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
run: |
pytest -n 1 --max-worker-restart=0 --dist=loadfile -m compile --make-reports=tests_torch_compile_cuda tests/
- name: Failure short reports
Expand All @@ -222,6 +225,7 @@ jobs:
run_examples_tests:
permissions:
contents: read
id-token: write
name: Examples PyTorch CUDA tests on Ubuntu

runs-on:
Expand Down Expand Up @@ -250,7 +254,7 @@ jobs:

- name: Run example tests on GPU
env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
run: |
uv pip install ".[training]"
pytest -n 1 --max-worker-restart=0 --dist=loadfile --make-reports=examples_torch_cuda examples/
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/push_tests_fast.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ jobs:
run_fast_tests:
permissions:
contents: read
id-token: write
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -58,7 +59,7 @@ jobs:
shell: bash

env:
HF_TOKEN: ${{ secrets.DIFFUSERS_HF_HUB_READ_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot

steps:
- name: Checkout diffusers
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/push_tests_mps.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ jobs:
run_fast_tests_apple_m1:
permissions:
contents: read
id-token: write
name: Fast PyTorch MPS tests on MacOS
runs-on: macos-13-xlarge

Expand Down Expand Up @@ -60,7 +61,7 @@ jobs:
shell: arch -arch arm64 bash {0}
env:
HF_HOME: /System/Volumes/Data/mnt/cache
HF_TOKEN: ${{ secrets.HF_TOKEN }}
HF_OIDC_RESOURCE: Diffusers-CI-Bot
run: |
${CONDA_RUN} python -m pytest -n 0 --make-reports=tests_torch_mps tests/

Expand Down
Loading
Loading