Skip to content

Report OpenMP blocktime in milliseconds - #302

Open
Aschenbath wants to merge 1 commit into
joblib:masterfrom
Aschenbath:feat/openmp-blocktime-270
Open

Aschenbath wants to merge 1 commit into
joblib:masterfrom
Aschenbath:feat/openmp-blocktime-270

Conversation

@Aschenbath

@Aschenbath Aschenbath commented Nov 7, 2026 •

Copy link
Copy Markdown

Expose blocktime_ms in OpenMP information from threadpool_info(), controller info(), and the CLI. LLVM/Intel OpenMP values are read from kmp_get_blocktime on every query and converted to milliseconds, preserving fractional milliseconds. Runtimes without that symbol, including GNU OpenMP, report None (null in JSON).

The dynamic property and OpenMPController.get_blocktime() reflect the calling thread's current value after kmp_set_blocktime, including when a controller is reused. The unit is cached per loaded runtime so later environment changes do not change the conversion.

Unit-detection limitation: the common LLVM/Intel kmp API does not export a unit getter. Conversion therefore uses a valid KMP_BLOCKTIME startup setting, which must remain available until the runtime is first inspected. Microsecond settings require a runtime that supports them. Changing units through kmp_set_defaults is unsupported. These conditions are documented in the README and API docstrings; this implementation does not provide unconditional native unit introspection.

Validation on this revision:

  • Fresh Python subprocesses load real OpenMP with OMP_DISPLAY_ENV=VERBOSE and KMP_SETTINGS=1, parse the effective runtime setting, and compare it with threadpool_info(). Cases cover the default, 37, 37ms, 35001us, and 35 us, then change the environment and call kmp_set_blocktime to check cached units with a fresh value. The two microsecond cases failed before conversion on Intel OpenMP.
  • Windows, CPython 3.12.12, Intel OpenMP 2026.2.0: focused OpenMP tests passed; full suite 64 passed, 52 skipped, 2 failed. Both failures are test_get_original_num_threads warning expectations with this NumPy/SciPy setup, and both also fail against unchanged master 13b1a129901f353ee4820a3fb7e9a6f7d5fcf4f4 in the same environment.
  • Native glibc Linux, CPython 3.13.18, LLVM OpenMP 19.1.7, NumPy/SciPy and compiled GNU OpenMP helpers: full suite 83 passed, 35 skipped with PYTHON_CPU_COUNT=4. The initial unrestricted run reached 82 passed and 35 skipped but timed out in the existing 10-second controller concurrency stress test, which creates four threads per reported CPU. No Linux baseline run is claimed.
  • All repository pre-commit hooks passed on an ext4 copy with tracked Git file modes, and all 43 tracked files matched the working tree bytes. git diff --check passed.

macOS and hosted CI results are not claimed here. AI assistance was used for investigation, implementation, testing, and this description.

Closes #270.

@ogrisel ogrisel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Here is a first pass of feedback.

Comment thread threadpoolctl.py Outdated
from ``kmp_get_blocktime`` (LLVM/Intel OpenMP), or None when unavailable.
Its units are milliseconds by default, or microseconds when configured
through ``KMP_BLOCKTIME`` on runtimes supporting that unit. This value is
queried on every call, including when reusing a controller.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather always report "ms" to avoid any ambiguity on how to correctly leverage this value. Is it possible to programmatically inspect the runtime and do the conversion in get_blocktime itself so that it always returns ms values?

Comment thread README.md Outdated
'user_api': 'blas',
'version': '2019.0.4'},
{'filepath': '/home/ogrisel/miniconda3/envs/tmp/lib/libiomp5.so',
{'blocktime': 200,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could even rename 'blocktime' to 'blocktime_ms' if we ensure that we always report this value in miliseconds.

Comment thread tests/test_openmp.py
assert get_blocktime() == 7
assert kmp_controller.info()["blocktime"] == 7
finally:
set_blocktime(original)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should include a test that imports openmp in a Python subprocess and the verbose env variable set and then check that the value reported by threadpool_info is consistent with the value parsed from stdout (including when manually set to non-ms units).

@Aschenbath
Aschenbath force-pushed the feat/openmp-blocktime-270 branch from 6a2e9fd to 3e86126 Compare November 14, 2026 09:59
@Aschenbath Aschenbath changed the title Report OpenMP blocktime in threadpool information Report OpenMP blocktime in milliseconds Nov 14, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make it possible to inspect active/spin wait behavior of LLVM/Intel OpenMP via kmp_get_blocktime

2 participants