logging: Logger.setLevel() and logging.disable() are O(number of loggers) #160936
Description
Bug report
Every Logger.setLevel() and logging.disable() call goes through Manager._clear_cache(), which walks the whole loggerDict and clears each logger's isEnabledFor cache, even the ones that never cached anything. So setting the level on N loggers one by one is quadratic. That shows up in apps or test suites with thousands of loggers (one per module/plugin/connection) that get configured individually.
import logging, time
for n in (1000, 2000, 4000, 8000):
logging.root.manager.loggerDict.clear()
loggers = [logging.getLogger(f"app.mod{i}") for i in range(n)]
for lg in loggers:
lg.isEnabledFor(logging.DEBUG)
t = time.perf_counter()
for lg in loggers:
lg.setLevel(logging.INFO)
print(f"{n:5d} loggers: {time.perf_counter() - t:.3f} s")On 3.14.7 this gives about 0.23 s at 2000 loggers, 0.89 s at 4000 and 4.4 s at 8000 (each doubling is about 4x).
A fix is to have the manager remember which loggers actually have something in their cache (a WeakSet filled on the isEnabledFor cache-miss path, which already holds the lock) and only clear those. The cache-hit path doesn't change. I have a patch ready and can open a PR.
CPython versions tested on:
3.14, CPython main branch
Operating systems tested on:
Linux
Activity
aisk commentedon Oct 30, 2026
Using thousands of loggers is not a common case in real world code, and
setLevel()/disable()is not called very often.So I'm curious, did you run into a real performance problem with these? If not, I doubt this optimization is worth it.
pythongh-160936: Clear all cached loggers unconditionally in _clear_c…
Ar-maan05 commentedon Nov 1, 2026
Good question! Not from an app of my own, I spotted it reading the code. It does match something people really write though: looping over
loggerDictand callingsetLevel()on each logger, like metavoice-src and ktrain do. It's about 75 ms at 2000 loggers and 1.5 s at 8000 on 3.14, so it only really bites big setups.The PR is small and doesn't touch the cache-hit path, and it now also fixes a stale cache bug on standalone loggers (
Logger("x").setLevel(ERROR)leavesisEnabledFor(DEBUG)True), so I think it's worth having even if the speedup is niche.pythongh-160936: Check cache state before and after in the new loggin…
pythongh-160936: Clear the logger cache after the new logging tests
aisk commentedon Nov 5, 2026
The examples you gave are all called only once when the process starts. And normally there are not so many loggers, so it only takes a few milliseconds. So if we add a lot of complexity for this, I think there is no strong reason to fix it.
For the bug you mentioned, maybe we can fix it separately.