Skip to content

Make Cache.set faster by running fewer statements and less Python per write - #37

Merged
avgvi merged 7 commits into
masterfrom
perf/set-fewer-statements
Nov 2, 2026
Merged

avgvi merged 7 commits into
masterfrom
perf/set-fewer-statements

Conversation

@avgvi

@avgvi avgvi commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Two changes that cut the SQL run on every write. Each one stands on its own and has its own commit.

  • Cache.volume() in one statement. Each set calls volume() to check the size limit, and it ran two statements: PRAGMA page_count and a read of the stored size. It now reads both in one statement through pragma_page_count(). It falls back to the old two statements on SQLite older than 3.16, and on a read-only snapshot without a size row. volume() still updates cache.size, which check() reads.
  • The earliest expire time comes with the row lookup in Cache.set. The query that finds the existing row for the key now also returns the earliest expire time in the cache. _cull skips its search for expired items when nothing can have expired yet, which includes a row written already expired. When something has expired, culling works exactly as before. Eviction timing doesn't change, and the check stays correct across processes because it reads inside the same transaction.

A third change, replacing the contextlib generator behind Cache._transact with a class, was measured at about 2% per write and then reverted: the gain doesn't justify replacing that code. There's a CHANGELOG.md entry under Unreleased.

Measured with interleaved runs: each build in its own venv on the Python 3.14.6 GIL build, SQLite 3.53.1, Apple M5 Pro, 20 rounds in random order. Medians are against PyPI diskcache 5.6.3; master was run twice to show the noise.

build set set, expire=300 Django set
5.6.3, second venv (noise) −0.1% −0.8% −1.6%
master, two venvs −11.8% / −11.7% +2.8% / +3.2% +3.0% / +3.9%
volume() change only −13.6% +1.2% +0.5%
set lookup change only −23.1% −7.1% −10.7%
both changes + the reverted transaction class −26.3% −8.9% −14.8%

The combined row was measured with the transaction class, which saved about 2 points of it, so this branch should land about 2 points short of that row; it wasn't re-measured after the revert. Cache.get is unchanged.

Limits of these numbers. The lookup change saves the expired-items search only on sets where nothing in the cache has expired. In these benchmarks nothing expired.

  • No gain when entries expire continuously: in a cache where entries expire about as often as new ones are written, most sets still run the search. This is common for a long-running Django cache, where every set carries a timeout, and for the refresh pattern where a get misses on an expired key and the same key is set again.
  • Worst case measured: every set finds one freshly expired entry. The lookup change alone was about 2–4% slower than master there, within noise (+4.2% against 5.6.3, master +0.5% / +1.0%); with all three changes it was level with master.
  • Realistic expectation: such caches should expect roughly the 2% from the volume() change. The Django figures above are the best case.

Possible follow-ups, not in this PR:

  • leave the row being written out of the earliest-expire lookup, so the refresh pattern can skip the search too
  • use DELETE … RETURNING to remove expired items in one statement

Checked:

  • The full test suite passes on Python 3.14 on macOS (814 passed, 3 skipped, 99% coverage).
  • New tests cover:
    • rows expired by another connection are culled by the next set
    • a set whose own row is already expired culls it
    • the expired search is skipped when nothing has expired
    • both volume() fallbacks
  • Dropping the own-row check, always skipping the search, or never skipping it each makes at least one test fail.
  • bluecheck, flake8, isortcheck, mypy and pylint pass.
  • Master, including PR Make a plain Cache.get read less and reuse a cursor per thread #38, is merged into the branch.

@avgvi
avgvi merged commit 69b3d74 into master Nov 2, 2026
22 of 23 checks passed
@avgvi
avgvi deleted the perf/set-fewer-statements branch November 2, 2026 21:08
@avgvi avgvi mentioned this pull request Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant