[avgvi]

Adds SafeDisk.with_signing(), an opt-in disk for libraries that cache their own objects, which no allowlist can hold: parsed databases whose pickles call functions, torch models, application objects, requests.Response. Testing diskcache-community 5.8.2 against 42 downstream libraries found five that need it (cantools, pywander, pyjinhx, fluke-fl, holobench). Each one's fallback on 5.8.3 narrows what it caches or puts HMAC code into the library itself. With this disk, the change on their side is a few lines.

Nothing existing changes unless with_signing() is called. No other disk behaves differently, and the cache.db schema and the file layout are the same. It is new, security-sensitive API in a patch release, and the changelog gives it a section of its own.

What it does

• Every value the disk pickles is stored as a full pickle with an HMAC-SHA256 tag over the pickle and the cache key. The tag is checked before anything is unpickled.
• A pickled value without a matching tag raises UnpicklingError, or reads as missing with on_error='miss'. That includes every unsigned pickle, plain data included, so the allowlist is not used for values on a signing disk.
• str, bytes, int and float values are stored as on every disk, unsigned.
• Keys stay on the allowlist, and with_types widens what they may hold.
• The cache key is part of the tag, so a value moved under another key is refused, and so is a filename pointed at another row's value file.
  • Numeric keys are encoded in one canonical form, because SQLite treats 1 and 1.0, and 0.0 and -0.0, as one key.
  • NaN keys raise TypeError, because SQLite stores them as NULL. That bug exists on every disk today.
• push hashes the pickle before its transaction and adds the key once the transaction has chosen it, so the write lock is held only for microseconds.

Row format

• The row's mode stays MODE_PICKLE.
• The value column always starts with 45 bytes:
  • a magic, b'\xffDCS';
  • a version (1);
  • an 8-byte key id;
  • the 32-byte tag over header ‖ pickle ‖ key ‖ len(key).
• The pickle follows inline, or sits alone in the value file.
• 0xff is not a pickle opcode, so diskcache 5.6.3 and every existing disk raise UnpicklingError: invalid load key, '\xff' on a signed row before reading anything else, or delete it under on_error='miss'. A new test in test_golden.py checks this with a real 5.6.3 interpreter in the coexist job.
• A SafeDisk without signing now says the row is a signed entry.
• File rows are checked in a streaming pass first, then read with a cap at the length that pass saw and checked again. A planted large file is therefore refused without being read into memory.

Keys

• Default key file: $DISKCACHE_SIGNING_KEY_FILE, otherwise signing.key in a diskcache-community folder of the user's local data directory:
  • $XDG_DATA_HOME or ~/.local/share on Linux;
  • ~/Library/Application Support on macOS;
  • %LOCALAPPDATA% on Windows.
• Creation: the key file is created on first use with mode 0600 in a 0700 folder. It is written to a temporary file and hard-linked into place, so processes that start together end up with the same key.
• Checks when a cache opens:
  • a key file inside the cache directory raises ValueError. The check compares device and inode, so another letter case on macOS and symlinks are caught, and FanoutCache also checks its own root;
  • on POSIX, a key file that other users can write is refused, and so is a folder they can write without the sticky bit;
  • a key file owned by another user is refused;
  • a user-owned key file that others can read is refused. One owned by root is accepted, which is how Docker and Kubernetes mount secrets.
• No home directory: opening the cache raises OSError and names DISKCACHE_SIGNING_KEY_FILE.
• key=: takes 32 or more bytes held only in memory, for caches that live in one process (fluke-fl). Pickling a cache that uses it raises TypeError.
• Keeping the key out of sight: the key never appears in a pickle of Cache or FanoutCache, in a repr, or as bytes in an attribute. The disk holds a keyed hmac.HMAC object instead.

Changes outside the new module

• core.py:
  • the seven fetch call sites in Cache pass the row's key only when the disk signs, and other disks are called exactly as before;
  • push signs once the key is chosen;
  • SafeDisk gains with_signing, and its put, store and fetch gain a branch for signing disks;
  • with_numpy() and with_signing() refuse to combine;
  • SafeJSONDisk.with_signing() raises.
• fanout.py checks the key file against the fanout root, since each shard's disk sees only its shard.
• .pylintrc raises max-module-lines from 4300 to 4400, since core.py is now 4359 lines.

Not in this PR

• Re-signing existing rows: a resign() was in an earlier draft and was removed after review, because running it on a shared or restored cache would sign planted rows.
• Accepting several keys while reading.
• Publisher signatures for prebuilt caches.
• Refusing raw values.
• A sign and verify helper for files outside diskcache.

Testing

• pytest, Python 3.12 (Django 5.2, numpy, pydantic, and the 5.6.3 interpreter for the coexist tests): 936 passed, 1 skipped, coverage 99.27%. signing.py is at 100%.
• pytest, Python 3.14: 934 passed, 3 skipped, coverage 99.29%.
• Linters, docs and stubs:
  • blue, isort, flake8, mypy (including tests/typecheck_stubs.py), pylint (10.00/10) and doc8 are clean;
  • the Sphinx build passes with -W;
  • stubtest reports the same three errors as master.
• Speed of disks that don't sign: a micro-benchmark of get hit, get miss and set against master is within noise. For a signing disk, a small get costs about 1 µs more. A 1 MiB file row reads in 0.8 ms instead of 0.06 ms, because of the two passes. A small set is faster than plain SafeDisk, because it skips the allowlist scan.
• Not run: Windows, the SQLITE_DQS=0 job, and the Django 4.2 and 6.0 jobs. On Windows only the key file's location is checked, not its permissions.

[read-the-docs-community]

Documentation build overview

📚 diskcache-community | 🛠️ Build #35626465 | 📁 Comparing f300cf4 against latest (ac57330)

  🔍 Preview build  

4 files changed

± api.html
± fix-cve-2025-69872.html
± genindex.html
± tutorial.html