Skip to content

Type the persistence layer, and guard what it writes - #414

Merged
dovvnloading merged 1 commit into
mainfrom
types/persistence-layer
Sep 4, 2026
Merged

Type the persistence layer, and guard what it writes#414
dovvnloading merged 1 commit into
mainfrom
types/persistence-layer

Conversation

@dovvnloading

Copy link
Copy Markdown
Owner

Problem

session_save.py held 246 of the 444 remaining mypy backend errors — 55%
of everything left, in one file — and session_load.py another 20. All the same
shape the rest of this sweep has been closing: SceneNode.state is typed
NodeState | None against a field-less marker, so a serializer reading
node.state.gitlink_repo cannot be checked.

The save side is the easiest instance of it there has ever been. Its serializers
are reached through a kind-keyed dispatch table, so the kind is settled before
the function is entered. Each one already knows exactly what it is looking at —
it just had no way to say so.

Change

Mapped before touching anything

serializers needing narrowing 16
distinct state-field reads 105
fields not declared on their kind's state class 0
bodies reading from two state classes 0
isinstance guards that had to be preserved 0

The thinking and conversation serializers read no state at all — those nodes
carry none — so they are untouched.

The change itself

node_access.with_state narrows a node whose kind is already decided. Each
serializer's first parameter is renamed and one line added:

def _serialize_chat_node(raw_node: SceneNode) -> dict[str, Any]:
    node = with_state(raw_node, ChatState)

Bodies are untouched. Verified: all 16 are AST-identical to their pre-change
versions apart from that single inserted statement, signatures differ only in
the first parameter's name, and the module's other 13 functions are unchanged.

session_load.py's 20 were more varied — nodes built directly and then written
to, two accumulators with no inferable element type, one loop variable reused
for a wider type, and one conditional only ever reached when its value is a
legal status string.

Behaviour change

with_state raises SceneError when the state is missing or of another class.

Before, such a node failed too — but partway through building its payload, on
whatever getattr happened to hit first. Node states are plain non-slotted
dataclasses, so a wrong-state node does not fail on the first access. This is
the write side of persistence
: a payload half-built from the wrong fields is a
saved session that cannot be loaded back. Nothing catches the exception either
way — build_chat_data has no per-node handler — so the save fails in both
cases. It now fails immediately, naming the node and the state class it wanted.

backend/tests/test_serializer_state_guards.py pins this for all 16, both for a
node with no state (what a row predating its kind's state class looks like) and
for one carrying another kind's. Verified non-vacuous: with with_state
stubbed to a pass-through, 32 of the 33 tests fail.

One gate change, deliberately

[tool.mypy] gains follow_imports = "silent". Without it, listing
session_save.py pulls its whole import closure — autosave, chat_library,
assets, api_provider — into the gate and reports 141 errors in modules
nobody put on the ratchet. That would mean the only way to add a clean module is
to first clean everything it touches, which is not how a ratchet is supposed to
move.

I checked this does not quietly neuter the gate: a deliberate type error
injected into session_save.py, domain/graph.py, settings_store/persistence.py
and events.py is still caught in all four.

Result

before this sweep before this PR now
mypy backend errors 1,059 444 179
session_save.py 246 246 0
session_load.py 20 20 0
CI-enforced scope 4 files 48 50

What's left is a long tail: api_provider.py 32, assets.py 19,
chat_library.py 14, then 43 files under 10 apiece. No single lever any more.

Test plan

  • Full suite: 3,280 passed, 20 skipped.
  • The 16 serializers verified AST-identical to their pre-change versions.
  • The new guard tests verified to fail (32/33) when the guard is removed.
  • ruff check . clean. mypy clean across 50 source files.

🤖 Generated with Claude Code

session_save.py held 246 of the 444 remaining `mypy backend` errors - 55%
of everything left in one file - and session_load.py another 20. Every one
was the same shape the rest of this sweep has been closing: SceneNode.state
is typed `NodeState | None` against a field-less marker, so a serializer
reading `node.state.gitlink_repo` cannot be checked.

The save side is the easiest case of it there has ever been. Its
serializers are reached through a kind-keyed dispatch table, so the kind is
settled before the function is entered - each one already knows exactly
what it is looking at, it just had no way to say so. node_access.with_state
is that way.

Mapped mechanically before changing anything: 16 serializers, 105 distinct
state-field reads, every field declared on its kind's own state class, no
body reading from two state classes, no isinstance guards to preserve. The
thinking and conversation serializers read no state at all - those nodes
carry none - so they are untouched.

Each serializer's first parameter is renamed to raw_node and one line
added:

    def _serialize_chat_node(raw_node: SceneNode) -> dict[str, Any]:
        node = with_state(raw_node, ChatState)

Bodies are untouched. Verified: every one of the 16 is AST-identical to its
pre-change version apart from that single inserted statement, signatures
differ only in the first parameter's name, and the 13 other functions in
the module are unchanged.

BEHAVIOUR CHANGE, and the reason this file got tests rather than just a
green suite. with_state raises SceneError when the state is missing or of
another class. Before, that node failed too - partway through building its
payload, on whatever getattr happened to hit first. Node states are plain
non-slotted dataclasses, so a wrong-state node does not fail on the first
access; it fails somewhere in the middle, and this is the write side of
persistence, where a payload half-built from the wrong fields is a saved
session that cannot be loaded back. Nothing catches the exception either
way - build_chat_data has no per-node handler - so the save fails in both
cases. It now fails immediately, saying which node and which state class.

backend/tests/test_serializer_state_guards.py pins that for all 16, both
for a node with no state (what a row predating its kind's state class looks
like) and for a node carrying another kind's. Verified non-vacuous: with
with_state stubbed to a pass-through, 32 of the 33 fail.

session_load.py's 20 were more varied - nodes built directly and then
written to, two accumulators with no inferable element type, one loop
variable reused for a wider type, and one conditional that is only ever
reached when its value is a legal status string.

[tool.mypy] gains follow_imports = "silent". Without it, listing
session_save.py pulls its whole import closure - autosave, chat_library,
assets, api_provider - into the gate and reports 141 errors in modules
nobody put on the ratchet, which would mean the only way to add a clean
module is to first clean everything it touches. Verified this does not
neuter the gate: a deliberate type error injected into session_save.py,
domain/graph.py, settings_store/persistence.py and events.py is still
caught in all four.

`mypy backend` on the same measure as the 444 baseline: 179. The enforced
gate covers 50 source files.

Test plan: full suite, 3,280 passed / 20 skipped. ruff clean, mypy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dovvnloading
dovvnloading merged commit f3e3b3d into main Sep 4, 2026
5 checks passed
@dovvnloading
dovvnloading deleted the types/persistence-layer branch September 4, 2026 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant