From 9b605c2dca4416d3ed536929beef53f3762add8f Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Thu, 6 Aug 2026 00:35:47 -0500 Subject: [PATCH 1/2] fix(secrets): the rotation fingerprinter skipped the SOAP body_secret_value class (BACKLOG #1009) connector_secret_env_values, the ASVS 13.3.4 runtime rotation fingerprinter, filtered connector settings on bare _SECRET_SETTING_KEYS membership. The SOAP body_secret_value_ settings reach secrecy only through the prefix branch of _is_secret_setting, so they fell through the filter: a rotation of a SOAP injected body secret would not be fingerprinted on first deployment, while every sibling connector credential is tracked. Fix (config/wiring.py): filter through _is_secret_setting, not the bare frozenset, so the prefix-only body_secret_value_ class is covered. The change is additive -- body_secret_value_ is not in _NON_ROTATABLE_SECRET_SETTING_KEYS and each is a bare EnvRef the isinstance guard already accepts -- so it enrols the class and moves nothing else. Reverse gate (tests/test_secret_rotation_inventory.py): the forward gate (test_secret_setting_keys_are_registered) proves every _SECRET_SETTING_KEYS member is registered but is blind to the other direction -- a registered connector secret that does not resolve through the fingerprinter. body_secret_value entered CRITICAL_SECRETS by hand without passing through _SECRET_SETTING_KEYS, exactly the direction the old comment wrongly promised "can never disagree". The new test_registered_connector_secrets_are_reachable_by_the_fingerprinter builds a probe outbound per registered connector secret and asserts its env key is returned, so a future hand-added registry entry cannot silently fall through again. Regression test (tests/test_secret_rotation_watcher.py): a real Soap(body_secrets=...) outbound exercising factory -> _hoist_body_secrets -> body_secret_value_0 -> fingerprint end to end. It fails on the pre-fix predicate and passes after. Deliberately out of scope, flagged for a separate follow-up item: the JWS signing-key passphrase is emitted as the connector setting sign_private_key_password (transports/signing.py), which is in neither _SECRET_SETTING_KEYS nor _is_secret_setting, so it falls through the same way. CRITICAL_SECRETS registers it under the SigningConfig field names private_key / private_key_password instead of the emitted sign_-prefixed names. That is a separate redaction-plus-fingerprint gap, not folded into this one-predicate fix; private_key_password is excused in the reverse gate with that reason. ASVS 13.3.4 stays partial. Cites ADR 0015 for the body-secret class (the wiring citation to an internal-ledger #236 does not resolve in public docs/BACKLOG.md). --- messagefoundry/config/wiring.py | 18 +++-- tests/test_secret_rotation_inventory.py | 103 +++++++++++++++++++++++- tests/test_secret_rotation_watcher.py | 27 +++++++ 3 files changed, 139 insertions(+), 9 deletions(-) diff --git a/messagefoundry/config/wiring.py b/messagefoundry/config/wiring.py index d23f892d..8b5c6de5 100644 --- a/messagefoundry/config/wiring.py +++ b/messagefoundry/config/wiring.py @@ -691,9 +691,13 @@ def _is_secret_setting(name: str) -> bool: #: They live in :data:`_SECRET_SETTING_KEYS` only so ``/metadata`` redacts them defence-in-depth (a #: username can leak directory structure). The **single source of truth** for "which secret settings are #: rotatable": imported by ``tests/test_secret_rotation_inventory.py`` (the ASVS-13.1.4 registration gate) -#: and read by :func:`connector_secret_env_values` (the ASVS-13.3.4 rotation fingerprinter), so the -#: redaction list, the doc registration gate, and the runtime fingerprint set can never disagree about -#: which members are credentials you rotate. +#: and read by :func:`connector_secret_env_values` (the ASVS-13.3.4 rotation fingerprinter). That the +#: redaction list, the doc registration gate, and the runtime fingerprint set agree about which members +#: are credentials you rotate is **enforced by two gates, not assumed**: the forward gate +#: (``test_secret_setting_keys_are_registered``) proves every rotatable key is registered, and the +#: reverse gate (``test_registered_connector_secrets_are_reachable_by_the_fingerprinter``, BACKLOG #1009) +#: proves every registered connector secret is reachable by the fingerprinter — the direction a +#: hand-added registry entry (the SOAP ``body_secret_value`` class) had slipped through. _NON_ROTATABLE_SECRET_SETTING_KEYS: frozenset[str] = frozenset( {"username", "basic_user", "proxy_user", "ws_username", "credential_username"} ) @@ -708,8 +712,10 @@ def connector_secret_env_values( each with the DEK-derived keyed MAC so a per-Connection connector credential is monitored for rotation exactly like the fixed ``MEFOR_*`` classes. - A setting is included when its key is a **rotatable** credential — in :data:`_SECRET_SETTING_KEYS` and - not a non-rotatable identifier in :data:`_NON_ROTATABLE_SECRET_SETTING_KEYS` — AND it is an ``env()`` + A setting is included when its key is a **rotatable** credential — recognised by + :func:`_is_secret_setting` (so the SOAP ``body_secret_value_`` prefix class is covered, not only + the fixed :data:`_SECRET_SETTING_KEYS` names) and not a non-rotatable identifier in + :data:`_NON_ROTATABLE_SECRET_SETTING_KEYS` — AND it is an ``env()`` ref whose key resolves to a **non-empty string** in ``env_values``. Values are returned **transiently** to be MAC'd — never persisted or logged; the map key is the operator-chosen env name, never the value. Connections sharing an env key collapse to one entry (one secret → one rotation clock). Inline (non- @@ -722,7 +728,7 @@ def connector_secret_env_values( specs += [c.spec for c in registry.outbound.values()] for spec in specs: for name, value in spec.settings.items(): - if name in _NON_ROTATABLE_SECRET_SETTING_KEYS or name not in _SECRET_SETTING_KEYS: + if name in _NON_ROTATABLE_SECRET_SETTING_KEYS or not _is_secret_setting(name): continue if isinstance(value, EnvRef): resolved = env_values.get(value.key) diff --git a/tests/test_secret_rotation_inventory.py b/tests/test_secret_rotation_inventory.py index a483e572..1e87927e 100644 --- a/tests/test_secret_rotation_inventory.py +++ b/tests/test_secret_rotation_inventory.py @@ -36,6 +36,7 @@ import re from pathlib import Path +from typing import Any _ROOT = Path(__file__).resolve().parent.parent _PKG = _ROOT / "messagefoundry" @@ -185,9 +186,12 @@ def test_secret_setting_keys_are_registered() -> None: # every one that is not a bare identifier in _NON_ROTATABLE_SECRET_SETTING_KEYS — must be a registered # critical secret with a rotation row, so a FUTURE connector credential added there trips this gate # until it is inventoried + documented. Both are imported from wiring (the single source of truth, - # ALSO read by the ASVS-13.3.4 runtime fingerprinter connector_secret_env_values, so the registration - # gate and the runtime rotation set can never disagree); imported locally so a collection-time wiring - # import error is contained to this one test. + # ALSO read by the ASVS-13.3.4 runtime fingerprinter connector_secret_env_values). This gate proves + # only the FORWARD direction (every _SECRET_SETTING_KEYS member is registered); that the registration + # gate and the runtime rotation set agree is ENFORCED, not self-evident -- the reverse is checked by + # test_registered_connector_secrets_are_reachable_by_the_fingerprinter (BACKLOG #1009), which is the + # direction body_secret_value slipped through (it entered CRITICAL_SECRETS by hand, never through + # _SECRET_SETTING_KEYS). Imported locally so a collection-time wiring import error is contained here. from messagefoundry.config.wiring import ( _NON_ROTATABLE_SECRET_SETTING_KEYS, _SECRET_SETTING_KEYS, @@ -210,6 +214,99 @@ def test_secret_setting_keys_are_registered() -> None: ) +# --- ASVS 13.3.4: the REVERSE of test_secret_setting_keys_are_registered (BACKLOG #1009) ---------- +# +# The forward gate proves _SECRET_SETTING_KEYS is registered. It is blind to the other direction: a +# registered CONNECTOR secret whose runtime setting never resolves through connector_secret_env_values +# (the ASVS-13.3.4 fingerprinter). The SOAP body_secret_value class entered CRITICAL_SECRETS by hand +# without passing through _SECRET_SETTING_KEYS, and the fingerprinter's bare-frozenset filter dropped it +# -- so a rotation of a SOAP injected body secret was not auto-detected, while the forward gate walked +# straight past (its comment even promised the two sets "can never disagree"). This gate closes that +# direction: every registered connector secret must be REACHABLE by the fingerprinter (ADR 0015). + +#: Registered connector secrets that name NO reachable connector setting, each with the reason. Kept +#: here (not only in prose) so an exclusion is a reviewed decision with a hygiene test behind it, exactly +#: like _NOT_FINGERPRINTED: silently removing a real reachable secret must not pass unnoticed. +_NOT_A_CONNECTOR_SETTING: dict[str, str] = { + "private_key_password": ( + "a per-message JWS SigningConfig passphrase (ADR 0018); the connector-level setting names are " + "sign_private_key / sign_private_key_password (transports/signing.py), neither a " + "_SECRET_SETTING_KEYS member, so connector_secret_env_values does not resolve it. Whether the " + "JWS signing key is itself fingerprinted/redacted is a real but SEPARATE question, deliberately " + "not folded into the one-predicate #1009 scope" + ), +} + +#: A registered connector secret whose runtime setting name is DYNAMIC -- the CRITICAL_SECRETS key is a +#: family label, not the literal setting the fingerprinter sees. Maps label -> a representative concrete +#: setting name in that family, so the reachability probe uses a name _is_secret_setting accepts. +_CONNECTOR_SETTING_REPRESENTATIVE: dict[str, str] = { + "body_secret_value": "body_secret_value_0", # Soap(body_secrets=...) -> body_secret_value_ +} + + +def test_registered_connector_secrets_are_reachable_by_the_fingerprinter() -> None: + """The missing reverse of test_secret_setting_keys_are_registered (BACKLOG #1009, ADR 0015). + + For every registered CONNECTOR secret (a CRITICAL_SECRETS key that is not a fixed ``MEFOR_*`` env + var), build a representative outbound whose settings name that secret and assert + ``connector_secret_env_values`` returns its env key -- so a future registry entry added by hand + cannot silently fall through the fingerprint filter the way ``body_secret_value`` did. + + Mutation (current bug): revert the ``config/wiring.py`` predicate to bare ``_SECRET_SETTING_KEYS`` + membership -> ``body_secret_value``'s probe key is absent from the result -> named in ``unreachable``. + Mutation (future-proofing): add a fabricated unreachable entry to ``CRITICAL_SECRETS`` -> it is named + in ``unreachable`` too. Both are genuine reds, not vacuous passes. + """ + from messagefoundry.config.models import ConnectorType + from messagefoundry.config.wiring import ( + ConnectionSpec, + EnvRef, + OutboundConnection, + Registry, + connector_secret_env_values, + ) + + connector_secrets = [ + k + for k in CRITICAL_SECRETS + if not k.startswith("MEFOR_") and k not in _NOT_A_CONNECTOR_SETTING + ] + reg = Registry() + env_values: dict[str, str] = {} + probes: dict[str, str] = {} # registered secret -> its unique probe env-value key + for secret in connector_secrets: + # The fingerprinter reads setting NAMES and ignores connector type, so a REST spec carrying an + # arbitrary secret setting name is a valid probe of the filter (no connector-type validation runs). + setting_name = _CONNECTOR_SETTING_REPRESENTATIVE.get(secret, secret) + probe = f"probe_env_{secret}" + settings: dict[str, Any] = {setting_name: EnvRef(key=probe)} + reg.outbound[f"OB_{secret}"] = OutboundConnection( + name=f"OB_{secret}", + spec=ConnectionSpec(type=ConnectorType.REST, settings=settings), + ) + env_values[probe] = f"SYNTH-{secret}" # synthetic; env key names are not PHI + probes[secret] = probe + + resolved = connector_secret_env_values(reg, env_values) + unreachable = sorted(secret for secret, probe in probes.items() if probe not in resolved) + assert not unreachable, ( + "registered connector secret(s) that connector_secret_env_values does not resolve -- a rotation " + "of them would not be fingerprinted (ASVS 13.3.4). Fix the wiring._is_secret_setting / " + "_SECRET_SETTING_KEYS coverage, or excuse each in _NOT_A_CONNECTOR_SETTING WITH the reason it " + f"names no connector setting: {unreachable}" + ) + + # Hygiene: the two curated maps must be disjoint and may name only registered CONNECTOR secrets, so + # neither can rot into a place to park an unreachable secret (mirrors the _NOT_FINGERPRINTED discipline). + overlap = sorted(set(_NOT_A_CONNECTOR_SETTING) & set(_CONNECTOR_SETTING_REPRESENTATIVE)) + assert not overlap, f"secret(s) both excused AND represented -- pick one: {overlap}" + for name in (*_NOT_A_CONNECTOR_SETTING, *_CONNECTOR_SETTING_REPRESENTATIVE): + assert name in CRITICAL_SECRETS and not name.startswith("MEFOR_"), ( + f"{name!r} in a curated connector-secret map is not a registered connector secret" + ) + + def test_scanner_flags_a_planted_secret(tmp_path: Path) -> None: # The guard must actually catch a new secret env var, so a real regression can't pass silently # (mirrors test_security_static.test_scanner_flags_a_planted_pattern). diff --git a/tests/test_secret_rotation_watcher.py b/tests/test_secret_rotation_watcher.py index 73a329a5..6925c384 100644 --- a/tests/test_secret_rotation_watcher.py +++ b/tests/test_secret_rotation_watcher.py @@ -23,7 +23,9 @@ InboundConnection, OutboundConnection, Registry, + Soap, connector_secret_env_values, + env, ) from messagefoundry.pipeline.secret_rotation import ( SecretRotationRunner, @@ -295,6 +297,31 @@ def test_connector_secret_env_values_skips_unresolved_and_empty() -> None: assert connector_secret_env_values(reg, {"present": "V", "blank": ""}) == {"present": "V"} +def test_connector_secret_env_values_includes_a_soap_body_secret() -> None: + # BACKLOG #1009 (ADR 0015): a SOAP ``body_secrets={token: env(...)}`` desugars to a top-level + # ``body_secret_value_`` EnvRef that reaches secrecy only through the prefix branch of + # ``_is_secret_setting`` -- it is NOT a literal ``_SECRET_SETTING_KEYS`` member. The fingerprinter + # must resolve it (a rotation of a SOAP injected body secret would otherwise go undetected while + # every sibling connector credential is tracked). Built from a real ``Soap()`` factory so the + # factory -> _hoist_body_secrets -> body_secret_value_0 -> fingerprint path is exercised end to end. + reg = Registry() + # Synthetic placeholder matching the .gitleaks.toml ``MF_IIS__`` allowlist (ADR 0015 + # amendment): a body-secret TOKEN is public by design -- it sits in committed Handler source and the + # transport swaps in the real env() credential at send time -- not a credential. >=16 chars to satisfy + # ``_BODY_SECRET_TOKEN_RE``. + token = "MF_IIS_BODY_9f2c41ab3d7e" + reg.outbound["OB_IIS"] = OutboundConnection( + name="OB_IIS", + spec=Soap( + url="https://api.example.com/svc", + body_secrets={token: env("iis_body_pw")}, + ), + ) + assert connector_secret_env_values(reg, {"iis_body_pw": "SYNTH-VAL"}) == { + "iis_body_pw": "SYNTH-VAL" + } + + # --- live-by-default: DEK tracked via the stamp when operator date unset ----- From c7abb13ba36b91028e168ac7a5e8ec18e2431632 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Thu, 6 Aug 2026 00:36:21 -0500 Subject: [PATCH 2/2] docs(backlog): flip #1009 to built (BACKLOG #1009) The body_secret_value rotation-fingerprint fix landed (config/wiring.py filters through _is_secret_setting, plus the reverse gate and the SOAP regression test), so #1009's banner flips from not-started to built. Banner line only. The ranked table, the four census distribution lines, and every other item's banner are untouched. The census was NOT recomputed. --- docs/BACKLOG.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index e3142125..217ec0ea 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -4378,14 +4378,15 @@ open. ## 1009. SOAP `body_secret_value_` is redacted, registered and documented — and never fingerprinted -> 🔢 **Filed 2026-08-04 — not started. Scored 2026-08-04 → P2.** Value **5/10** · Difficulty -> **2/10** · _fill-in_. `connector_secret_env_values`, the ASVS 13.3.4 runtime rotation -> fingerprinter, filters on bare `_SECRET_SETTING_KEYS` membership at `config/wiring.py:725`, -> while `body_secret_value_` reaches secrecy only through the prefix branch of -> `_is_secret_setting` (`:686`) — so a rotation of a SOAP injected body secret is not -> auto-detected the way every sibling class is, and the registration gate whose comment promises -> the two sets "can never disagree" walks straight past it; the fix is one line plus the reverse -> assertion that gate is missing. +> ✅ **Built 2026-08-05 — Scored 2026-08-04, P2.** Value **5/10** · Difficulty +> **2/10**. `connector_secret_env_values`, the ASVS 13.3.4 runtime rotation fingerprinter, now +> filters connector secrets through `_is_secret_setting` (`config/wiring.py:725`) instead of bare +> `_SECRET_SETTING_KEYS` membership, so the prefix-only `body_secret_value_` SOAP body-secret +> class is fingerprinted and a rotation of it is auto-detected the way every sibling class is. The +> missing reverse gate (`test_registered_connector_secrets_are_reachable_by_the_fingerprinter`) now +> asserts every registered connector secret is reachable by the fingerprinter, so the "can never +> disagree" invariant is enforced rather than assumed and a future hand-added registry entry cannot +> slip through (ADR 0015). **Cluster:** Security & Compliance. **Priority:** P2. **Verdict:** build. **Severity:** low — a monitoring gap on an opt-in connector secret class, **not** a disclosure.