Add SafeDict/SafeList pillar wrapping with Pydantic secrets, output redaction, and no_log#68907
Open
Akm0d wants to merge 9 commits intosaltstack:masterfrom
Open
Add SafeDict/SafeList pillar wrapping with Pydantic secrets, output redaction, and no_log#68907Akm0d wants to merge 9 commits intosaltstack:masterfrom
Akm0d wants to merge 9 commits intosaltstack:masterfrom
Conversation
…edaction, and no_log - Declare pydantic in base requirements and document behavior in the pillar tutorial. - Introduce salt.utils.safepillar: SafeDict, SafeList, SecretStr/SecretBytes, unwrap for cache, literal collection, and redaction helpers. - Wrap compiled pillar on minion-facing paths (remote/async pillar, cache read path, State pillar, gen_modules, pillar.items / pillar.ext). - Redact known pillar string literals in per-state returns and in state.* job returns/logs on the minion. - Add runtime no_log on state chunks (via STATE_RUNTIME_KEYWORDS) to mask comment / changes. - Add unit tests for safe containers and integration tests for redaction / no_log (run under full salt-factories CI).
Unwrap minion_blackout_whitelist before comparing to function_name so SecretStr entries match str job names.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR adds context-aware masking so sensitive pillar data is much harder to leak into state return payloads, minion logs, and event/log forwarding, while keeping explicit pillar APIs useful for debugging.
Highlights
pydantic>=2.4in base requirements;SecretStr/SecretBytesfor string/byte leaves.salt.utils.safepillar:SafeDict/SafeList,wrap_pillar_tree/unwrap_pillar_tree, literal collection, substring redaction,no_logmasking.no_log: trueon a state chunk (runtime keyword) masks that chunk’scommentandchangesregardless of whether the secret text appears in pillar literals.salt-callfrom thin keeps working (fixesModuleNotFoundError: pydantic/ follow-on import errors in CI).pillar.itemsandpillar.getreturn plain structures (unwrap_pillar_tree) so operators still see real values when they intentionally call those functions; in-memory pillar and accidental stringification stay protected.pillar.rstupdates.Fixes: #67367 (mask pillar in state output / logs).
Previous Behavior
{'name': 'curl https://api.example.com', 'comment': 'Request failed: invalid key sk-live-abc123', 'changes': {'stdout': 'error: sk-live-abc123'}, 'result': False}New Behavior
{'name': 'curl https://api.example.com', 'comment': 'Request failed: invalid key **********', 'changes': {'stdout': 'error: **********'}, 'result': False}2)
no_log: trueon a stateBefore:
commentandchangesare whatever the state module returned (may include secrets not present as pillar literals).After: both are masked to the placeholder shape:
3) In-memory pillar vs
pillar.items/pillar.getstr(pillar['db']['password'])→**********(safe if something logs or stringifies the object).pillar.items-style unwrap →{'db': {'password': 'plain-in-pillar'}}(plain values for intentional inspection).Merge requirements
pillar.rst)test_thin_dir)Reviewer notes
isinstance(x, str)for every pillar leaf may need to acceptSecretStror call.get_secret_value()after unwrap at boundaries.