Skip to content

feat(data-app): let secrets-set write plain (unencrypted) env vars (#738) - #741

Draft
padak wants to merge 1 commit into
mainfrom
claude/issue-738-plaintext-env-vars
Draft

feat(data-app): let secrets-set write plain (unencrypted) env vars (#738)#741
padak wants to merge 1 commit into
mainfrom
claude/issue-738-plaintext-env-vars

Conversation

@padak

@padak padak commented Sep 4, 2026

Copy link
Copy Markdown
Member

What

data-app secrets-set now accepts a bare KEY=VALUE alongside #KEY=VALUE. The # prefix decides encryption instead of being mandatory, mirroring the UI's "optional encryption" checkbox.

kbagent data-app secrets-set --project P --app-id 74021026 \
  --secret '#API_KEY=s3cr3t' \
  --secret 'SCORING_BACKUP_TAG=ppl-assessment-db'

Why

parameters.dataApp.secrets has always held both kinds of entry, and secrets-list / secrets-get / secrets-remove have handled both since 0.43.9. Only writing a plain key was impossiblesecrets-set demanded the # and encrypted unconditionally.

That left two bad options for a non-secret setting:

  • Encrypt it. This is write-only: the Encryption API has no decrypt endpoint, so once a Storage Files tag is encrypted, nobody — operator, UI or CLI — can read back what it was set to. secrets-list and config detail both show <encrypted>.
  • sync pull, hand-edit _config.yml, sync push.

Design notes

  • A payload with no # key never calls the Encryption API. An unreachable Encryption API must not fail a write that needs no encryption.
  • A KBC:: value is rejected under EITHER form. Under # it would double-encrypt; under a bare key it would park a foreign project's ciphertext in the config where nothing can ever decrypt it. (Previously this guard existed only for # keys, since bare keys could not be written.)
  • --allow-plaintext-on-encrypt-failure is unrelated and unchanged. It covers a # key whose encryption failed, still warns loudly, and its plaintext_written response key stays distinct from the new plaintext_keys (deliberate plain entries).
  • Additive response shape. secrets_set still lists every written key, so existing parsers are unaffected; encrypted_keys / plaintext_keys are new. secrets-list entries gain encrypted and valuevalue is null for every encrypted entry, preserving the never-decrypt boundary.
  • --secrets-file follows the same rule (a JSON object whose #-prefixed keys are encrypted and whose bare keys are written in clear).

File placement

The key/value validation loop moved out of set_data_app_secrets into partition_secret_entries in services/_data_app_bodies.py: services/data_app_service.py is grandfathered by scripts/check_file_size.py and may only shrink, so the feature had to make room for itself (1641 → 1635 code lines).

Behavior change

One existing test parametrization changed: "BAD" (no #) is no longer a malformed key for secrets-set — it is a valid plain env var. _validate_secret_key(require_hash=True) still exists and still rejects it; secrets-set simply no longer passes that mode. Covered by a renamed test that says so.

How it was tested

  • 9 new unit tests: plain-only write skips the Encryption API entirely, mixed payload encrypts only the # half, pre-existing entries of both kinds survive the read-modify-write, KBC:: refusal on a bare key, dry-run shows the plain value while masking the encrypted one, secrets-list tagging, and CLI plumbing (bare key forwarded; an entry without = still exits 2).
  • One live E2E case (test_data_app_secrets_set_writes_plain_env_vars) that writes a mixed payload against a real project KMS and asserts via config detail that the plain value is stored verbatim while its sibling is KBC:: ciphertext.
  • make check green (6576 tests).

Docs updated: CLAUDE.md, context.py AGENT_CONTEXT, SKILL.md (regenerated), commands-reference.md, gotchas.md. Version-gated as vNEXT per the release process.

Fixes #738

…738)

`parameters.dataApp.secrets` has always held two kinds of entry, and the
Keboola UI offers exactly that choice ("optional encryption"): a `#`-prefixed
key is encrypted under the project KMS, a bare key is a plain env var whose
value stays readable. `secrets-list` / `secrets-get` / `secrets-remove` have
handled both since 0.43.9 -- only WRITING a plain key was impossible, because
`secrets-set` demanded the `#` and encrypted unconditionally.

Storing a non-secret (a Storage Files tag, a feature flag, a log level) then
had two bad options: encrypt it -- write-only, since the Encryption API has no
decrypt endpoint, so nobody can ever read back what it was set to -- or
`sync pull`, hand-edit `_config.yml`, `sync push`.

- The `#` now DECIDES encryption instead of being mandatory: `#KEY=VALUE` is
  encrypted (unchanged), `KEY=VALUE` is written verbatim. Same rule for
  `--secrets-file`.
- A payload with no `#` key never calls the Encryption API at all, so an
  unreachable API cannot fail a write that needs no encryption.
- A `KBC::` value is still rejected under EITHER form: under `#` it would
  double-encrypt, under a bare key it would park a foreign project's
  ciphertext where nothing can decrypt it.
- `--allow-plaintext-on-encrypt-failure` is unrelated and unchanged -- it
  covers a `#` key whose encryption FAILED, and `plaintext_written` stays
  distinct from the new `plaintext_keys`.
- Response gains `encrypted_keys` / `plaintext_keys`; `secrets_set` still
  lists both, so existing parsers are unaffected. `secrets-list` entries gain
  `encrypted` and `value` (`null` for every encrypted entry) and human output
  prints plain values inline.

The validation loop moved to `partition_secret_entries` in
`services/_data_app_bodies.py` -- `data_app_service.py` is grandfathered by
the file-size gate and may only shrink.

Tested: 9 new unit tests (plain-only write skips encryption, mixed payload
encrypts only the `#` half, existing entries of both kinds survive, `KBC::`
refusal on a bare key, dry-run masking, list tagging, CLI plumbing) plus a
live E2E case asserting the plain value lands verbatim in the config while its
sibling lands as `KBC::` ciphertext. `make check` green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

data-app secrets-set: support plaintext (unencrypted) environment variables

1 participant