Skip to content

feat(project): accept a Storage token without the command line - #728

Open
soustruh wants to merge 2 commits into
mainfrom
feat/token-input-without-shell-history
Open

feat(project): accept a Storage token without the command line#728
soustruh wants to merge 2 commits into
mainfrom
feat/token-input-without-shell-history

Conversation

@soustruh

@soustruh soustruh commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Why

A project authenticates in one of two ways: a browser-login session (kbagent auth login, then the interactive auth register-projects picker), or a static Storage token. For a person at a browser the session path needs no token on the command line.

The session path does not cover every case. A static token is still used for:

  • CI, containers, and headless runs
  • an agent that must set or replace one project's token
  • the conversion of a session project to a static one, through project edit --token

On the static-token path, project edit --project X --token <value> was the only way to enter the token. The value then sits in three places the user did not choose: the shell history, the kbagent REPL history file, and a process listing. project add was better. On a terminal it prompts with hidden input. But it could not read a token from a file or a pipe, so the two commands did not match.

This matters most for agents. An agent cannot type a token for the person, and the session path needs a human at a browser. On the static-token path it therefore writes the command instead, and the obvious command teaches the unsafe habit.

What changed

Both commands now take the same four token sources. If you pass two of them, the command exits 2.

flag behavior
--token-stdin Shows a hidden prompt on a terminal. Reads to EOF on a pipe. Uses the existing read_password_stdin(), which already backs --password-stdin.
--token-file PATH Reads the file, then deletes it once the command succeeds. Warns when other users can read the file.
--token-env NAME Reads the variable you name. kbagent reads no variable unless you name it.
--token VALUE Works as before. On a terminal it warns that the token is now in the shell history.

--keep-token-file stops the delete. Use it for a read-only mount, or for a file you read more than once.

AGENT_CONTEXT, gotchas.md and keboola-expert.md now carry one ordered rule:

  1. If the person can run the command, give them --token-stdin.
  2. If you must run it yourself, ask them to write the token to a file and pass --token-file.
  3. In CI, pass --token-env.
  4. Use --token VALUE only when nothing else works.

Nothing breaks

project add still reads KBC_TOKEN on its own. The existing test for that path passes unchanged.

KBAGENT_PROJECT_FROM_ENV=1 plus KBC_TOKEN plus KBC_STORAGE_API_URL is a different code path in ConfigStore.load(). This PR does not change it. Its tests pass unchanged.

Three deliberate deviations, for the reviewer

  1. project edit still ignores KBC_TOKEN, and now says why. edit_project() overwrites the alias's project_id and project_name with the values the new token verifies as. Nothing checks that the token belongs to the same project. An exported KBC_TOKEN could therefore repoint an alias at a different project during a plain --new-alias, and report success. --token-env KBC_TOKEN reads the variable on request.
  2. --token warns at run time. auth login-password states the same risk in the help text of --password only. Help text reaches the person who reads it. The warning reaches the person who does not.
  3. --token-file deletes its input. No other -file flag does this: --git-pat-file, --secrets-file and --public-key-file all keep the file. Here the file carries a credential exactly once. This is also the only path where an agent can take a token from a person without a cleanup step for the person to remember. The help text states that unlink removes the directory entry and does not scrub the bytes.

Not in this PR

serve keeps one plain token field on ProjectCreate and ProjectEdit. A pipe, a local file path and a local variable name have no meaning over HTTP. No route changes, so there is no endpoint reference to regenerate.

This PR bumps no version and adds no changelog.py entry, per CONTRIBUTING. New gotchas.md entries carry the (since vNEXT) tag.

One trap worth reading

token_came_from_command_line() matches the ParameterSource member by name. Typer bundles its own copy of Click, so click.core.ParameterSource and the enum a running context returns are different classes. A == between them is always False. The first version compared them that way. The warning never appeared, --token no longer conflicted with the other sources, and every test still passed. TestTokenCameFromCommandLine pins this.

Testing

make check passes. New coverage:

  • every source resolves to the token, and every pair of sources conflicts
  • the file goes away on success, and survives --dry-run, a failure, and --keep-token-file
  • a rejected unlink warns instead of failing the command
  • the session-project conversion warning still appears when the token arrives by pipe
  • no source prints the token to stdout or stderr

@soustruh soustruh left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of #728 — feat(project): accept a Storage token without the command line

Generated by kbagent-pr-reviewer subagent. Verdict and findings below are advisory; the human author retains every veto. CI-coverable issues (lint, format, tests) are confirmed via make check, not duplicated here.

Summary

The PR gives project add and project edit three new mutually-exclusive ways to supply a Storage token (--token-stdin, --token-file, --token-env) alongside the existing --token, so an agent never has to hand a person a command with the token typed on the line. No CLI command is added, removed, or renamed — only flags on two existing commands — so the high-risk "new command" branch of the Plugin synchronization map does not apply, and the PR still updates every hand-maintained doc surface that does apply (CLAUDE.md, commands/context.py, keboola-expert.md §1/§2, commands-reference.md, gotchas.md with (since vNEXT) tags). make check passes clean (6454 tests). Verdict: COMMENT — no blocking findings, two non-blocking items worth a look before or shortly after merge.

Verdict

  • Verdict: COMMENT
  • Blocking findings: 0
  • Non-blocking findings: 2
  • Nits: 2

Blocking findings

(none)

Non-blocking findings

[NB-1] src/keboola_agent_cli/commands/_helpers.py:204-225_read_token_file's permission check is not skipped on Windows, unlike its two siblings in this codebase

auth/state_store.py:_fix_permissions_if_needed and services/doctor_service.py's config-file check both special-case os.name == "nt" before inspecting st_mode, because Windows does not expose POSIX permission bits the same way (Path.stat().st_mode on Windows commonly reports something like 0o666 for an ordinary file, independent of any real ACL). _read_token_file has no such guard, so mode & 0o077 will very likely be non-zero for every --token-file invocation on Windows (a documented, supported install platform per README.md "### Windows"), producing a spurious "is readable by other users... should be 0600" warning on every use. Not tested on an actual Windows box, but the inference follows directly from Python's documented stat() behavior there and the precedent already in this repo. Fix: mirror the os.name == "nt" skip used by the two sibling checks.

[NB-2] src/keboola_agent_cli/commands/project.py (project_add/project_edit) — no E2E test exercises the new token-input flags

tests/test_e2e.py uses project add --token ... pervasively as setup scaffolding for other tests, but no new/updated E2E test drives --token-stdin, --token-file, or --token-env against a real Keboola stack. The new resolution logic is covered thoroughly at the CLI (tests/test_cli.py, tests/test_project_edit_cli.py) and helper (tests/test_helpers.py) layers with a mocked client, which is good coverage for the branching logic itself, but per CONTRIBUTING.md "every CLI command must have E2E coverage" the new input paths on these two commands have never been exercised against the real Storage API verify-token call. Given the mocked-layer coverage is already extensive, this is a reasonable one-cycle deferral rather than a merge blocker.

Nits

  • [NIT-1] CLAUDE.md:449-462 — the terse ## All CLI Commands signature lines for project add / project edit list --token-stdin | --token-file PATH | --token-env NAME | --token TOKEN but omit --keep-token-file, unlike sibling entries in the same file that tend to be exhaustive (e.g. org setup's [--dry-run] [--yes] [--token-description PREFIX] [--refresh]). It is documented in commands/context.py and commands-reference.md, so this is cosmetic only.
  • [NIT-2] src/keboola_agent_cli/commands/_helpers.py:216-224 — the world-readable check mode & 0o077 also fires on a group/other write-without-read permission combination (e.g. mode 0620), where the printed message "is readable by other users" is not quite accurate (it would actually be writable, not readable, by that class). Edge case, wording only.

Verification log

  • Read CONTRIBUTING.md §"Checklist: Adding a New CLI Command", §"Plugin synchronization map", §"Releasing a new version" → loaded, confirmed no new/removed/renamed command in this diff so the command-surface rows do not trigger.
  • Read CLAUDE.md convention #17 + ## All CLI Commands → loaded.
  • Read plugins/kbagent/agents/keboola-expert.md §1 Rule 6 and §3 → loaded; confirmed Rule 6's project add --token reference was updated to --token-stdin in the diff.
  • gh auth status → logged in as soustruh, token scopes repo, read:org, gist
  • gh pr view 728 --json title,body,files,additions,deletions,baseRefName,headRefName,labels,statestate: OPEN, 11 files, +1012/-78, title feat(project): accept a Storage token without the command line (conventional, type matches: new flag behavior = feat) ✓
  • git rev-parse --abbrev-ref HEADfeat/token-input-without-shell-history, matches <branch>; git log -1f8b587c..., matches PR head ✓
  • gh pr diff 728 → saved, 1441 lines, read in full ✓
  • git diff main...HEAD -- src/keboola_agent_cli/cli.py 'src/keboola_agent_cli/commands/**/*.py' | grep '@.*_app\.command' → empty ✓ (confirms no command added/removed/renamed, so OPERATION_REGISTRY / server routers / hints-definitions rows of the sync map are not triggered by this PR)
  • Layer-violation greps (typer/click in services, httpx in commands, formatter/typer in clients) → all empty ✓ no layer violation
  • grep "project.add\|project.edit" src/keboola_agent_cli/permissions.py"project.add": "admin", "project.edit": "admin" already present (pre-existing, unaffected) ✓
  • find src/keboola_agent_cli -iname "*hint*" → no hints/ module exists in this repo; that row of the review playbook does not apply here
  • grep "token" src/keboola_agent_cli/server/routers/projects.py → plain token: str field unchanged, matches PR description's disclosed "Not in this PR" deviation ✓
  • make check (full target: lint, format-check, typecheck, skill-check, version-check, version-gate-check, command-sync-check, endpoints-check, changelog-check, check-error-codes, check-sentinel-guards, loc-check, test) → exit 0, 6454 passed, 12 skipped; scripts/check_command_sync.py → "OK: all 268 CLI commands are registered (OPERATION_REGISTRY) and documented" ✓; check_file_size.py flags commands/project.py as 989 lines over the 800 soft ceiling, but this is a pre-existing WARN (non-fatal) and the PR's net addition to that file is small (+58/-43) — not treated as a finding
  • uv run kbagent project add --help → confirmed rendered help text matches the diff's option declarations exactly (label text, mutual-exclusion wording) ✓
  • Reproduced manually against a scratch config dir: kbagent project add --token X --token-file Y → exit 2, Specify exactly one of --token / --token-file, file survives ✓ matches test_project_add_rejects_two_token_sources
  • Reproduced: --token-file on a 0644 file against a project add that fails verification (fake token, real API call) → prints "is readable by other users (mode 0644)" warning, error message masks the token (sometoken-...epro), and the token file is not deleted on failure ✓ matches test_a_failed_edit_keeps_the_token_file semantics
  • Reproduced: --token-env NONEXISTENT_VAR → exit 2, Environment variable NONEXISTENT_VAR_XYZ is unset or empty
  • grep read_password_stdin( src/ tests/ → all three existing call sites (dev_portal.py x2, auth.py) call it with zero args, compatible with the new label: str = "Password: " default parameter — no backward-compat break ✓
  • gotchas.md new entries verified individually: all three carry (since vNEXT) and are bullet points, not headings, so they are unaffected by the "no vNEXT in headings" anchor-slug rule ✓
  • Security/convention greps (magic numbers, bare except, raw error-code literals outside tests, print() in production code, token leakage without masking) → all clean; the one error_code="INVALID_TOKEN" hit is in a test file, consistent with the existing test-only pattern used across 4 other test files in this repo ✓

Open questions for the author

(none)

`project add` and `project edit` now take --token-stdin, --token-file
and --token-env. None of them writes the token to the shell history, to
the kbagent REPL history file, or to a process listing. --token still
works. On a terminal it now warns that the token is in the history.

--token-file deletes the file after the command succeeds. It keeps the
file on --dry-run, after a failure, and with --keep-token-file.

`project edit` does not read KBC_TOKEN. A new token overwrites the
alias's project_id and project_name with the values the token verifies
as. Nothing checks that the token belongs to the same project, so an
exported variable could repoint an alias during a plain --new-alias.
`project add` reads KBC_TOKEN as before.
@soustruh
soustruh force-pushed the feat/token-input-without-shell-history branch from f8b587c to 9d8ab30 Compare September 1, 2026 22:50
@soustruh

soustruh commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Addressed in 9d8ab30; make check stays green.

  • NB-1 (Windows mode check): fixed. _read_token_file now skips the permission check when os.name == "nt", the same guard auth/state_store.py and services/doctor_service.py already use. A new test (test_permission_check_is_skipped_on_windows) pins it.
  • NIT-2 (warning wording): fixed. The message now says "accessible to other users", which also fits a write-without-read mode such as 0620.
  • NIT-1 (--keep-token-file in CLAUDE.md): fixed. Both project add and project edit signature lines now list it.
  • NB-2 (no E2E on the new flags): deferred one cycle, as suggested. The branching logic is covered at the CLI and helper layers with a mocked client. The new input paths need a real Storage verify-token call, so an E2E test lands separately rather than in this PR.

@soustruh
soustruh marked this pull request as ready for review September 2, 2026 11:43
The Windows CI job runs the suite, and three of the new token-input
tests failed there. Each assumed POSIX filesystem behavior that Windows
does not share. Production code is unchanged; the nt guard added in the
previous commit is correct.

- test_unreadable_file_is_rejected used chmod(0o000), which does not make
  a file unreadable on Windows. It now passes a directory, so read_text
  raises OSError on every platform.
- test_world_readable_file_warns and test_private_file_does_not_warn
  forced os.name to "posix", which defeated the nt guard and read
  Windows' fabricated 0o666 bits. They now skip on Windows, as
  test_auth_state_store.py and test_doctor_service.py already do for
  POSIX-permission tests.
- test_permission_check_is_skipped_on_windows monkeypatched os.name to
  "nt". It now runs only on a real Windows host.
- test_unlink_failure_warns_but_does_not_raise used a read-only
  directory, which does not block deletion on Windows. It now passes a
  directory, so unlink raises OSError on every platform.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant