Repository navigation
Tell AI agents to pass the device code login on to the user - #7235
Conversation
|
Commit: Regression test reportSelected 2 changed tests (limit=20, 0 not selected) 2 tests: regression=2 unreleased=0 coverage=0 failing=0
TestAccept/cmd/auth/login/device-code/non-interactive/errors: branch ✅ | main (df305f2) ❌ | latest (v1.23.0) ❌main (df305f2): latest (v1.23.0): TestAccept/cmd/auth/login/device-code/non-interactive/nominal: branch ✅ | main (df305f2) ❌ | latest (v1.23.0) ❌main (df305f2): latest (v1.23.0): |
Integration test reportCommit: db9fab4
Top 6 slowest tests (at least 2 minutes):
|
6636d4c to
03b79ec
Compare
03b79ec to
35d4fe6
Compare
35d4fe6 to
c8a6169
Compare
c8a6169 to
03acd36
Compare
03acd36 to
46f6bad
Compare
simonfaltum
left a comment
There was a problem hiding this comment.
Thanks, the note reads well and AgentNotice prints the same text as before. The main thing is the terminal-flow note, which I don't think will work for agents (inline). The rest are small.
| agentNote := deviceCodeAgentNote + "Ask the user for the confirmation code their browser shows, and enter it\n" + | ||
| `at the "Confirmation code" prompt of this command.` |
There was a problem hiding this comment.
An agent won't have an interactive terminal, so I don't think this note will work for it. Without a terminal it already gets the two-step flow and never sees this text. If the command does run in a PTY (some IDE agent terminals), the agent usually can't type into a running command, so it waits at the prompt until the login times out.
Could agent detection pick the two-step flow (start, then rerun with --user-code and --confirmation-code) even when stdin is a terminal? Then this variant of the note can go, along with the README line that says agent detection doesn't pick the flow. (If the note stays, "Confirmation code" here duplicates the AskCancellable label below, and .agents/rules/style-guide-go.md asks for a shared constant.)
There was a problem hiding this comment.
Some agents are able to start commands in a terminal mode, e.g. codex:
I started the command in an interactive terminal (tty: true). When it paused for input, the tool returned a session ID while keeping the
process running.
I then used write_stdin with that session ID to send your code followed by \n, which acts like pressing Enter:
write_stdin({
session_id: 48408,
chars: "XXX\n"
})
f52b31d to
1f1f5fe
Compare
1f1f5fe to
9702fd7
Compare
simonfaltum
left a comment
There was a problem hiding this comment.
No blocking findings. Reviewed against device-code-agent-login; the change preserves both login flows and the existing AgentNotice output. The PTY explanation addresses the earlier terminal-flow concern: Codex can keep the command running and supply the confirmation code through write_stdin.
Validation: go test ./libs/agent ./cmd/auth ./internal/testutil and the changed non-interactive/nominal and non-interactive/errors acceptance tests pass locally. CI is green, including the cross-platform test matrix and integration tests.
Non-blocking nit at cmd/auth/login_device_code.go:261: the new wrong-code agent note makes the returned error end with a period, contrary to AGENTS.md's error-string rule. Drop the final period and update the snapshot.
When an AI agent is detected, the first step of the non-interactive login also prints a note that tells it to give the URL and the user code to the user, not to approve the login itself, and to ask the user for the confirmation code. Co-authored-by: Isaac <no-reply@databricks.com>
The device code login built the "Note for AI agents (<name>): " prefix by hand, like AgentNotice does. agent.Note now adds it for both, and returns an empty string when no agent is detected. AgentNotice prints the same text as before. Co-authored-by: Isaac <no-reply@databricks.com>
An agent that runs the login in a terminal, e.g. through a pseudo-terminal, got the interactive prompt without the note, so nothing told it to pass the login on to the user. The note is now part of the approval instructions that both logins print. The prompt tests clear the agent environment, so that running them from an agent does not change the output. Co-authored-by: Isaac <no-reply@databricks.com>
The interactive device code login printed the note of the non-interactive one, which tells the agent to ask the user for the confirmation code but not what to do with it. That login waits for the code at its prompt, so its note now tells the agent to enter the code there. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Each note is used once, so it moves next to its use. Co-authored-by: Isaac <no-reply@databricks.com>
The fake server now issues a new user code for every device code login, so the agent login in the nominal test gets WDJB-0004. Co-authored-by: Isaac <no-reply@databricks.com>
The SDK reports no agent for an empty AGENT, so setting it unconditionally keeps the behavior and makes t.Setenv reject parallel tests, as the comment says. Co-authored-by: Isaac <no-reply@databricks.com>
SDK v0.191.0 detects Pi through PI_CODING_AGENT, so the nominal device code login test showed (pi) instead of (test-agent) when run inside Pi. Co-authored-by: Isaac <no-reply@databricks.com>
…code The agent runs the second step of the non-interactive login, so it reads the error that says to approve the login. The note tells it to ask the user to approve it instead. Co-authored-by: Isaac <no-reply@databricks.com>
Error strings must not end with punctuation. Co-authored-by: Isaac <no-reply@databricks.com>
793e6dd to
db9fab4
Compare
Stacked on #7220.
Changes
When an AI agent is detected (
useragent.AgentProvider()), device code login prints a note after the user code that tells the agent to give the URL and the code to the user, not to open the URL or approve the login itself, and to ask the user for the confirmation code. In the interactive login, the note also tells the agent to enter the code at theConfirmation codeprompt. Agent detection does not change the flow.The note uses the new
agent.AgentNote, whichAgentNoticenow uses too, with its text unchanged. The newtestutil.SetAgentkeeps the agent running the tests out of them.Why
The login must be approved by the user, but an agent that runs it gets the URL and the code and could try to approve it itself, or not pass it on. Split out of #7220 to keep that PR smaller.
Tests
The
device-code/non-interactive/nominalacceptance test runs both steps withAGENTset.TestPromptForConfirmationCode_Agentchecks the interactive note, andlibs/agenttests coverAgentNoteandAgentNotice.This pull request and its description were written by Isaac.