Distinguish proxied tool exits from lstk errors in command telemetry - #499
Draft
joe4dev wants to merge 2 commits into
Draft
Distinguish proxied tool exits from lstk errors in command telemetry#499joe4dev wants to merge 2 commits into
joe4dev wants to merge 2 commits into
Conversation
Co-Authored-By: Claude <noreply@anthropic.com>
joe4dev
force-pushed
the
devx-1004-distinguish-proxied-tool-exits-from-lstk-errors-in-command
branch
2 times, most recently
from
September 10, 2026 12:48
483514f to
e2f5cec
Compare
Co-Authored-By: Claude <noreply@anthropic.com>
joe4dev
force-pushed
the
devx-1004-distinguish-proxied-tool-exits-from-lstk-errors-in-command
branch
from
September 10, 2026 13:00
e2f5cec to
8a19bc0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
lstk awsfails for two very different reasons: an lstk preflight error, or the AWS CLI itself rejecting the user's command. Telemetry recorded both identically, so the error dashboard ranks users' own CLI mistakes as lstk errors — that is how "exit status 252" became the no. 1 lstk error (DEVX-1003).Solution
lstk_commandevents gainresult.proxy_error, true when the failure came from a wrapped tool the user asked for. It describes the error's origin, not whether a tool was proxied — a successfullstk aws s3 lsisfalse.The discriminator is not an
*exec.ExitErrorin the chain, as the ticket proposed: lstk shells out for its own purposes too, and those exits are lstk's failures. Instead each proxy exec site declares ownership viaproc.MarkUserToolExit. Forgetting it on a future proxy only loses a data point; marking one of lstk's own execs would hide an lstk bug behind the user's name.That forced one split:
azurecli.Execserved both thelstk azpassthrough and lstk's ownsetup azure/interception calls. It is nowExec(the user's, marks) andRun(lstk's, does not) over a sharedexecAz.proxy_erroris deliberately notomitempty, so absence means only "emitted before this field existed".The user's tool failed →
proxy_error: trueexit_codelstk aws s3 lsslstk aws s3 lsagainst a service errorlstk terraform applyon a bad configlstk <extension> …exiting non-zeroThe exit code is always the wrapped tool's own, unchanged.
lstk failed →
proxy_error: falseexit_codelstk aws s3 ls, Docker downruntime not healthy— never reached the toollstk aws s3 ls, emulator not runninglstk aws s3 ls, successlstk shells out, but the failure is still lstk's →
proxy_error: falseThese are the cases the rejected
*exec.ExitErrordiscriminator would have got wrong. The user never typed these invocations and cannot fix them by changing their command line.error_msglstk setup azureaz cloud listcould not list Azure clouds: az […]: exit status 1lstk az stop-interceptionaz cloud showcould not determine the active Azure cloud: az […]: exit status 1lstk update(Homebrew install)brew upgradeupdate failed: exit status 1lstk terraform init(S3 backend)aws s3api create-bucketcreating state bucket …: exit status NNote the contrast:
lstk awswith the AWS CLI exiting 252 giveserror='exit status 252'andproxy_error: true, whilelstk updatewithbrewexiting 1 giveserror='update failed: exit status 1'andproxy_error: false. Both carry an*exec.ExitErrorand both read "exit status N" — only the call site knows which is the user's.For the analytics side
proxy_error, notproxiedas the ticket specifies.proxiedreads as "this invocation was proxied", which is false for a successfullstk aws s3 lsthat the field would still markfalse. Renaming needs a nod from whoever builds the dashboard panel.lstk terraform apply, CItimeout) recordsproxy_error: true, exit_code: -1. Not an lstk error, but not a usage error either — excludeexit_code == -1.proxy_erroralone, since the field is failure-only per the ticket. It still needs the proxy command-name list.Also
The integration env inherited an ambient
LOCALSTACK_DISABLE_EVENTS=1, silently disabling every telemetry assertion. 43 integration tests that fail locally for developers who export it now pass. Separate commit.Docs
Nothing user-facing. No command, flag, output, env var, or documented behavior changes — this adds a field to the internal
lstk_commandanalytics payload, which no file underdocs/describes. The consumer is the analytics dashboard, not the CLI user.Review
Human review advised — it changes what the analytics error ranking means, and the
azureclisplit toucheslstk setup azureand interception, whose integration tests skip without a working Azure CLI.Related
Resolves DEVX-1004