telemetry: emit standard toolkit auth metrics for parity with other IDE extensions - #577
Draft
laileni-aws wants to merge 1 commit into
Draft
telemetry: emit standard toolkit auth metrics for parity with other IDE extensions#577laileni-aws wants to merge 1 commit into
laileni-aws wants to merge 1 commit into
Conversation
…uration, didLoadModule login) The plugin did not report the standard toolkit auth telemetry that the VS Code and JetBrains extensions report, so the authentication experience of the Eclipse plugin could not be compared with theirs. Emit the same events: - aws_loginWithBrowser on every login and re-authentication, with the result, the start url, the credential type, whether it renewed an existing connection, and the duration of the previous session for that start url. The latest successful login is recorded in the plugin store so the next login can report how long the session lived for; a first login, and the first login after a sign out, report no duration because there is no previous session to measure. A failed login reports a short reason code, never an error message or a path. - auth_userState with source startup, reporting whether the plugin started connected, expired, or not connected. The state restored from the plugin store is optimistic, so the metric is reported for the first state that follows it, which is the outcome of the re-authentication performed on start up. - toolkit_didLoadModule with module login, reporting that the login view loaded or the reason code of the failure that stopped it from loading. The re-authentication view of this plugin is built from native widgets rather than a web view, so it has no equivalent load step and reports no toolkit_didLoadModule event. The telemetry definitions this plugin generates its metric builders from predated toolkit_didLoadModule and the session duration of aws_loginWithBrowser, so both are added from the shared definitions. An optional reason is added to both metrics as well, matching what the other extensions report on them. The generated builder types session duration as a primitive int, which cannot hold a duration of more than roughly twenty five days in milliseconds and which serializes to zero when it is not set. Sessions are expected to live for months, so the entry is written with the exact millisecond value, and left out entirely when no previous session is known.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Eclipse plugin does not emit the standard toolkit authentication telemetry that the VS Code and JetBrains extensions emit, so its authentication experience cannot be compared with theirs — in particular how long an authentication session lives before the user has to sign in again, and whether the login UI loads.
Solution
Emit the same events the other extensions do:
aws_loginWithBrowseron every interactive login and re-authentication, withresult,credentialStartUrl,credentialType,isReAuth, andsessionDuration. The instant of the latest successful browser login is kept in the plugin store per start URL, so the next login reports how long the previous session lived. A first login, and the first login after a sign-out, report no duration because there is no previous session to measure; a sign-out clears the recorded login. A failed login reports a short reason code (the exception type name) — never an error message or a file path.loginOnInvalidToken=false, no browser involved) does not emit this event and does not reset the recorded login, so start-ups are not counted as browser logins.auth_userStatewithsource=startup, reporting whether the plugin startedconnected,expired, ornotConnected. The state restored from the plugin store is optimistic, so the metric is reported for the first state that follows it — the outcome of the re-authentication performed on start-up. Emitted once per plugin session.toolkit_didLoadModulewithmodule=login, reporting that the login view loaded (Succeeded, on the webview'sonLoadcallback) or the reason code of the failure that prevented it (DependencyMissing,AssetLoadFailed).The re-authentication view in this plugin is built from native SWT widgets rather than a webview, so it has no equivalent load step and emits no
toolkit_didLoadModuleevent.Telemetry definitions
The bundled
commonDefinitions.jsonpredatedtoolkit_didLoadModuleand thesessionDurationfield ofaws_loginWithBrowser, so both are added from the shared definitions, along with an optionalreasonon both metrics to match what the other extensions report.The generated builder types
sessionDurationas a primitiveint, which cannot hold a duration longer than ~25 days in milliseconds and serializes to0when unset. Sessions are expected to live for months, so the entry is written with the exact millisecond value and omitted entirely when no previous session is known.Testing
mvn package— BUILD SUCCESS, 524 tests, 0 failures (checkstyle and coverage checks included).New unit tests cover:
sessionDurationreported only when a previous login exists for the same start URL; omitted on first login; not overwritten by a failed loginaws_loginWithBrowserSucceeded / Failed (with exception-type reason) /isReAuthaws_loginWithBrowserand no timestamp update on the silent start-up token refreshauth_userStateemitted once on start-up with the correct status (notConnected immediately when nothing is stored; connected/expired after the start-up re-authentication resolves)toolkit_didLoadModulelogin failure paths (assets missing / cannot be served)AuthPluginStoretimestamp persistence, start-URL matching, and clearing on sign-outNot covered by an automated test: the
onLoadsuccess emission inLoginViewActionHandler, because that handler depends on Eclipse preference services that are unavailable under plain surefire. It was verified by inspection.