Skip to content

[MVP] Windows installer funnel telemetry (off by default) - #141

Draft
IliaMManolov wants to merge 6 commits into
mainfrom
ccr-324b73e0-ynx0j5-windows-installer-tracking
Draft

IliaMManolov wants to merge 6 commits into
mainfrom
ccr-324b73e0-ynx0j5-windows-installer-tracking

Conversation

@IliaMManolov

@IliaMManolov IliaMManolov commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

MVP for Ilia's review. Do not merge. It needs a GDPR/compliance review before any release build turns it on.

Summary

Users log in on the dashboard but drop off before the desktop app is installed and running. This PR adds anonymous funnel events so that we can see where a Windows install stops.

Event Sent by When
desktop_installer_download_clicked dashboard, Edison-Watch/edison-watch#1846 A desktop app download button is clicked
installer_started NSIS installer End of .onInit
installer_step install_clicked NSIS installer The user leaves the folder page
installer_step old_version_handled + exit_code NSIS installer The previous version's uninstaller finished (also fires on a fresh install)
installer_step files_installed NSIS installer End of the install section
installer_finished success / failed / cancelled (+ exit_code / error_level / blocking_dialog) NSIS installer The wizard closes or the install fails
installer_previous_run_incomplete NSIS installer The next run finds an earlier run that never finished (hard Quit paths, killed process)
app_first_launch app main process First start after install, with the same install_id
  • Off by default. It is compiled in only when SG_INSTALL_TELEMETRY_URL and SG_INSTALL_TELEMETRY_KEY are set at build time (scripts/installer-telemetry-config.mjs writes a git-ignored include; the build:win* scripts run it first). Without them, the installer logic is unchanged. No CI workflow sets them, so dev, demo and release all stay off until the review.
  • Own PostHog project. The events go to a separate installer funnel project with "Discard client IP data" on, never to the dashboard project. The docs list the settings to create ("Setting up the PostHog project").
  • Anonymous. install_id is a random GUID for each installer run. Events set $process_person_profile: false and $geoip_disable: true. No user, machine, path or account data is sent.
  • Cannot hang the install. The installer sends with WinHTTP in-process and uses the user's proxy settings. Timeouts are 1 to 1.5 s, including the separate response-header timeout, which defaults to 90 s. After one failed send, or one that takes over 1 s, the run sends nothing more. The app uses net.fetch with a 3 s timeout and does not block startup.
  • Opt-out: /NOTELEMETRY, DO_NOT_TRACK=1 or SEALGATE_NO_TELEMETRY=1. The app also sends nothing for custom, local and dev backends.

The full design, privacy notes, known gaps and a manual Windows VM test plan (cases A to N) are in packages/desktop/docs/installer-telemetry.md.

Per-download id

The edison-watch side (being built on #1846) serves the installer as SealGate-<ver>-<arch>-setup_dl-<16 hex>.exe, with a random id per download. The id is not stored and not linked to the user.

  • The installer reads the id from its own file name. A browser (1) suffix does not matter.
  • Fallback: the same _dl-<id> token in the HostUrl or ReferrerUrl line of its Zone.Identifier stream.
  • With an id, every installer event carries download_id and download_id_source, and the marker passes both to app_first_launch.
  • No id, or a malformed one, sends no field and changes nothing else.

Dashboard side

This is Edison-Watch/edison-watch#1846. It instruments the shared downloadClient helper, with the properties platform, found, app_version and asset. The per-download id joins the download to the install for each download. The dashboard event that is linked to the logged-in user does not carry the id.

Findings

  • electron-builder's hard Quit paths ("SealGate cannot be closed", locked files, decompression failed) skip all NSIS callbacks. They are now reported on the next attempt as installer_previous_run_incomplete.
  • If the old uninstaller fails, electron-builder shows a MessageBox with no /SD default, so a silent auto-update can wait on a dialog. The event carries blocking_dialog=true. The fix is the separate PR Stop silent updates from waiting on the "Failed to uninstall" dialog #142. Both PRs define customUnInstallCheck, so the second to merge gets a small conflict, which I will resolve.
  • customUnInstallCheck replaces electron-builder's default handling of the uninstaller result. The macro reproduces the app-builder-lib 26.x default, and nsisUninstallCheck.test.ts pins the upstream function.

Code review

A strict maintainability review ran on the diff. Fixed in f76282d:

  • The build:win* scripts now regenerate the config, so a stale test endpoint cannot ship.
  • seconds_since_install is now correct on retries.
  • installer.nsh is now the one owner of the electron-builder hooks.
  • The upstream-copy pin test.
  • The slow-send breaker.
  • upgrade no longer depends on the /currentuser flag.
  • No run record in the UAC inner instance.
  • The step is renamed to old_version_handled.
  • The app send is simpler.

Not taken: the app does not move to the env-config PostHog key. The marker keeps the installer's endpoint, so that the installer and the app always report to the same project.

Test plan

  • npm run typecheck, npx vitest run: 230 tests pass, 20 of them new.
  • Cross-built the NSIS installer on Linux (electron-builder + wine, -WX), with telemetry on and off.
  • Under wine, against a local mock endpoint (scripts/mock-capture-server.mjs), never the production PostHog or Sentry:
    • The clean install sends all steps and success. The upgrade sends upgrade=true. The cancel sends cancelled.
    • The forced old-uninstaller failure sends failed with exit_code=1.
    • A killed run is reported by the next run.
    • The telemetry-off build, /NOTELEMETRY and DO_NOT_TRACK all send 0 events.
    • A refused connection adds no time. An endpoint that never answers adds about 2 s and gets one request only.
    • Download id from the file name:
      • _dl-<16 hex>: found.
      • With a (1) suffix: found.
      • Plain name, a non-hex character, 17 digits: no field.
    • The full flow from download to first launch carries one id.
  • app_first_launch: Electron's main loop hangs under wine, so this step ran on the Linux build of the same app, reading the marker that the wine install wrote.
  • Manual run on a real Windows VM (docs, cases A to N). Case N, the Zone.Identifier fallback, cannot run under wine.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WiFcRTVRzq2KVKYAgvKyNK

The installer now reports anonymous funnel events (installer_started,
installer_step, installer_finished with outcome and exit code) to a PostHog
capture endpoint, and the app sends app_first_launch with the same random
install_id. It is compiled in only when SG_INSTALL_TELEMETRY_URL and
SG_INSTALL_TELEMETRY_KEY are set at build time, so release builds are
unchanged until the compliance review.

- build/installerTelemetry.nsh: WinHTTP sender with short timeouts, NSIS
  hooks, /NOTELEMETRY and DO_NOT_TRACK opt-outs
- scripts/installer-telemetry-config.mjs: writes the git-ignored switch file
- scripts/mock-capture-server.mjs: local stand-in for PostHog in tests
- src/main/infra/installTelemetry.ts: first-launch event via net.fetch
- docs/installer-telemetry.md: funnel, privacy notes, Windows VM test plan

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WiFcRTVRzq2KVKYAgvKyNK
- installer_previous_run_incomplete: the run id and last step stay in
  HKCU\Software\SealGate\InstallTelemetry until the run reports its end, so
  the hard Quit paths (locked files, app cannot be closed, killed process)
  show up on the next attempt
- cap WINHTTP_OPTION_RECEIVE_RESPONSE_TIMEOUT (90 s by default) and set
  timeouts on the request handle; an endpoint that never answers hung the
  installer before this
- stop sending for the rest of the run after the first failed request
- use the user's proxy settings (WINHTTP_ACCESS_TYPE_AUTOMATIC_PROXY)
- blocking_dialog on a silent run whose old uninstaller failed
- real uninstall removes the telemetry registry key and marker
- mock server: MOCK_CAPTURE_HANG=1 black-hole mode
- docs: per-build PostHog plan, more VM test cases

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WiFcRUnhb9PzJb7nDzwNL9
- build:win scripts run the config script first, so a stale test config
  can never ship in a later local build
- seconds_since_install keeps the install time across retries
- one owner for electron-builder hooks: installer.nsh defines
  customInstall/customPageAfterChangeDir and calls sgIt* macros that are
  empty when telemetry is off; drop the SG_IT_HOOK_INIT indirection
- pin electron-builder's handleUninstallResult in a test, since
  customUnInstallCheck reproduces its default by hand
- circuit breaker also trips on a send slower than 1 s (slow proxy discovery)
- upgrade comes from the registry folders, not the /currentuser flags
- no run record in the elevated inner instance
- rename old_version_checked to old_version_handled (fires on fresh installs)
- simpler app send: typed PostFn, no Promise.race, one log line

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WiFcRTVRzq2KVKYAgvKyNK
The installer and first-launch events never use the dashboard key. The docs
list the project settings to create (Discard client IP data on, no
autocapture or replay, short retention) and how a workflow turns it on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WiFcRTVRzq2KVKYAgvKyNK
The dashboard can serve the installer as
SealGate-<ver>-<arch>-setup_dl-<16 hex>.exe. The installer reads the id from
its file name (Zone.Identifier HostUrl/ReferrerUrl as a fallback) and sends
download_id on every event; the marker passes it to app_first_launch. No id,
or a malformed one, sends no field and changes nothing else.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WiFcRTVRzq2KVKYAgvKyNK
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants