Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ When this plugin is hosted by different CLIs:
- `aioTelemetry` (optional object in the root `package.json` of the **host CLI** — the same `pjson` oclif passes into the init hook):
- `postUrl` (optional string): HTTPS URL of the telemetry proxy that receives POSTed metric batches. Use this when your CLI should send telemetry to a different App Builder action or gateway than the plugin default.
- `fetchHeaders`: Optional extra headers merged into telemetry requests (`Content-Type` is always set)
- `productPrivacyPolicyLink`: A link to display to users when prompting to opt in
- `productName`: How to refer to the CLI when the user is prompted to enable telemetry (from `displayName` or `name` in `package.json`)
- `productPrivacyPolicyLink`: A link shown in the one-time telemetry notice (what is collected and how to opt out)
- `productName`: How to refer to the CLI in the telemetry notice (from `displayName` or `name` in `package.json`)
- `productBin`: Shown in help text (from `bin` in `package.json`; if several bins exist, the first is used). Example: run `${productBin} telemetry on`

### Overriding the telemetry POST URL
Expand Down Expand Up @@ -80,7 +80,7 @@ The resolved URL is passed to the flush worker on each telemetry send; it applie

Telemetry is suppressed when `AIO_TELEMETRY_DISABLED` is set to one of **`true`**, **`1`**, or **`yes`** (exact match; case-sensitive). Other values such as `0`, `false`, `no`, or an empty string do **not** disable telemetry via this variable.

This does not change the persisted opt-in state. Useful for CI pipelines and scripted environments.
This does not change the persisted opt-out state. Useful for CI pipelines and scripted environments.

```sh
AIO_TELEMETRY_DISABLED=true aio app deploy
Expand All @@ -94,7 +94,7 @@ Telemetry is **best-effort**: events are not persisted when the proxy is down or

On **`postrun`**, any in-memory metrics from earlier hooks in the same command are merged with the `postrun` metric and the combined batch is handed off to a **fire-and-forget detached subprocess** (`src/flush-worker.js`). The parent CLI spawns the worker and immediately `unref()`s it, so the CLI can exit without waiting for the HTTP POST. If the POST fails (network error or non-2xx response), the batch is dropped; telemetry must not block or slow normal CLI use.

Non-`postrun` events (for example `command-error`, `telemetry-prompt`) are held in an **in-memory buffer** until that flush. If the process exits before `postrun` (crash, `SIGKILL`), buffered events are lost. The buffer is cleared when telemetry is disabled or when `init` runs again (new command session).
Non-`postrun` events (for example `command-error`, `telemetry-notice`) are held in an **in-memory buffer** until that flush. If the process exits before `postrun` (crash, `SIGKILL`), buffered events are lost. The buffer is cleared when telemetry is disabled or when `init` runs again (new command session).

## Agent detection

Expand Down Expand Up @@ -122,7 +122,7 @@ To opt into agent tracking without setting a tool-specific variable, set `AIO_IN

## POST data

The `eventData` attribute is always a string. Objects and arrays are stored as a JSON text (e.g. `"{}"`, `"{\"message\":\"…\"}"`). String payloads (such as telemetry prompt outcomes `accepted` / `declined`) are stored as that plain text without an extra layer of JSON quoting. Numbers and booleans use their usual string forms (`"0"`, `"false"`).
The `eventData` attribute is always a string. Objects and arrays are stored as a JSON text (e.g. `"{}"`, `"{\"message\":\"…\"}"`). String payloads (such as the telemetry notice outcome `shown`) are stored as that plain text without an extra layer of JSON quoting. Numbers and booleans use their usual string forms (`"0"`, `"false"`).

Example shape of the metric payload:

Expand Down
3 changes: 1 addition & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@
"@adobe/aio-lib-core-networking": "^5.0.4",
"@oclif/core": "^4",
"ci-info": "^4.0.0",
"debug": "^4.1.1",
"inquirer": "^8.2.1"
"debug": "^4.1.1"
},
"devDependencies": {
"@adobe/eslint-config-aio-lib-config": "5.0.0",
Expand Down
13 changes: 6 additions & 7 deletions src/hooks/init.js
Original file line number Diff line number Diff line change
Expand Up @@ -37,17 +37,16 @@ module.exports = async function (opts) {
opts.argv.filter(arg => arg.indexOf('-') === 0).join(','),
global.prerunTimer)

// init event does not post telemetry, it stores some info that will be used later
// this will prompt to optIn/Out if telemetry.optIn is undefined
// Telemetry is opt-out (on by default). On the first run we show a one-time,
// non-blocking notice instead of an opt-in prompt. isNull() is true only until the
// notice records the default, so this shows once.
if ((opts.argv.indexOf('--no-telemetry') < 0) &&
!inCI &&
telemetryLib.isNull()) {
// let's ask!
// unfortunately the `oclif-dev readme` run by prepack fires this event, which hangs CI
// Also we don't prompt for telemetry if the first command is a telemetry command because they
// are probably setting it on or off already
// skip in CI (handled above) and when oclif-dev readme runs (it fires this event);
// also skip for telemetry commands, where the user is already setting state.
if (['readme', 'telemetry'].indexOf(opts.id) < 0) {
return telemetryLib.prompt(productName, binName, opts?.config?.pjson?.aioTelemetry?.productPrivacyPolicyLink)
telemetryLib.notice(productName, opts?.config?.pjson?.aioTelemetry?.productPrivacyPolicyLink)
}
}
}
41 changes: 12 additions & 29 deletions src/telemetry-lib.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,10 @@ const path = require('path')
const os = require('os')
const config = require('@adobe/aio-lib-core-config')

const inquirer = require('inquirer')
const debug = require('debug')('aio-telemetry:telemetry-lib')

/** Adobe I/O App Builder web action that forwards CLI metrics to New Relic (ingest key stays server-side). */
const DEFAULT_TELEMETRY_POST_URL = 'https://53444-aioclitelemetryproxy-stage.adobeio-static.net/api/v1/web/dx-excshell-1/telemetry'
const DEFAULT_TELEMETRY_POST_URL = 'https://53444-aioclitelemetryproxy.adobeio-static.net/api/v1/web/dx-excshell-1/telemetry'

/** @returns {boolean} Whether `AIO_TELEMETRY_DISABLED` opts out (only the literal string `"true"`). */
function isEnvTelemetryDisabled () {
Expand Down Expand Up @@ -120,6 +119,10 @@ const getOnMessage = (productName, binName) => {
const getOffMessage = (binName) => {
return `\nTelemetry is off.\nIf you would like to turn telemetry on, simply run \`${binName} telemetry on\``
}
const getNoticeMessage = (productName, privacyPolicyLink) => {
return `${productName} collects anonymous usage data to help us improve our products.\n` +
`Telemetry is on by default; read what we collect and how it is used here: ${privacyPolicyLink || defaultPrivacyPolicyLink}`
}

/**
* Builds the value stored in the metric `eventData` attribute. For `postrun`, an empty object is
Expand Down Expand Up @@ -186,7 +189,7 @@ async function trackEvent (eventType, rawEventData = {}) {

const optedOut = isDisabledForCommand || isEnvTelemetryDisabled() || config.get(`${configKey}.optOut`, 'global') === true
const willSend = !optedOut
debug('trackEvent %s eventData=%j postUrl=%s willSend=%s', eventType, eventData, postUrl, willSend)
debug(`trackEvent ${eventType} eventData=${JSON.stringify(eventData)} postUrl=${postUrl} willSend=${willSend}`)

if (optedOut) {
pendingCommandMetrics.length = 0
Expand Down Expand Up @@ -276,7 +279,7 @@ module.exports = {
config.set(`${configKey}.optOut`, true)
},
isEnabled: () => {
return !isDisabledForCommand && !isEnvTelemetryDisabled() && config.get(`${configKey}.optOut`, 'global') === false
return !isDisabledForCommand && !isEnvTelemetryDisabled() && config.get(`${configKey}.optOut`, 'global') !== true
},
disableForCommand: () => {
isDisabledForCommand = true
Expand All @@ -296,30 +299,10 @@ module.exports = {
},
getOnMessage,
getOffMessage,
prompt: async (productName, binName, privacyPolicyLink) => {
console.log(`
How you use ${productName} provides us with important data that we can use
to make our products better. Please read our guide for more information on
the data we anonymously collect, and how we use it.
${privacyPolicyLink || defaultPrivacyPolicyLink}
`)

const response = await inquirer.prompt([{
name: 'accept',
type: 'confirm',
message: `Would you like to allow ${productName} to collect anonymous usage data?`
}])
if (response.accept) {
config.set(`${configKey}.optOut`, false)
console.log(getOnMessage(productName, binName))
trackEvent('telemetry-prompt', 'accepted')
} else {
// we will set optOut to true after tracking this one event
// todo: check if tty error
config.set(`${configKey}.optOut`, false)
console.log(getOffMessage(binName))
trackEvent('telemetry-prompt', 'declined')
config.set(`${configKey}.optOut`, true)
}
getNoticeMessage,
notice: (productName, privacyPolicyLink) => {
console.log(getNoticeMessage(productName, privacyPolicyLink))
config.set(`${configKey}.optOut`, false)
trackEvent('telemetry-notice', 'shown')
}
}
86 changes: 26 additions & 60 deletions test/hooks.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,8 @@
*/

const { createFetch } = require('@adobe/aio-lib-core-networking')
const inquirer = require('inquirer')
const config = require('@adobe/aio-lib-core-config')

jest.mock('inquirer')
jest.mock('@adobe/aio-lib-core-config')
jest.mock('child_process', () => ({
spawn: jest.fn(() => ({ unref: jest.fn() }))
Expand All @@ -31,10 +29,16 @@ const mockPackageJson = {
}

describe('hook interfaces', () => {
let noticeSpy
beforeEach(() => {
fetch.mockReset()
spawn.mockClear()
config.get.mockReset()
config.set.mockClear()
noticeSpy = jest.spyOn(telemetryLib, 'notice')
})
afterEach(() => {
noticeSpy.mockRestore()
})

test('command-error', async () => {
Expand Down Expand Up @@ -65,114 +69,77 @@ describe('hook interfaces', () => {
expect(bodyNf[0].metrics.map((m) => m.attributes.eventType)).toEqual(['command-not-found', 'postrun'])
})

/**
* Should prompt when config.get(optOut) returns undefined
* post results
*/
test('init prompt accept:true', async () => {
test('init shows one-time notice on first run', async () => {
const preEnv = process.env
process.env = { ...preEnv, CI: undefined, GITHUB_ACTIONS: undefined }
const hook = require('../src/hooks/init')
expect(typeof hook).toBe('function')
inquirer.prompt = jest.fn().mockResolvedValue({ accept: true })
config.get = jest.fn().mockReturnValue(undefined)
await hook({ config: { name: 'name', version: '0.0.1', pjson: mockPackageJson }, argv: [] })
expect(inquirer.prompt).toHaveBeenCalled()
expect(noticeSpy).toHaveBeenCalled()
expect(config.set).toHaveBeenCalledWith('aio-cli-telemetry.optOut', false)
expect(spawn).not.toHaveBeenCalled()
await telemetryLib.trackEvent('postrun')
expect(spawn).toHaveBeenCalledTimes(1)
const flushPayloadAcc = JSON.parse(spawn.mock.calls[0][1][1])
const bodyAcc = JSON.parse(flushPayloadAcc.body)
expect(bodyAcc[0].metrics.map((m) => m.attributes.eventType)).toEqual(['telemetry-prompt', 'postrun'])
expect(bodyAcc[0].metrics[0].attributes.eventData).toBe('accepted')
expect(bodyAcc[0].metrics.map((m) => m.attributes.eventType)).toEqual(['telemetry-notice', 'postrun'])
expect(bodyAcc[0].metrics[0].attributes.eventData).toBe('shown')
process.env = preEnv
})

test('init prompt - full coverage when run by gh actions', async () => {
test('init - no notice for telemetry commands', async () => {
const preEnv = process.env
process.env = { ...preEnv, CI: undefined, GITHUB_ACTIONS: undefined }
const hook = require('../src/hooks/init')
expect(typeof hook).toBe('function')
inquirer.prompt = jest.fn().mockResolvedValue({ accept: true })
config.get = jest.fn().mockReturnValue(undefined)
await hook({ id: 'telemetry', config: { name: 'name', version: '0.0.1' }, argv: [] })
expect(inquirer.prompt).not.toHaveBeenCalled()
expect(noticeSpy).not.toHaveBeenCalled()
expect(spawn).not.toHaveBeenCalled()
process.env = preEnv
})

test('init prompt - dont ask for telemetry for telemetry commands', async () => {
const hook = require('../src/hooks/init')
expect(typeof hook).toBe('function')
inquirer.prompt = jest.fn().mockResolvedValue({ accept: true })
config.get = jest.fn().mockReturnValue(undefined)
await hook({ id: 'telemetry', config: { name: 'name', version: '0.0.1' }, argv: [] })
expect(inquirer.prompt).not.toHaveBeenCalled()
expect(spawn).not.toHaveBeenCalled()
})

test('init prompt - dont run when oclif is generating readme', async () => {
const hook = require('../src/hooks/init')
expect(typeof hook).toBe('function')
inquirer.prompt = jest.fn().mockResolvedValue({ accept: true })
config.get = jest.fn().mockReturnValue(undefined)
await hook({ id: 'readme', config: { name: 'name', version: '0.0.1' }, argv: [] })
expect(inquirer.prompt).not.toHaveBeenCalled()
expect(spawn).not.toHaveBeenCalled()
})

test('init prompt - dont run when oclif is generating readme and CI is off', async () => {
test('init - no notice when oclif is generating readme', async () => {
const preEnv = process.env
process.env = { ...preEnv, CI: undefined }
process.env = { ...preEnv, CI: undefined, GITHUB_ACTIONS: undefined }
const hook = require('../src/hooks/init')
expect(typeof hook).toBe('function')
inquirer.prompt = jest.fn().mockResolvedValue({ accept: true })
config.get = jest.fn().mockReturnValue(undefined)
await hook({ id: 'readme', config: { name: 'name', version: '0.0.1' }, argv: [] })
expect(inquirer.prompt).not.toHaveBeenCalled()
expect(noticeSpy).not.toHaveBeenCalled()
expect(spawn).not.toHaveBeenCalled()
process.env = preEnv
})

test('no prompt when process.env.CI', async () => {
test('init - no notice when process.env.CI', async () => {
const preEnv = process.env
process.env = { ...preEnv, CI: 'true' }
let hook
jest.isolateModules(() => {
hook = require('../src/hooks/init')
})

expect(typeof hook).toBe('function')
inquirer.prompt = jest.fn().mockResolvedValue({ accept: false })
config.get = jest.fn().mockReturnValue(undefined)
expect(inquirer.prompt).not.toHaveBeenCalled()
await hook({ config: { name: 'name', version: '0.0.1' }, argv: ['--verbose'] })
expect(noticeSpy).not.toHaveBeenCalled()
expect(spawn).not.toHaveBeenCalled()
expect(inquirer.prompt).not.toHaveBeenCalled()
process.env = preEnv
})

/**
* Should prompt when config.get(optOut) returns undefined
* should still post after prompt even though it is declined, this is the last post
* When the user has already chosen a state (optOut defined), isNull() is false,
* so the notice is not shown again.
*/
test('init prompt accept:false', async () => {
test('init - no notice when telemetry state already set', async () => {
const preEnv = process.env
process.env = { ...preEnv, CI: undefined, GITHUB_ACTIONS: undefined }
const hook = require('../src/hooks/init')
expect(typeof hook).toBe('function')
inquirer.prompt = jest.fn().mockResolvedValue({ accept: false })
config.get = jest.fn().mockReturnValue(undefined)
await hook({ config: { name: 'name', version: '0.0.1', pjson: mockPackageJson }, argv: ['--verbose'] })
expect(inquirer.prompt).toHaveBeenCalled()
config.get = jest.fn().mockReturnValue(false) // optOut already set -> isNull() false
await hook({ config: { name: 'name', version: '0.0.1', pjson: mockPackageJson }, argv: [] })
expect(noticeSpy).not.toHaveBeenCalled()
expect(spawn).not.toHaveBeenCalled()
telemetryLib.enable()
await telemetryLib.trackEvent('postrun')
expect(spawn).toHaveBeenCalledTimes(1)
const flushPayloadDec = JSON.parse(spawn.mock.calls[0][1][1])
const bodyDec = JSON.parse(flushPayloadDec.body)
expect(bodyDec[0].metrics.map((m) => m.attributes.eventType)).toEqual(['telemetry-prompt', 'postrun'])
expect(bodyDec[0].metrics[0].attributes.eventData).toBe('declined')
process.env = preEnv
})

Expand Down Expand Up @@ -210,13 +177,12 @@ describe('hook interfaces', () => {
* Should NOT prompt even though config.get(optOut) returned undefined
* --no-telemetry flag wins
*/
test('init --no-telemetry no prompt', async () => {
test('init --no-telemetry no notice', async () => {
const hook = require('../src/hooks/init')
expect(typeof hook).toBe('function')
inquirer.prompt = jest.fn()
config.get = jest.fn().mockReturnValue(undefined)
await hook({ config: { name: 'name', version: '0.0.1' }, argv: ['--no-telemetry'] })
expect(inquirer.prompt).not.toHaveBeenCalled()
expect(noticeSpy).not.toHaveBeenCalled()
expect(spawn).not.toHaveBeenCalled()
})

Expand Down
1 change: 0 additions & 1 deletion test/index.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ const TheCommand = require('../src/commands/telemetry')
const { stdout } = require('stdout-stderr')
const config = require('@adobe/aio-lib-core-config')

jest.mock('inquirer')
jest.mock('@adobe/aio-lib-core-config')

const fetch = createFetch()
Expand Down
12 changes: 12 additions & 0 deletions test/telemetry-lib.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,18 @@ describe('telemetry-lib', () => {

expect(telemetryLib.getOnMessage).toBeDefined()
expect(telemetryLib.getOnMessage).toBeInstanceOf(Function)

expect(telemetryLib.getNoticeMessage).toBeInstanceOf(Function)
expect(telemetryLib.notice).toBeInstanceOf(Function)
})

test('getNoticeMessage uses the default privacy link, or a provided one', async () => {
const withDefault = telemetryLib.getNoticeMessage('Adobe Developer CLI')
expect(withDefault).toMatch('on by default')
expect(withDefault).toMatch('developer.adobe.com/app-builder/docs/guides/telemetry')

const withLink = telemetryLib.getNoticeMessage('Adobe Developer CLI', 'https://example.com/privacy')
expect(withLink).toMatch('https://example.com/privacy')
})

test('exports init function', async () => {
Expand Down
Loading