[flaky-ci] Make logcat capture bounded and non-gating - #12708
simonrozsival wants to merge 6 commits into
Conversation
Bound adb device discovery and logcat collection inside a reusable helper so stalled diagnostics preserve partial output without setting the job to SucceededWithIssues. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
simonrozsival
added
the
flaky-ci
Copy redirected adb streams explicitly and give them a bounded completion window after process exit or termination so complete and partial captures are not truncated. Add repeated burst-output and timeout-partial regression coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
/review |
|
✅ Android PR Reviewer completed successfully!
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ LGTM
Findings: 0 errors · 0 warnings · 1 suggestion
The bounded process handling preserves complete output on success and partial output on timeout, while the pipeline integration keeps diagnostic capture non-gating. The existing timeout, failure, output-volume, and partial-output cases are well covered; I left one inline suggestion to cover the inherited-pipe/output-drain edge case. All 44 checks are green.
Generated by Android PR Reviewer for #12708 · gpt56 · 90.6 AIC · ⌖ 13.8 AIC · ⊞ 25.7K
Comment /review to run again
Cover a fake adb descendant that inherits stdout after its parent exits, proving the output drain remains bounded and preserves output written before the pipe stays open. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…timeout-resilience
The production helper runs in every APK instrumentation lane, so avoid adding a separate fake-adb test step to each package-test pipeline run. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
simonrozsival
added
the
ready-to-review
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The new Azure Pipelines warning helper doesn’t fully escape reserved characters in ##vso[...] logging commands, which can corrupt or truncate warnings for some messages.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Lite
Findings: 1
New issues introduced by this change (1)
| Severity | Finding |
|---|---|
 |
build-tools/automation/scripts/CaptureLogcat.ps1 — ❌ Azure Pipelines logging commands require escaping ] and ; in addition to %, CR, and LF;… |
What changed in this PR
This PR updates the APK instrumentation pipeline diagnostics so adb logcat -d capture is explicitly time-bounded and cannot change the job’s final gating status, addressing CI flakiness where logcat capture can stall.
Changes:
- Replaces the inline bash-based
adb devices/adb logcat -dcapture with a reusable PowerShell helper invocation. - Adds
CaptureLogcat.ps1, which bounds device discovery and logcat collection, attempts bounded termination of hungadbprocesses, and emits Azure Pipelines warnings while always exiting successfully.
| File | Description |
|---|---|
| build-tools/automation/yaml-templates/apk-instrumentation.yaml | Switches logcat capture to the new bounded PowerShell helper so diagnostics aren’t job-gating. |
| build-tools/automation/scripts/CaptureLogcat.ps1 | Implements bounded adb devices and adb logcat -d collection with timeout/termination handling and warning-only reporting. |
| [string] $Message | ||
| ) | ||
|
|
||
| $escapedMessage = $Message.Replace('%', '%AZP25').Replace("`r", '%0D').Replace("`n", '%0A') |
jonathanpeppers
left a comment
jonathanpeppers
left a comment
There was a problem hiding this comment.
This has a merge conflict, maybe a recent PR from mmitche also tried to fix this?
Resolve the logcat capture conflict by preserving the diagnostics added in #12744 while keeping capture-only timeouts non-gating. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
| [int] $OutputDrainTimeoutSeconds = 5 | ||
| ) | ||
|
|
||
| $ErrorActionPreference = 'Stop' |
There was a problem hiding this comment.
Same question here, should we introduce powershell? Or use inline bash in yaml?
adb logcat -dcan stall even afteradb devicesreports a connected emulator. Azure Pipelines then applies the one-minute task timeout added by #12471, records an error, and converts the task toSucceededWithIssuesbecause it usescontinueOnError. The finalfail-on-issue.yamlstep intentionally turns that status into a job failure. This occurred in builds 1583273 and 1585745. The log-volume reduction in #12620 does not prevent adb or emulator communication from stalling.Add a reusable PowerShell helper that bounds device discovery to 10 seconds and logcat collection to 45 seconds. It redirects logcat directly to the existing artifact path, preserving complete output on success and partial output on timeout, kills timed-out process trees with a bounded grace period, emits an explicit Azure warning, and exits successfully for capture-only failures.
The pipeline keeps
condition: always()but no longer relies on task-level timeout orcontinueOnError, so diagnostic capture cannot changeAgent.JobStatus.fail-on-issue.yamlremains unchanged and continues to gate unrelated build and test failures.Related: #12704
Prior mitigations: #12471, #12620