Skip to content

[flaky-ci] Stabilize AndroidMessageHandler cancellation rewind test - #12712

Open
simonrozsival wants to merge 9 commits into
mainfrom
simonrozsival-message-handler-cancellation
Open

simonrozsival wants to merge 9 commits into
mainfrom
simonrozsival-message-handler-cancellation

Conversation

@simonrozsival

@simonrozsival simonrozsival commented Sep 8, 2026

Copy link
Copy Markdown
Member

Summary

  • run the cancellation/rewind regression and focused helpers from the non-SSL AndroidMessageHandlerCancellationTests fixture so NativeAOT includes them
  • use a controlled seekable stream that gates the first CopyToAsync only after its first destination write is flushed
  • require both a client-side flushed-write signal and a server-owned validated body-prefix read before cancellation
  • require the first request to observe OperationCanceledException while that content copy is explicitly blocked
  • abort the first HttpListenerResponse after client cancellation instead of draining an incomplete fixed-length body to EOF
  • retain the stream-position rewind assertion and make the second content copy ungated
  • read exactly the retry Content-Length, byte-validate the reused body, then respond without an additional EOF-dependent read
  • await and bound all request/server teardown, while only allowing cancellation-side cleanup failures
  • add focused regressions for buffered destination flushing and aborting a blocked HttpListener request-body read

Failure analysis

In Azure DevOps build 1584741, the stream-position assertion had already passed. The reported Broken pipe occurred on the second SendAsync because the retry server closed its response without consuming the 1 MB upload. This change removes that server/client race rather than changing AndroidMessageHandler product behavior.

Private review identified two synchronization pitfalls in earlier revisions:

  • ByteArrayContent.ReadAsStreamAsync() returns MemoryStream, whose optimized CopyToAsync can advance Position to Length before its destination write completes. The test now uses an explicit copy gate instead of inferring active upload state from Position.
  • Android HttpURLConnection/Okio can buffer the initial 4096-byte destination write. The controlled stream now awaits cancellation-aware FlushAsync before signaling progress, ensuring headers/body can reach the server while further copying remains blocked.

Azure DevOps build 1587086 then failed identically in Debug, Release, NoAab, and CoreCLRTrimmable because the first server handler drained the canceled fixed-length request to EOF. Managed HttpListener only reports body EOF after the declared Content-Length is consumed, and client cancellation does not guarantee an immediate FIN/RST, so its InputStream.ReadAsync can remain pending indefinitely. The server now validates one bounded body prefix, waits for the client to observe cancellation, and calls HttpListenerResponse.Abort() to terminate the context without draining.

The tests were also moved out of the SSL-tagged AndroidMessageHandlerTests fixture. TestInstrumentation excludes SSL when PublishAot is enabled, while the destination fixture's AndroidMessageHandlerCancellation and InetAccess categories are not excluded, so the regression now runs in NativeAOT.

References #12704.

Validation

  • git diff --check
  • project XML parse and explicit compile-item wiring for AndroidMessageHandlerCancellationTests.Rewind.cs
  • source/category check confirms the moved tests exist only in the non-SSL cancellation fixture; TestInstrumentation excludes SSL, not AndroidMessageHandlerCancellation or InetAccess, for PublishAot
  • standalone .NET 11 controlled-stream and server-abort harness: 20/20 iterations passed
    • verifies the progress gate fires only after buffered bytes are flushed and visible
    • holds a partial fixed-length request connection open, proves the next server read blocks, and verifies response abort terminates it
  • specialist review of the fixture move and the server/cancellation protocol: no findings
  • attempted ./dotnet-local.sh build -c Debug tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj; blocked by NETSDK1147 because this worktree's local SDK has no android workload installed
  • focused on-device execution was unavailable because ANDROID_HOME, adb, and an Android emulator/device are not configured on this machine

Track both server handlers, synchronize cancellation after upload begins, and fully validate the retried request body before responding.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@simonrozsival simonrozsival added the flaky-ci Intermittent CI failures and work to improve CI reliability label Sep 8, 2026
simonrozsival and others added 3 commits September 8, 2026 12:56
Replace MemoryStream position inference with an explicit destination-write gate and add focused coverage for cancellation and retry behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Flush the controlled stream's initial destination write before signaling progress and cover buffered destinations that hide unflushed bytes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use a server-observed body prefix before cancellation, abort the first response instead of waiting for EOF, and read exactly Content-Length on retry.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@simonrozsival

Copy link
Copy Markdown
Member Author

/review

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Android PR Reviewer completed successfully!

Generated by Android PR Reviewer for #12712

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Needs Changes

Findings: 0 errors, 1 warning, 0 suggestions.

The revised synchronization is substantially stronger: cancellation is gated on an actual flushed upload plus a server-validated prefix, retry consumption is exact and byte-validated, and cleanup bounds all outstanding work. The remaining issue is configuration coverage: the cancellation regression is still hosted in an SSL-tagged fixture and is therefore excluded from NativeAOT.

CI build 1587386 is still in progress; the completed CLA and Android Tools/Java.Interop checks are green, with platform builds currently running.

Generated by Android PR Reviewer for #12712 · gpt56 · 132.9 AIC · ⌖ 13.4 AIC · ⊞ 25.7K
Comment /review to run again

Move the cancellation rewind regression and its focused helpers out of the SSL-tagged handler fixture so PublishAot lanes execute them.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@simonrozsival

Copy link
Copy Markdown
Member Author

/review

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Android PR Reviewer completed successfully!

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection results could not be parsed.

Review the workflow run logs for details.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • azcliprod.blob.core.windows.net

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "azcliprod.blob.core.windows.net"

See Network Configuration for more information.

Generated by Android PR Reviewer for #12712

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection results could not be parsed.

Review the workflow run logs for details.

⚠️ Needs Changes

Findings: 0 errors, 0 warnings, 1 suggestion.

The move into AndroidMessageHandlerCancellationTests fixes the previous NativeAOT coverage problem: PublishAot excludes the old fixture’s SSL category, but not the destination fixture’s categories. The revised synchronization also waits for both client write progress and a server-validated prefix before cancellation, verifies rewind before retry, and bounds cleanup.

Azure build 1587832 is not green: Linux and macOS builds failed, while the Windows build is still running. Detailed Azure timeline/log access is unavailable here because Azure authentication is not configured, so I could not determine whether those failures are related to this PR; CI must be green before LGTM.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • azcliprod.blob.core.windows.net

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "azcliprod.blob.core.windows.net"

See Network Configuration for more information.

Generated by Android PR Reviewer for #12712 · gpt56 · 156.1 AIC · ⌖ 21.9 AIC · ⊞ 25.7K
Comment /review to run again

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@simonrozsival
simonrozsival marked this pull request as ready for review September 9, 2026 15:08
Copilot AI lite review requested due to automatic review settings September 9, 2026 15:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The changes are test-only and materially improve determinism; the remaining feedback is minor clarity/maintainability in the new test code.

Review tier: Lite
Findings: 2 Low severity

New issues introduced by this change (2)
Severity Finding
Low severity tests/​Mono.Android-Tests/​Mono.Android-Tests/​Xamarin.Android.Net/​AndroidMessageHandlerCancellationTests.Rewind.cs — The timeout-path failure message mentions "after releasing the request body", but this branch never…
Low severity tests/​Mono.Android-Tests/​Mono.Android-Tests/​Xamarin.Android.Net/​AndroidMessageHandlerCancellationTests.Rewind.cs — cancellationObserved.TrySetResult(true) is called in both the OperationCanceledException catch and…
What changed in this PR

Moves and rewrites the cancellation/rewind regression test to make it deterministic and runnable under NativeAOT lanes, eliminating a server/client race that was causing flaky failures during retry uploads.

Changes:

  • Replaces the previous rewind-after-cancellation test with a gated, seekable stream and a tighter client/server synchronization protocol (including server-side abort on cancellation).
  • Adds focused regressions covering buffered destination flushing and HttpListenerResponse.Abort() behavior for blocked request-body reads.
  • Wires the new rewind test file into the on-device test project and makes the cancellation fixture partial to host the new file.
File Description
tests/​Mono.Android-Tests/​Mono.Android-Tests/​Xamarin.Android.Net/​AndroidMessageHandlerTests.cs Removes the older rewind-after-cancellation test from the SSL-tagged fixture.
tests/​Mono.Android-Tests/​Mono.Android-Tests/​Xamarin.Android.Net/​AndroidMessageHandlerCancellationTests.Rewind.cs Adds the new deterministic rewind/cancellation regression plus helper regressions and controlled stream implementations.
tests/​Mono.Android-Tests/​Mono.Android-Tests/​Xamarin.Android.Net/​AndroidMessageHandlerCancellationTests.cs Makes the fixture partial so the rewind tests can live in a separate file.
tests/​Mono.Android-Tests/​Mono.Android-Tests/​Mono.Android.NET-Tests.csproj Adds an explicit <Compile Include=...Rewind.cs /> so the moved tests are included in the on-device test assembly.
Suppressed comments (1)

tests/Mono.Android-Tests/Mono.Android-Tests/Xamarin.Android.Net/AndroidMessageHandlerCancellationTests.Rewind.cs:344

  • ReleaseFirstCopy() unconditionally calls firstWriteCompleted.TrySetCanceled(); when the first write already completed successfully this no-ops, but it obscures the intent (only cancel the signal if it hasn't fired). Making it conditional reduces confusion when reasoning about task states in failures.
			public void ReleaseFirstCopy ()
			{
				releaseFirstCopy.TrySetResult (true);
				firstWriteCompleted.TrySetCanceled ();
			}

simonrozsival and others added 3 commits September 9, 2026 17:12
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

flaky-ci Intermittent CI failures and work to improve CI reliability needs-review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants