Skip to content

Add reproducer for check raw file injection smoke-test flake#11361

Closed
bm1549 wants to merge 1 commit into
masterfrom
brian.marks/reproduce-raw-file-injection-flake
Closed

Add reproducer for check raw file injection smoke-test flake#11361
bm1549 wants to merge 1 commit into
masterfrom
brian.marks/reproduce-raw-file-injection-flake

Conversation

@bm1549
Copy link
Copy Markdown
Contributor

@bm1549 bm1549 commented May 13, 2026

What Does This Do

Adds OutputThreadsTest, a JUnit 5 reproducer for the dominant flake of check raw file injection in LogInjectionSmokeTest (31 of 41 reports in CI Visibility):

java.lang.IndexOutOfBoundsException: toIndex = 3
  at java.util.AbstractList.subListRangeCheck(...)
  at datadog.smoketest.LogInjectionSmokeTest.parseTraceFromStdOut(LogInjectionSmokeTest.groovy:416)

The test exercises the buggy path deterministically via a ChunkedInputStream that returns one chunk per read() call. Both tests pass today and will start failing once OutputThreads is fixed. That makes them a ready regression test for the fix.

Motivation

The proximate trigger is a partial-read bug in OutputThreads.ProcessOutputRunnable.run(): when rc.read(buffer) returns a chunk with no newline AND the inner loop has consumed no lines yet, the fall-through branch decodes the partial buffer and adds it to testLogMessages as if it were a complete line. Under CI load the OS pipe splits the child's System.out.println("THIRDTRACEID <traceId> <spanId>") write. The smoke test then parses a truncated "THIRDTRACEID 12345" chunk via split(" ")[1..2]. The Groovy range translates to subList(1, 3) on a 2-element list, hence toIndex = 3.

Prior fixes (#10920, #10999, #11075) ruled out the assertion bug, the BaseApplication timeout, and process liveness, but left the partial-read issue in place. This reproducer isolates that issue without touching the production code path, so the fix can be reviewed against a deterministic test.

Additional Notes

  • The reproducer lives in dd-smoke-tests/src/test/java/, a new test source set for the parent module. Adding testImplementation(libs.junit.jupiter) makes the JUnit 5 API available at test-compile time; the JUnit Platform runner is already enabled by the project's default Gradle setup.
  • Verified deterministic across 5 consecutive runs locally.
  • A follow-up PR will fix OutputThreads and flip these tests to assert correct behavior.

Contributor Checklist

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

The dominant failure mode (31 of 41 reports in CI Visibility) is an
IndexOutOfBoundsException with toIndex=3 inside parseTraceFromStdOut.
The proximate trigger is a partial-read bug in OutputThreads: when
rc.read(buffer) returns a chunk with no newline and the inner loop has
consumed no lines yet, the fall-through branch adds the partial buffer
to testLogMessages as if it were a complete line. The OS pipe splits
the child's THIRDTRACEID println under CI load, leaving the smoke test
to parse a truncated "THIRDTRACEID 12345" chunk.

OutputThreadsTest exercises the buggy path deterministically via a
ChunkedInputStream that returns one chunk per read() call. Both tests
pass today and will start failing once OutputThreads is fixed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bm1549 bm1549 added type: enhancement Enhancements and improvements comp: testing Testing tag: no release notes Changes to exclude from release notes tag: flaky test Flaky tests tag: ai generated Largely based on code generated by an AI or LLM labels May 13, 2026
@bm1549 bm1549 closed this May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: testing Testing tag: ai generated Largely based on code generated by an AI or LLM tag: flaky test Flaky tests tag: no release notes Changes to exclude from release notes type: enhancement Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant