[Bug]: Java ValidatesRunner tests cannot run on Windows, beamTestPipelineOptions loses its JSON quotes

[Eliaaazzz]

What happened?

On Windows, every Java ValidatesRunner test fails during TestPipeline.create() before any pipeline runs, because the beamTestPipelineOptions system property arrives at the test JVM with its JSON quotes stripped.

Reproduce on Windows (any runner's VR task, this is the portable Spark one):

gradlew.bat :runners:spark:3:job-server:validatesPortableRunnerBatch --tests "*SplittableDoFnTest*"

All tests fail identically:

java.lang.RuntimeException at SplittableDoFnTest.java:140
  Caused by: com.fasterxml.jackson.databind.JsonMappingException
    Caused by: com.fasterxml.jackson.core.JsonParseException:
      Unexpected character ('-' (code 45)) in numeric value: expected digit (0-9) to follow minus sign
      at [Source: UNKNOWN; line: 1, column: 4] (through reference chain: java.lang.Object[][0])

SplittableDoFnTest.java:140 is @Rule public final transient TestPipeline p = TestPipeline.create().

Cause

BeamModulePlugin passes the options as compact JSON:

config.systemProperties.put("beamTestPipelineOptions", JsonOutput.toJson(beamTestPipelineOptions))

JsonOutput.toJson emits ["--runner=org.apache.beam...","--jobServerDriver=..."] with no spaces. Gradle forks the test worker with that as a -D argument, and Java's ProcessBuilder on Windows only quotes arguments that contain whitespace. The argument is therefore passed unquoted and the Windows C runtime strips the inner " characters, so TestPipeline reads:

[--runner=org.apache.beam...,--jobServerDriver=...]

TestPipeline.testingPipelineOptions() then does MAPPER.readValue(beamTestPipelineOptions, List.class) (TestPipeline.java:394-397). Jackson reads [, -, -, and reports "expected digit to follow minus sign" at column 4, which matches the error above.

This does not affect CI, which runs on Linux.

Prior art in the same file

BeamModulePlugin already handles this for one code path, wrapping each option so a layer of quoting survives:

// Windows handles quotation marks differently
if (pipelineOptionsString && System.properties['os.name'].toLowerCase().contains('windows')) {
  def allOptionsListFormatted = allOptionsList.collect { "\"$it\"" }
  pipelineOptionsStringFormatted = JsonOutput.toJson(allOptionsListFormatted)
} else if (pipelineOptionsString) {
  pipelineOptionsStringFormatted = JsonOutput.toJson(allOptionsList)
}

The ValidatesRunner task paths do not do this:

• createPortableValidatesRunnerTask, config.systemProperties.put("beamTestPipelineOptions", JsonOutput.toJson(beamTestPipelineOptions))
• createCrossLanguageValidatesRunnerTask, systemProperty "beamTestPipelineOptions", JsonOutput.toJson(config.javaPipelineOptions)

Applying the same treatment in those paths looks like it would fix it, though writing the options to a file and pointing the property at it would avoid the command-line quoting question entirely.

I ran into this while working on #19468 and could not get any local Java VR signal on Windows, so I had to rely on CI. Happy to put up a patch if the approach sounds right.

Issue Priority

Priority: 3 (minor)

Issue Components

• Component: Java SDK
• Component: Infrastructure

[Eliaaazzz]

.add-labels bug,'awaiting triage'

[Eliaaazzz]

Some follow-up, since the report above was based on a single run.

It is deterministic, not flaky

Three runs of the portable Spark batch VR on Windows, every test failing the same way at TestPipeline.create():

run	filter	result
1	*SplittableDoFnTest*	9 tests, 9 failed, 9 JsonParseException
2	*SplittableDoFnTest*	9 tests, 9 failed, 9 JsonParseException
3	*ParDoTest*	59 tests, 59 failed, 59 JsonParseException

77 of 77, and every failure carries the same through reference chain: java.lang.Object[][0]. It is not specific to a test class, and nothing passes.

Minimal reproduction of the mechanism

The quote loss can be reproduced without Gradle or Beam. This mimics how the Test task forks a worker: a -D property whose value is compact JSON, passed through ProcessBuilder.

// PropPrinter.java
public class PropPrinter {
  public static void main(String[] args) {
    System.out.println("CHILD_RECEIVED=" + System.getProperty("beamTestPipelineOptions"));
  }
}

// Launcher.java
import java.io.File;
import java.util.Arrays;

public class Launcher {
  public static void main(String[] args) throws Exception {
    String json = "[\"--runner=org.apache.beam.runners.portability.testing.TestPortableRunner\",\"--experiments=beam_fn_api\"]";
    System.out.println("PARENT_SENT   =" + json);
    String java = System.getProperty("java.home") + File.separator + "bin" + File.separator + "java";
    ProcessBuilder pb = new ProcessBuilder(
        Arrays.asList(java, "-DbeamTestPipelineOptions=" + json, "-cp", System.getProperty("user.dir"), "PropPrinter"));
    pb.redirectErrorStream(true);
    Process p = pb.start();
    System.out.print(new String(p.getInputStream().readAllBytes()));
    p.waitFor();
  }
}

On Windows with JDK 11:

PARENT_SENT   =["--runner=org.apache.beam.runners.portability.testing.TestPortableRunner","--experiments=beam_fn_api"]
CHILD_RECEIVED=[--runner=org.apache.beam.runners.portability.testing.TestPortableRunner,--experiments=beam_fn_api]

Every quote is gone, which is exactly the [--runner=... that Jackson rejects at the start of the array. ProcessBuilder only quotes arguments containing whitespace, and JsonOutput.toJson emits no whitespace, so the argument goes through unquoted and the C runtime strips the quotes. The same value survives on Linux, which is why CI never sees this.