test: verify native lifecycle recovery and Android accessibility

demo/tests/native-embedding/android/README.md

@@ -10,7 +10,7 @@ node run.mjs emulator-5554
 
 The runner uses the repository's Gradle wrapper to publish the SDK into `build/maven`, then builds the consumer without including the SDK source project. Dependencies resolve from the published AAR, POM/Gradle metadata, and Maven Central. Set `JAVA_HOME` to Java 21 and configure `ANDROID_HOME` (or `local.properties`) for your SDK installation.
 
-After a release reaches Maven Central, run `node run.mjs emulator-5554 --central`
+After a release reaches Maven Central, run `WORKERS_ANDROID_VERSION=0.1.2 node run.mjs emulator-5554 --central`
 to verify the released coordinate using the same acceptance checks. This skips
 local SDK publication and removes the local Maven repository from dependency
 resolution; only Google and Maven Central are used.
@@ -26,3 +26,82 @@ To verify the published JavaScript alongside the Maven SDK, set
 package before running with `--central`. The fixture uses that package's runtime
 and esbuild plugin instead of local `dist`. Match its source revision to the
 Maven artifact using the [dependency matrix](../../../../docs/native-embedding.md#dependency-setup).
+
+## Lifecycle and connection recovery
+
+Start `node ../network-server.mjs` in a separate terminal, then add `--lifecycle`
+to the runner invocation. The test sends the app to Home and restores its existing
+task by tapping its card in Android Recents, checks the Worker page, then verifies native HTTP
+success, a connection closed before response headers, and successful retry.
+ADB reverse temporarily exposes loopback port 18764 to the emulator and is removed
+after the run. Reserve that port for the fixture. Stop the server after testing.
+This models connection loss, not airplane mode or an offline cloud binding.
+
+For published npm 0.1.3, set `WORKERS_ANDROID_VERSION=0.1.2` with `--central` to
+select the compatible released Maven artifact. Without an override, local SDK
+publication and the consumer follow the repository package version.
+
+`build/acceptance/startup.json` records the selected build configuration and elapsed time including
+accessibility polling and tap dispatch; it is not a precise startup benchmark.
+`memory-ready.txt` is a `dumpsys meminfo` snapshot of the host application process.
+It does not include the entire WebView renderer process tree. Neither measurement
+establishes physical-device or production performance.
+
+If UIAutomator exposes an empty WebView hierarchy after navigation, the runner
+checks the visible DOM element through WebView debugging and uses its viewport
+bounds for the native tap. Each use is logged and `dom-visibility.png` records the
+screen. Native host/isolation checks continue to require Android accessibility
+nodes. This fallback is functional verification, not an accessibility acceptance.
+
+For a separate Release startup/memory sample, run with
+`WORKERS_BUILD_CONFIGURATION=release` and `--measure-only` (without `--lifecycle`).
+The fixture signs this local Release APK with its debug key for emulator install;
+it does not enable application debugging. This mode checks initial Worker Home,
+saves `measurement.png` and the metrics, then closes back to the host. It is not
+the full acceptance suite, which requires Debug WebView inspection. Preserve
+`build/acceptance/` from the full run before this measurement overwrites its files.
+
+## Investigate Android accessibility separately
+
+`accessibility-run.mjs` distinguishes the rendered DOM from the Android
+accessibility tree. A missing accessibility heading is a failure even when its
+DOM or screenshot is correct; it never uses the functional runner's DOM fallback
+to pass this test.
+
+Build and install the debug-only plain WebView comparator and instrumentation:
+
+```sh
+# Run from the repository root with Java 21 and ANDROID_HOME configured.
+demo/android/gradlew -p demo/tests/native-embedding/android -PcentralOnly -PsdkVersion=0.1.2 :app:assembleDebug :app:assembleDebugAndroidTest
+adb -s emulator-5556 install -r demo/tests/native-embedding/android/app/build/outputs/apk/debug/app-debug.apk
+adb -s emulator-5556 install -r demo/tests/native-embedding/android/app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk
+
+# One persistent UiAutomation connection, native taps, strict accessibility checks.
+node demo/tests/native-embedding/android/accessibility-run.mjs emulator-5556 sdk 20 --persistent
+node demo/tests/native-embedding/android/accessibility-run.mjs emulator-5556 default 20 --persistent
+
+# A fresh `uiautomator dump` connection at each observation; DOM checked separately.
+node demo/tests/native-embedding/android/accessibility-run.mjs emulator-5556 sdk 10
+```
+
+Modes are `sdk`, `default` (plain WebView, default profile), and `profile` (plain
+WebView, separate profile). The comparator has no Workers bridge or Service Worker.
+It is included only in Debug APKs. Each round visits Next and submits a form, so
+20 rounds assert 40 transitions. `WORKERS_A11Y_OUTPUT` selects the evidence directory;
+the default is `artifacts/android-accessibility/<connection>-<mode>` relative to the
+current directory. The runner validates the instrumentation result rather than
+trusting `am instrument`'s shell exit status. Failure evidence includes the native
+tree, visible DOM and screenshot. DOM capture is diagnostic, never a success oracle.
+
+### WebView provider requirements for verification
+
+Record the provider version with every run (`webview.txt`). WebView 134.0.6998.135
+can expose an empty native accessibility tree even in the plain WebView comparator.
+Use the verified 155.0.8059.43 provider to reproduce this environment's acceptance.
+Updating the verification environment resolved the observed failure; this does not
+introduce a new SDK minimum WebView version or certify TalkBack. Android remains
+in the verification scope. If the failure recurs on the verified provider, compare
+plain WebView and SDK results before attributing it to the SDK.
+
+Keep run results and investigation evidence in the pull request and generated
+artifacts, as required by the repository's contribution guidelines.

demo/tests/native-embedding/android/accessibility-run.mjs

@@ -0,0 +1,259 @@
+import { execFileSync } from "node:child_process";
+import { mkdir, writeFile } from "node:fs/promises";
+import { setTimeout as delay } from "node:timers/promises";
+import path from "node:path";
+
+// Run after assembling/installing the Debug comparator app. This deliberately
+// reconnects UiAutomation for every dump; DOM reads are a separate diagnostic.
+const [device, mode = "sdk", count = "10"] = process.argv.slice(2);
+if (
+  !device?.startsWith("emulator-") ||
+  !["sdk", "default", "profile"].includes(mode)
+)
+  throw new Error(
+    "Usage: node accessibility-run.mjs emulator-ID sdk|default|profile [rounds]",
+  );
+const rounds = Number(count);
+if (!Number.isInteger(rounds) || rounds < 1 || rounds > 100)
+  throw new Error("Invalid rounds");
+const connection = process.argv.includes("--persistent")
+  ? "persistent"
+  : "stateless";
+const output = path.resolve(
+  process.env.WORKERS_A11Y_OUTPUT ||
+    `artifacts/android-accessibility/${connection}-${mode}`,
+);
+await mkdir(output, { recursive: true });
+const adb = (...args) =>
+  execFileSync("adb", ["-s", device, ...args], {
+    encoding: "utf8",
+    timeout: 30000,
+  });
+if (connection === "persistent") {
+  const log = execFileSync(
+    "adb",
+    [
+      "-s",
+      device,
+      "shell",
+      "am",
+      "instrument",
+      "-w",
+      "-e",
+      "mode",
+      mode,
+      "-e",
+      "rounds",
+      String(rounds),
+      "probe.workersembed.test/probe.AccessibilityProbe",
+    ],
+    { encoding: "utf8", timeout: 180000 },
+  );
+  await writeFile(path.join(output, "run.log"), log);
+  await writeFile(
+    path.join(output, "webview.txt"),
+    adb("shell", "dumpsys", "webviewupdate"),
+  );
+  console.log(log);
+  const passed = log
+    .split("\n")
+    .filter((line) => line.startsWith("{"))
+    .map((line) => JSON.parse(line))
+    .some(
+      (result) =>
+        result.mode === mode &&
+        result.status === "passed" &&
+        result.rounds === rounds,
+    );
+  if (!passed) {
+    for (const name of [
+      "accessibility-failure.json",
+      "accessibility-failure-dom.json",
+      "accessibility-failure.png",
+    ]) {
+      const data = execFileSync("adb", [
+        "-s",
+        device,
+        "exec-out",
+        "run-as",
+        "probe.workersembed",
+        "cat",
+        `files/${name}`,
+      ]);
+      if (name.endsWith(".json")) JSON.parse(data.toString());
+      else if (
+        !data
+          .subarray(0, 8)
+          .equals(Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]))
+      )
+        throw new Error("Invalid screenshot evidence");
+      await writeFile(path.join(output, name), data);
+    }
+  }
+  process.exit(passed ? 0 : 1);
+}
+async function dump(label) {
+  const file = `/sdcard/workers-a11y-${label}.xml`;
+  adb("shell", "rm", "-f", file);
+  const log = adb("shell", "uiautomator", "dump", file);
+  if (!log.includes("dumped to:"))
+    throw new Error(`UIAutomator failed: ${log}`);
+  const xml = adb("exec-out", "cat", file);
+  await writeFile(path.join(output, `${label}.xml`), xml);
+  return xml;
+}
+adb("shell", "am", "force-stop", "probe.workersembed");
+adb("logcat", "-c");
+adb(
+  "shell",
+  "am",
+  "start",
+  "-W",
+  "-n",
+  `probe.workersembed/probe.${mode === "sdk" ? "Launcher" : "WebViewControl"}`,
+  "--ez",
+  "profile",
+  String(mode === "profile"),
+);
+if (mode === "sdk") {
+  let button;
+  for (let i = 0; i < 10 && !button; i++) {
+    const xml = await dump(`host-${i}`);
+    button = [...xml.matchAll(/<node\b[^>]*>/g)].find(([node]) =>
+      node.includes('text="Open Worker"'),
+    )?.[0];
+  }
+  const bounds = button?.match(/bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"/);
+  if (!bounds) throw new Error("No native Open Worker button");
+  adb(
+    "shell",
+    "input",
+    "tap",
+    String((+bounds[1] + +bounds[3]) / 2),
+    String((+bounds[2] + +bounds[4]) / 2),
+  );
+}
+const pid = adb("shell", "pidof", "probe.workersembed").trim();
+const port = adb(
+  "forward",
+  "tcp:0",
+  `localabstract:webview_devtools_remote_${pid}`,
+).trim();
+async function evaluate(expression) {
+  const targets = await (
+    await fetch(`http://127.0.0.1:${port}/json`, {
+      signal: AbortSignal.timeout(3000),
+    })
+  ).json();
+  const target = targets.find(
+    (t) =>
+      t.type === "page" &&
+      (t.url.startsWith("https://control.invalid/") ||
+        t.url.startsWith("https://workers-sdk.local/")) &&
+      !t.url.includes("__workers_capacitor_host"),
+  );
+  if (!target) return null;
+  return new Promise((resolve, reject) => {
+    const socket = new WebSocket(target.webSocketDebuggerUrl);
+    const timer = setTimeout(() => {
+      socket.close();
+      reject(new Error("CDP timeout"));
+    }, 5000);
+    socket.onopen = () =>
+      socket.send(
+        JSON.stringify({
+          id: 1,
+          method: "Runtime.evaluate",
+          params: { expression, returnByValue: true },
+        }),
+      );
+    socket.onmessage = ({ data }) => {
+      const message = JSON.parse(data);
+      if (message.id === 1) {
+        clearTimeout(timer);
+        socket.close();
+        if (message.error || message.result?.exceptionDetails)
+          reject(new Error(JSON.stringify(message)));
+        else resolve(message.result?.result?.value);
+      }
+    };
+    socket.onerror = () => {
+      clearTimeout(timer);
+      socket.close();
+      reject(new Error("CDP connection failed"));
+    };
+  });
+}
+async function awaitHeading(expected) {
+  for (let i = 0; i < 100; i++) {
+    let actual;
+    try {
+      actual = await evaluate("document.querySelector('h1')?.textContent");
+    } catch {
+      await delay(100);
+      continue;
+    }
+    if (actual === expected) return;
+    await delay(100);
+  }
+  throw new Error(`DOM heading missing: ${expected}`);
+}
+const results = [];
+try {
+  await awaitHeading("Worker Home");
+  for (let round = 0; round < rounds; round++) {
+    for (const [action, expected] of [
+      ["document.querySelector('a').click()", "Worker Next"],
+      ["document.querySelector('form').requestSubmit()", "Posted verified"],
+    ]) {
+      await evaluate(action);
+      await awaitHeading(expected);
+      const label = `${round}-${expected.split(" ")[0]}`;
+      const xml = await dump(label);
+      const visible = xml.includes(`text="${expected}"`);
+      const result = {
+        round,
+        expected,
+        domPresent: true,
+        accessibilityPresent: visible,
+        retries: [],
+      };
+      if (!visible) {
+        await writeFile(
+          path.join(output, `${label}.png`),
+          execFileSync("adb", ["-s", device, "exec-out", "screencap", "-p"]),
+        );
+        for (let retry = 0; retry < 3; retry++) {
+          const recovered = (await dump(`${label}-retry-${retry}`)).includes(
+            `text="${expected}"`,
+          );
+          result.retries.push(recovered);
+          if (recovered) break;
+        }
+      }
+      results.push(result);
+      console.log(JSON.stringify(result));
+      await writeFile(
+        path.join(output, "results.json"),
+        JSON.stringify(results, null, 2),
+      );
+    }
+  }
+} finally {
+  adb("forward", "--remove", `tcp:${port}`);
+  await writeFile(path.join(output, "runtime.log"), adb("logcat", "-d"));
+  await writeFile(
+    path.join(output, "webview.txt"),
+    adb("shell", "dumpsys", "webviewupdate"),
+  );
+}
+console.log(
+  JSON.stringify({
+    mode,
+    observations: results.length,
+    missing: results.filter((r) => !r.accessibilityPresent).length,
+  }),
+);
+
+if (results.some((result) => !result.accessibilityPresent))
+  process.exitCode = 1;

demo/tests/native-embedding/android/app/build.gradle

@@ -2,12 +2,14 @@ apply plugin: 'com.android.application'
 android {
     namespace 'probe'
     compileSdk 36
-    defaultConfig { applicationId 'probe.workersembed'; minSdk 24; targetSdk 36 }
+    buildTypes { release { signingConfig signingConfigs.debug } }
+    defaultConfig { applicationId 'probe.workersembed'; minSdk 24; targetSdk 36; testInstrumentationRunner 'probe.AccessibilityProbe' }
     compileOptions { sourceCompatibility JavaVersion.VERSION_21; targetCompatibility JavaVersion.VERSION_21 }
     packaging.resources { excludes += 'META-INF/INDEX.LIST'; merges += 'META-INF/io.netty.versions.properties' }
 }
 dependencies {
-    def sdkVersion = new groovy.json.JsonSlurper().parse(rootProject.file('../../../../package.json')).version
+    def sdkVersion = providers.gradleProperty('sdkVersion').getOrElse(new groovy.json.JsonSlurper().parse(rootProject.file('../../../../package.json')).version)
     implementation "io.github.rdlabo:cf-workers-native-android:${sdkVersion}"
+    debugImplementation 'androidx.webkit:webkit:1.14.0'
     implementation 'androidx.activity:activity:1.10.1'
 }