feat(codex): add compact observations with private JSON evidence

plugins/opengui/.codex-plugin/plugin.json

@@ -1,17 +1,17 @@
 {
   "name": "opengui",
   "version": "0.2.0",
-  "description": "Control authorized local Android devices from Codex on macOS, with screenshot-guided actions and a read-only device wall.",
+  "description": "Control authorized local Android or HarmonyOS devices from Codex on macOS, with screenshot-guided actions and a read-only device wall.",
   "author": { "name": "Core-Mate", "url": "https://github.com/Core-Mate" },
   "homepage": "https://github.com/Core-Mate/OpenGUI/tree/main/plugins/opengui",
   "repository": "https://github.com/Core-Mate/OpenGUI",
   "license": "MIT",
-  "keywords": ["android", "mobile", "local", "automation"],
+  "keywords": ["android", "harmonyos", "mobile", "local", "automation"],
   "skills": "./skills/",
   "interface": {
     "displayName": "OpenGUI",
-    "shortDescription": "Local Android control for Codex on macOS",
-    "longDescription": "Use screenshot-guided actions on one to four authorized local Android devices. Includes a read-only device wall. Requires local Codex on macOS, USB debugging, and an initial verified runtime download. Does not support cloud device access or concurrent control of the same phone by another agent.",
+    "shortDescription": "Local phone control for Codex on macOS",
+    "longDescription": "Use screenshot-guided actions on one to four authorized local Android or HarmonyOS devices. Includes a read-only device wall. Requires local Codex on macOS, USB debugging, and an initial verified runtime download. Does not support cloud device access or concurrent control of the same phone by another agent.",
     "developerName": "Core-Mate",
     "category": "Developer Tools",
     "capabilities": ["Interactive", "Read", "Write"],

plugins/opengui/README.md

@@ -41,3 +41,7 @@ pnpm package
 ```
 
 The browser test additionally needs development-only `agent-browser` and `ffmpeg`. It validates actual H.264 decode and canvas changes, first-frame receipts, continued playback after completion, and subscriber cleanup. End users do not need those tools. See the [candidate acceptance report](../../docs/plans/2026-09-13-viewer-candidate-acceptance.md) for separate host, device, performance, installation and release evidence. Candidate packaging alone does not pass those gates.
+
+## HarmonyOS screenshot control
+
+The source plugin also provides an experimental HDC + UiTest driver for native HarmonyOS. Android remains the default and retains its video-readiness gate. HarmonyOS uses screenshot sessions and a read-only screenshot wall, without scrcpy. See [setup and limits](docs/harmonyos.md) and [verification](docs/harmonyos-testing.md). This applies only to the standalone Codex plugin, not WorkBuddy, DSH, or the full stack.

plugins/opengui/README.zh-CN.md

@@ -27,3 +27,7 @@ bash scripts/install-macos.command --archive /绝对路径/opengui-codex-0.2.0.t
 回退前结束当前控制任务、关闭展示；使用保留的旧版安装器和旧归档重新安装。安装目录保留旧包及配置恢复记录。不要强杀其他宿主进程，不要整体覆盖配置或删除无关插件。
 
 两端分别独立构建和打包，不依赖 DSH。当前验证结果及尚未通过的项目见候选验收报告。浏览器支持解码、模拟视频通过、真机播放、宿主自动操作、发布上线是不同证据。
+
+## 鸿蒙截图控制
+
+源码插件提供实验性的 HDC + UiTest 原生鸿蒙驱动。Android 仍为默认平台并保留视频首帧门槛；鸿蒙使用截图会话和只读截图预览，不准备 scrcpy。见[使用与限制](docs/harmonyos.zh-CN.md)及[验证说明](docs/harmonyos-testing.md)。范围仅为独立 Codex 插件，不包含 WorkBuddy、DSH 或完整客户端/服务端移植。

plugins/opengui/docs/compact-observations.md

@@ -0,0 +1,42 @@
+# Compact observations and complete evidence
+
+Use `--compact` before the interface name:
+
+```sh
+node lib/cli.js --compact opengui_observe '{"sessionId":"returned-session-id"}'
+```
+
+The compact response preserves the current observation ID, JPEG coordinate space,
+layout stability, focused editable nodes and action assessment. It is primarily
+useful for HarmonyOS advisory layouts. Non-observation results and results without
+layouts remain unchanged. Do not interpret omitted nodes as absent elements.
+
+Complete observations are written to `observationPath` with owner-only read/write
+permissions. The JSON references the screenshot file without embedding its base64
+bytes. It can contain hidden advisory node text: keep it private and query only
+necessary fields. Default CLI output remains complete.
+
+Closing or cancelling a session removes its JPEG and JSON together. Expired
+session directories retain the existing pruning policy. Copy authorized evidence
+before closing. If the host reports a running process, wait for that same process;
+do not parse partial output or repeat an uncertain action.
+
+Before trying a changed source build, finish old sessions and stop its idle daemon;
+use a dedicated runtime directory during development. These source changes do not
+claim that an already published package has been upgraded.
+
+## Candidate verification
+
+On 2026-10-16, the full suite passed 138 tests across 22 files using two workers,
+followed by TypeScript/CLI build, package validation, packaging, installer shell
+syntax and isolated packaged-install checks. Local Node.js was 24.15.0 and pnpm
+was 10.31.0; the remote three-platform CI and its pinned versions remain a
+separate pending gate.
+
+Source commit `386e81e` was checked on TYR-AL00 through a new screenshot session.
+The compact observation omitted the 104 non-focused layout nodes while the
+complete private JSON retained them. The JSON had mode 0600 and matched the
+response's observation ID and screenshot metadata. The screenshot was inspected;
+closing the session removed both JPEG and JSON and the task-only daemon was
+stopped. This check did not repeat the driver's navigation smoke test or perform
+external business actions. Device content is not committed.

plugins/opengui/docs/harmonyos-testing.md

@@ -0,0 +1,56 @@
+# HarmonyOS driver verification
+
+## Reproduce the automated checks
+
+From `plugins/opengui`, run `pnpm check`, `pnpm package`,
+`bash -n scripts/install-macos.command`, and `node scripts/test-installer.mjs`.
+The existing CI matrix covers Linux, macOS arm64 and macOS Intel. Local macOS
+results do not establish that the remote matrix has passed.
+
+The tests cover Android regressions, HDC failure acknowledgments and argument
+quoting, device identity, layout limits and changes, coordinate scaling,
+API-dependent text input, permission assessment, cancellation and owned artifact
+cleanup. Android still requires video readiness; HarmonyOS explicitly rejects
+Android video preparation and opens a screenshot session.
+
+## Device evidence
+
+Historical physical-device verification used a TYR-AL00 at API 24, HDC 3.2.0d,
+and UiTest 6.0.2.3. It exercised observation, launch, Unicode input, long press,
+navigation, cancellation and session cleanup. Those records predate the current
+upstream merge and do not establish verification of this candidate.
+
+On 2026-10-16, source commit `371481a` passed a short physical-device smoke test
+on TYR-AL00 with HDC 3.2.0f. The workflow opened a screenshot control session
+without an Android video viewer, observed a stable launcher, swiped to the
+adjacent home page, inspected the resulting image, swiped back and inspected the
+restored home page. Closing the session removed all three runtime screenshots;
+the task-only idle daemon was then stopped. No messages or external business
+data were created or deleted. This is a narrow navigation check, not a repeat of
+the historical Unicode, launch or cancellation scenarios. The current API level
+and UiTest version were not separately recorded; capture passed the driver's
+capability probe. Raw device content is not committed.
+
+Local validation used Node.js 24.15.0 and the installed pnpm 10.31.0 runner with
+the existing locked dependencies. The first suite run passed 134 of 135 tests;
+one unchanged launcher fixture exceeded its five-second timeout. All eight
+launcher tests passed in a focused rerun with two workers. TypeScript/CLI build,
+package validation, packaging, installer shell syntax and isolated packaged
+installation passed. The installer uses fixture Codex/private Node hosts and a
+verified official scrcpy cache. Its first live download attempt timed out; the
+same pinned asset was fetched with curl, checked against its pinned SHA-256 and
+validated by the actual cache installer before reuse. This does not establish
+that a first-use video download succeeds on every network.
+
+The remote CI matrix uses Node.js 22.23.2 and pnpm 11.19.0 and remains a separate
+pending gate. No physical Android check was run in this smoke test.
+
+Suggested short workflow: open a screenshot session, observe the actual phone,
+perform ordinary authorized navigation, inspect the returned image, then close
+the session and verify that its owned runtime artifacts are removed. Do not send
+messages or delete unrelated data to prove driver behavior.
+
+API 17 hardware, physical multi-device operation, landscape/foldable layouts,
+actual unplug/reconnect, protected screens and long-running unattended operation
+are unverified. These limits should be disclosed rather than expanded into a
+claim that every supported phone or business workflow has passed.

plugins/opengui/docs/harmonyos.md

@@ -0,0 +1,96 @@
+# HarmonyOS screenshot driver
+
+The standalone Codex plugin can use a local HDC + UiTest driver instead of ADB.
+This scope is limited to `plugins/opengui`; it does not port WorkBuddy, DSH,
+the Android client, or the backend to HarmonyOS.
+
+## Requirements and source build
+
+- macOS arm64/x64 and Node.js 22.19 or newer.
+- A locally installed DevEco Studio/HDC SDK and an authorized USB-debugging phone.
+- Native HarmonyOS API 17 or newer with UiTest. Historical physical-device
+  coverage is API 24; API 17 has mocked coverage and is not a physical-device claim.
+
+From the plugin directory:
+
+```sh
+pnpm install --frozen-lockfile --ignore-scripts
+pnpm check
+export OPENGUI_PLATFORM=harmonyos
+# For a non-standard SDK location:
+# export OPENGUI_HDC_PATH=/absolute/path/to/sdk/toolchains/hdc
+node lib/cli.js --doctor
+node lib/cli.js opengui_list_devices '{}'
+node lib/cli.js opengui_open_session '{}'
+```
+
+Keep `OPENGUI_PLATFORM=harmonyos` on every invocation and retain the host-provided
+`CODEX_THREAD_ID`. Select the returned opaque device ID explicitly when more than
+one phone is attached. The default platform remains Android. Unsupported platform
+values fail rather than selecting another driver. Platform-specific daemon and
+startup-lock paths prevent Android and HarmonyOS from connecting to the same daemon.
+
+HDC is resolved from PATH, a standard DevEco Studio path, or the absolute
+`OPENGUI_HDC_PATH`. The plugin does not distribute HDC, install a phone agent,
+prepare scrcpy, or restart the system HDC service. Source functionality does not
+imply that an existing published installer includes this driver.
+
+## Screenshot workflow and supported operations
+
+HarmonyOS opens screenshot sessions directly. It does not provide an Android
+scrcpy video viewer or accept a video `viewerId`. `opengui_open_viewer` and
+`--prepare-video` reject HarmonyOS explicitly. Android retains its verified
+first-video-frame requirement. The HarmonyOS session's read-only device wall is
+an optional screenshot preview, not a real-time-video acceptance claim.
+
+Observe and view the returned JPEG before acting. Coordinates, target boxes and
+advisory layout bounds use the returned JPEG's pixel space. Every action consumes
+the latest `observationId`; failures require a fresh observation.
+
+| Action | Mapping and limits |
+| --- | --- |
+| `tap` / `long_press` | UiTest click / longClick at the visible target box center |
+| `swipe` | Screenshot coordinates are mapped to device pixels; duration becomes a clamped UiTest velocity |
+| `text` | Requires one focused editable node; API 18+ uses focused text input and API 17 uses coordinate input |
+| `key` | Back, Home and Enter; AppSwitch is explicitly unsupported |
+| `launch` | Requires both `packageName` (bundle) and `abilityName` (UIAbility) |
+| `wait` | Cancellable bounded wait followed by a fresh observation |
+
+Use `--interfaces` for the exact action schema. Classify consequential actions
+accurately and retain normal native one-action approval. This driver does not
+add pre-authorized history deletion or a test cleanup grant.
+
+## Advisory layouts and text results
+
+Screenshot capture is bracketed by layout fingerprints with bounded retries.
+Unstable layouts or a changed layout before coordinate/text dispatch are rejected.
+The layout is advisory: hidden Flutter nodes can be reported as visible, and
+fingerprints cannot detect every Canvas, animation, video, or purely visual change.
+Always confirm the actual target and result in the screenshot.
+
+`actionAssessment` distinguishes `performed`, `text_present`,
+`permission_required`, and `text_unconfirmed`. `performed` is an acknowledged
+command, not business success. `text_present` requires checking the complete
+field value. Unicode input may overwrite the clipboard or cause a permission
+prompt; never automatically grant permanent permission or blindly repeat input.
+
+## Cleanup and validation
+
+Close or cancel the session on completion. Runtime screenshots are removed with
+the session; copy authorized evidence beforehand. Temporary device artifacts use
+unique names and bounded cleanup. A disconnect or forced process termination can
+leave an artifact: clean only the specific run-owned file, never other tools' files.
+
+See [verification and limits](harmonyos-testing.md) and the
+[Chinese guide](harmonyos.zh-CN.md). The implementation relies on the official
+[UiTest](https://github.com/openharmony/docs/blob/master/zh-cn/application-dev/application-test/uitest-guidelines.md)
+and [HDC](https://github.com/openharmony/docs/blob/master/zh-cn/application-dev/dfx/hdc.md) interfaces.
+
+## Compact observations
+
+Pass `--compact` before the interface name to retain focused input nodes, layout
+stability, permission state and screenshot coordinates without printing the full
+layout. `observationPath` points to an owner-readable JSON containing the complete
+observation. Read only the fields needed and still inspect the screenshot.
+Default output remains complete; results without layouts are not shortened.
+See [output and evidence lifecycle](compact-observations.md).